IMAGE ENCODING / DECODING METHODS AND COMPRESSED VIDEO DATA TRANSMISSION DEVICES

Abstract

The invention proposes a method and apparatus for encoding / decoding images. The image decoding method includes steps to decode the current block's sub-block merge mode indicator, decode the current block's normal merge mode indicator if the sub-block merge mode indicator does not indicate sub-block merge mode, and decode the current block's sub-block merge mode information if the sub-block merge mode indicator does indicate sub-block merge mode.

Description

Method and device for encoding / decoding inter-screen prediction information

[0001] The present invention relates to a video encoding / decoding method, device, and recording medium storing a bitstream. Specifically, the present invention relates to a video encoding / decoding method and device for encoding / decoding various inter-screen prediction modes.

[0002] Recently, the demand for high-resolution, high-quality images, such as HD (High Definition) and UHD (Ultra High Definition) images, is increasing across various application fields. As image data becomes higher in resolution and quality, the relative amount of data increases compared to conventional image data. Therefore, transmitting image data using existing media such as wired or wireless broadband lines or storing it using existing storage media leads to increased transmission and storage costs. To address these issues arising as image data becomes higher in resolution and quality, highly efficient image encoding / decoding technologies for higher-resolution and higher-quality images are required.

[0003] There are various technologies for image compression, such as inter-picture prediction technology that predicts pixel values ​​included in the current picture from pictures before or after the current picture, intra-picture prediction technology that predicts pixel values ​​included in the current picture using pixel information in the current picture, transformation and quantization technology for compressing the energy of the residual signal, and entropy coding technology that assigns short codes to values ​​with high frequency of appearance and long codes to values ​​with low frequency of appearance. Using these image compression technologies, image data can be effectively compressed and transmitted or stored.

[0004] As inter-screen prediction modes become more diverse to improve the prediction performance of video encoding devices, the amount of data required to express the inter-screen prediction modes increases, which may result in compression bit rate overhead.

[0005] The present invention aims to provide a method capable of efficiently encoding / decoding various inter-screen prediction modes.

[0006] In addition, the present invention aims to provide a recording medium storing a bitstream generated by the video encoding / decoding method or device of the present invention.

[0007] A video decoding method according to the present invention may include a step of decoding a sub-block merge mode indicator of a current block, a step of decoding a regular merge mode indicator of the current block when the sub-block merge mode indicator indicates that the current block is not a sub-block merge mode, and a step of decoding sub-block merge mode information of the current block when the sub-block merge mode indicator indicates a sub-block merge mode.

[0008] In the above video decoding method, the regular merge mode indicator, when it is a first value, may indicate that a regular merge mode or a merge mode with motion vector difference is used to derive inter-screen prediction information of the current block, and when it is a second value, may indicate that an intra-screen combined merge mode or a geometric segmentation merge mode is used to derive inter-screen prediction information of the current block.

[0009] In the above image decoding method, whether or not to decode the regular merge mode indicator can be determined based on the indicator of whether or not to use combined prediction within the screen of the current block.

[0010] In the above image decoding method, whether or not to decode the regular merge mode indicator can be determined based on the skip indicator of the current block.

[0011] In the above video decoding method, whether or not to decode the regular merge mode indicator can be determined based on the slice type of the current block.

[0012] In the above image decoding method, the regular merge mode indicator may not be decoded if at least one of the horizontal size of the current block and the vertical size of the current block is greater than or equal to a predefined value.

[0013] In the above video decoding method, if the regular merge mode indicator is the first value, the step of decoding a merge mode indicator having a motion vector difference of the current block may be further included, and if the regular merge mode indicator is the second value, the step of decoding a merge mode indicator combined within the screen of the current block may be further included.

[0014] In the above image decoding method, the merge mode indicator with the motion vector difference may be regarded as a predefined value indicating a regular merge mode rather than a merge mode with the motion vector difference if the merge mode indicator with the motion vector difference is not decoded in the step of decoding the merge mode indicator with the motion vector difference.

[0015] In the above image decoding method, the merge mode indicator combined within the screen is:

[0016] In the step of decoding the combined merge mode indicator within the screen, if it is not decrypted, it can be set based on at least one of the regular merge mode indicator, the combined merge mode use indicator within the screen, the skip indicator, and the size of the current block.

[0017] A video encoding method according to the present invention may include a step of encoding a sub-block merge mode indicator of a current block, a step of encoding a regular merge mode indicator of the current block when the sub-block merge mode indicator indicates that the current block is not a sub-block merge mode, and a step of encoding sub-block merge mode information of the current block when the sub-block merge mode indicator indicates a sub-block merge mode.

[0018] In the above video encoding method, the regular merge mode indicator, when it is a first value, may indicate that a regular merge mode or a merge mode with motion vector difference is used to derive inter-screen prediction information of the current block, and when it is a second value, may indicate that an intra-screen combined merge mode or a geometric segmentation merge mode is used to derive inter-screen prediction information of the current block.

[0019] In the above image encoding method, whether or not to encode the regular merge mode indicator can be determined based on the indicator of whether or not to use combined prediction within the screen of the current block.

[0020] In the above video encoding method, whether or not the regular merge mode indicator is encoded can be determined based on the skip indicator of the current block.

[0021] In the above video encoding method, whether or not to encode the regular merge mode indicator can be determined based on the slice type of the current block.

[0022] In the above image encoding method, the regular merge mode indicator may not be encoded if at least one of the horizontal size of the current block and the vertical size of the current block is greater than or equal to a predefined value.

[0023] In the above video encoding method, if the regular merge mode indicator is the first value, the step of encoding a merge mode indicator having a motion vector difference of the current block may be further included, and if the regular merge mode indicator is the second value, the step of encoding a merge mode indicator combined within the screen of the current block may be further included.

[0024] In addition, the recording medium according to the present invention can store a bitstream generated by the image encoding method according to the present invention.

[0025] According to the present invention, encoding efficiency can be improved by defining the encoding / decoding order of various inter-screen prediction mode information.

[0026] In addition, according to the present invention, encoding efficiency can be improved by encoding / decoding inter-screen prediction mode information based on inter-screen prediction mode information encoded / decoded at a higher level.

[0027] In addition, according to the present invention, a recording medium storing a bitstream generated by the image encoding / decoding method or device of the present invention can be provided.

[0028] In addition, according to the present invention, the encoding and decoding efficiency of an image can be improved.

[0029] Figure 1 is a block diagram showing the configuration according to one embodiment of an encoding device to which the present invention is applied.

[0030] Figure 2 is a block diagram showing the configuration according to one embodiment of a decryption device to which the present invention is applied.

[0031] Figure 3 is a diagram schematically showing the division structure of an image when encoding and decoding an image.

[0032] Figure 4 is a diagram for explaining an embodiment of an on-screen prediction process.

[0033] Figure 5 is a diagram for explaining an embodiment of a screen-to-screen prediction process.

[0034] Figure 6 is a diagram for explaining the process of transformation and quantization.

[0035] Figure 7 is a diagram illustrating reference samples available for prediction within the screen.

[0036] FIG. 8 is a flowchart illustrating a method for encoding / decoding inter-screen prediction information according to an embodiment of the present invention.

[0037] FIGS. 9 to 12 are drawings for explaining the inter-screen prediction information syntax of the merge mode according to various embodiments of the present invention.

[0038] Figure 13 is a diagram for explaining an indicator for using non-regular merge mode with entropy decoding of a sequence parameter set.

[0039] FIG. 14 is a diagram for explaining the inter-screen prediction information syntax of the merge mode according to one embodiment of the present invention.

[0040] Figure 15 is a diagram for explaining an indicator for using affine adaptive motion vector difference of a sequence parameter set.

[0041] FIG. 16 is a diagram for explaining the inter-screen prediction information syntax of the merge mode according to one embodiment of the present invention.

[0042] FIG. 17 is a diagram illustrating syntax elements representing various inter-screen prediction modes according to one embodiment of the present invention.

[0043] Figure 18 is a flowchart illustrating an image decoding method according to an embodiment of the present invention.

[0044] FIG. 19 is a flowchart illustrating an image encoding method according to an embodiment of the present invention.

[0045] The present invention is susceptible to various modifications and embodiments. Therefore, specific embodiments are illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the present invention to specific embodiments, but rather to include all modifications, equivalents, and substitutes included within the spirit and scope of the present invention. In the drawings, similar reference numerals designate the same or similar functions throughout. The shape and size of elements in the drawings may be exaggerated for clarity. The detailed description of the exemplary embodiments described below refers to the accompanying drawings, which illustrate specific embodiments by way of example. These embodiments are described in sufficient detail to enable those skilled in the art to practice the embodiments. It should be understood that the various embodiments, while different, are not necessarily mutually exclusive. For example, specific shapes, structures, and characteristics described herein may be implemented in other embodiments without departing from the spirit and scope of the present invention. Furthermore, it should be understood that the location or arrangement of individual components within each disclosed embodiment may be changed without departing from the spirit and scope of the embodiment. Accordingly, the detailed description set forth below is not intended to be taken in a limiting sense, and the scope of the exemplary embodiments, if properly described, is defined only by the appended claims, along with the full scope equivalents to which such claims are entitled.

[0046] In the present invention, terms such as first, second, etc. may be used to describe various components, but the components should not be limited by these terms. These terms are used solely to distinguish one component from another. For example, without departing from the scope of the present invention, the first component may be referred to as the second component, and similarly, the second component may also be referred to as the first component. The term "and / or" includes a combination of multiple related described items or any of multiple related described items.

[0047] When a component of the present invention is referred to as being "connected" or "connected" to another component, it should be understood that it may be directly connected or connected to that other component, but that other components may also be present in between. Conversely, when a component is referred to as being "directly connected" or "directly connected" to another component, it should be understood that no other components are present in between.

[0048] The components shown in the embodiments of the present invention are independently depicted to represent different characteristic functions, and do not mean that each component is composed of separate hardware or a single software component. That is, each component is listed and included as a separate component for convenience of explanation, and at least two components among each component may be combined to form a single component, or a single component may be divided into multiple components to perform a function, and such integrated and separate embodiments of each component are also included in the scope of the present invention as long as they do not deviate from the essence of the present invention.

[0049] The terminology used in the present invention is only used to describe specific embodiments and is not intended to limit the present invention. The singular expression includes plural expressions unless the context clearly indicates otherwise. In the present invention, it should be understood that the terms "comprise" or "have" are intended to specify the presence of a feature, number, step, operation, component, part, or combination thereof described in the specification, but do not exclude in advance the possibility of the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof. In other words, the description of a specific configuration in the present invention as "comprising" does not exclude configurations other than the specified configuration, and means that additional configurations may be included in the scope of the implementation of the present invention or the technical idea of ​​the present invention.

[0050] Some components of the present invention may not be essential components that perform essential functions of the present invention, but may be optional components merely for performance enhancement. The present invention may be implemented by including only components essential to realizing the essence of the present invention, excluding components used solely for performance enhancement. A structure that includes only essential components, excluding optional components used solely for performance enhancement, is also within the scope of the present invention.

[0051] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. In describing the embodiments of this specification, if it is determined that a detailed description of a related known configuration or function may obscure the gist of this specification, the detailed description will be omitted. The same reference numerals will be used for identical components in the drawings, and duplicate descriptions of identical components will be omitted.

[0052] Hereinafter, the term "video" may refer to a single picture constituting a video, or may refer to the video itself. For example, "encoding and / or decoding of a video" may refer to "encoding and / or decoding of a video," or may refer to "encoding and / or decoding of one of the videos constituting the video."

[0053] Hereinafter, the terms “movie” and “video” may be used interchangeably and have the same meaning.

[0054] Hereinafter, the target image may be an encoding target image, which is the target of encoding, and / or a decoding target image, which is the target of decoding. Furthermore, the target image may be an input image input to an encoding device, or an input image input to a decoding device. Here, the target image may have the same meaning as the current image.

[0055] Hereinafter, the terms “image”, “picture”, “frame” and “screen” may be used interchangeably and have the same meaning.

[0056] Hereinafter, the target block may be an encoding target block that is the target of encoding and / or a decoding target block that is the target of decoding. Furthermore, the target block may be a current block that is the target of current encoding and / or decoding. For example, the terms "target block" and "current block" may be used interchangeably and have the same meaning.

[0057] Hereinafter, the terms "block" and "unit" may be used interchangeably and have the same meaning. Alternatively, "block" may refer to a specific unit.

[0058] Hereinafter, the terms “region” and “segment” may be used interchangeably.

[0059] Hereinafter, a specific signal may be a signal representing a specific block. For example, the original signal may be a signal representing a target block. The prediction signal may be a signal representing a predicted block. The residual signal may be a signal representing a residual block.

[0060] In the embodiments, each of the specified information, data, flags, indices, and elements, attributes, etc., can have a value. The value "0" of the information, data, flags, indices, and elements, attributes, etc., can represent logical false or a first predefined value. That is, the values ​​"0", false, logical false, and the first predefined value can be used interchangeably with each other. The value "1" of the information, data, flags, indices, and elements, attributes, etc., can represent logical true or a second predefined value. That is, the values ​​"1", true, logical true, and the second predefined value can be used interchangeably with each other.

[0061] When a variable such as i or j is used to represent a row, column, or index, the value of i can be an integer greater than or equal to 0, or an integer greater than or equal to 1. That is, in the embodiments, rows, columns, and indices can be counted from 0, or can be counted from 1.

[0062]

[0063] Glossary of Terms

[0064] Encoder: refers to a device that performs encoding. In other words, it can mean an encoding device.

[0065] Decoder: Refers to a device that performs decoding. In other words, it can refer to a decoding device.

[0066] Block: An MxN array of samples. Here, M and N can represent positive integer values, and a block can commonly represent a two-dimensional sample array. A block can represent a unit. The current block can represent an encoding target block that is the target of encoding during encoding, and a decoding target block that is the target of decoding during decoding. In addition, the current block can be at least one of an encoding block, a prediction block, a residual block, and a transform block.

[0067] Sample: The basic unit that constitutes a block. Bit depth (B) d ) from 0 to 2 Bd - It can be expressed as a value up to 1. In the present invention, a sample can be used with the same meaning as a pixel or a pixel. That is, a sample, a pixel, and a pixel can have the same meaning.

[0068] Unit: This may refer to a unit of image encoding and decoding. In image encoding and decoding, a unit may be a region into which one image is divided. In addition, a unit may refer to a divided unit when one image is divided into detailed units and encoded or decoded. That is, one image may be divided into multiple units. In image encoding and decoding, predefined processing may be performed for each unit. One unit may be further divided into sub-units having a smaller size than the unit. Depending on the function, a unit may refer to a block, a macroblock, a coding tree unit, a coding tree block, a coding unit, a coding block, a prediction unit, a prediction block, a residual unit, a residual block, a transform unit, a transform block, etc. In addition, a unit may mean a luminance component block, a corresponding chroma component block, and a syntax element for each block to distinguish it from a block. The unit may have various sizes and shapes, and in particular, the shape of the unit may include a geometric shape that can be expressed in two dimensions, such as a square, a rectangle, a trapezoid, a triangle, a pentagon, etc. In addition, the unit information may include at least one or more of the type of the unit indicating an encoding unit, a prediction unit, a residual unit, a transformation unit, etc., the size of the unit, the depth of the unit, the encoding and decoding order of the unit, etc.

[0069] Coding Tree Unit: It consists of one luminance component (Y) coding tree block and two chrominance component (Cb, Cr) coding tree blocks related to it. It may also mean including the blocks and syntax elements for each block. Each coding tree unit may be split using one or more splitting methods such as a quad tree, a binary tree, a ternary tree, etc. to form lower units such as a coding unit, a prediction unit, a transformation unit, etc. It may be used as a term to refer to a sample block that becomes a processing unit in the process of decoding / encoding an image, such as splitting an input image. Here, a quad tree may mean a quarternary tree.

[0070] If the size of the coding block falls within a predetermined range, splitting may only be possible using a quadtree. Here, the predetermined range may be defined as at least one of the maximum size and the minimum size of the coding block that can be split using only a quadtree. Information indicating the maximum / minimum size of the coding block that allows splitting using a quadtree form may be signaled through a bitstream, and the information may be signaled in units of at least one of a sequence, a picture parameter, a tile group, or a slice (segment). Alternatively, the maximum / minimum size of the coding block may be a fixed size pre-set in the encoder / decoder. For example, if the size of the coding block is 256x256 to 64x64, splitting may only be possible using a quadtree. Alternatively, if the size of the coding block is larger than the size of the maximum transform block, splitting may only be possible using a quadtree. In this case, the block to be split may be at least one of the coding block or the transform block. In such cases, the information indicating the division of the coding block (e.g., split_flag) may be a flag indicating whether or not a quadtree is split. If the size of the coding block falls within a certain range, splitting may only be possible using a binary tree or a 3-split tree. In this case, the above description regarding the quadtree may be equally applicable to the binary tree or the 3-split tree.

[0071] Coding Tree Block: This term can be used to refer to any one of the Y coding tree block, Cb coding tree block, and Cr coding tree block.

[0072] Neighbor block: This may refer to a block adjacent to the current block. A block adjacent to the current block may refer to a block whose boundary is adjacent to the current block or a block located within a certain distance from the current block. A neighboring block may refer to a block adjacent to a vertex of the current block. Here, a block adjacent to a vertex of the current block may refer to a block vertically adjacent to a neighboring block horizontally adjacent to the current block, or a block horizontally adjacent to a neighboring block vertically adjacent to the current block. A neighboring block may also refer to a restored neighboring block.

[0073] Reconstructed Neighbor Block: This may refer to a neighboring block that has already been spatially / temporally encoded or decoded around the current block. In this case, the reconstructed neighboring block may refer to a reconstructed neighboring unit. The reconstructed spatial neighboring block may be a block within the current picture that has already been reconstructed through encoding and / or decoding. The reconstructed temporal neighboring block may be a reconstructed block or a neighboring block of a location corresponding to the current block of the current picture within the reference image.

[0074] Unit Depth: This can indicate the degree to which a unit has been divided. In a tree structure, the root node corresponds to the initial, undivided unit. This root node can be referred to as the root node. Furthermore, the root node can have the minimum depth. In this case, the root node can have a depth of level 0. A node with a depth of level 1 can represent a unit created by splitting the initial unit once. A node with a depth of level 2 can represent a unit created by splitting the initial unit twice. A node with a depth of level n can represent a unit created by splitting the initial unit n times. A leaf node can be the lowest node and cannot be further divided. The depth of a leaf node can be the maximum level. For example, the default value for the maximum level can be 3. The root node can be said to have the shallowest depth, and the leaf node can have the deepest depth. Additionally, when units are represented in a tree structure, the level at which a unit exists can indicate unit depth.

[0075] Bitstream: This can mean a string of bits containing encoded image information.

[0076] Parameter Set: Corresponds to header information among the structures in the bitstream. At least one of a video parameter set, a sequence parameter set, a picture parameter set, and an adaptation parameter set may be included in the parameter set. In addition, the parameter set may also include tile group, slice header, and tile header information. In addition, the tile group may mean a group including multiple tiles, and may have the same meaning as a slice.

[0077] An adaptation parameter set may refer to a parameter set that can be shared and referenced by different pictures, subpictures, slices, tile groups, tiles, or bricks. Furthermore, subpictures, slices, tile groups, tiles, or bricks within a picture may reference different adaptation parameter sets and use information within the adaptation parameter set.

[0078] Additionally, adaptation parameter sets can reference different adaptation parameter sets using identifiers of different adaptation parameter sets in subpictures, slices, tile groups, tiles, or bricks within a picture.

[0079] Additionally, adaptation parameter sets can reference different adaptation parameter sets within a slice, tile group, tile, or brick within a subpicture using identifiers of different adaptation parameter sets.

[0080] Additionally, different adaptation parameter sets can be referenced by using identifiers of different adaptation parameter sets within tiles or bricks within a slice.

[0081] Additionally, the adaptation parameter sets can reference different adaptation parameter sets using identifiers of different adaptation parameter sets within the bricks within the tile.

[0082] By including information about an adaptation parameter set identifier in the parameter set or header of the above subpicture, an adaptation parameter set corresponding to the adaptation parameter set identifier can be used in the subpicture.

[0083] By including information about an adaptation parameter set identifier in the parameter set or header of the above tile, an adaptation parameter set corresponding to the adaptation parameter set identifier can be used in the tile.

[0084] By including information about an adaptation parameter set identifier in the header of the above brick, an adaptation parameter set corresponding to the adaptation parameter set identifier can be used in the brick.

[0085] The above picture can be divided into one or more rows of tiles and one or more columns of tiles.

[0086] The above subpicture may be divided into one or more tile rows and one or more tile columns within the picture. The above subpicture is a rectangular / square region within the picture and may include one or more CTUs. Additionally, one subpicture may include at least one tile / brick / slice.

[0087] The above tile is a rectangular / square area within the picture and may include one or more CTUs. In addition, the tile may be divided into one or more bricks.

[0088] The above brick may represent one or more CTU rows within a tile. A tile may be divided into one or more bricks, and each brick may have at least one CTU row. A tile that is not divided into two or more tiles may also represent a brick.

[0089] The above slice may include one or more tiles within a picture, and may include one or more bricks within a tile.

[0090] Parsing: This can mean entropy decoding a bitstream to determine the value of a syntax element, or it can mean entropy decoding itself.

[0091] Symbol: It can mean at least one of the syntactic elements of the target unit for encoding / decoding, the coding parameter, the value of the transform coefficient, etc. In addition, the symbol can mean the target of entropy encoding or the result of entropy decoding.

[0092] Prediction Mode: This can be information indicating a mode in which encoding / decoding is performed with intra-screen prediction or a mode in which encoding / decoding is performed with inter-screen prediction.

[0093] Prediction Unit: This can refer to the basic unit used when performing predictions, such as inter-screen prediction, intra-screen prediction, inter-screen compensation, intra-screen compensation, and motion compensation. A single prediction unit can be divided into multiple partitions or multiple sub-prediction units of smaller sizes. Multiple partitions can also be the basic unit for performing prediction or compensation. Partitions created by dividing a prediction unit can also be prediction units.

[0094] Prediction Unit Partition: This can refer to the form in which the prediction unit is divided.

[0095] Reference Picture List: This may refer to a list containing one or more reference pictures used for inter-frame prediction or motion compensation. The types of reference picture lists may include LC (List Combined), L0 (List 0), L1 (List 1), L2 (List 2), L3 (List 3), etc., and one or more reference picture lists may be used for inter-frame prediction.

[0096] Inter-Prediction Indicator: This can indicate the inter-prediction direction (unidirectional prediction, bidirectional prediction, etc.) of the current block. Alternatively, it can indicate the number of reference images used to generate the prediction block for the current block. Alternatively, it can indicate the number of prediction blocks used when performing inter-prediction or motion compensation for the current block.

[0097] Prediction list utilization flag: Indicates whether a prediction block is generated using at least one reference picture within a specific reference picture list. The prediction list utilization flag can be used to derive an inter-screen prediction indicator, and conversely, the prediction list utilization flag can be derived using the inter-screen prediction indicator. For example, when the prediction list utilization flag indicates a first value of 0, it can indicate that a prediction block is not generated using a reference picture within the reference picture list, and when it indicates a second value of 1, it can indicate that a prediction block can be generated using the reference picture list.

[0098] Reference Picture Index: This may refer to an index that indicates a specific reference picture in the reference picture list.

[0099] Reference Picture: This may refer to an image referenced by a specific block for inter-frame prediction or motion compensation. Alternatively, the reference picture may be an image that includes a reference block referenced by the current block for inter-frame prediction or motion compensation. Hereinafter, the terms "reference picture" and "reference picture" may be used interchangeably and have the same meaning.

[0100] Motion Vector: This can be a two-dimensional vector used for inter-frame prediction or motion compensation. A motion vector can indicate the offset between a target block to be encoded / decoded and a reference block. For example, (mvX, mvY) can represent a motion vector. mvX can represent the horizontal component, and mvY can represent the vertical component.

[0101] Search Range: The search range can be a two-dimensional region where motion vectors are searched during inter-frame prediction. For example, the search range can have dimensions MxN, where M and N can each be positive integers.

[0102] Motion Vector Candidate: When predicting a motion vector, this can refer to a block that becomes a prediction candidate or the motion vector of that block. Additionally, the motion vector candidate can be included in a motion vector candidate list.

[0103] Motion Vector Candidate List: This may refer to a list constructed using one or more motion vector candidates.

[0104] Motion Vector Candidate Index: This may refer to an indicator pointing to a motion vector candidate within the motion vector candidate list. It may be the index of a motion vector predictor.

[0105] Motion Information: may mean information including at least one of a motion vector, a reference image index, an inter-screen prediction indicator, a prediction list utilization flag, reference image list information, a reference image, a motion vector candidate, a motion vector candidate index, a merge candidate, and a merge index.

[0106] Merge Candidate List: This may refer to a list constructed using one or more merge candidates.

[0107] Merge Candidate: This can refer to a spatial merge candidate, a temporal merge candidate, a combined merge candidate, a combined bi-prediction merge candidate, a zero-merge candidate, etc. Merge candidates can include motion information such as inter-frame prediction indicators, reference image indices for each list, motion vectors, prediction list utilization flags, and inter-frame prediction indicators.

[0108] Merge Index: This may refer to an indicator that points to a merge candidate within the merge candidate list. Furthermore, the merge index may indicate the block that derived the merge candidate among the restored blocks spatially and temporally adjacent to the current block. Furthermore, the merge index may indicate at least one piece of motion information associated with the merge candidate.

[0109] Transform Unit: This can refer to a basic unit when performing residual signal encoding / decoding, such as transform, inverse transform, quantization, inverse quantization, and transform coefficient encoding / decoding. One transform unit can be divided into multiple lower transform units having smaller sizes. Here, the transform / inverse transform can include at least one of a primary transform / inverse transform and a secondary transform / inverse transform.

[0110] Scaling: This can refer to the process of multiplying a quantized level by a factor. The result of scaling the quantized level can be a transform coefficient. Scaling can also be called dequantization.

[0111] Quantization Parameter: This can refer to a value used when generating quantized levels using transform coefficients in quantization. Alternatively, it can refer to a value used when generating transform coefficients by scaling quantized levels in dequantization. The quantization parameter can be a value mapped to the quantization step size.

[0112] Delta Quantization Parameter: This can mean the difference value between the predicted quantization parameter and the quantization parameter of the target unit to be encoded / decoded.

[0113] Scan: This can refer to a method for arranging the order of coefficients within a unit, block, or matrix. For example, sorting a two-dimensional array into a one-dimensional array is called a scan. Alternatively, sorting a one-dimensional array into a two-dimensional array can also be called a scan or inverse scan.

[0114] Transform Coefficient: This may refer to a coefficient value generated after performing a transform in an encoder. Alternatively, it may refer to a coefficient value generated after performing at least one of entropy decoding and dequantization in a decoder. The meaning of the transform coefficient may also include a quantized level or a quantized transform coefficient level that applies quantization to a transform coefficient or residual signal.

[0115] Quantized Level: This can refer to the value generated by performing quantization on a transform coefficient or residual signal in an encoder. Alternatively, it can refer to the value that is the target of dequantization before dequantization is performed in a decoder. Similarly, the quantized transform coefficient level, which is the result of transformation and quantization, can also be included in the meaning of the quantized level.

[0116] Non-zero Transform Coefficient: This can mean a transform coefficient whose magnitude is not 0, a transform coefficient level whose magnitude is not 0, or a quantized level.

[0117] Quantization Matrix: This can refer to a matrix used in the quantization or dequantization process to improve the subjective or objective image quality. The quantization matrix can also be called a scaling list.

[0118] Quantization Matrix Coefficient: This can refer to each element within a quantization matrix. Quantization matrix coefficients can also be called matrix coefficients.

[0119] Default Matrix: This can refer to a predetermined quantization matrix that is predefined in the encoder and decoder.

[0120] Non-default Matrix: This can refer to a quantization matrix that is not predefined in the encoder and decoder and is signaled by the user.

[0121] Statistical value: The statistical value for at least one variable, encoding parameter, constant, etc. having specific operable values ​​may be at least one of the mean, weighted mean, weighted sum, minimum, maximum, mode, median, and interpolation values ​​of the specific values.

[0122] Figure 1 is a block diagram showing the configuration according to one embodiment of an encoding device to which the present invention is applied.

[0123] The encoding device (100) may be an encoder, a video encoding device, or an image encoding device. A video may include one or more images. The encoding device (100) may sequentially encode one or more images.

[0124] Referring to FIG. 1, the encoding device (100) may include a motion prediction unit (111), a motion compensation unit (112), an intra prediction unit (120), a switch (115), a subtractor (125), a transformation unit (130), a quantization unit (140), an entropy encoding unit (150), an inverse quantization unit (160), an inverse transformation unit (170), an adder (175), a filter unit (180), and a reference picture buffer (190).

[0125] The encoding device (100) can perform encoding on an input image in intra mode and / or inter mode. In addition, the encoding device (100) can generate a bitstream including encoded information through encoding the input image and output the generated bitstream. The generated bitstream can be stored in a computer-readable recording medium or can be streamed through a wired / wireless transmission medium. When the intra mode is used as the prediction mode, the switch (115) can be switched to intra, and when the inter mode is used as the prediction mode, the switch (115) can be switched to inter. Here, the intra mode can mean an intra-screen prediction mode, and the inter mode can mean an inter-screen prediction mode. The encoding device (100) can generate a prediction block for an input block of the input image. In addition, after the prediction block is generated, the encoding device (100) can encode a residual block using a residual of the input block and the prediction block. The input image may be referred to as the current image, which is the current encoding target. The input block may be referred to as the current block, which is the current encoding target, or the encoding target block.

[0126] When the prediction mode is intra mode, the intra prediction unit (120) can use samples of blocks already encoded / decoded around the current block as reference samples. The intra prediction unit (120) can perform spatial prediction on the current block using the reference samples, and can generate prediction samples for the input block through spatial prediction. Here, intra prediction can mean prediction within the screen.

[0127] When the prediction mode is inter mode, the motion prediction unit (111) can search for an area that best matches the input block from the reference image during the motion prediction process and derive a motion vector using the searched area. At this time, the area can be used as a search area. The reference image can be stored in the reference picture buffer (190). Here, when encoding / decoding for the reference image is processed, it can be stored in the reference picture buffer (190).

[0128] The motion compensation unit (112) can generate a prediction block for the current block by performing motion compensation using a motion vector. Here, inter prediction may mean inter-screen prediction or motion compensation.

[0129] The above motion prediction unit (111) and motion compensation unit (112) can generate a prediction block by applying an interpolation filter to a portion of the reference image when the value of the motion vector does not have an integer value. In order to perform inter-screen prediction or motion compensation, it is possible to determine whether the motion prediction and motion compensation method of the prediction unit included in the corresponding encoding unit is a skip mode, a merge mode, an advanced motion vector prediction (AMVP) mode, or a current picture reference mode based on the encoding unit, and perform inter-screen prediction or motion compensation according to each mode.

[0130] The subtractor (125) can generate a residual block using the difference between the input block and the predicted block. The residual block may also be referred to as a residual signal. The residual signal may refer to the difference between the original signal and the predicted signal. Alternatively, the residual signal may be a signal generated by transforming, quantizing, or transforming and quantizing the difference between the original signal and the predicted signal. The residual block may be a residual signal in block units.

[0131] The transform unit (130) can perform a transform on the residual block to generate a transform coefficient and output the generated transform coefficient. Here, the transform coefficient may be a coefficient value generated by performing a transform on the residual block. When the transform skip mode is applied, the transform unit (130) may also skip the transform on the residual block.

[0132] Quantized levels can be generated by applying quantization to transform coefficients or residual signals. In the following embodiments, quantized levels may also be referred to as transform coefficients.

[0133] The quantization unit (140) can generate a quantized level by quantizing a transform coefficient or residual signal according to a quantization parameter, and can output the generated quantized level. At this time, the quantization unit (140) can quantize the transform coefficient using a quantization matrix.

[0134] The entropy encoding unit (150) can generate a bitstream by performing entropy encoding according to a probability distribution on values ​​produced by the quantization unit (140) or coding parameter values ​​produced during the encoding process, and can output the bitstream. The entropy encoding unit (150) can perform entropy encoding on information about image samples and information for decoding the image. For example, the information for decoding the image can include syntax elements, etc.

[0135] When entropy encoding is applied, a small number of bits are allocated to symbols with a high occurrence probability, and a large number of bits are allocated to symbols with a low occurrence probability, thereby representing the symbols, whereby the size of the bit string for the symbols to be encoded can be reduced. The entropy encoding unit (150) can use an encoding method such as exponential Golomb, Context-Adaptive Variable Length Coding (CAVLC), or Context-Adaptive Binary Arithmetic Coding (CABAC) for entropy encoding. For example, the entropy encoding unit (150) can perform entropy encoding using a Variable Length Coding / Code (VLC) table. In addition, the entropy encoding unit (150) may perform arithmetic encoding using the binarization method, probability model, and context model derived from the binarization method of the target symbol and the probability model of the target symbol / bin.

[0136] The entropy encoding unit (150) can change a two-dimensional block form coefficient into a one-dimensional vector form through a transform coefficient scanning method to encode a transform coefficient level (quantized level).

[0137] Coding parameters may include not only information (flags, indices, etc.) encoded by the encoder and signaled to the decoder, such as syntax elements, but also information derived during the encoding or decoding process, and may mean information required when encoding or decoding an image. For example, unit / block size, unit / block depth, unit / block partition information, unit / block shape, unit / block partition structure, whether to partition in quadtree shape, whether to partition in binary tree shape, partition direction in binary tree shape (horizontal direction or vertical direction), partition type in binary tree shape (symmetric partition or asymmetric partition), whether to partition in 3-partition tree shape, partition direction in 3-partition tree shape (horizontal direction or vertical direction), partition type in 3-partition tree shape (symmetric partition or asymmetric partition), whether to partition in composite tree shape, partition direction in composite tree shape (horizontal direction or vertical direction), partition type in composite tree shape (symmetric partition or asymmetric partition), partition tree in composite tree shape (binary tree or 3-partition tree), prediction mode (intra-screen prediction or inter-screen prediction), intra-screen luminance prediction mode / direction, intra-screen chrominance prediction mode / direction, intra-screen partition information, inter-screen partition information, encoding block partition flag, prediction block partition flag, transform block partition flag, reference sample filtering method, reference sample filter tab, reference sample filter coefficients, prediction block filtering method, prediction block filter Tab, prediction block filter coefficient, prediction block boundary filtering method, prediction block boundary filter tab, prediction block boundary filter coefficient, intra-screen prediction mode, inter-screen prediction mode, motion information, motion vector, motion vector difference, reference picture index, inter-screen prediction direction, inter-screen prediction indicator, prediction list utilization flag, reference picture list, reference picture, motion vector prediction index, motion vector prediction candidate, motion vector candidate list, whether to use merge mode, merge index, merge candidate, merge candidate list, whether to use skip mode,Interpolation filter type, interpolation filter tap, interpolation filter coefficient, motion vector size, motion vector representation accuracy, transform type, transform size, information on whether to use the first transform, information on whether to use the second transform, first transform index, second transform index, information on whether there is a residual signal, coded block pattern, coded block flag, quantization parameter, residual quantization parameter, quantization matrix, whether to apply an intra-screen loop filter, intra-screen loop filter coefficients, intra-screen loop filter tap, intra-screen loop filter shape / shape, whether to apply a deblocking filter, deblocking filter coefficients, deblocking filter tap, deblocking filter strength, deblocking filter shape / shape, whether to apply an adaptive sample offset, adaptive sample offset value, adaptive sample offset category, adaptive sample offset type, whether to apply an adaptive loop filter, adaptive loop filter coefficients, adaptive loop filter tap, adaptive loop filter shape / shape, binarization / inverse binarization method, context model determination method, context model update method, whether to perform a regular mode, Whether to perform bypass mode, context bin, bypass bin, significant coefficient flag, last significant coefficient flag, coefficient group unit encoding flag, last significant coefficient position, flag whether coefficient value is greater than 1, flag whether coefficient value is greater than 2, flag whether coefficient value is greater than 3, remaining coefficient value information, sign information, restored luminance sample, restored chrominance sample, residual luminance sample, residual chrominance sample, luminance transform coefficient, chrominance transform coefficient, luminance quantized level, chrominance quantized level, transform coefficient level scanning method, size of decoder lateral motion vector search region, shape of decoder lateral motion vector search region, number of decoder lateral motion vector searches, CTU size information, minimum block size information, maximum block size information, maximum block depth information, minimum block depth information, image display / output order, slice identification information, slice type,At least one value or a combined form of slice division information, tile group identification information, tile group type, tile group division information, tile identification information, tile type, tile division information, picture type, input sample bit depth, restored sample bit depth, residual sample bit depth, transform coefficient bit depth, quantized level bit depth, information about a luminance signal, and information about a chrominance signal may be included in the encoding parameter.

[0138] Here, signaling a flag or index may mean that the encoder entropy encodes the flag or index and includes it in the bitstream, and that the decoder entropy decodes the flag or index from the bitstream.

[0139] When the encoding device (100) performs encoding through inter prediction, the encoded current image can be used as a reference image for other images to be processed later. Accordingly, the encoding device (100) can reconstruct or decode the encoded current image again and store the reconstructed or decoded image as a reference image in the reference picture buffer (190).

[0140] The quantized level can be dequantized in the dequantization unit (160) and inversely transformed in the inverse transformation unit (170). The dequantized and / or inversely transformed coefficients can be combined with a prediction block through an adder (175), and a reconstructed block can be generated by combining the dequantized and / or inversely transformed coefficients and the prediction block. Here, the dequantized and / or inversely transformed coefficients refer to coefficients on which at least one of dequantization and inverse transformation has been performed, and may refer to a reconstructed residual block.

[0141] The restoration block may pass through a filter unit (180). The filter unit (180) may apply at least one of a deblocking filter, a sample adaptive offset (SAO), an adaptive loop filter (ALF), etc. to the restoration sample, restoration block, or restoration image. The filter unit (180) may also be referred to as an in-loop filter.

[0142] A deblocking filter can remove block distortion that occurs at the boundaries between blocks. Whether to apply a deblocking filter to the current block can be determined based on the samples contained in several columns or rows within the block. When applying a deblocking filter to a block, different filters can be applied depending on the required deblocking filtering strength.

[0143] Sample adaptive offset can be used to compensate for encoding errors by adding an appropriate offset value to sample values. Sample adaptive offset can compensate for the offset from the original image on a sample-by-sample basis in a deblocked image. This can be done by dividing the samples contained in the image into a fixed number of regions, determining the regions to be offset, and applying the offset to those regions, or by applying the offset based on the edge information of each sample.

[0144] An adaptive loop filter can perform filtering based on a comparison between a reconstructed image and the original image. By dividing the samples contained in the image into predetermined groups and determining the filter to be applied to each group, filtering can be performed differentially for each group. Information regarding whether to apply an adaptive loop filter can be signaled for each coding unit (CU), and the shape and filter coefficients of the adaptive loop filter applied to each block can vary.

[0145] The restored block or restored image that has passed through the filter unit (180) may be stored in the reference picture buffer (190). The restored block that has passed through the filter unit (180) may be a part of the reference image. In other words, the reference image may be a restored image composed of restored blocks that have passed through the filter unit (180). The stored reference image may be used for inter-screen prediction or motion compensation thereafter.

[0146] Figure 2 is a block diagram showing the configuration according to one embodiment of a decryption device to which the present invention is applied.

[0147] The decoding device (200) may be a decoder, a video decoding device, or an image decoding device.

[0148] Referring to FIG. 2, the decoding device (200) may include an entropy decoding unit (210), an inverse quantization unit (220), an inverse transformation unit (230), an intra prediction unit (240), a motion compensation unit (250), an adder (255), a filter unit (260), and a reference picture buffer (270).

[0149] The decoding device (200) can receive a bitstream output from the encoding device (100). The decoding device (200) can receive a bitstream stored in a computer-readable recording medium, or a bitstream streamed through a wired / wireless transmission medium. The decoding device (200) can perform decoding on the bitstream in intra mode or inter mode. In addition, the decoding device (200) can generate a restored image or a decoded image through decoding, and can output the restored image or the decoded image.

[0150] If the prediction mode used for decryption is intra mode, the switch can be switched to intra. If the prediction mode used for decryption is inter mode, the switch can be switched to inter.

[0151] The decoding device (200) can decode the input bitstream to obtain a reconstructed residual block and generate a prediction block. Once the reconstructed residual block and the prediction block are obtained, the decoding device (200) can generate a reconstructed block to be decoded by adding the reconstructed residual block and the prediction block. The block to be decoded may be referred to as a current block.

[0152] The entropy decoding unit (210) can generate symbols by performing entropy decoding according to a probability distribution for the bitstream. The generated symbols may include symbols in the form of quantized levels. Here, the entropy decoding method may be the reverse process of the entropy encoding method described above.

[0153] The entropy decoding unit (210) can change a one-dimensional vector-shaped coefficient into a two-dimensional block-shaped coefficient through a transform coefficient scanning method to decode a transform coefficient level (quantized level).

[0154] The quantized level can be dequantized in the dequantization unit (220) and inversely transformed in the inverse transformation unit (230). The quantized level can be generated as a restored residual block as a result of performing the dequantization and / or inverse transformation. At this time, the inverse quantization unit (220) can apply a quantization matrix to the quantized level.

[0155] When the intra mode is used, the intra prediction unit (240) can generate a prediction block by performing spatial prediction on the current block using sample values ​​of already decoded blocks surrounding the block to be decoded.

[0156] When the inter mode is used, the motion compensation unit (250) can generate a prediction block by performing motion compensation using a motion vector and a reference image stored in the reference picture buffer (270) on the current block. The motion compensation unit (250) can generate a prediction block by applying an interpolation filter to a portion of the reference image when the value of the motion vector does not have an integer value. In order to perform motion compensation, it is possible to determine whether the motion compensation method of the prediction unit included in the corresponding encoding unit is skip mode, merge mode, AMVP mode, or current picture reference mode based on the encoding unit, and motion compensation can be performed according to each mode.

[0157] The adder (255) can add the restored residual block and the prediction block to generate a restored block. The filter unit (260) can apply at least one of a deblocking filter, a sample adaptive offset, and an adaptive loop filter to the restored block or restored image. The filter unit (260) can output the restored image. The restored block or restored image can be stored in the reference picture buffer (270) and used for inter prediction. The restored block that has passed through the filter unit (260) can be a part of the reference image. In other words, the reference image can be a restored image composed of restored blocks that have passed through the filter unit (260). The stored reference image can be used for inter-screen prediction or motion compensation thereafter.

[0158] Figure 3 is a diagram schematically illustrating an image segmentation structure when encoding and decoding an image. Figure 3 schematically illustrates an embodiment in which one unit is divided into multiple sub-units.

[0159] In order to efficiently segment an image, a coding unit (CU) may be used in encoding and decoding. The coding unit may be used as a basic unit for image encoding / decoding. In addition, the coding unit may be used as a unit for distinguishing an intra-screen prediction mode and an inter-screen prediction mode during image encoding / decoding. The coding unit may be a basic unit used for the process of prediction, transformation, quantization, inverse transformation, inverse quantization, or encoding / decoding of transform coefficients.

[0160] Referring to FIG. 3, an image (300) is sequentially divided into Largest Coding Unit (LCU) units, and a division structure is determined for each LCU unit. Here, LCU may be used in the same sense as Coding Tree Unit (CTU). Division of a unit may mean division of a block corresponding to the unit. Block division information may include information regarding the depth of the unit. The depth information may indicate the number of times and / or the degree to which the unit is divided. One unit may be hierarchically divided into multiple sub-units with depth information based on a tree structure. In other words, a unit and a sub-unit generated by division of the unit may correspond to a node and a child node of the node, respectively. Each divided sub-unit may have depth information. The depth information may be information indicating the size of a CU and may be stored for each CU. Since the unit depth indicates the number of times and / or the degree to which the unit is divided, the division information of a sub-unit may include information regarding the size of the sub-unit.

[0161] The partition structure may refer to the distribution of coding units (CUs) within the CTU (310). This distribution may be determined depending on whether to partition one CU into a plurality of CUs (a positive integer greater than or equal to 2, including 2, 4, 8, 16, etc.). The horizontal and vertical sizes of the CU generated by the partition may be half the horizontal size and half the vertical size of the CU before the partition, respectively, or may be smaller than the horizontal size and vertical size of the CU before the partition, depending on the number of partitions. The CU may be recursively partitioned into a plurality of CUs. By recursive partitioning, at least one of the horizontal size and the vertical size of the partitioned CU may be reduced compared to at least one of the horizontal size and the vertical size of the CU before the partitioning. The partitioning of the CU may be performed recursively up to a predefined depth or a predefined size. For example, the depth of a CTU may be 0, and the depth of a Smallest Coding Unit (SCU) may be a predefined maximum depth. Here, the CTU may be a coding unit having the largest coding unit size as described above, and the SCU may be a coding unit having the smallest coding unit size. Splitting starts from the CTU (310), and each time the horizontal and / or vertical size of the CU is reduced by the splitting, the depth of the CU increases by 1. For example, for each depth, a CU that is not split may have a size of 2Nx2N. Additionally, for a CU that is split, a CU of size 2Nx2N may be split into four CUs of size NxN. The size of N may be halved each time the depth increases by 1.

[0162] Additionally, information on whether a CU is split can be expressed through the CU's split information. The split information can be 1-bit information. All CUs except SCUs can include split information. For example, if the split information value is the first value, the CU may not be split, and if the split information value is the second value, the CU may be split.

[0163] Referring to Figure 3, a CTU with depth 0 can be a 64x64 block. 0 can be the minimum depth. An SCU with depth 3 can be an 8x8 block. 3 can be the maximum depth. CUs of 32x32 blocks and 16x16 blocks can be represented as depths 1 and 2, respectively.

[0164] For example, when one coding unit is split into four coding units, the horizontal and vertical sizes of the four split coding units may be half the sizes of the horizontal and vertical sizes of the coding unit before being split. For example, when a coding unit having a size of 32x32 is split into four coding units, each of the four split coding units may have a size of 16x16. When one coding unit is split into four coding units, it can be said that the coding unit is split in the form of a quad-tree (quad-tree partition).

[0165] For example, when one coding unit is split into two coding units, the horizontal or vertical size of the two split coding units may be half the size compared to the horizontal or vertical size of the coding unit before being split. For example, when a coding unit having a size of 32x32 is split vertically into two coding units, each of the two split coding units may have a size of 16x32. For example, when a coding unit having a size of 8x32 is split horizontally into two coding units, each of the two split coding units may have a size of 8x16. When one coding unit is split into two coding units, the coding unit can be said to be split in the form of a binary tree (binary-tree partition).

[0166] For example, when one coding unit is split into three coding units, the coding unit can be split into three coding units by splitting the horizontal or vertical size of the coding unit before splitting at a ratio of 1:2:1. For example, when a coding unit having a size of 16x32 is split horizontally into three coding units, the three split coding units can have sizes of 16x8, 16x16, and 16x8 from the top, respectively. For example, when a coding unit having a size of 32x32 is split vertically into three coding units, the three split coding units can have sizes of 8x32, 16x32, and 8x32 from the left, respectively. When one coding unit is split into three coding units, the coding unit can be said to have been split in the form of a ternary-tree (ternary-tree partition).

[0167] CTU (320) of Fig. 3 is an example of a CTU to which quadtree partitioning, binary tree partitioning, and 3-split tree partitioning are all applied.

[0168] As described above, at least one of quadtree partitioning, binary tree partitioning, and 3-partition tree partitioning may be applied to partition a CTU. Each partitioning may be applied based on a predetermined priority. For example, quadtree partitioning may be preferentially applied to a CTU. A coding unit that cannot be further quadtree partitioned may correspond to a leaf node of a quadtree. A coding unit corresponding to a leaf node of a quadtree may be a root node of a binary tree and / or a 3-partition tree. That is, a coding unit corresponding to a leaf node of a quadtree may be binary-tree partitioned, 3-partition tree partitioned, or may not be partitioned any further. At this time, by preventing quadtree partitioning from being performed again on a coding unit generated by binary-tree partitioning or 3-partition tree partitioning of a coding unit corresponding to a leaf node of a quadtree, partitioning of a block and / or signaling of partition information may be effectively performed.

[0169] The splitting of a coding unit corresponding to each node of a quadtree can be signaled using quad splitting information. Quad splitting information having a first value (e.g., '1') can indicate that the corresponding coding unit is quadtree split. Quad splitting information having a second value (e.g., '0') can indicate that the corresponding coding unit is not quadtree split. The quad splitting information can be a flag having a predetermined length (e.g., 1 bit).

[0170] There may be no priority between binary tree partitioning and 3-partition tree partitioning. That is, a coding unit corresponding to a leaf node of a quadtree may be partitioned either by a binary tree or by a 3-partition tree. Furthermore, a coding unit generated by a binary tree partitioning or a 3-partition tree partitioning may be partitioned again by a binary tree or a 3-partition tree, or may not be partitioned any further.

[0171] A partition in which there is no priority between a binary tree partition and a 3-partition tree partition can be called a multi-type tree partition. That is, a coding unit corresponding to a leaf node of a quadtree can become a root node of a multi-type tree. The partitioning of a coding unit corresponding to each node of a multi-type tree can be signaled using at least one of information on whether the multi-type tree is partitioned, information on the direction of partitioning, and information on the partition tree. For the partitioning of a coding unit corresponding to each node of the multi-type tree, information on whether the multi-type tree is partitioned, information on the direction of partitioning, and information on the partition tree may be sequentially signaled.

[0172] Information on whether a composite tree with a first value (e.g., '1') is split may indicate that the corresponding encoding unit is split into a composite tree. Information on whether a composite tree with a second value (e.g., '0') may indicate that the corresponding encoding unit is not split into a composite tree.

[0173] When an encoding unit corresponding to each node of a composite tree is split into a composite tree, the encoding unit may further include split direction information. The split direction information may indicate a split direction of the composite tree split. Split direction information having a first value (e.g., '1') may indicate that the corresponding encoding unit is split in a vertical direction. Split direction information having a second value (e.g., '0') may indicate that the corresponding encoding unit is split in a horizontal direction.

[0174] When an encoding unit corresponding to each node of a composite tree is split into a composite tree, the encoding unit may further include split tree information. The split tree information may indicate a tree used for splitting the composite tree. Split tree information having a first value (e.g., '1') may indicate that the encoding unit is split into a binary tree. Split tree information having a second value (e.g., '0') may indicate that the encoding unit is split into a 3-split tree.

[0175] The information on whether to split, the information on the split tree, and the information on the split direction may each be flags having a predetermined length (e.g., 1 bit).

[0176] At least one of quad partition information, information on whether a composite tree is partitioned, partition direction information, and partition tree information can be entropy encoded / decoded. For entropy encoding / decoding of the above information, information of a peripheral encoding unit adjacent to a current encoding unit can be used. For example, the partitioning shape (whether to be partitioned, partition tree, and / or partitioning direction) of a left encoding unit and / or an upper encoding unit is likely to be similar to the partitioning shape of the current encoding unit. Therefore, based on the information of the peripheral encoding unit, context information for entropy encoding / decoding of the information of the current encoding unit can be derived. At this time, the information of the peripheral encoding unit can include at least one of quad partition information of the corresponding encoding unit, information on whether a composite tree is partitioned, partition direction information, and partition tree information.

[0177] In another embodiment, among binary tree partitioning and 3-partition tree partitioning, binary tree partitioning may be performed preferentially. That is, binary tree partitioning may be applied first, and the encoding unit corresponding to the leaf node of the binary tree may be set as the root node of the 3-partition tree. In this case, quadtree partitioning and binary tree partitioning may not be performed for the encoding unit corresponding to the node of the 3-partition tree.

[0178] A coding unit that cannot be further split by quadtree splitting, binary tree splitting, and / or 3-tree splitting can become a unit of encoding, prediction, and / or transformation. That is, a coding unit may not be further split for prediction and / or transformation. Accordingly, a split structure, split information, etc. for splitting a coding unit into prediction units and / or transformation units may not exist in the bitstream.

[0179] However, if the size of the encoding unit that is the unit of division is larger than the size of the largest transform block, the encoding unit may be recursively divided until the size becomes equal to or smaller than the size of the largest transform block. For example, if the size of the encoding unit is 64x64 and the size of the largest transform block is 32x32, the encoding unit may be divided into four 32x32 blocks for transformation. For example, if the size of the encoding unit is 32x64 and the size of the largest transform block is 32x32, the encoding unit may be divided into two 32x32 blocks for transformation. In this case, whether or not to split the encoding unit for transformation is not separately signaled, but may be determined by comparing the width or height of the encoding unit with the width or height of the largest transform block. For example, if the width of the encoding unit is larger than the width of the largest transform block, the encoding unit may be divided into two vertically. Additionally, if the height of the encoding unit is greater than the height of the maximum transform block, the encoding unit may be divided into two halves horizontally.

[0180] Information about the maximum and / or minimum size of the coding unit and information about the maximum and / or minimum size of the transform block can be signaled or determined at a higher level of the coding unit. The higher level can be, for example, a sequence level, a picture level, a tile level, a tile group level, a slice level, etc. For example, the minimum size of the coding unit can be determined as 4x4. For example, the maximum size of the transform block can be determined as 64x64. For example, the minimum size of the transform block can be determined as 4x4.

[0181] Information about the minimum size of a coding unit corresponding to a leaf node of a quadtree (quadtree minimum size) and / or information about the maximum depth from a root node to a leaf node of a composite tree (composite tree maximum depth) may be signaled or determined at a higher level of the coding unit. The higher level may be, for example, a sequence level, a picture level, a slice level, a tile group level, a tile level, etc. Information about the quadtree minimum size and / or information about the composite tree maximum depth may be signaled or determined for each of an intra-screen slice and an inter-screen slice.

[0182] The differential information about the size of a CTU and the maximum size of a transform block can be signaled or determined at a higher level of the coding unit. The higher level can be, for example, a sequence level, a picture level, a slice level, a tile group level, a tile level, etc. Information about the maximum size of a coding unit corresponding to each node of a binary tree (binary tree maximum size) can be determined based on the size of the coding tree unit and the differential information. The maximum size of a coding unit corresponding to each node of a 3-partition tree (3-partition tree maximum size) can have different values ​​depending on the type of slice. For example, in the case of an intra-screen slice, the maximum size of the 3-partition tree can be 32x32. Also, for example, in the case of an inter-screen slice, the maximum size of the 3-partition tree can be 128x128. For example, the minimum size of the encoding unit corresponding to each node of the binary tree (binary tree minimum size) and / or the minimum size of the encoding unit corresponding to each node of the 3-partition tree (3-partition tree minimum size) can be set as the minimum size of the encoding block.

[0183] As another example, the maximum size of a binary tree and / or the maximum size of a 3-partition tree may be signaled or determined at the slice level. Additionally, the minimum size of a binary tree and / or the minimum size of a 3-partition tree may be signaled or determined at the slice level.

[0184] Based on the size and depth information of the various blocks described above, quad partition information, information on whether a composite tree is partitioned, partition tree information, and / or partition direction information may or may not be present in the bitstream.

[0185] For example, if the size of the encoding unit is not greater than the quadtree minimum size, the encoding unit does not include quad partition information, and the quad partition information can be inferred as the second value.

[0186] For example, if the size (width and height) of an encoding unit corresponding to a node of a composite tree is larger than the maximum size (width and height) of a binary tree and / or the maximum size (width and height) of a 3-partition tree, the encoding unit may not be split into a binary tree and / or a 3-partition tree. Accordingly, information on whether the composite tree is split is not signaled, but can be inferred as a second value.

[0187] Alternatively, if the size (width and height) of the encoding unit corresponding to the node of the composite tree is equal to the minimum size (width and height) of the binary tree, or if the size (width and height) of the encoding unit is equal to twice the minimum size (width and height) of the 3-partition tree, the encoding unit may not be subjected to binary tree splitting and / or 3-partition tree splitting. Accordingly, information on whether the composite tree is split or not may be inferred as a second value without being signaled. This is because, if the encoding unit is subjected to binary tree splitting and / or 3-partition tree splitting, an encoding unit smaller than the minimum size of the binary tree and / or the minimum size of the 3-partition tree is generated.

[0188] Alternatively, the binary tree splitting or the 3-split tree splitting may be limited based on the size of the virtual pipeline data unit (hereinafter, the pipeline buffer size). For example, if the binary tree splitting or the 3-split tree splitting splits the encoding unit into sub-coding units that do not fit the pipeline buffer size, the binary tree splitting or the 3-split tree splitting may be limited. The pipeline buffer size may be the size of the largest transform block (e.g., 64X64). For example, when the pipeline buffer size is 64X64, the following splitting may be limited.

[0189] - 3-partition tree partitioning for NxM (N and / or M are 128) coding units

[0190] - Horizontal binary tree partitioning for 128xN (N <= 64) encoding units

[0191] - Vertical binary tree partitioning for Nx128 (N <= 64) coding units

[0192] Alternatively, if the depth of the composite tree of the encoding unit corresponding to the node of the composite tree is equal to the maximum depth of the composite tree, the encoding unit may not be split into a binary tree and / or a 3-partition tree. Accordingly, information on whether the composite tree is split is not signaled, but can be inferred as a second value.

[0193] Alternatively, information on whether the composite tree is split may be signaled only if at least one of vertical binary tree splitting, horizontal binary tree splitting, vertical 3-partition tree splitting, and horizontal 3-partition tree splitting is possible for the encoding unit corresponding to the node of the composite tree. Otherwise, the encoding unit may not be split into a binary tree and / or a 3-partition tree. Accordingly, information on whether the composite tree is split may not be signaled, but may be inferred as a second value.

[0194] Alternatively, the split direction information may be signaled only when both vertical binary tree splitting and horizontal binary tree splitting are possible for the encoding unit corresponding to the node of the composite tree, or when both vertical 3-partition tree splitting and horizontal 3-partition tree splitting are possible. Otherwise, the split direction information may not be signaled and may be inferred as a value indicating a possible splitting direction.

[0195] Alternatively, the split tree information may be signaled only if both vertical binary tree splitting and vertical 3-partition tree splitting are possible for the encoding unit corresponding to the node of the composite tree, or if both horizontal binary tree splitting and horizontal 3-partition tree splitting are possible. Otherwise, the split tree information may not be signaled and may be inferred as a value indicating a tree that can be split.

[0196] Figure 4 is a diagram for explaining an embodiment of an on-screen prediction process.

[0197] The arrows from the center to the periphery of Figure 4 may indicate the prediction directions of the prediction modes within the screen.

[0198] In-frame encoding and / or decoding may be performed using reference samples from neighboring blocks of the current block. The neighboring blocks may be reconstructed neighboring blocks. For example, in-frame encoding and / or decoding may be performed using values ​​or encoding parameters of reference samples contained in the reconstructed neighboring blocks.

[0199] A prediction block may refer to a block generated as a result of performing prediction within a screen. The prediction block may correspond to at least one of a CU, a PU, and a TU. The unit of the prediction block may be the size of at least one of a CU, a PU, and a TU. The prediction block may be a square block having a size such as 2x2, 4x4, 16x16, 32x32, or 64x64, or may be a rectangular block having a size such as 2x8, 4x8, 2x16, 4x16, or 8x16.

[0200] In-screen prediction can be performed based on the in-screen prediction mode for the current block. The number of in-screen prediction modes that the current block can have can be a predefined fixed value, or it can be determined differently based on the properties of the prediction block. For example, the properties of the prediction block can include the size and shape of the prediction block.

[0201] The number of prediction modes within a picture may be fixed to N regardless of the size of the block. Or, for example, the number of prediction modes within a picture may be 3, 5, 9, 17, 34, 35, 36, 65, or 67. Or, the number of prediction modes within a picture may vary depending on the size of the block and / or the type of the color component. For example, the number of prediction modes within a picture may vary depending on whether the color component is a luma signal or a chroma signal. For example, the number of prediction modes within a picture may increase as the size of the block increases. Or, the number of prediction modes within a luma component block may be greater than the number of prediction modes within a chroma component block.

[0202] The in-screen prediction mode may be a non-directional mode or a directional mode. The non-directional mode may be a DC mode or a planar mode, and the angular mode may be a prediction mode having a specific direction or angle. The in-screen prediction mode may be expressed by at least one of a mode number, a mode value, a mode number, a mode angle, and a mode direction. The number of in-screen prediction modes may be one or more M including the non-directional and directional modes. A step of checking whether samples included in reconstructed neighboring blocks for in-screen prediction of the current block can be used as reference samples of the current block may be performed. If there is a sample that cannot be used as a reference sample of the current block, the sample value of at least one of the samples included in the reconstructed neighboring blocks is copied and / or interpolated to replace the sample value of the sample that cannot be used as a reference sample, and then the sample value can be used as a reference sample of the current block.

[0203] Figure 7 is a diagram illustrating reference samples available for prediction within the screen.

[0204] As illustrated in FIG. 7, at least one of reference sample lines 0 to 3 may be used for intra-screen prediction of the current block. In FIG. 7, samples of segments A and F may be padded with the closest samples of segments B and E, respectively, instead of being taken from reconstructed neighboring blocks. Index information indicating reference sample lines to be used for intra-screen prediction of the current block may be signaled. For example, in FIG. 7, reference sample line indicators 0, 1, and 2 may be signaled as index information indicating reference sample lines 0, 1, and 2. If the upper boundary of the current block is the boundary of a CTU, only reference sample line 0 may be available. Therefore, in this case, the index information may not be signaled. If a reference sample line other than reference sample line 0 is used, filtering on the prediction block described below may not be performed.

[0205] When making in-screen predictions, a filter can be applied to at least one of the reference sample or the prediction sample based on at least one of the in-screen prediction mode and the size of the current block.

[0206] In planar mode, when generating a prediction block of the current block, the sample value of the prediction target sample can be generated using a weighted sum of the top and left reference samples of the current sample, the top-right and bottom-left reference samples of the current block, depending on the position of the prediction target sample within the prediction block. In addition, in DC mode, when generating a prediction block of the current block, the average value of the top and left reference samples of the current block can be used. In addition, in directional mode, the prediction block can be generated using the top, left, top-right and / or bottom-left reference samples of the current block. Real-valued interpolation can also be performed to generate the prediction sample value.

[0207] In the case of intra-screen prediction between color components, a prediction block for a current block of a second color component can be generated based on a corresponding reconstruction block of a first color component. For example, the first color component can be a luminance component, and the second color component can be a chrominance component. For intra-screen prediction between color components, parameters of a linear model between the first color component and the second color component can be derived based on a template. The template can include top and / or left peripheral samples of the current block and top and / or left peripheral samples of a reconstruction block of the first color component corresponding thereto. For example, the parameters of the linear model can be derived using a sample value of the first color component having a maximum value among the samples in the template and a sample value of the second color component corresponding thereto, and a sample value of the first color component having a minimum value among the samples in the template and a sample value of the second color component corresponding thereto. Once the parameters of the linear model are derived, a prediction block for the current block can be generated by applying a corresponding reconstruction block to the linear model. Depending on the video format, subsampling may be performed on the surrounding samples of the reconstructed block of the first color component and the corresponding reconstructed block. For example, if one sample of the second color component corresponds to four samples of the first color component, the four samples of the first color component may be subsampled to compute one corresponding sample. In this case, parameter derivation of the linear model and intra-picture prediction between color components may be performed based on the subsampled corresponding samples. Whether intra-picture prediction between color components is performed and / or the range of the template may be signaled as an intra-picture prediction mode.

[0208] A current block can be split into two or four sub-blocks in the horizontal or vertical direction. The split sub-blocks can be sequentially reconstructed. That is, intra-picture prediction can be performed on the sub-blocks to generate sub-prediction blocks. In addition, inverse quantization and / or inverse transformation can be performed on the sub-blocks to generate sub-residual blocks. A reconstructed sub-block can be generated by adding the sub-prediction block to the sub-residual block. The reconstructed sub-block can be used as a reference sample for intra-picture prediction of a subsequent sub-block. A sub-block can be a block including a predetermined number of samples (e.g., 16). Therefore, for example, if the current block is an 8x4 block or a 4x8 block, the current block can be split into two sub-blocks. In addition, if the current block is a 4x4 block, the current block cannot be split into sub-blocks. If the current block has a size other than that, the current block can be split into four sub-blocks. Information regarding whether or not to perform the sub-block-based intra-screen prediction and / or the splitting direction (horizontal or vertical) may be signaled. The sub-block-based intra-screen prediction may be restricted to be performed only when using reference sample line 0. When the sub-block-based intra-screen prediction is performed, filtering on the prediction block described below may not be performed.

[0209] A final prediction block can be generated by performing filtering on a predicted prediction block within a screen. The filtering can be performed by applying predetermined weights to a filtering target sample, a left reference sample, an upper reference sample, and / or an upper left reference sample. The weights and / or reference samples (range, position, etc.) used for the filtering can be determined based on at least one of a block size, an within-screen prediction mode, and a position of the filtering target sample within the prediction block. The filtering can be performed only for a predetermined within-screen prediction mode (e.g., DC, planar, vertical, horizontal, diagonal, and / or adjacent diagonal mode). The adjacent diagonal mode can be a mode obtained by adding or subtracting k from the diagonal mode. For example, k can be a positive integer less than or equal to 8.

[0210] The intra-screen prediction mode of the current block can be predicted from the intra-screen prediction modes of blocks existing around the current block and entropy encoded / decoded. If the intra-screen prediction modes of the current block and the surrounding blocks are the same, information indicating that the intra-screen prediction modes of the current block and the surrounding blocks are the same can be signaled using predetermined flag information. In addition, indicator information regarding an intra-screen prediction mode that is the same as the intra-screen prediction mode of the current block among the intra-screen prediction modes of a plurality of surrounding blocks can be signaled. If the intra-screen prediction modes of the current block and the surrounding blocks are different, entropy encoding / decoding can be performed based on the intra-screen prediction modes of the surrounding blocks to entropy encode / decode intra-screen prediction mode information of the current block.

[0211] Figure 5 is a diagram for explaining an embodiment of a screen-to-screen prediction process.

[0212] The rectangles illustrated in Figure 5 can represent images. Additionally, the arrows in Figure 5 can indicate prediction directions. Each image can be classified into an I-picture (Intra Picture), a P-picture (Predictive Picture), a B-picture (Bi-predictive Picture), etc., depending on its encoding type.

[0213] An I picture can be encoded / decoded through intra-picture prediction without inter-picture prediction. A P picture can be encoded / decoded through inter-picture prediction using only reference pictures existing in a unidirectional direction (e.g., forward or backward). A B picture can be encoded / decoded through inter-picture prediction using reference pictures existing in both directions (e.g., forward and backward). In addition, in the case of a B picture, it can be encoded / decoded through inter-picture prediction using reference pictures existing in both directions or inter-picture prediction using reference pictures existing in one of the forward and backward directions. Here, the bidirectional direction can be forward and backward. Here, when inter-picture prediction is used, the encoder can perform inter-picture prediction or motion compensation, and the decoder can perform corresponding motion compensation.

[0214] Below, inter-screen prediction according to an embodiment is described in detail.

[0215] Inter-screen prediction or motion compensation can be performed using reference images and motion information.

[0216] Motion information for a current block can be derived through inter-frame prediction by each of the encoding device (100) and the decoding device (200). The motion information can be derived using motion information of a reconstructed surrounding block, motion information of a collocated block (col block), and / or a block adjacent to the collocated block. The collocated block may be a block corresponding to the spatial position of the current block within an already reconstructed collocated picture (col picture). Here, the collocated picture may be one picture from at least one reference picture included in a reference picture list.

[0217] The method for deriving motion information may vary depending on the prediction mode of the current block. For example, prediction modes applied for inter-frame prediction may include AMVP mode, merge mode, skip mode, merge mode with motion vector difference, sub-block merge mode, geometric segmentation mode, inter-intra joint prediction mode, and affine inter mode. Here, the merge mode may be referred to as motion merge mode.

[0218] For example, when AMVP is applied as a prediction mode, a motion vector candidate list can be generated by determining at least one of the motion vector of the reconstructed neighboring block, the motion vector of the called block, the motion vector of the block adjacent to the called block, and the (0, 0) motion vector as a motion vector candidate. A motion vector candidate can be derived using the generated motion vector candidate list. Motion information of the current block can be determined based on the derived motion vector candidate. Here, the motion vector of the called block or the motion vector of the block adjacent to the called block can be referred to as a temporal motion vector candidate, and the motion vector of the reconstructed neighboring block can be referred to as a spatial motion vector candidate.

[0219] The encoding device (100) can calculate the motion vector difference (MVD) between the motion vector of the current block and the motion vector candidate, and can entropy encode the MVD. In addition, the encoding device (100) can generate a bitstream by entropy encoding the motion vector candidate index. The motion vector candidate index can indicate an optimal motion vector candidate selected from among the motion vector candidates included in the motion vector candidate list. The decoding device (200) can entropy decode the motion vector candidate index from the bitstream, and select a motion vector candidate of a block to be decoded from among the motion vector candidates included in the motion vector candidate list using the entropy-decoded motion vector candidate index. In addition, the decoding device (200) can derive a motion vector of a block to be decoded by summing the entropy-decoded MVD and the motion vector candidate.

[0220] Meanwhile, the encoding device (100) can entropy encode the resolution information of the calculated MVD. The decoding device (200) can adjust the resolution of the entropy-decoded MVD using the MVD resolution information.

[0221] Meanwhile, the encoding device (100) can calculate the motion vector difference (MVD) between the motion vector of the current block and the motion vector candidate based on the affine model, and can entropy encode the MVD. The decoding device (200) can derive the affine control motion vector of the block to be decoded through the sum of the entropy-decoded MVD and the affine control motion vector candidate, thereby deriving the motion vector in units of sub blocks.

[0222] The bitstream may include a reference picture index indicating a reference picture, etc. The reference picture index may be entropy encoded and signaled from the encoding device (100) to the decoding device (200) via the bitstream. The decoding device (200) may generate a prediction block for the block to be decoded based on the derived motion vector and the reference picture index information.

[0223] Another example of a method for deriving motion information is merge mode. Merge mode may refer to merging motions of multiple blocks. Merge mode may refer to a mode in which motion information of a current block is derived from motion information of surrounding blocks. When merge mode is applied, a merge candidate list may be generated using motion information of reconstructed surrounding blocks and / or motion information of a call block. The motion information may include at least one of 1) a motion vector, 2) a reference image index, and 3) an inter-frame prediction indicator. The prediction indicator may be unidirectional (L0 prediction, L1 prediction) or bidirectional.

[0224] The merge candidate list may represent a list in which motion information is stored. The motion information stored in the merge candidate list may be at least one of motion information of a neighboring block adjacent to the current block (spatial merge candidate), motion information of a block collocated with the current block in a reference image (temporal merge candidate), new motion information generated by a combination of motion information already existing in the merge candidate list, motion information of a block encoded / decoded before the current block (history-based merge candidate), and a zero merge candidate.

[0225] The encoding device (100) can generate a bitstream by entropy encoding at least one of a merge flag and a merge index, and then signal it to the decoding device (200). The merge flag can be information indicating whether to perform a merge mode for each block, and the merge index can be information on which block among the neighboring blocks adjacent to the current block will be merged. For example, the neighboring blocks of the current block can include at least one of the left neighboring block, the upper neighboring block, and the temporal neighboring block of the current block.

[0226] Meanwhile, the encoding device (100) can entropy encode correction information for correcting the motion vector among the motion information of the merge candidate and signal it to the decoding device (200). The decoding device (200) can correct the motion vector of the merge candidate selected by the merge index based on the correction information. Here, the correction information can include at least one of correction status information, correction direction information, and correction size information. As described above, a prediction mode that corrects the motion vector of the merge candidate based on the signaled correction information can be referred to as a merge mode with motion vector difference.

[0227] Skip mode may be a mode that applies motion information of surrounding blocks as is to the current block. When skip mode is used, the encoding device (100) may entropy encode information on which block's motion information is to be used as the motion information of the current block and signal it to the decoding device (200) through a bitstream. At this time, the encoding device (100) may not signal a syntax element regarding at least one of motion vector difference information, an encoding block flag, and a transform coefficient level (quantized level) to the decoding device (200).

[0228] Subblock merge mode may refer to a mode for deriving motion information in units of subblocks of a coding block (CU). When the subblock merge mode is applied, a subblock merge candidate list may be generated using motion information (subblock-based temporal merge candidate) of a subblock collocated with a current subblock in a reference image and / or an affine control point motion vector merge candidate.

[0229] Geometric partitioning mode may refer to a mode in which the current block is divided in a predetermined direction to derive motion information for each block, each prediction sample is derived using each derived motion information, and each derived prediction sample is weighted and combined to derive a prediction sample of the current block.

[0230] Inter-intra combined prediction mode may mean a mode in which a prediction sample of the current block is derived by weighting prediction samples generated through inter-screen prediction and prediction samples generated through intra-screen prediction.

[0231] The decoding device (200) can independently correct the derived motion information. The decoding device (200) can search a predefined area based on a reference block indicated by the derived motion information to derive the motion information with the minimum SAD as corrected motion information.

[0232] The decoding device (200) can compensate for the prediction sample derived through inter-screen prediction using optical flow.

[0233] Figure 6 is a diagram for explaining the process of transformation and quantization.

[0234] As illustrated in FIG. 6, a quantized level can be generated by performing a transformation and / or quantization process on the residual signal. The residual signal can be generated as a difference between an original block and a prediction block (an intra-screen prediction block or an inter-screen prediction block). Here, the prediction block can be a block generated by intra-screen prediction or inter-screen prediction. Here, the transformation can include at least one of a primary transformation and a secondary transformation. A transformation coefficient can be generated by performing a primary transformation on the residual signal, and a secondary transformation can be performed on the transformation coefficient to generate a secondary transformation coefficient.

[0235] The primary transform can be performed using at least one of a plurality of pre-defined transform methods. For example, the plurality of pre-defined transform methods can include a discrete cosine transform (DCT), a discrete sine transform (DST), or a Karhunen-Loeve transform (KLT)-based transform. A secondary transform can be performed on transform coefficients generated after the primary transform is performed. The transform method applied during the primary transform and / or the secondary transform can be determined according to at least one of the encoding parameters of the current block and / or the neighboring blocks. Alternatively, transform information indicating the transform method can be signaled. The DCT-based transform can include, for example, DCT2, DCT-8, etc. The DST-based transform can include, for example, DST-7.

[0236]

[0237] Quantization can be performed on the result or residual signal of the first transform and / or the second transform to generate a quantized level. The quantized level can be scanned according to at least one of an up-right diagonal scan, a vertical scan, and a horizontal scan based on at least one of an within-picture prediction mode or a block size / shape. For example, the coefficients of a block can be converted into a one-dimensional vector form by scanning them using an up-right diagonal scan. Depending on the size of the transform block and / or the within-picture prediction mode, a vertical scan that scans the two-dimensional block shape coefficients in the column direction, or a horizontal scan that scans the two-dimensional block shape coefficients in the row direction may be used instead of the up-right diagonal scan. The scanned quantized level can be entropy encoded and included in a bitstream.

[0238] A decoder can entropy-decode a bitstream to generate quantized levels. These quantized levels can be aligned into a two-dimensional block shape through inverse scanning. At least one of the following methods can be used for inverse scanning: upper-right diagonal scanning, vertical scanning, or horizontal scanning.

[0239] Inverse quantization can be performed on a quantized level, a second inverse transform can be performed depending on whether a second inverse transform is performed, and a first inverse transform can be performed depending on whether a first inverse transform is performed on the result of the second inverse transform, thereby generating a restored residual signal.

[0240] Inverse mapping of the dynamic range may be performed on the luminance component restored through intra-picture prediction or inter-picture prediction before in-loop filtering. The dynamic range may be divided into 16 equal pieces, and a mapping function for each piece may be signaled. The mapping function may be signaled at the slice level or the tile group level. The inverse mapping function for performing the inverse mapping may be derived based on the mapping function. In-loop filtering, storage of reference pictures, and motion compensation are performed in the inverse mapped area, and the prediction block generated through inter-picture prediction may be converted to the mapped area by mapping using the mapping function, and then used to generate a reconstructed block. However, since intra-picture prediction is performed in the mapped area, the prediction block generated through intra-picture prediction may be used to generate a reconstructed block without mapping / inverse mapping.

[0241] If the current block is a residual block of chrominance components, the residual block can be converted into an inverse mapped region by performing scaling on the chrominance components of the mapped region. The availability of the scaling can be signaled at the slice level or the tile group level. The scaling can be applied only when the mapping for the luma component is available and the partitioning of the luminance component and the partitioning of the chrominance component follow the same tree structure. The scaling can be performed based on the average of the sample values ​​of the luminance prediction block corresponding to the chrominance block. In this case, if the current block uses inter-screen prediction, the luminance prediction block can mean the mapped luminance prediction block. The value required for the scaling can be derived by referring to a lookup table using the index of the piece to which the average of the sample values ​​of the luminance prediction block belongs. Finally, by scaling the residual block using the derived value, the residual block can be converted into an inverse mapped region. Restoration of the subsequent chrominance component blocks, intra-screen prediction, inter-screen prediction, in-loop filtering, and storage of reference pictures can be performed in the reverse-mapped area.

[0242] Information indicating whether mapping / demapping of the luminance component and chrominance component is available can be signaled via a sequence parameter set.

[0243] A prediction block of a current block can be generated based on a block vector indicating a displacement between the current block and a reference block within the current picture. In this way, a prediction mode that generates a prediction block by referencing the current picture can be called an Intra Block Copy (IBC) mode. The IBC mode can be applied to an MxN (M<=64, N<=64) coding unit. The IBC mode can include a skip mode, a merge mode, an AMVP mode, etc. In the case of the skip mode or the merge mode, a merge candidate list is configured, and a merge index is signaled so that one merge candidate can be specified. The block vector of the specified merge candidate can be used as the block vector of the current block. The merge candidate list can include at least one or more of a spatial candidate, a history-based candidate, an average-of-two-candidate candidate, or a zero-merge candidate. In the case of the AMVP mode, a differential block vector can be signaled. Additionally, the predicted block vector can be derived from the left neighboring block and the upper neighboring block of the current block. An index indicating which neighboring block to use can be signaled. The predicted block in the IBC mode can be limited to blocks included in the current CTU or the left CTU and within the previously reconstructed region. For example, the block vector value of the predicted block of the current block can be limited so that it is located within the three 64x64 block regions preceding the 64x64 block to which the current block belongs in the encoding / decoding order. By limiting the block vector value in this way, the memory consumption and device complexity associated with the IBC mode implementation can be reduced.

[0244]

[0245] Hereinafter, a method for encoding / decoding inter-screen prediction information according to the present invention will be described.

[0246]

[0247] FIG. 8 is a flowchart illustrating a method for encoding / decoding inter-screen prediction information according to an embodiment of the present invention.

[0248] Referring to FIG. 8, inter-screen prediction information encoding / decoding can be composed of a [D1] prediction mode information encoding / decoding step and a [D2] inter-screen prediction information encoding / decoding step. Each step is described in detail below.

[0249]

[0250] [D1] Prediction mode information encoding / decoding step

[0251] When encoding and decoding inter-screen prediction information for a current encoding / decoding target block (hereinafter referred to as “current block”), encoding and decoding of prediction mode information may be performed using at least one or more of a skip indicator (e.g., cu_skip_flag), a prediction mode indicator (e.g., pred_mode_flag), and a prediction mode IBC indicator (e.g., pred_mode_ibc_flag).

[0252] Prediction mode information according to one embodiment of the present invention may indicate whether skip mode, inter-screen prediction mode, intra-screen prediction mode, intra-screen block copy-based prediction mode, and palette mode are performed for the current block. In addition, the prediction mode information may include at least one or more of a skip indicator, a prediction mode indicator, and a prediction mode IBC indicator.

[0253] The skip flag (cu_skip_flag) may indicate that there is no residual signal for the current block. Accordingly, entropy encoding / decoding, quantization / dequantization, and transformation / inverse transformation may not be performed on the residual signal. The skip flag may be a single bit with a value of '0' or '1'. Meanwhile, the skip flag may indicate whether to perform inter-frame prediction or intra-frame block copy-based prediction for the current block.

[0254] For example, if the decoded skip indicator 'cu_skip_flag' in the decoder is '1', inter-screen prediction information (e.g., merge_idx) can be decoded.

[0255] For example, if the decoded skip indicator 'cu_skip_flag' in the decoder is '1' and the current slice type is an 'I' (in-screen) slice, the intra-screen block copy-based prediction skip information can be decoded.

[0256] If the slice type of the current block is an 'I' slice and the horizontal or vertical length of the target block is greater than 64, the skip indicator may not be entropy decoded. In other words, this may mean that the block copy-based prediction skip mode within the screen is not allowed for blocks with the above conditions.

[0257] For example, if the decoded skip indicator 'cu_skip_flag' in the decoder is '1', entropy decoding for the residual signal may not be performed.

[0258] For example, if the decoded skip indicator 'cu_skip_flag' in the decoder is '1', inverse quantization and inverse transformation may not be performed.

[0259] For example, if the decrypted skip indicator 'cu_skip_flag' in the decoder is '0', the prediction mode indicator or the prediction mode IBC indicator can be decrypted.

[0260]

[0261] The prediction mode indicator (pred_mode_flag) can indicate whether inter-screen or intra-screen prediction is performed for the current block. The prediction mode indicator can be a single bit with a value of 0 or 1.

[0262] For example, if the decoded prediction mode indicator in the decoder is '0', entropy decoding can be performed on information related to the inter-screen prediction mode.

[0263] For example, if the decoded prediction mode indicator in the decoder is '1', entropy decoding can be performed on the prediction mode-related information within the screen.

[0264]

[0265] The prediction mode IBC indicator (pred_mode_ibc_flag) can indicate whether the current block is predicted based on block copy within the screen. The prediction mode indicator can be a single bit with a value of 0 or 1.

[0266] For example, if the decoded block copy prediction mode indicator (pred_mode_ibc_flag) in the decoder is '1', entropy decoding for the block copy prediction mode in the decoder can be performed.

[0267] For example, if the decoded intra-screen block copy prediction mode indicator (pred_mode_ibc_flag) in the decoder is '0', entropy decoding can be performed on information related to inter-screen prediction or intra-screen prediction mode.

[0268] If the width or height of the current block is greater than 64, the entropy decoding of the intra-screen block copy prediction mode indicator may not be performed. In other words, this may mean that intra-screen block copy-based prediction is not allowed if the width or height of the target block is greater than 64.

[0269] When performing entropy encoding / decoding of prediction mode information, entropy encoding / decoding can be performed adaptively and differently using prediction information of spatial neighboring blocks. The prediction information of the spatial neighboring blocks can be at least one of a skip indicator, a prediction mode indicator, a prediction mode IBC indicator, and a residual signal presence indicator (cbf) of the spatial neighboring blocks.

[0270] For example, in CABAC entropy encoding / decoding of a skip indicator, different probability models can be used depending on the number of cases in which the skip indicator of a spatial neighboring block is '1'.

[0271] For example, in entropy encoding / decoding of a skip indicator, CABAC can be performed if the number of cases where the skip indicator of a spatial surrounding block is '1' is greater than a specific threshold, and bypass coding can be performed otherwise.

[0272]

[0273] [D2] Inter-screen prediction information encoding / decoding step

[0274] Encoding and decoding of inter-screen prediction information for the current block can be performed.

[0275]

[0276] The inter-screen prediction mode may include at least one of a general merge mode that infers motion information of a current block from motion information of surrounding blocks, and an Inter mode (AMVP mode, Advance Motion Vector Prediction mode) that derives motion information of a current block by combining entropy-decoded motion vector difference information and predicted motion vector.

[0277] General merge mode can be indicated by the value of a general merge mode indicator (e.g., general_merge_flag).

[0278] For example, if the entropy-decoded 'general_merge_flag' is the first value '1', it may mean that the current block is decoded in the general merge mode, and may indicate that the current block is decoded in at least one merge mode among the intra-screen block copy-based merge mode, the regular merge mode, the merge mode with motion vector difference, the sub-block merge mode, the intra-screen combined merge mode, and the triangulation mode.

[0279] For example, if the entropy decrypted 'general_merge_flag' has a second value of '0', it may mean that the current block is decrypted in AMVP mode or in-screen block copy-based inter mode.

[0280] If the decrypted skip indicator 'cu_skip_flag' in the decoder is the first value '1', the general merge mode indicator (general_merge_flag) can be inferred to be the first value '1' without entropy decryption.

[0281] Prediction modes applicable to the above general merge mode may include a block copy-based merge mode within a screen, a regular merge mode, a merge mode with motion vector difference (MMVD mode, Merge with MVD mode), a subblock merge mode, an intra-screen combined merge mode (or Combined Inter Intra Prediction mode, CIIP mode), a triangulation mode (or a geometric partitioning mode, GPM, Geometric Partitioning Mode), etc.

[0282] Meanwhile, prediction modes applicable to the above skip mode may include a block copy-based merge mode within the screen, a regular merge mode, a merge mode with motion vector difference, a sub-block merge mode, a triangulation mode (or a geometric segmentation mode), etc.

[0283]

[0284] Regular merge mode can use the motion information of a merge candidate selected by a decrypted merge index as the motion information of the current block. Here, regular merge mode can refer to the merge mode described in FIG. 5.

[0285] Regular merge mode can be indicated by the value of a regular merge mode indicator (e.g., regular_merge_flag), which can be entropy encoded / decoded.

[0286] For example, if the entropy-decoded 'regular_merge_flag' is the first value '1', it may mean that the current block is decrypted in regular merge mode. If the 'regular_merge_flag' is the first value '1', the merge index information may be entropy encoded / decrypted.

[0287] For example, if the entropy-decoded 'regular_merge_flag' is the second value '0', it may mean that the current block is decoded in at least one merge mode other than the regular merge mode, such as the merge mode with motion vector difference, the sub-block merge mode, the intra-screen combined merge mode, or the geometric segmentation mode.

[0288] The regular merge mode indicator can be derived to the first value '1' without entropy decoding / decoding if the general merge mode indicator (general_merge_flag) has the first value '1' and satisfies at least one of the conditions below (condition 1 or condition 2). In other words, it can be indicated that the current block is always in regular merge mode.

[0289] Condition 1: If all of the entropy-decoded indicator values ​​at the upper level, such as Sequence Parameter Set (SPS), Picture Parameter Set (PPS), Adaptation Parameter Set (APS), subpicture, tile group, and slice header, have the second value of '0',

[0290] Indicator whether to use merge mode with motion vector differences (e.g., sps_mmvd_enabled_flag),

[0291] Indicator for whether to use sub-block-level temporal motion vector prediction (e.g., sps_sbtmvp_enabled_flag)

[0292] Indicator for whether to use affine transformation model prediction (e.g., sps_affine_enabled_flag)

[0293] Indicator for whether to use combined predictions within the screen (e.g., sps_ciip_enabled_flag)

[0294] Indicator for whether to use triangulation merge mode (e.g., sps_triangle_enabled_flag)

[0295] Meanwhile, if the above indicator values ​​are the second value '0', it may mean that the merge mode is not used.

[0296]

[0297] Condition 2: If the merge mode indicator (sps_mmvd_enabled_flag) with entropy-decoded motion vector differences at the upper level, such as Sequence Parameter Set (SPS), Picture Parameter Set (PPS), Adaptation Parameter Set (APS), subpicture, tile group, and slice header, has the second value of '0', and the product of the width and height of the current block is 32.

[0298]

[0299] As another example, the regular merge mode indicator can be derived to the first value '1' without entropy encoding / decoding when the general merge mode indicator (general_merge_flag) value is the first value '1' and at least one of the conditions below (condition 1 or condition 2) is satisfied. In other words, it can be indicated that the current block is always in regular merge mode.

[0300] Condition 1: When all of the entropy-decoded indicator values ​​at the upper level, such as Sequence Parameter Set (SPS), Picture Parameter Set (PPS), Adaptation Parameter Set (APS), subpicture, tile group, and slice header, have the second value '0', and the maximum number of candidates for the sub-block merge mode (MaxNumSubblockMergeCand) is 0 (MaxNumSubblockMergeCand = 0),

[0301] Indicator whether to use merge mode with motion vector differences (e.g., sps_mmvd_enabled_flag),

[0302] Indicator for whether to use combined predictions within the screen (e.g., sps_ciip_enabled_flag)

[0303] Indicator for whether to use triangulation merge mode (e.g., sps_triangle_enabled_flag)

[0304]

[0305] Condition 2: If the merge mode indicator (sps_mmvd_enabled_flag) with entropy-decoded motion vector differences at the upper level, such as Sequence Parameter Set (SPS), Picture Parameter Set (PPS), Adaptation Parameter Set (APS), subpicture, tile group, and slice header, has the second value of '0', and the product of the width and height of the current block is 32.

[0306] As another example, the regular merge mode indicator can be derived to the first value '1' without entropy encoding / decoding when the general merge mode indicator (general_merge_flag) value is the first value '1' and at least one of the conditions below (condition 1 or condition 2) is satisfied. In other words, it can be indicated that the current block is always in regular merge mode.

[0307] Condition 1: If all of the entropy-decoded indicator values ​​at the upper level, such as Sequence Parameter Set (SPS), Picture Parameter Set (PPS), Adaptation Parameter Set (APS), subpicture, tile group, and slice header, have the second value '0', and the maximum number of candidates for the triangulation merge mode (MaxNumTriangleMergeCand) is 0 (MaxNumTriangleMergeCand = 0),

[0308] Indicator whether to use merge mode with motion vector differences (e.g., sps_mmvd_enabled_flag),

[0309] Indicator for whether to use sub-block-level temporal motion vector prediction (e.g., sps_sbtmvp_enabled_flag)

[0310] Indicator for whether to use affine transformation model prediction (e.g., sps_affine_enabled_flag)

[0311] Indicator for whether to use combined predictions within the screen (e.g., sps_ciip_enabled_flag)

[0312]

[0313] Condition 2: If the merge mode indicator (sps_mmvd_enabled_flag) with entropy-decoded motion vector differences at the upper level, such as Sequence Parameter Set (SPS), Picture Parameter Set (PPS), Adaptation Parameter Set (APS), subpicture, tile group, and slice header, has the second value of '0', and the product of the width and height of the current block is 32.

[0314]

[0315] As another example, the regular merge mode indicator can be derived to the first value '1' without entropy encoding / decoding when the general merge mode indicator (general_merge_flag) value is the first value '1' and at least one of the conditions below (condition 1 or condition 2) is satisfied. In other words, it can be indicated that the current block is always in regular merge mode.

[0316] Condition 1: If all of the entropy-decoded indicator values ​​at the upper level, such as Sequence Parameter Set (SPS), Picture Parameter Set (PPS), Adaptation Parameter Set (APS), subpicture, tile group, and slice header, have the second value '0', and the maximum candidate number information of the sub-block merge mode (MaxNumSubblockMergeCand) is '0' (MaxNumSubblockMergeCand = 0) and the maximum candidate number information of the triangulation merge mode (MaxNumTriangleMergeCand) is '0' (MaxNumTriangleMergeCand = 0),

[0317] Indicator whether to use merge mode with motion vector differences (e.g., sps_mmvd_enabled_flag),

[0318] Indicator for whether to use combined predictions within the screen (e.g., sps_ciip_enabled_flag)

[0319]

[0320] Condition 2: If the merge mode indicator (sps_mmvd_enabled_flag) with entropy-decoded motion vector differences at the upper level, such as Sequence Parameter Set (SPS), Picture Parameter Set (PPS), Adaptation Parameter Set (APS), subpicture, tile group, and slice header, has the second value of '0', and the product of the width and height of the current block is 32.

[0321]

[0322] As another example, the regular merge mode indicator can be derived to the first value '1' without entropy encoding / decoding when the general merge mode indicator (general_merge_flag) has a first value of '1' and the sub-block merge mode indicator (e.g., merge_subblock_flag) has a second value of '0' indicating that it is not a sub-block merge mode. In other words, it can be indicated that the current block is always in regular merge mode.

[0323]

[0324] Meanwhile, the regular merge mode indicator (e.g., regular_merge_flag) can indicate regular merge mode or merge mode with motion vector difference.

[0325] For example, if the entropy-decoded 'regular_merge_flag' is the first value of '1', it may mean that the current block is decoded in regular merge mode or merge mode with motion vector difference.

[0326]

[0327] The block copy-based skip / merge mode within the screen can be determined based on the entropy-decoded prediction mode information (skip indicator or prediction mode IBC indicator), and the motion information of the merge candidate selected by the decoded merge index can be used as the motion information of the current block, i.e., the block vector.

[0328] In the case of block copy-based skip / merge mode within the above screen, only merge index information can be entropy decoded.

[0329]

[0330] FIGS. 9 to 12 are drawings for explaining the inter-screen prediction information syntax of the merge mode according to various embodiments of the present invention.

[0331] Figure 9 is an example of a syntax table regarding inter-screen prediction mode information to be entropy decoded when the current block is encoded / decoded in general merge mode (i.e., when general_merge_flag = 1).

[0332] As in Fig. 9, the regular merge mode indicator (regular_merge_flag) can be entropy decoded only when at least one of the entropy-decoded 'sps_mmvd_enabled_flag', 'sps_sbtmvp_enabled_flag', 'sps_affine_enabled_flag', 'sps_ciip_enabled_flag' or 'sps_triangle_enabled_flag' indicators has a value of '1' at a higher level (e.g., sequence level), i.e., when the use of the corresponding mode is permitted. In other cases, the regular merge mode indicator value can always be inferred to be '1' without performing entropy decoding.

[0333] If at least one of the 'sps_mmvd_enabled_flag', 'sps_sbtmvp_enabled_flag', 'sps_affine_enabled_flag', 'sps_ciip_enabled_flag' or 'sps_triangle_enabled_flag' directives has a value of '1', then entropy decoding for regular merge mode directives can additionally be determined based on the width and height of the current block.

[0334] For example, as in Fig. 9, the regular merge mode indicator can be entropy decoded only if the product of the width and height of the current block is not 32. In other words, if the product of the width and height is 32, entropy decoding is not performed and it can always be inferred as '1'. In other words, it can be inferred that the current block is in regular merge mode.

[0335] If at least one of the indicators 'sps_mmvd_enabled_flag', 'sps_sbtmvp_enabled_flag', 'sps_affine_enabled_flag', 'sps_ciip_enabled_flag' or 'sps_triangle_enabled_flag' has a value of '1', and additionally, if 'sps_mmvd_enabled_flag' has a first value of '1', the regular merge mode indicator can always be entropy decoded regardless of the horizontal and vertical lengths of the current block, as shown in FIG. 9.

[0336]

[0337] Figure 10 is another example of a syntax table for inter-screen prediction mode information to be entropy decoded when the current block is encoded / decoded in general merge mode (i.e., when general_merge_flag = 1).

[0338] Unlike in FIG. 9, whether to entropy decode the regular merge mode indicator (regular_merge_flag) can be determined based on the maximum number of entropy-decoded sub-block merge modes (MaxNumSubblockMergeCand) at another higher level (e.g., sequence level, picture level, or slice header) rather than directly using the entropy-decoded 'sps_sbtmvp_enabled_flag', 'sps_affine_enabled_flag' indicators at a higher level (e.g., sequence level).

[0339] The above MaxNumSubblockMergeCand can have a value from 0 to 5, and can be derived as follows by the 'five_minus_max_num_subblock_merge_cand' information.

[0340] MaxNumSubblockMergeCand = 5- five_minus_max_num_subblock_merge_cand

[0341] If the above MaxNumSubblockMergeCand has a value of '0', it may mean that the sub-block unit merge mode is not used for the current slice.

[0342] As in Fig. 10, if the maximum number of sub-block merge mode information of the current block is greater than 0, or if at least one of the entropy-decoded 'sps_mmvd_enabled_flag', 'sps_ciip_enabled_flag', or 'sps_triangle_enabled_flag' indicators at a higher level has a value of '1', entropy decoding for the regular merge mode indicator (regular_merge_flag) can be performed.

[0343]

[0344] Figure 11 is another example of a syntax table for inter-screen prediction mode information to be entropy decoded when the current block is encoded / decoded in general merge mode (i.e., when general_merge_flag = 1).

[0345] Unlike in Fig. 9, whether to entropy decode the regular merge mode indicator (regular_merge_flag) may be determined based on the maximum number of entropy-decoded triangle merge modes (MaxNumTriangleMergeCand) at another higher level (e.g., slice header) rather than directly using the entropy-decoded 'sps_triangle_enabled_flag' indicator at a higher level (e.g., sequence level). Here, the triangle merge mode may be a geometric merge mode.

[0346] The above MaxNumTriangleMergeCand can be derived as follows by the decrypted 'max_num_merge_cand_minus_max_num_triangle_cand' information at the slice level.

[0347] MaxNumTriangleMergeCand = MaxNumMergeCand- max_num_merge_cand_minus_max_num_triangle_cand

[0348] The above MaxNumMergeCand may mean the number of merge candidates derived from 'six_minus_max_num_merge_cand' decrypted at the slice level.

[0349] If the above MaxNumTriangleMergeCand has a value of '0', it may mean that the triangle segmentation merge mode is not used for the current slice.

[0350] As in Fig. 11, if the maximum number of information of the triangulation merge mode of the current block is greater than 1, or if at least one of the entropy-decoded 'sps_mmvd_enabled_flag', 'sps_sbtmvp_enabled_flag', 'sps_affine_enabled_flag', and 'sps_ciip_enabled_flag' at the upper level has a value of '1', entropy decoding for the regular merge mode indicator (regular_merge_flag) can be performed.

[0351]

[0352] Figure 12 is another example of a syntax table for inter-screen prediction mode information to be entropy decoded when the current block is encoded / decoded in general merge mode (i.e., when general_merge_flag = 1).

[0353] Unlike in FIG. 9, whether to entropy decode the regular merge mode indicator (regular_merge_flag) can be determined based on the maximum number information of entropy-decoded sub-block merge modes (MaxNumSubblockMergeCand) and the maximum number information of triangulation merge modes (MaxNumTriangleMergeCand) at other higher levels (e.g., sequence level, picture level, or slice header), rather than directly using the entropy-decoded 'sps_sbtmvp_enabled_flag', 'sps_affine_enabled_flag', and 'sps_triangle_enabled_flag' indicators at a higher level (e.g., sequence level).

[0354] The above MaxNumSubblockMergeCand can have a value from 0 to 5, and can be derived as follows by the 'five_minus_max_num_subblock_merge_cand' information.

[0355] MaxNumSubblockMergeCand = 5- five_minus_max_num_subblock_merge_cand

[0356] If the above MaxNumSubblockMergeCand has a value of '0', it may mean that the sub-block unit merge mode is not used for the current block.

[0357] The above MaxNumTriangleMergeCand can be derived as follows by the decrypted 'max_num_merge_cand_minus_max_num_triangle_cand' information at the slice level.

[0358] MaxNumTriangleMergeCand = MaxNumMergeCand- max_num_merge_cand_minus_max_num_triangle_cand

[0359] The above MaxNumMergeCand means the number of merge candidates derived from 'six_minus_max_num_merge_cand' decrypted at the slice level.

[0360] If the above MaxNumTriangleMergeCand has a value of '0', it may mean that the triangle segmentation merge mode is not used for the current slice.

[0361] As in Fig. 12, if the maximum number of triangle division merge modes of the current block is greater than 1, or the maximum number of sub-block merge modes is greater than 0, or at least one of the entropy-decoded 'sps_mmvd_enabled_flag' and 'sps_ciip_enabled_flag' at the upper level has a value of '1', entropy decoding for the regular merge mode indicator (regular_merge_flag) can be performed.

[0362]

[0363] Figure 13 is an example syntax table for an indicator indicating whether to use non-regular merge mode with entropy decoding in a sequence parameter set.

[0364] When the value of the non-regular merge mode enablement indicator (e.g., sps_non_regular_merge_enabled_flag) is the first value '1', at least one indicator among the indicators of the merge mode with motion vector difference enablement indicator, the indicator of the sub-block unit temporal motion vector prediction enablement indicator, the indicator of the affine model transform-based prediction enablement indicator, the indicator of the intra-picture combined merge mode enablement indicator, and the indicator of the triangulation merge mode enablement indicator can be entropy decoded.

[0365] When the above non-regular merge mode use indicator value is the second value '0', the merge mode use indicator with motion vector difference, the sub-block unit temporal motion vector prediction use indicator, the affine model transform-based prediction use indicator, the intra-screen combined merge mode use indicator, and the triangulation merge mode use indicator can be inferred to be the second value '0' that can indicate that the corresponding prediction modes are not used without entropy decoding.

[0366]

[0367] Figure 14 is another example of a syntax table for inter-screen prediction mode information to be entropy decoded when the current block is encoded / decoded in general merge mode (i.e., when general_merge_flag = 1).

[0368] Whether to use entropy decoding for the regular merge mode indicator (regular_merge_flag) can be determined based on the value of the indicator indicating whether to use entropy decoded non-regular merge mode at a higher level, such as the Sequence Parameter Set (SPS), Picture Parameter Set (PPS), Adaptation Parameter Set (APS), subpicture, tile group, and slice header.

[0369] For example, the regular merge mode indicator (regular_merge_flag) can be entropy decoded only if the non-regular merge mode enable indicator (e.g., sps_non_regular_merge_enabled_flag) has a primary value of '1' and satisfies at least one of the two conditions (the decrypted sps_mmvd_enabled_flag at the upper level has a primary value of '1', and the product of the width and height of the current block is not 32).

[0370]

[0371] Below, the merge mode with motion vector difference, sub-block merge mode, intra-screen combined merge mode and triangulation mode are described in detail.

[0372]

[0373] Merge with motion vector difference (MMVD mode) can use the motion vector among the motion information of the merge candidate selected by the decoded merge index as the motion information of the current block after correcting it based on the correction information. Here, the correction information can include at least one of the correction direction information (mmvd_direction_idx) and the correction size information (mmvd_distance_idx). The motion vector difference information can be derived through the correction information, and the motion information obtained by adding the derived motion vector difference information to the motion vector among the motion information of the merge candidate selected by the merge index can be used as the motion information of the current block.

[0374] The merge mode with the motion vector difference can be indicated by the value of the merge mode indicator with the motion vector difference (e.g., mmvd_merge_flag), and the indicator can be entropy encoded / decoded.

[0375] For example, if the entropy-decoded 'mmvd_merge_flag' is the first value '1', it may mean that the current block is decoded in merge mode with motion vector difference. If the 'mmvd_merge_flag' is the first value '1', 'mmvd_cand_flag' corresponding to merge index information, correction direction information (mmvd_distance_idx), and correction size information (mmvd_direction_idx) may be entropy-decoded.

[0376] For example, if the entropy decoded 'mmvd_merge_flag' has a second value of '0', it may mean that the current block is decoded in at least one other merge mode, such as regular merge mode, sub-block merge mode, intra-frame combined merge mode, or triangulation merge mode, rather than the merge mode with motion vector difference.

[0377] For example, if the entropy decoded 'mmvd_merge_flag' has a second value of '0', it may mean that the current block is decoded in regular merge mode, not merge mode with motion vector differences.

[0378] The above directive may not perform entropy decoding based on the value of the directive indicating whether to use entropy-decoded motion vector differential merge mode at a higher level, such as a sequence parameter set (SPS), a picture parameter set (PPS), an adaptation parameter set (APS), a subpicture, a tile group, or a slice header.

[0379] For example, in SPS, 'mmvd_merge_flag' can be entropy decoded only if the indicator (e.g., sps_mmvd_enabled_flag) for enabling merge mode with entropy-decoded motion vector differences has the first value '1' indicating the use of merge mode with motion vector differences.

[0380] If the above indicator (mmvd_merge_flag) is not entropy decoded, it can be set to a value of '0'. In other words, it can indicate that the merge mode with motion vector difference is not applied to the current decoding target block.

[0381]

[0382] Subblock merge mode may refer to a mode that derives motion information in units of subblocks of the current block. When the subblock merge mode is applied, a subblock merge candidate list may be generated using motion information (subblock-based temporal merge candidate) of a subblock collocated with the current subblock in a reference image and / or an affine control point motion vector merge candidate.

[0383] The above sub-block merge mode can be indicated by the value of the sub-block merge mode indicator (e.g., merge_subblock_flag), and the indicator can be entropy encoded / decoded.

[0384] For example, if the entropy-decrypted 'merge_subblock_flag' is the first value '1', it may mean that the current block is decrypted in sub-block merge mode. If the 'merge_sublock_flag' is the first value '1', the sub-block merge index information (merge_subblock_idx) may be entropy-decrypted.

[0385] For example, if the entropy decoded 'merge_subblock_flag' is the second value '0', it may mean that the current block is decoded in at least one other merge mode, such as regular merge mode, merge mode with motion vector difference, intra-screen combined merge mode, or triangulation merge mode, rather than sub-block merge mode.

[0386] The above directive may not perform entropy decoding based on at least one of an indicator indicating whether to use entropy-decoded sub-block unit temporal motion vector prediction at a higher level, such as a sequence parameter set (SPS), a picture parameter set (PPS), an adaptation parameter set (APS), a sub-picture, a tile group, a slice header, etc., and an indicator indicating whether to use affine transform model-based prediction.

[0387] For example, if the indicator 'merge_subblock_flag' in SPS has a first value of '1' indicating the use of sub-block unit temporal motion vector prediction, the indicator 'merge_subblock_flag' can be entropy decoded.

[0388] For example, if the 'merge_subblock_flag' directive in SPS has a first value of '1' indicating that the affine transformation model is used, the 'merge_subblock_flag' directive can be entropy decoded.

[0389] For example, if both 'sps_sbtmvp_enabled_flag' and 'sps_affine_enabled_flag' in SPS, which are entropy decoded, have a second value of '0' indicating that sub-block unit temporal motion vector prediction and affine transform model prediction are not used, the above-mentioned indicator 'merge_subblock_flag' may not be entropy decoded.

[0390] The above directive can perform entropy decoding based on the maximum number of sub-block merge modes (MaxNumSubblockMergeCand). For example, if MaxNumSubblockMergeCand is '0', the directive 'merge_subblock_flag' may not be entropy decoded.

[0391] If the above indicator (merge_subblock_flag) is not entropy decoded, it can be set to a value of '0'. In other words, it can indicate that the sub-block unit merge mode is not applied to the current decryption target block.

[0392]

[0393] The combined inter-intra prediction mode (or CIIP mode, Combined Inter Intra Prediction mode) may refer to a mode that derives a prediction sample of the current block by weighting and adding prediction samples generated by inter-prediction and prediction samples generated by intra-prediction.

[0394] The combined merge mode within the above screen can be indicated by the value of the combined merge mode indicator within the screen (e.g., ciip_flag), and the indicator can be entropy encoded / decoded.

[0395] For example, if the entropy-decoded 'ciip_flag' is the first value '1', it may mean that the current block is decoded in the merge mode combined within the screen. If the 'ciip_flag' is the first value '1', the merge index information (merge_idx) for generating inter-screen prediction samples may be entropy-decoded.

[0396] For example, if the entropy decoded 'ciip_flag' has a second value of '0', it may mean that the current block is decoded in at least one other merge mode, such as regular merge mode, merge mode with motion vector difference, sub-block merge mode, or triangulation merge mode, rather than the in-screen combined merge mode.

[0397] The above directive may not perform entropy decoding based on the value of the directive indicating whether to use the combined merge mode within the entropy-decoded picture at a higher level, such as the Sequence Parameter Set (SPS), Picture Parameter Set (PPS), Adaptation Parameter Set (APS), subpicture, tile group, slice header, etc.

[0398] For example, in SPS, 'ciip_flag' can be entropy decoded only if the combined merge mode use indicator (e.g., sps_ciip_enabled_flag) within the entropy decoded screen has the first value '1' indicating the use of combined merge mode within the screen.

[0399] If the above directive is not entropy decoded, the value of the combined merge mode directive within the screen may be set based on at least one of the combined merge mode use directive (sps_ciip_enabled_flag), the regular merge mode directive (regular_merge_flag), the skip directive (cu_skip_flag), and the size of the current block.

[0400]

[0401] The geometric partitioning mode (GPM) may refer to a mode in which the current block is divided to derive motion information for each block, each prediction sample is derived using each derived motion information, and each derived prediction sample is weighted and added to derive a prediction sample of the current block. Here, the geometric partitioning mode may refer to a mode in which the current block is divided into asymmetric sub-blocks and prediction is performed.

[0402] The above triangulation merge mode can be indicated by the value of the triangulation merge mode indicator (e.g., triangle_merge_flag), and the indicator can be entropy encoded / decoded.

[0403] For example, if the entropy decrypted 'triangle_merge_flag' is the first value '1', it may mean that the current block is decrypted in triangulation merge mode.

[0404] For example, if the entropy decoded 'triangle_merge_flag' is the second value '0', it may mean that the current block is decoded in at least one other merge mode, such as regular merge mode, merge mode with motion vector difference, sub-block merge mode, or intra-screen combined merge mode, rather than triangulation merge mode.

[0405] The above directive can perform entropy decoding based on the value of the directive indicating whether to use entropy-decoded triangulation merge mode at a higher level, such as a sequence parameter set (SPS), a picture parameter set (PPS), an adaptation parameter set (APS), a subpicture, a tile group, or a slice header.

[0406] For example, in SPS, 'trianlge_merge_flag' can be entropy decoded only if the enable entropy decoded triangulation merge mode indicator (e.g., sps_triangle_enabled_flag) has the first value '1' indicating the use of triangulation merge mode.

[0407] If the above indicator is not entropy decoded, it can be set to a value of '0'. In other words, it can indicate that the triangulation merge mode is not applied to the current decryption target block.

[0408] Meanwhile, whether the triangulation merge mode is performed can be determined based on the value of the combined merge mode indicator (e.g., ciip_flag) within the screen. For example, if the combined merge mode indicator (ciip_flag) within the screen is a second value of '0' that does not indicate the combined merge mode within the screen, the current block can be determined to be in the triangulation merge mode, and the triangulation merge prediction information can be entropy decoded.

[0409] Additionally, whether the geometric segmentation merge mode is performed can be determined based on the value of the combined merge mode indicator (e.g., ciip_flag) within the screen. For example, if the combined merge mode indicator (ciip_flag) within the screen is '0', which is a second value that does not indicate the combined merge mode within the screen, the geometric segmentation merge prediction information can be entropy decoded.

[0410]

[0411] The triangulation merge mode can be indicated by the triangulation merge mode indicator value, and the indicator can set the indicator (e.g., MergeTriangleFlag) value of the current block to '1' when the following conditions are satisfied without performing entropy encoding / decoding. Here, the triangulation merge mode can mean the geometric merge mode. That is, the indicator indicating whether to use the triangulation merge mode, the maximum number of the triangulation merge mode, and the triangulation merge mode indicator can mean the indicator indicating whether to use the geometric merge mode, the maximum number of the geometric merge mode, and the geometric merge mode indicator, respectively.

[0412] If at least one of the conditions below is not met, the 'MergeTriangleFlag' value can be set to '0'. In other words, it can mean that the triangulation merge mode is not allowed for the current block.

[0413] If the value of the indicator indicating whether to use entropy-decoded triangulation merge mode at the top level (e.g., sps_triangle_enabled_flag) is '1',

[0414] If the slice type of the current block is B slice,

[0415] If the general merge mode indicator (general_merge_flag) value is '1',

[0416] If the maximum allowable number of candidates in the triangle merge mode (MaxNumTriangleMergeCand) is greater than or equal to 2,

[0417] If the product of the width and height of the current block is greater than or equal to 64,

[0418] If the regular merge mode indicator (regular_merge_flag) value is '0',

[0419] If the merge mode indicator (mmvd_merge_flag) with motion vector difference is '0',

[0420] If the sub-block merge mode indicator (merge_subblock_flag) value is '0',

[0421] If the value of the merge mode indicator (ciip_flag) within the screen is '0',

[0422] If the above MergeTriangleFlag is the first value '1', the split direction information indicator (merge_triangle_split_dir) and the merge index information (merge_triangle_idx0, merge_triangle_idx1) for each of the two sub-blocks can be entropy encoded / decoded.

[0423] When the maximum allowable number of candidates (MaxNumTriangleMergeCand) in the triangle segmentation merge mode is 2, 'merge_triangle_idx1' can use the remaining candidates except for the candidates indicated by 'merge_triangle_idx0' without performing entropy encoding / decoding.

[0424]

[0425] If the encoded / decoded general merge mode (general_merge_flag) for the current block is the first value, '1', and all of the merge mode indicator values ​​below are the second value, '0', the merge index information (merge_idx) for the current block can be inferred as '0' without entropy decoding.

[0426] Regular merge mode indicator (e.g., regular_merge_flag)

[0427] Merge mode indicator with motion vector difference (e.g., mmvd_merge_flag)

[0428] Sub-block merge mode indicator (e.g., merge_subblock_flag)

[0429] Combined merge mode indicators within the screen (e.g., ciip_flag)

[0430] Triangulation merge mode indicator (e.g., MergeTriangleFlag)

[0431] If the general merge mode (general_merge_flag) is the first value '1' and the merge index information is inferred as '0' under the above conditions, the motion information of the first candidate among the motion information stored in the merge candidate list can be used as the motion information of the current block.

[0432]

[0433] The motion information stored in the merge candidate list may be at least one of motion information of a neighboring block adjacent to the current block (spatial merge candidate), motion information of a block collocated with the current block in a reference image (temporal merge candidate), new motion information generated by a combination of motion information already existing in the merge candidate list, motion information of a block encoded / decoded before the current block (history-based merge candidate), and a zero merge candidate.

[0434]

[0435] As another example, if the encoded / decoded general merge mode (general_merge_flag) for the current block is the first value, '1', and the merge mode indicator values ​​below are all the second value, '0', the current block can be set to fallback merge mode, a merge list can be constructed with only zero merge candidates, and then the motion information of the first candidate can be used as the motion information of the current block.

[0436] Regular merge mode indicator (regular_merge_flag)

[0437] Merge mode indicator with motion vector difference (mmvd_merge_flag)

[0438] Subblock merge mode indicator (merge_subblock_flag)

[0439] Combined merge mode indicator (ciip_flag) within the screen

[0440] Triangulation merge mode indicator (MergeTriangleFlag)

[0441] If the general merge mode (general_merge_flag) is the first value '1' and the merge index information is inferred as '0' under the above conditions, additional correction may not be performed based on the motion vector derived from the index information.

[0442] If the general merge mode (general_merge_flag) is the first value '1' and the current block is set to fallback merge mode under the above conditions, no additional correction may be performed.

[0443] The above additional correction may mean a process of correcting motion information with the minimum SAD by searching a pre-defined area based on a reference block indicated by motion information derived from merge index information '0' in the decoder (Decoder Motion Vector Refinement, DMVR).

[0444] The above additional compensation may mean a process (Bi-Directional Optical Flow, BDOF) of compensating for prediction samples derived through inter-screen prediction using optical flow based on motion information derived from merge index information '0' in the decoder.

[0445]

[0446] Prediction modes applicable to Inter mode (or AMVP mode, Advance Motion Vector Prediction mode) may include inter mode based on block copying within the screen, general AMVP mode, affine inter mode using an affine transformation model, symmetric MVD mode (SMVD, Symmetric Motion Vector Difference) that derives MVD in the L1 direction based on MVD in the L0 direction, and AMVR mode (Adaptive Motion Vector Resolution) that adjusts the resolution of MVD.

[0447]

[0448] The block copy inter mode within the screen can be determined based on the entropy decoded prediction mode information (prediction mode IBC indicator, e.g., pred_mode_ibc_flag), and a motion vector obtained by adding the decoded block vector difference (BVD) to the motion vector prediction candidate selected by the decoded motion vector index (e.g., mvp_l0_flag) can be used as the motion vector of the current block, i.e., the block vector.

[0449]

[0450] The general AMVP mode can use the motion vector obtained by adding the decoded L0 / L1 motion vector differential to the motion vector prediction candidate selected by the decoded motion vector index (e.g., mvp_lX_flag, X=0 or 1) for each of the L0 direction and L1 direction as the L0 / L1 motion vector of the current block.

[0451] Inter-screen prediction indicator (inter_pred_idc) indicating whether L0, L1 unidirectional inter-screen prediction or L0 / L1 bidirectional inter-screen prediction is performed, reference picture index (ref_idx) for L0 and L1 directions, and horizontal / vertical motion vector difference value information for L0 and L1 directions can be entropy encoded / decoded.

[0452] If at least one of the horizontal or vertical motion vector difference values ​​for the L0 and L1 directions is not '0', the adaptive motion vector difference indicator (amvr_flag) can be entropy decoded, and if the indicator value is '1', the indicator (amvr_precision_flag) for 1 / 16-pixel, 1 / 2-pixel, 1-pixel or 4-pixel resolution can be entropy decoded.

[0453] For example, if 'amver_flag' is the first value '1' and 'amvr_precision_flag' is the second value '0', it can mean that the motion vector difference has a resolution of 1 (integer) pixel. If 'amver_flag' is the first value '1' and 'amvr_precision_flag' is the first value '1', it can mean that the motion vector difference has a resolution of 4 (integer) pixels. If 'amvr_flag' is the second value '0', the motion vector difference has a resolution of 1 / 4 pixel.

[0454]

[0455] The affine inter mode can use the motion vector obtained by adding the decoded L0 / L1 affine control point motion vector difference to the affine control point motion vector prediction candidate selected by the decoded motion vector index (e.g., mvp_lX_flag, X=0 or 1) for each of the L0 direction and the L1 direction as the L0 / L1 affine control point motion vector of the current block.

[0456] The affine inter mode indicator (inter_affine_flag) that indicates whether the current block is in affine inter mode can be entropy encoded / decoded.

[0457] When the above indicator (inter_affine_flag) is the first value '1', the affine type information (cu_affine_type_flag) indicating whether it is a 4-parameter using 2 affine control point motion vectors or a 6-parameter using 3 affine control point motion vectors can be entropy encoded / decoded.

[0458] If 'cu_affine_type_flag' is the first value '1', it can mean a 6-parameter using 3 affine control point motion vectors.

[0459] If 'cu_affine_type_flag' is the second value '0', it can mean a 4-parameter using two affine control point motion vectors.

[0460] The above affine type information (cu_affine_type_flag) can determine whether to entropy encode / decode based on an indicator (e.g., sps_affine_type_flag) of whether to use 6 entropy-decoded parameters at a higher level, such as a sequence parameter set (SPS), a picture parameter set (PPS), an adaptation parameter set (APS), a subpicture, a tile group, and a slice header.

[0461] For example, if the entropy-decoded sps_affine_type_flag in SPS is '1' and the affine inter mode indicator 'inter_affine_flag' of the current decryption target block is 1, cu_affine_type_flag can be added / decoded.

[0462] If the above affine type information does not exist, it can be inferred as the second value, '0', i.e. 4-parameter.

[0463] Inter-screen prediction indicator (inter_pred_idc) indicating whether L0, L1 unidirectional inter-screen prediction or L0 / L1 bidirectional inter-screen prediction is performed, and reference image index (ref_idx) for L0 and L1 directions, based on affine type information, horizontal / vertical motion vector difference value information for two or three L0 and L1 directions can be entropy encoded / decoded.

[0464]

[0465] If at least one of the horizontal or vertical motion vector difference values ​​for the above two or three L0 and L1 directions is not '0', the adaptive motion vector difference indicator (amvr_flag) can be entropy decoded, and if the indicator value is '1', the indicator (amvr_precision_flag) for 1-pixel or 1 / 16-pixel resolution can be entropy decoded.

[0466] If 'amver_flag' is the first value '1' and 'amvr_precision_flag' is the second value '0', it can mean that the motion vector difference value is 1 / 16 pixel resolution.

[0467] If 'amver_flag' is the first value '1' and 'amvr_precision_flag' is the first value '1', it can mean that the motion vector difference value has a resolution of 1 (integer) pixel.

[0468] If 'amvr_flag' is the second value '0', the motion vector difference may mean 1 / 4 pixel resolution.

[0469]

[0470] The above adaptive motion vector difference indicator (amvr_flag) can determine whether to entropy encode / decode based on an indicator (e.g., sps_affine_amvr_enabled_flag) indicating whether to use entropy-decoded affine adaptive motion vector difference at a higher level, such as a sequence parameter set (SPS), a picture parameter set (PPS), an adaptation parameter set (APS), a subpicture, a tile group, or a slice header.

[0471] The above affine adaptive motion vector difference use indicator (sps_affine_amvr_enabled_flag) can be entropy encoded / decoded only when the affine transformation model prediction use indicator (sps_affine_enabled_flag) value is the first value, '1', as shown in FIG. 15.

[0472] For example, in SPS, if the encoded / decoded 'sps_affine_amvr_enabled_flag' is the first value '1', the current block's affine inter mode indicator 'inter_affine_flag' is the first value '1', and at least one of the horizontal or vertical motion vector difference values ​​for two or three L0 and L1 directions is not '0', the adaptive motion vector difference indicator (amvr_flag) can be entropy encoded / decoded.

[0473]

[0474] SMVD mode can use reference image index information derived at the slice level without performing entropy encoding / decoding on the reference image index (ref_idx) information for each of the L0 and L1 directions. In addition, the horizontal / vertical motion vector differential values ​​in the L1 direction can be used as the horizontal / vertical motion vector differential values ​​in the L1 direction, which are symmetrical values ​​of the entropy-encoded / decoded L0-direction horizontal / vertical motion vector differential values, without performing entropy encoding / decoding on the horizontal / vertical motion vector differential values ​​in the L1 direction.

[0475] A symmetric MVD mode indicator (sym_mvd_flag) indicating whether the current block is in SMVD mode can be entropy encoded / decoded.

[0476] The above indicator (sym_mvd_flag) can determine whether to entropy encode / decode based on at least one of an indicator (sps_smvd_enabled_flag) indicating whether to use entropy-decoded symmetric MVD mode at a higher level such as a Sequence Parameter Set (SPS), a Picture Parameter Set (PPS), an Adaptation Parameter Set (APS), a subpicture, a tile group, a slice header, etc.; an inter-screen prediction indicator (inter_pred_idc) indicating whether to use L0, L1 unidirectional inter-screen prediction or L0 / L1 bidirectional inter-screen prediction; and mvd_l1_zero_flag indicating whether to use the motion vector difference in the L1 direction as '0' without entropy decoding by entropy-decoding / decoding in the slice header.

[0477] For example, if the decoded 'sps_smvd_enabled_flag' in SPS is the first value '1' that allows symmetric MVD, and the inter-screen prediction indicator of the current block is bidirectional prediction, and the decoded 'mvd_l1_zero_flag' is the second value '0', then the indicator (sym_mvd_flag) can be entropy encoded / decoded.

[0478] For example, if the decoded 'sps_smvd_enabled_flag' in SPS is the first value '1' that allows symmetric MVD, and the inter-screen prediction indicator of the current block is bidirectional prediction, and the decoded 'mvd_l1_zero_flag' is the first value '1', the indicator (sym_mvd_flag) may not be entropy encoded / decoded.

[0479] If the above 'sym_mvd_flag' value does not exist, the indicator value is inferred to be '0', which may mean that symmetric MVD is not allowed for the current block.

[0480]

[0481] FIG. 16 is a diagram for explaining the inter-screen prediction information syntax of the merge mode according to one embodiment of the present invention.

[0482] Figure 16 is an example of a syntax table regarding inter-screen prediction mode information to be entropy decoded when the current block is encoded / decoded in general merge mode (i.e., when general_merge_flag = 1).

[0483] The sub-block merge indicator (merge_subblock_flag) can determine whether to perform entropy decoding based on at least one of the maximum number of entropy-decoded sub-block merge modes (MaxNumSubblockMergeCand) at a higher level (e.g., sequence level or picture level), the horizontal size of the current block, and the vertical size of the current block.

[0484] For example, if MaxNumSubblockMergeCand is greater than '0' and both the width of the current block and the height of the current block are greater than or equal to 8, the sub-block merge directive can be entropy decoded.

[0485] The regular merge mode indicator (regular_merge_flag) may determine whether to perform entropy decoding based on at least one of the sub-block merge mode indicator (merge_subblock_flag), the skip indicator (cu_skip_flag), the indicator indicating whether to use combined prediction within the screen (sps_ciip_enabled_flag), the horizontal size of the current block (cbWidth), the vertical size of the current block (cbHeight), the area of ​​the current block (cbWidth*cbHeight), the slice type (slice_type), and the maximum number of triangle merge candidates (MaxNumTriangleMergeCand).

[0486] For example, if the sub-block merge mode indicator (merge_subblock_flag) is '1', which is the first value indicating the sub-block merge mode, the regular merge mode indicator (regular_merge_flag) may not be entropy decoded.

[0487] For example, if the in-screen combined prediction enablement indicator (sps_ciip_enabled_flag) has a first value of '1' indicating that in-screen combined prediction is allowed, and the skip indicator (cu_skip_flag) has a second value of '0' indicating that skip mode is not used, then the regular merge mode indicator (regular_merge_flag) can be entropy decoded.

[0488]

[0489] For example, if the slice type is B slice and the maximum number of triangulation merge candidates is greater than or equal to a predefined value (e.g., 2), the regular merge mode indicator (regular_merge_flag) can be entropy decoded. If the maximum number of triangulation merge candidates is less than the predefined value, it may mean that the triangulation merge mode is not allowed. For example, if the slice type is B slice and the triangulation merge mode is allowed, the regular merge mode indicator (regular_merge_flag) can be entropy decoded.

[0490] For example, if the slice type indicates an I slice or a P slice and the skip indicator (cu_skip_flag) is the first value '1' indicating skip mode, the regular merge mode indicator (regular_merge_flag) may not be entropy decoded.

[0491] For example, if the slice type indicates an I-slice or a P-slice and the in-picture combined prediction use indicator (sps_ciip_enabled_flag) is a second value of '0', which does not indicate that in-picture combined prediction use is allowed, the regular merge mode indicator (regular_merge_flag) may not be entropy decoded.

[0492] For example, if at least one of the width of the current block and the height of the current block is greater than or equal to a predefined value (e.g., 128), the regular merge mode indicator (regular_merge_flag) may not be entropy decoded.

[0493] Additionally, if the conditions described in FIG. 16 are satisfied, the regular merge mode indicator (regular_merge_flag) can be entropy decoded.

[0494] The merge mode indicator (mmvd_merge_flag) with motion vector difference can be entropy decoded when the regular merge mode indicator is '1', which is the first value indicating regular merge mode, and the merge mode with motion vector difference use indicator (sps_mmvd_enabled_flag) is '1', which is the first value indicating use of merge mode with motion vector difference.

[0495] The combined merge mode indicator (ciip_flag) in the picture may be entropy decoded based on at least one of the regular merge mode indicator (regular_merge_flag), the skip indicator (cu_skip_flag), the combined prediction use indicator (sps_ciip_enabled_flag), the width of the current block (cbWidth), the height of the current block (cbHeight), the slice type (slice_type), and the maximum number of triangle merge candidates (MaxNumTriangleMergeCand). The combined merge mode indicator (ciip_flag) in the picture may be entropy decoded if the regular merge mode indicator has a second value of '0' indicating that it is not regular merge mode.

[0496] If the maximum number of triangulation merge candidates (MaxNumTriangleMergeCand) is less than 2, the merge mode indicator (ciip_flag) combined within the screen may not be entropy decoded. If the maximum number of triangulation merge candidates (MaxNumTriangleMergeCand) is less than 2, it may mean that the triangulation merge mode is not allowed.

[0497] If the slice type indicates an I-slice or a P-slice, the merge mode indicator (ciip_flag) within the screen may not be entropy decoded.

[0498] Merge index information (merge_idx) can be entropy decoded when the combined merge mode indicator (ciip_flag) within the screen is the first value '1' indicating the combined merge mode within the screen, and the maximum number of merge candidates (MaxNumMergeCand) is greater than 1.

[0499] If the combined merge mode indicator (ciip_flag) within the screen is not entropy decoded, the value of the combined merge mode indicator within the screen may be set based on at least one of the combined merge mode use indicator (sps_ciip_enabled_flag), the regular merge mode indicator (regular_merge_flag), the skip indicator (cu_skip_flag), and the size of the current block.

[0500] Therefore, even if the combined merge mode indicator (ciip_flag) within the screen is not entropy decoded, the value of the combined merge mode indicator (ciip_flag) within the screen can be set to the first value, '1', so that the merge index information (merge_idx) can be entropy decoded even if the combined merge mode indicator within the screen is not entropy decoded.

[0501] Whether the triangulation merge mode is performed can be determined based on the value of the combined merge mode indicator (ciip_flag) within the screen. For example, if the combined merge mode indicator (ciip_flag) within the screen is a second value of '0' that does not indicate the combined merge mode within the screen, the triangulation merge prediction information (merge_tranagle_split_dir, merge_triangle_idx0, merge_triagle_idx1) can be entropy decoded. Here, the triangulation merge mode can be a geometric split merge mode.

[0502]

[0503] FIG. 17 is a diagram illustrating syntax elements representing various inter-screen prediction modes according to one embodiment of the present invention.

[0504] A value of '1' for the regular merge mode indicator (regular_merge_flag) may indicate that regular merge mode or merge mode with motion vector difference is used to derive inter-screen prediction information. A value of '0' for the regular merge mode indicator (regular_merge_flag) may indicate that intra-screen combined merge mode or geometry segmentation merge mode is used to derive inter-screen prediction information.

[0505] A value of the merge mode indicator (mmvd_merge_flag) with motion vector difference may indicate that the merge mode with motion vector difference is used to derive inter-frame prediction information. A value of the merge mode indicator (mmvd_merge_flag) with motion vector difference may indicate that the regular merge mode is used to derive inter-frame prediction information.

[0506] Meanwhile, if the merge mode indicator (mmvd_merge_flag) with motion vector difference is not entropy decoded, it can be set to the value of '0'. In other words, it can indicate that the merge mode with motion vector difference is not applied to the current decoding target block and that the regular merge mode is applied.

[0507] The combined merge mode indicator (ciip_flag) within a screen can indicate whether the combination of inter-screen prediction and intra-screen prediction is applied.

[0508] Meanwhile, if the combined merge mode indicator (ciip_flag) within the screen is not entropy decoded, the value of the combined merge mode indicator within the screen may be set based on at least one of the combined merge mode use indicator (sps_ciip_enabled_flag), the regular merge mode indicator (regular_merge_flag), the skip indicator (cu_skip_flag), and the size of the current block.

[0509]

[0510] Figure 18 is a flowchart illustrating an image decoding method according to an embodiment of the present invention.

[0511] Referring to FIG. 18, the decoder can decode the sub-block merge mode indicator (e.g., merge_subblock_flag) of the current block (S1801).

[0512] And, if the sub-block merge mode indicator indicates that the sub-block merge mode is not a sub-block merge mode (S1802-No), the decoder can decode the regular merge mode indicator (e.g., regular_merge_flag) of the current block (S1803).

[0513] Here, the regular merge mode indicator may, when it is a first value (e.g., '1'), indicate that a regular merge mode or a merge mode with motion vector difference is used to derive inter-screen prediction information of the current block, and when it is a second value (e.g., '0'), indicate that an intra-screen combined merge mode or a geometric segmentation merge mode is used to derive inter-screen prediction information of the current block.

[0514] The above regular merge mode indicator can determine whether to decode based on an indicator (e.g., sps_ciip_enabled_flag) of whether to use combined prediction within the screen of the current block.

[0515] The above regular merge mode indicator can determine whether to decrypt based on the skip indicator (e.g., cu_skip_flag) of the current block.

[0516] The above regular merge mode indicator can determine whether to decrypt based on the slice type (e.g., slice_type) of the current block.

[0517] The above regular merge mode directive may not be decrypted if at least one of the horizontal size of the current block and the vertical size of the current block is greater than or equal to a predefined value (e.g., 128).

[0518] And, if the sub-block merge mode indicator indicates the sub-block merge mode (S1802-Yes), the decoder can decode the sub-block merge mode information of the current block (S1804). Here, the sub-block merge mode information may be a sub-block merge index (e.g., merge_subblock_idx).

[0519] Meanwhile, the decoder can decode a merge mode indicator (mmvd_merge_flag) having a motion vector difference of the current block when the regular merge mode indicator is a first value (e.g., '1'). And, the decoder can decode a merge mode indicator (e.g., ciip_flag) combined within the screen of the current block when the regular merge mode indicator is a second value (e.g., '0').

[0520] If the merge mode indicator with the motion vector difference is not entropy decoded, the merge mode indicator with the motion vector difference may be considered as a predefined value (e.g., '0') indicating a regular merge mode rather than a merge mode with the motion vector difference.

[0521] If the combined merge mode indicator within the screen is not entropy decoded, the combined merge mode indicator within the screen may be set based on at least one of a regular merge mode indicator, an indicator indicating whether to use the combined merge mode within the screen, a skip indicator, and the size of the current block.

[0522]

[0523] FIG. 19 is a flowchart illustrating an image encoding method according to an embodiment of the present invention.

[0524] Referring to FIG. 19, the encoder can encode a sub-block merge mode indicator (e.g., merge_subblock_flag) of the current block (S1901).

[0525] And, if the sub-block merge mode indicator indicates that the sub-block merge mode is not a sub-block merge mode (S1902-No), the encoder can encode the regular merge mode indicator (regular_merge_flag) of the current block (S1903).

[0526] Here, the regular merge mode indicator may, when it is a first value (e.g., '1'), indicate that a regular merge mode or a merge mode with motion vector difference is used to derive inter-screen prediction information of the current block, and when it is a second value (e.g., '0'), indicate that an intra-screen combined merge mode or a geometric segmentation merge mode is used to derive inter-screen prediction information of the current block.

[0527] The above regular merge mode indicator may determine whether to encode based on an indicator (e.g., sps_ciip_enabled_flag) indicating whether to use combined prediction within the screen of the current block.

[0528] The above regular merge mode indicator can determine whether to encode based on the skip indicator (e.g., cu_skip_flag) of the current block.

[0529] The above regular merge mode indicator can determine whether to encode based on the slice type (e.g., slice_type) of the current block.

[0530] The above regular merge mode indicator may not be encoded if at least one of the horizontal size of the current block and the vertical size of the current block is greater than or equal to a predefined value (e.g., 128).

[0531] And, if the sub-block merge mode indicator indicates the sub-block merge mode (S1902-Yes), the encoder can encode the sub-block merge mode information of the current block (S1904). Here, the sub-block merge mode information can be a sub-block merge index (e.g., merge_subblock_idx).

[0532] Meanwhile, the encoder can encode a merge mode indicator (mmvd_merge_flag) having a motion vector difference of the current block when the regular merge mode indicator is a first value (e.g., '1'). And, the encoder can encode a merge mode indicator (e.g., ciip_flag) combined within the screen of the current block when the regular merge mode indicator is a second value (e.g., '0').

[0533] A computer-readable non-transitory recording medium according to the present invention can store a bitstream generated by the image encoding method described in FIG. 19.

[0534] Specifically, in a non-transitory computer-readable recording medium storing a bitstream generated by an image encoding method, the image encoding method may include a step of encoding a sub-block merge mode indicator of a current block, a step of encoding a regular merge mode indicator of the current block when the sub-block merge mode indicator indicates that it is not a sub-block merge mode, and a step of encoding sub-block merge mode information of the current block when the sub-block merge mode indicator indicates a sub-block merge mode.

[0535]

[0536] The above embodiments can be performed in the same manner in the encoder and decoder.

[0537] An image can be encoded / decoded using at least one or a combination of at least one of the above embodiments.

[0538] The order of applying the above embodiments may be different in the encoder and the decoder, and the order of applying the above embodiments may be the same in the encoder and the decoder.

[0539] The above embodiments can be performed for each of the luminance and chrominance signals, and the above embodiments can be performed equally for the luminance and chrominance signals.

[0540] The shape of the block to which the above embodiments of the present invention are applied may have a square shape or a non-square shape.

[0541] At least one of the syntax elements (flags, indices, etc.) that are entropy encoded in the encoder and entropy decoded in the decoder can use at least one of the binarization methods below.

[0542] Truncated Rice binarization method

[0543] K-th order Exp_Golomb binarization method

[0544] Restricted K-th order Exp_Golomb binarization method

[0545] Fixed-length binarization method

[0546] Unary binarization method

[0547] Truncated unary binarization method

[0548] Truncated Binary Binarization Method

[0549] The above embodiments of the present invention can be applied according to the size of at least one of an encoding block, a prediction block, a transform block, a block, a current block, an encoding unit, a prediction unit, a transform unit, a unit, and a current unit. Here, the size may be defined as a minimum size and / or a maximum size for applying the above embodiments, or may be defined as a fixed size for which the above embodiments are applied. In addition, the first embodiment of the above embodiments may be applied to a first size, and the second embodiment may be applied to a second size. That is, the above embodiments may be applied in a complex manner depending on the size. In addition, the above embodiments of the present invention may be applied only when the size is greater than or equal to the minimum size and less than or equal to the maximum size. That is, the above embodiments may be applied only when the block size is included within a certain range.

[0550] For example, the above embodiments can be applied only when the size of the current block is 8x8 or larger. For example, the above embodiments can be applied only when the size of the current block is 4x4. For example, the above embodiments can be applied only when the size of the current block is 16x16 or smaller. For example, the above embodiments can be applied only when the size of the current block is 16x16 or larger and 64x64 or smaller.

[0551] The above embodiments of the present invention can be applied according to temporal layers. A separate identifier is signaled to identify the temporal layer to which the above embodiments are applicable, and the above embodiments can be applied to the temporal layer specified by the identifier. The identifier here may be defined as the lowest layer and / or highest layer to which the above embodiments are applicable, or may be defined as indicating a specific layer to which the above embodiments are applicable. In addition, a fixed temporal layer to which the above embodiments are applicable may be defined.

[0552] For example, the above embodiments may be applied only when the temporal layer of the current image is the lowest layer. For example, the above embodiments may be applied only when the temporal layer identifier of the current image is 1 or greater. For example, the above embodiments may be applied only when the temporal layer of the current image is the highest layer.

[0553] The slice type or tile group type to which the above embodiments of the present invention are applied is defined, and the above embodiments of the present invention can be applied according to the slice type or tile group type.

[0554] In the embodiments described above, the methods are described based on a flowchart as a series of steps or units. However, the present invention is not limited to the order of the steps, and some steps may occur in a different order or simultaneously with other steps described above. Furthermore, those skilled in the art will understand that the steps depicted in the flowchart are not exclusive, and that other steps may be included, or one or more steps in the flowchart may be deleted without affecting the scope of the present invention.

[0555] The above-described embodiments include examples of various aspects. While not all possible combinations to illustrate the various aspects can be described, those skilled in the art will recognize that other combinations are possible. Accordingly, the present invention is intended to encompass all other alterations, modifications, and variations within the scope of the following claims.

[0556] The embodiments of the present invention described above may be implemented in the form of program commands that can be executed through various computer components and recorded on a computer-readable recording medium. The computer-readable recording medium may include program commands, data files, data structures, etc., either singly or in combination. The program commands recorded on the computer-readable recording medium may be those specially designed and configured for the present invention or may be those known and available to those skilled in the art of computer software. Examples of the computer-readable recording medium include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical recording media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and hardware devices specially configured to store and execute program commands, such as ROMs, RAMs, and flash memories. Examples of program commands include not only machine language codes generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter, etc. The hardware devices may be configured to operate as one or more software modules to perform processing according to the present invention, and vice versa.

[0557] Although the present invention has been described above with specific details such as specific components and limited examples and drawings, these are provided only to help a more general understanding of the present invention, and the present invention is not limited to the above examples, and those with ordinary knowledge in the technical field to which the present invention pertains can make various modifications and variations from this description.

[0558] Therefore, the idea of ​​the present invention should not be limited to the embodiments described above, and all things that are modified equally or equivalently to the following claims as well as the claims are considered to fall within the scope of the idea of ​​the present invention.

[0559] The present invention can be used in a device for encoding / decoding an image.

Claims

1. In the video decryption method, A step of decoding the sub-block merge mode indicator of the current block; If the sub-block merge mode indicator indicates that the current block is not in sub-block merge mode, a step of decoding the regular merge mode indicator of the current block; and An image decoding method comprising a step of decoding sub-block merge mode information of the current block when the sub-block merge mode indicator indicates a sub-block merge mode.

2. In paragraph 1, The above regular merge mode directive is, If the first value is 0, it indicates that the regular merge mode or the merge mode with motion vector difference is used to derive the inter-screen prediction information of the current block. A video decoding method characterized in that, in the case of the second value, an intra-screen combined merge mode or a geometric segmentation merge mode is used to derive inter-screen prediction information of the current block.

3. In paragraph 1, The above regular merge mode directive is, A video decoding method characterized in that whether to decode is determined based on an indicator of whether to use combined prediction within the screen of the current block.

4. In paragraph 1, The above regular merge mode directive is, An image decoding method characterized in that whether or not the skip indicator of the current block is decoded is determined based on the skip mode.

5. In paragraph 1, The above regular merge mode directive is, An image decoding method characterized in that whether to decode is determined based on the slice type of the current block.

6. In paragraph 1, The above regular merge mode directive is, An image decoding method characterized in that, if at least one of the horizontal size of the current block and the vertical size of the current block is greater than or equal to a predefined value, the image is not decoded.

7. In paragraph 2, If the regular merge mode indicator is the first value, a step of decoding the merge mode indicator having the motion vector difference of the current block; and A video decoding method, characterized in that it further comprises a step of decoding a merge mode indicator combined within the screen of the current block when the regular merge mode indicator is the second value.

8. In paragraph 7, The merge mode indicator with the above motion vector difference is, A video decoding method characterized in that, if the merge mode indicator with the motion vector difference is not decoded in the step of decoding the above, it is regarded as a predefined value indicating a regular merge mode rather than a merge mode with a motion vector difference.

9. In paragraph 7, The merge mode indicators combined within the above screen are: A video decoding method characterized in that, in the step of decoding the combined merge mode indicator within the screen, if not decoded, it is set based on at least one of the regular merge mode indicator, the combined merge mode use indicator within the screen, the skip indicator, and the size of the current block.

10. In the video encoding method, A step of encoding a sub-block merge mode indicator of the current block; If the sub-block merge mode indicator indicates that the current block is not in sub-block merge mode, encoding the regular merge mode indicator of the current block; and An image encoding method comprising a step of encoding sub-block merge mode information of the current block when the sub-block merge mode indicator indicates a sub-block merge mode.

11. In paragraph 10, The above regular merge mode directive is, If the first value is 0, it indicates that the regular merge mode or the merge mode with motion vector difference is used to derive the inter-screen prediction information of the current block. A video encoding method characterized in that, in the case of the second value, it indicates that an intra-screen combined merge mode or a geometric segmentation merge mode is used to derive inter-screen prediction information of the current block.

12. In paragraph 10, The above regular merge mode directive is, A video encoding method characterized in that whether to encode is determined based on an indicator indicating whether to use combined prediction within the screen of the current block.

13. In paragraph 10, The above regular merge mode directive is, A video encoding method characterized in that whether or not the skip indicator of the current block is encoded is determined based on the skip mode.

14. In paragraph 10, The above regular merge mode directive is, An image encoding method characterized in that whether to encode is determined based on the slice type of the current block.

15. In paragraph 10, The above regular merge mode directive is, An image encoding method characterized in that, if at least one of the horizontal size of the current block and the vertical size of the current block is greater than or equal to a predefined value, encoding is not performed.

16. In paragraph 11, If the regular merge mode indicator is the first value, a step of encoding a merge mode indicator having a motion vector difference of the current block; and A video encoding method, characterized in that it further comprises a step of encoding a merge mode indicator combined within the screen of the current block when the regular merge mode indicator is the second value.

17. In a non-transitory computer-readable recording medium storing a bitstream generated by a video encoding method, The above image encoding method is, A step of encoding a sub-block merge mode indicator of the current block; If the sub-block merge mode indicator indicates that the current block is not in sub-block merge mode, encoding the regular merge mode indicator of the current block; and A recording medium characterized in that it comprises a step of encoding sub-block merge mode information of the current block when the sub-block merge mode indicator indicates the sub-block merge mode.

Images