Method, device, and recording medium for image encoding / decoding based on plurality of prediction modes

Abstract

Provided is an image decoding method for decoding a target block on the basis of a plurality of prediction modes. The image decoding method comprises the steps of: determining a prediction mode of the current block; deriving transform information of the current block on the basis of the prediction mode of the current block and the size of the current block; and carrying out transform of the current block on the basis of the transform information of the current block.

Description

Method, apparatus, and recording medium for image encoding / decoding based on multiple prediction modes

[0001] The present disclosure relates to a method, apparatus, and recording medium for image encoding / decoding. Specifically, the present invention relates to a method, apparatus, and recording medium for image encoding / decoding based on a plurality of prediction modes.

[0002] With the continuous development of the information and communication industry, services providing video through broadcasting and the Internet have spread globally.

[0003] Users demand videos with higher resolution and quality. To meet these user demands, video encoding and decoding technologies suitable for such videos are required. Video encoding technology can generate compressed video by compressing the video representing the images to have a smaller amount of data. Video decoding technology can generate reconstructed images using the compressed video.

[0004] Regarding video encoding and decoding technologies, various techniques exist, such as segmentation, prediction, transformation, quantization, filtering, and entropy encoding and decoding. By introducing, modifying, improving, and combining these various techniques, video and images can be compressed, transmitted, and stored more effectively.

[0005] One embodiment of the present disclosure provides a method and apparatus for image encoding / decoding based on a plurality of prediction modes to improve encoding efficiency.

[0006] In addition, one embodiment of the present disclosure provides a method for transmitting or storing a bitstream generated by an image encoding method, or a recording medium for storing a bitstream.

[0007] One aspect of the present disclosure provides an image decoding method for decoding a target block using a plurality of prediction modes. The method may include the steps of: determining a prediction mode of a current block; deriving transformation information of the current block based on the prediction mode of the current block and the size of the current block; and performing a transformation on the current block based on the transformation information of the current block.

[0008] Another aspect of the present disclosure provides an image encoding method for encoding a target block using a plurality of prediction modes. The method may include the steps of: determining a prediction mode of a current block; deriving transformation information of a current block based on the prediction mode of the current block and the size of the current block; performing a transformation on the current block based on the transformation information of the current block; and encoding the transformation information of the current block.

[0009] Another aspect of the present disclosure provides a method for providing a bitstream generated by the image encoding method to an image decoding device or storing it in a computer-readable recording medium.

[0010] Another aspect of the present disclosure provides a computer-readable recording medium that stores a bitstream generated by the image encoding method.

[0011] FIG. 1 shows a system for video coding according to one embodiment.

[0012] Figure 2 shows a segmentation structure of an image according to one embodiment.

[0013] Figure 3 shows the structure of an intra prediction according to one embodiment.

[0014] FIG. 4 shows the structure of an inter prediction to explain an inter prediction process according to one embodiment.

[0015] FIG. 5 shows the order of addition of spatial candidates to the candidate list according to one embodiment.

[0016] Figure 6 shows a plurality of in-loop filters according to one example.

[0017] Figure 7 shows the structure of entropy encoding and entropy decoding according to one example.

[0018] FIG. 8 illustrates a flowchart of one embodiment of the image encoding / decoding method and apparatus of the present disclosure.

[0019] FIG. 9 illustrates an example of deriving a gradient histogram.

[0020] Figure 10 illustrates an example of a template configuration.

[0021] Figure 11 illustrates an example of a juxtaposed block.

[0022] Figure 12 illustrates examples of surrounding blocks in the current image and surrounding blocks in the reference image.

[0023] Figure 13 illustrates an example of surrounding blocks adjusted (moved) using block vectors.

[0024] FIG. 14 illustrates an example where the upper right or lower left adjacent block is not available.

[0025] FIGS. 15 and 16 illustrate an example of a reference template for the current block.

[0026] Figure 17 illustrates an example of an in-screen sub-partition (ISP) prediction method.

[0027] Figures 18 and 19 illustrate examples of sequential in-frame sub-partition (ISP) prediction.

[0028] Figure 20 illustrates an example of a template for a sub-partition.

[0029] Figure 21 illustrates examples of deriving gradient histograms in a restored reference region.

[0030] Figure 22 illustrates an example of a TIMD list.

[0031] FIGS. 23 and FIGS. 24 illustrate an example of a merge list.

[0032] Figure 25 illustrates an example of the accumulation of HoG or occurrence frequency by mode.

[0033] Figure 26 illustrates an example of a change in a reference sample or line based on a comparison of the sum of HoG or occurrence frequencies by mode and a threshold.

[0034] Figure 27 illustrates examples of a smoothing filter and a sharpening filter.

[0035] Figure 28 illustrates examples of a 2x2 filter and a 3x3 filter.

[0036] FIG. 29 illustrates an embodiment of deriving a new candidate (vector) using a block vector or motion vector stored in a block at a location indicated by a previously derived block vector or motion vector in a list.

[0037] FIG. 30 is an example diagram showing a set of conversion types by block size according to one embodiment.

[0038] FIG. 31 is an example diagram showing a mapping table between an in-screen prediction mode and a transformation set index according to one embodiment.

[0039] FIG. 32 is a diagram illustrating a method for deriving conversion kernel set candidates according to one embodiment.

[0040] FIG. 33 is a diagram illustrating a method for deriving a set of conversion kernels for each mode group according to one embodiment.

[0041] Various modifications may be applied to the present invention. Additionally, the present invention may have various embodiments. Specific embodiments are described by the drawings and the detailed description.

[0042] Specific embodiments are not intended to limit the invention to specific embodiments, and it should be understood that all modifications, equivalents, and substitutions that fall within the spirit and scope of the invention are included as embodiments of the invention.

[0043] The embodiments are described in sufficient detail to enable those skilled in the art to practice the embodiments. It should be understood that the various embodiments are different but need not be mutually exclusive. For example, it should be understood that the shapes, structures, and characteristics described in relation to one embodiment may be applied to or implemented in other embodiments without departing from the spirit and scope of the invention. It should also be understood that the location or arrangement of components within one embodiment may be changed without departing from the spirit and scope of the invention. Accordingly, the following detailed description is not intended to be limiting, and the scope of the exemplary embodiments is limited only by the appended claims and all equivalents to the scope claimed by such claims, provided that they are appropriately described.

[0044] The detailed description of the embodiments described below may refer to the drawings relating to the embodiments. Descriptions described in the drawings or descriptions represented by the drawings may be considered part of the detailed description. In the drawings, similar reference numerals may refer to the same or similar functions for various aspects. Dependencies between components may not be limited to those depicted in the drawings.

[0045] In the embodiments, singular expressions may include plural expressions and may be limited to and / or limited to plural expressions unless the context clearly excludes plural expressions. That is to say, in the embodiments, expressions such as 'at least one' and 'one or more' may be replaced with 'plural'. Terms such as ' / ', 'and / or', 'at least one of' and 'one or more of' described for plural items may mean 1) one of the plural items, 2) some of the plural items, 3) a combination of some of the plural items, or 4) a combination of the plural items. Additionally, plural expressions may be replaced with singular expressions. Plural may mean an integer of 1, 2, 3, 4, or 5 or more.

[0046] In the embodiments, numbered terms such as 'first' and 'second' may be used to describe various components. These terms are used solely for the purpose of distinguishing one component from another and do not limit the components. For example, without departing from the scope of the present invention, the first component may be named the second component, and similarly, the second component may be named the first component.

[0047] The statement that a first component transmits (or provides) information to a second component may mean that the first component directly transmits information to the second component, or it may mean that the first component transmits information to the second component through another third component. Here, the information received (or acquired) by the second component may be information transmitted by the first component, or information generated by applying a specific processing to information transmitted by the first component.

[0048] The components of the embodiments may be illustrated independently to represent different characteristic functions, and this does not imply that each component corresponds to a separate hardware or a single software unit. That is, the components of the embodiments may be classified and enumerated for convenience of description. Two or more components described in the embodiments may be regarded as a single component. Furthermore, a single component described in the embodiments may be separated into multiple components that perform the functions of the said component separately. Embodiments in which such components are integrated and embodiments in which components are separated are also included within the scope of the present invention, provided that they do not depart from the essence of the invention.

[0049] The terms used in the embodiments are used merely to describe specific embodiments and are not intended to limit the invention. In the embodiments, terms such as "comprising" or "having" indicate the presence of features, numbers, steps, actions, components, parts, or combinations thereof described in the embodiments. The existence or addition of other features, numbers, steps, actions, components, parts, or combinations thereof not explicitly described in the embodiments is not excluded by these terms. That is, the description of a specific component of an embodiment as "comprising" does not exclude components other than the specific component, and means that additional components may also be included within the scope of the embodiments or the technical concept of the invention.

[0050] Some of the components of the embodiments may be optional components that are not essential for performing the essential functions of the invention. Such optional components may be used to enhance performance. The embodiments may be implemented as a structure comprising only the essential components required to realize the essence of the embodiments, excluding the optional components. Such a structure is also included within the scope of the embodiments.

[0051] In the following, embodiments are described in detail with reference to the attached drawings so that a person skilled in the art can easily implement the embodiments. In describing the embodiments, if it is determined that a detailed description of related known configurations or known functions could obscure the gist of this specification, such detailed description is omitted. Additionally, the same reference numerals are used for identical components within the drawings, and redundant descriptions of identical components are omitted.

[0052]

[0053] Replacement of terms in the examples

[0054] Below, terms listed in a single line may be used with the same meaning in the examples and may be used interchangeably in the examples.

[0055] - 'one or more', 'at least one'

[0056] - 'two or more', 'a plurality of', 'multiple', 'multiple'. (In the examples, 'one or more' or 'at least one' may be further limited to 'two or more', 'multiple', or 'multiple'.)

[0057] - 'Information', 'Signal'

[0058] - 'value', 'predefined value', 'specific value', 'threshold', 'threshold value', 'baseline value', 'reference value'

[0059] - 'statistical value', 'statistics value'

[0060] - 'indicator', 'index', 'index', 'flag', 'information'

[0061] - 'encoder', 'encoding apparatus'

[0062] - 'decoder', 'decoding apparatus'

[0063] - 'Entropy encoding', 'encoding', 'encoding'

[0064] - 'Entropy decoding', 'decoding', 'decoding'

[0065] - 'Coding', 'Encoding and / or decoding'

[0066] - 'video', 'moving picture', 'image', 'picture', 'picture', 'frame', 'screen'

[0067] - 'Reference picture', 'Reference video'

[0068] - 'Reference Picture List (RPL), 'Reference Image List'

[0069] - 'original', 'input', 'source'

[0070] - 'Block', 'Unit', 'Signal'

[0071] - 'square', 'square shape'

[0072] - 'pixel', 'pixel', 'sample', 'pel'

[0073] - 'region', 'area', 'part', 'segment'

[0074] - 'partition', 'split', 'divide'

[0075] - 'quad', 'quadronary'

[0076] - 'Luma component', 'Luma', 'luminance component', 'luminance', 'Y'

[0077] - 'Chroma component', 'Chroma', 'chrominance', 'chrominance component', 'Cb and Cr', 'Cb or Cr', 'Cb', 'Cr', 'U and V', 'U or V', 'U', 'V'

[0078] - 'target', 'current' (e.g., target block and current block, or target image and current image)

[0079] - 'neighbor', 'neighboring', 'adjacent', 'neighbor / neighboring' (e.g., neighbor block, adjacent block, and neighboring block)

[0080] - 'collocated', 'COL'

[0081] - 'reconstruction', 'reconstruction', 'decoding'

[0082] - 'reconstructed', 'reconstructed', 'decoded'

[0083] - 'Difference', 'Difference', 'Difference', 'Error', 'Residual', 'Residual'

[0084] - Largest Coding Unit (LCU), Coding Tree Unit (CTU)

[0085] - 'inter', 'inter-screen'

[0086] - 'Inter prediction', 'inter prediction', 'motion compensation'

[0087] - 'Inter Mode', 'Inter Prediction Mode', 'Inter-frame Mode', 'Inter-frame Prediction Mode'

[0088] - 'Motion Vector', 'Predicted Motion Vector', 'Advanced Motion Vector Prediction (AMVP)'

[0089] - 'List', 'Candidate List'

[0090] - 'spatial candidate', 'spatial merge candidate'

[0091] - 'temporal candidate', 'temporal merge candidate'

[0092] - 'prediction motion vector candidate', 'motion vector predictor'

[0093] - 'Prediction method', 'Prediction mode'

[0094] - 'Intra', 'Inside the screen'

[0095] - 'Intra prediction', 'Intra prediction'

[0096] - 'Intra Mode', 'Intra Prediction Mode'

[0097] - 'Dequantization', 'Scaling'

[0098] - 'Quantization matrix', 'Scaling list'

[0099] - 'Quantization matrix coefficients', 'Matrix coefficients'

[0100] - 'transform coefficient level', 'quantized level', 'quantized coefficient', 'quantized transform coefficient', 'quantized transform coefficient level'

[0101] - 'dequantized coefficient', 'dequantized transform coefficient'

[0102] - 'Scanning type', 'Scanning direction'

[0103] - 'directional mode', 'angle mode', 'angular mode', 'intra-prediction mode'

[0104] - 'Intra-prediction mode (mode) number', 'Intra-prediction mode (mode) index', 'Intra-prediction mode (mode) value', 'Intra-prediction mode (mode) angle', 'Intra-prediction mode (mode) direction', 'Intra-prediction direction (mode) number', 'Intra-prediction direction (mode) index', 'Intra-prediction direction (mode) value', 'Intra-prediction direction (mode) angle'

[0105] - 'Merge Mode', 'Motion Merge Mode'

[0106] - 'Geometric Partitioning Mode (GPM)', 'Triangle Partitioning Mode'

[0107] In addition to the terms exemplified above, terms having the same meaning according to the ordinary knowledge of the technical field may be used interchangeably in the embodiments.

[0108]

[0109] Information and range of values ​​of information described in the embodiments

[0110] In the embodiments, information may include a constant, a flag, an index, a variable, a coding parameter, an element, a syntax element, motion information, an attribute, an entity, an object, and data, etc. That is to say, the term 'information' may be interchangeable with 'data', 'flag', 'index', 'variable', 'element', 'syntax element', 'motion information', 'attribute', or 'entity'.

[0111] Information can have one of multiple values. 'The nth value' can mean the nth value among multiple values.

[0112] For example, the first value can represent '0' or (logical) false. The second value can represent '1' or (logical) true. Or, the first value can represent '1' or (logical) true. The second value can represent '0' or (logical) false.

[0113] A flag may be information having a value of either '0' or '1'. In the embodiments, the values ​​'0' and '1' of the flag may be replaced with '1' and '0', respectively. For example, information indicating whether a specific process is performed or information indicating whether a specific process is applied may be considered as a flag.

[0114] When a variable such as i or j is used to represent a row, column, or index, the variable may be an integer between 0 and n - 1 inclusive. Or, the variable may be an integer between 1 and n inclusive. Here, n may be the number of rows, the number of columns, or the number of entities pointed to by the index.

[0115]

[0116] Concepts related to coding

[0117] Concepts related to coding are explained below. The descriptions disclosed below may be applied to embodiments.

[0118] Predefined value: A predefined value may refer to a value commonly used by the encoding device and the decoder. For example, a predefined value may be interpreted as being limited to a fixed value. Alternatively, a predefined value may be a value shared by the encoding device and the decoder through signaling. Alternatively, a predefined value may be a value derived through the same procedure in the encoding device and the decoder so that the encoding device and the decoder have a common value. Alternatively, a predefined value may be a common value possessed by the encoding device and the decoder. The above description of a predefined value may also apply to predefined information. In the above descriptions, 'value' may be replaced with 'information'.

[0119] - Values ​​derived through the same procedure in the above-mentioned encoding device and decoding device may include values ​​derived through the same procedure for the same value and / or the same information in the encoding device and decoding device.

[0120] - Values ​​derived through the same procedure in the above-mentioned encoding device and decoding device may include values ​​derived using the same conditional statement for the same value and / or the same information in the encoding device and decoding device.

[0121] - The description of the predefined values ​​above may also apply to predefined information. In the descriptions above, 'value' may be replaced with 'information'.

[0122] Availability: The availability of specific modes for a specific target may mean that a selected mode among the specific modes is used for that specific target. Other modes belonging to the category of specific modes may be non-available modes. Non-available modes may not be used for a specific target. The above description of specific modes may also apply to other specific information. In the above descriptions, 'mode' may be replaced with 'information'.

[0123] Adjacency: 'Direction' for 'First Object'. 'Second Object' may refer to a 'Second Object' adjacent to the 'Direction' corner / face of the First Object. For example, the 'Top-left Block' for a 'Target Block' may be a block adjacent to the top-left of the Target Block. Here, the 'First Object' may be a Target Unit, Target Block, or Target Sample. 'Direction' may be one of left-above, above, right-above, left, right, left-below, below, and right-below. The 'Second Object' may be a Unit, Block, or Sample. For the directions of top-left, top-right, bottom-left, and bottom-right, the corner of the First Object and the corner of the Second Object may be diagonally adjacent. For the directions of top, left, right, and bottom, one face of the First Object and one face of the Second Object may be in contact with each other.

[0124] - For example, the block adjacent to the top-left of the target block may be the block adjacent to the top of the block adjacent to the left of the target block. The block adjacent to the top-right of the target block may be the block adjacent to the right of the block adjacent to the top of the target block. The block adjacent to the bottom-left of the target block may be the block adjacent to the bottom of the block adjacent to the left of the target block.

[0125] Coding: Coding can refer to encoding and / or decoding of an image.

[0126] Signal: A signal can represent information about an image, unit, or block. A specific signal can represent a specific image, a specific unit, or a specific block.

[0127] Image: An image can refer to a single picture constituting a video, or it can represent the video itself. For example, "encoding and / or decoding of an image" can mean "encoding and / or decoding of a video," or it can mean "encoding and / or decoding of one of the images constituting a video."

[0128] - An image can refer to the entirety of a picture, or it can refer to a part of a picture, such as a block.

[0129] Target image: The target image may be an encoding target image that is the subject of encoding and / or a decoding target image that is the subject of decoding. Additionally, the target image may be an input image processed by an encoding device and a restored image processed by a decoding device. The target image may be an image containing a target block.

[0130] Subpicture: A picture can be divided into one or more subpictures.

[0131] - A subpicture may be a square or rectangular area within the picture. A subpicture may include one or more CTUs.

[0132] - A subpicture may include one or more slices and / or one or more tiles. For example, a subpicture may consist of one or more slice rows and one or more slice columns. Alternatively, each subpicture may consist of one or more tile rows and one or more tile columns.

[0133] - A subpicture may include one or more slices that collectively cover a rectangular area within the picture. Accordingly, the boundary of each subpicture can always be the boundary of a slice. Additionally, each vertical subpicture boundary can always be the boundary of a vertical tile.

[0134] Slice: A slice may include one or more tiles within a picture. A slice may consist of one or more rows of tiles and one or more columns of tiles.

[0135] Tile: A tile can be a square or rectangular area within a picture. A tile can contain one or more CTUs. A picture can be divided into one or more tile rows and one or more tile columns.

[0136] CTU: An image can be divided into multiple Coding Tree Units (CTUs).

[0137] - A CTU may include one Y Coding Tree Block (CTB) and at least one of a Cb CTB and a Cr CTB associated with the Y CTB, and may include information for each CTB. The information may include syntax elements.

[0138] - Each CTU may be partitioned using one or more partitioning methods to form sub-units such as Coding Units (CU), Prediction Units (PU), and Transform Units (TU). One or more partitioning methods may include Quad Tree (QT) partitioning, Binary Tree (BT) partitioning, and Ternary Tree (TT) partitioning. Additionally, each CTU may be partitioned using Multi-Type Tree (MTT) partitioning, which uses a combination of multiple partitioning methods.

[0139] CTB: CTB can refer to one of Y CTB, Cb CTB, and Cr CTB.

[0140] Unit: A unit can be determined for specific processing in coding. A unit may be information about a specific region within an image. For specific processing in coding, the image may be recursively divided into multiple parts. A unit may represent the region to which the specific processing is applied and information about the aforementioned region.

[0141] - The unit type may represent a specific process applied to the unit. Depending on the unit type, a specific process may be applied to the unit. The 'specific' unit may be a unit for the process named 'specific' in the coding. For example, the unit may be at least one of the source unit, CTU, coding unit, prediction unit, residual unit, restored residual unit, transformation unit, and restored unit.

[0142] - A unit may include samples having a two-dimensional form or arrangement. In this respect, a 'unit' may mean a 'block'. For example, a block may be at least one of an original block, a CTB, a coding block (CB), a prediction block (PB), a residual block, a restored residual block, a transform block (TB), and a restored block. For example, a partition of a unit may mean a partition of a block corresponding to the unit.

[0143] - A unit may include syntactic elements. In other words, a block and the syntactic elements for the block can be combined and referred to as a unit.

[0144] - A block is an MxN array of samples. Here, M and N can represent positive integer values, and a block can commonly represent a two-dimensional array of samples. The current block can represent the encoding target block that is the subject of encoding during encoding, or the decoding target block that is the subject of decoding during decoding. Additionally, the current block can be at least one of a coding block, a prediction block, a residual block, a transformation block, or a restoration block. Blocks can have various sizes and shapes. For example, the shape of a block can be one or more of a tetragon, a rectangular, a square, a rectangle where the width differs from the height (i.e., an oblong), a trapezoid, a triangle, a right-angled triangle, and a pentagon. Here, the width and height of the rectangle can differ from each other. Additionally, the shape of a block may include other geometric figures that can be represented in two dimensions. For example, the shape of the block may be a square or a pentagon defined by subtracting the area of ​​a right triangle from the area of ​​a rectangle. Here, the right-angled vertex of the right triangle may be one of the vertices of the rectangle. Additionally, the shape of the block may be a combination of two or more of the aforementioned shapes. Additionally, the shape of the block may be the remainder of one of the aforementioned shapes after another shape has been subtracted.

[0145] - In the embodiments, the rectangle may be limited to a non-square rectangle. When the shape of a specific object in the embodiments is described as a rectangle, this description may additionally imply that the width and height of the specific object are different from each other.

[0146] - In the embodiments, the block may be limited to at least one of a vertically oriented block and a horizontally oriented block. A vertically oriented block may mean a block in which the vertical length is greater than the horizontal length. A horizontally oriented block may mean a block in which the horizontal length is greater than the vertical length.

[0147] - The unit may include a luma component block (i.e., a Y block) and two chroma component blocks (i.e., at least one of a Cb block and a Cr block), and may include information for each block. The information may include syntax elements.

[0148] - The unit information may include the unit type, unit size, unit depth, unit encoding order, and unit decoding order.

[0149] Target Unit: The target unit may be a block that is the target of encoding, an encoding target unit, and / or a decoding target unit that is the target of decoding. The target unit may be a specific region within the target picture to which one or more specific processing steps of coding are applied. By applying a specific processing step to the target unit, a unit of a specific type may be generated. Alternatively, the target unit may represent a unit having a specific type for a specific processing step of coding.

[0150] Depth: A block can be hierarchically divided into multiple sub-blocks with depth according to a tree structure. The multiple sub-blocks created by the division of a block can be referred to as partitions.

[0151] - The block depth can represent the level of the node corresponding to the block when the blocks constituting the image are represented as a tree structure. Alternatively, the block depth can represent the number of divisions applied until the block is determined. The block depth can increase by 1 as the block is further divided.

[0152] - In a tree structure, the root node can be considered to have the smallest level, and the leaf node the largest level. The root node may be the top node of the tree structure and may correspond to the first undivided block. The level of the root node may be 0 or 1. When the level of the root node is 0, a node with level 1 may represent the block determined by the first block being divided once. A node with level n may represent the block determined by the first block being divided n times. A leaf node may be the lowest node of the tree structure. A leaf node may be a node that cannot be further divided. The depth of a leaf node may be a predefined maximum depth. For example, the maximum depth may be a positive integer such as 3. The root node may represent a CTU. A leaf node may represent at least one of CU, PU, ​​or TU.

[0153] - Depth can have a type depending on the type of partition. QT depth can represent the depth for quadtree partitioning. BT depth can represent the depth for binary partitioning. TT depth can represent the depth for ternary partitioning.

[0154] Sample: A sample can be a base unit that constitutes a block. A sample can consist of one or more bits. Bit depth can be the number of bits that make up the sample. Samples range from 0 to 2 depending on the bit depth. Bd It can be expressed as values ​​up to -1.

[0155] PU: PU may refer to a base unit for processing related to prediction. For example, processing related to prediction may include inter-prediction, intra-prediction, intra-block copy (IBC) prediction, intra-compensation, and motion compensation.

[0156] A single PU can be divided into multiple sub-PUs that are smaller in size compared to the PU. These multiple sub-PUs can also serve as base units for processing related to prediction. In other words, a prediction unit partition generated by the division of the prediction unit can also be a prediction unit.

[0157] TU: A TU may be a base unit for processing related to a residual block. Processing related to a residual block may include at least one of transform, inverse transform, quantization, inverse quantization, transform coefficient encoding, transform coefficient decoding, entropy encoding, and entropy decoding. A single TU may be divided into a plurality of sub-transform units having a size smaller than that of the TU. The plurality of sub-TUs may also be base units for processing related to a residual block. That is to say, a transform unit partition generated by the division of the transform unit may also be a transform unit.

[0158] - The transformation may include one or more of a primary transformation and a secondary transformation, and the inverse transformation may include one or more of a primary inverse transformation and a secondary inverse transformation.

[0159] Parameter set: The parameter set can correspond to header information within the structure of the bitstream.

[0160] - The parameter set may include at least one of a Video Parameter Set (VPS), a Sequence Parameter Set (SPS), a Picture Parameter Set (PPS), an Adaptation Parameter Set (APS), and a Decoding Parameter Set (DPS).

[0161] Information signaled through a parameter set can be applied to pictures that reference the parameter set. For example, information within a VPS can be applied to pictures that reference the VPS. Information within an SPS can be applied to pictures that reference the SPS. Information within a PPS can be applied to pictures that reference the PPS. A parameter set can reference a higher-level parameter set. For example, a PPS can reference an SPS. An SPS can reference a VPS.

[0162] - Additionally, the parameter set may include tile group information, slice header information, and tile header information. A tile group may refer to a group or slice containing multiple tiles.

[0163] MPM (Most Probable Mode): MPM may represent an intra prediction mode that is likely to be used for intra prediction of a target block.

[0164] - One or more different MPMs can be determined based on coding parameters related to the target block and attributes of objects related to the target block.

[0165] - One or more MPMs may be determined based on the intra prediction mode of a reference block. There may be multiple reference blocks. One or more different MPMs may be determined depending on which intra prediction modes are used for one or more reference blocks. Reference blocks may include spatial neighbor blocks.

[0166] MPM List: An MPM list may be a list containing one or more MPMs. The number of one or more MPMs in an MPM list may be predefined.

[0167] MPM Index: The MPM index can indicate one or more MPMs in the MPM list that are used for intra prediction for the target block.

[0168] MPM Usage Indicator: The MPM Usage Indicator can indicate whether an MPM list is used for prediction regarding a target block.

[0169] Prediction mode: The prediction mode may be information indicating a prediction method for a target block, such as a mode used for intra-prediction or a mode used for inter-prediction. The prediction mode may refer to one of the prediction-related modes described in the embodiments. Additionally, the prediction mode may include at least one of an intra-mode, an inter-mode, and an intra-block copy mode.

[0170] Reference image list: The reference image list may be a list containing one or more reference images used for prediction of the target block.

[0171] - There may be multiple reference image lists. Multiple reference image lists may include List 0 (List 0; L0), List 1 (List 1; L1), etc.

[0172] - One or more reference image lists may be used for inter prediction for the target block. Parts such as 'L0' and 'L1' in the names of the information related to inter prediction may refer to the reference image lists associated with the information.

[0173] Reference picture: The reference picture may be an image referenced for prediction regarding the target block. Alternatively, the reference picture may be an image containing the reference block. The reference picture may include an image prior to the target image, the target image, and an image following the target image.

[0174] Reference image index: The reference image index may be an index indicating one reference image among one or more reference images in the reference image list that is used for prediction of the target block.

[0175] Reference Block: A reference block may be a block referenced for encoding / decoding of a target block, such as for prediction and filtering. For example, a reference block may include a reference sample referenced to derive a prediction sample, and may refer to a block that provides information used for decoding the target block.

[0176] Reference Sample: A reference sample may be a sample referenced for encoding / decoding of a target block, such as prediction and filtering.

[0177] Inter prediction indicator: The inter prediction indicator may indicate the direction of inter prediction for the target block. Inter prediction may be one of unidirectional prediction and bidirectional prediction. Alternatively, the inter prediction indicator may indicate the number of reference images used when generating prediction blocks for the target block. Alternatively, the inter prediction indicator may indicate the number of prediction blocks used for inter prediction for the target block. The reference direction may refer to the inter prediction indicator. For example, the inter prediction indicator may indicate either unidirectional or bidirectional. Alternatively, for an inter mode that uses only reference images within the L0 reference image list, the inter prediction indicator may have a first value of '0'; for an inter mode that uses only reference images within the L1 reference image list, the inter prediction indicator may have a second value of '1'; and for an inter mode that uses at least two of the reference images within the L0 reference image list and the L1 reference image list, the inter prediction indicator may have a third value of '2'.

[0178] Prediction List Utilization Flag: The prediction list utilization flag for a specific reference image list may indicate whether at least one reference image within that specific reference image list is used to generate the prediction block of the target block. For example, a value of '0' for the prediction list utilization flag for a specific reference image list may indicate that the prediction block is not generated using the reference images within that specific reference image list. A value of '1' for the prediction list utilization flag for a specific reference image list may indicate that the prediction block is generated using the reference images within that specific reference image list.

[0179] - An inter-prediction indicator can be derived using prediction list utilization flags. Conversely, an inter-prediction indicator can be derived using prediction list utilization flags. For example, an inter-prediction indicator can be derived using prediction list utilization flags for multiple reference image lists. If the inter-prediction indicator indicates that specific reference lists among the multiple reference image lists are being used, the prediction list utilization flags of the specific reference lists pointed to by the inter-prediction indicator among the prediction list utilization flags of the multiple reference image lists can be set to '1', and the prediction list utilization flags of the remaining reference image lists not pointed to by the inter-prediction indicator can be set to '0'.

[0180] Reference Direction: The reference direction may point to a list of reference images used for the prediction of the target block. For example, the reference direction may point to one or more of reference image list L0 and reference image list L1. The reference direction may be used interchangeably with "inter-frame prediction direction" and may be substituted for each other.

[0181] - The reference direction merely refers to the list of reference images used for prediction of the target block, and does not indicate that the directions of the reference images within the list are restricted to a forward direction or a backward direction. That is to say, each of the reference image list L0 and the reference image list L1 may include forward images and backward images, respectively. Here, the forward direction may indicate a direction from the target image to the image preceding the target image. Forward inter-prediction may be an inter-prediction that uses the image preceding the target image as a reference image. The backward direction may indicate a direction from the target image to the image following the target image. Backward inter-prediction may be an inter-prediction that uses the image following the target image as a reference image.

[0182] - A unidirectional reference direction may mean that a single reference image list is used. A bidirectional reference direction may mean that two reference image lists are used. For example, the reference direction may indicate one of the following: that only reference image list L0 is used, that only reference image list L1 is used, or that two reference image lists are used. Additionally, the reference direction may be indicated by an inter-predictor.

[0183] Picture Order Count (POC): The POC of a picture can represent the display order or output order of the picture.

[0184] Motion information: Motion information may be information used to specify a reference block. Motion information may include information used for inter prediction, such as a motion vector (MV), reference image index, reference image, inter prediction indicator, prediction list utilization flag, etc. Additionally, motion information may include information used in a specific inter prediction mode, such as an MV candidate, MV candidate index, merge candidate, and merge index. Additionally, motion information may include information related to the block vector described below. Information related to the block vector may mean information including at least one of a block vector, a block vector candidate, and a block vector candidate index.

[0185] - Multiple motion information for multiple reference image lists may be used for inter-prediction of the target block. Motion information for a specific reference image list may be used for prediction using that specific reference image list. Multiple (intermediate) prediction blocks may be derived from the multiple motion information. A (final) prediction block for the target block may be generated using statistical values ​​for the multiple (intermediate) prediction blocks.

[0186] MV: MV can be a 2-dimensional vector used in inter-prediction. MV can represent the offset between the target block and the reference block. Alternatively, MV can represent the difference between the location of the target block and the location of the reference block.

[0187] - For example, MV is (mv x , mv y It can be expressed in the form of ). mv x can represent a horizontal component, and mv yIt can represent a vertical component.

[0188] - The zero vector can be (0, 0) MV.

[0189] Block Vector (BV): A BV can be a two-dimensional vector used in intra-block copy prediction. A BV can represent the offset between a target block within a target image and a reference block within a target image. In other words, a BV can represent the displacement between a target block and a reference block within a target image.

[0190] - For example, BV is similar to MV (bv x , bv y It can be expressed in the form of ). bv x can represent a horizontal component, and bv y It can represent a vertical component.

[0191] - The zero vector can be (0, 0) BV.

[0192] Motion Information Candidates: In a specific prediction, motion information of the target block can be selected from motion information candidates determined by a specific method. A motion information candidate may refer to the motion information of a reference block, or it may refer to the reference block itself that possesses motion information. Here, the reference block may be a block determined by a specific method to select motion information candidates.

[0193] Candidate List: A candidate list may be a list containing one or more candidates. For example, a candidate list may include a motion information candidate list, a merge candidate list, an MV candidate list, an MPM list, etc. A candidate list may be generated in the same manner in both the encoding device and the decoder. That is to say, the candidate list used in the encoding device and the candidate list used in the decoder may be identical, and the same candidate list may be shared between the encoding device and the decoder. The encoding device may select a candidate from among the candidates in the candidate list to be used for processing the target block. An indicator pointing to the selected candidate may be signaled from the encoding device to the decoder. The decoder may use the indicator to identify the candidate from among the candidates in the candidate list to be used for processing the target block. Alternatively, the encoding device and the decoder may identify the candidate from among the candidates in the candidate list to be used for processing the target block by the same rule.

[0194] Motion Information Candidate List: A motion information candidate list may refer to a list constructed using one or more motion information candidates.

[0195] Motion Information Candidate Index: The motion information candidate index may be an identifier or indicator pointing to a motion information candidate among the motion information candidates in the motion information candidate list that is used for prediction regarding the target block.

[0196] - In a specific inter-prediction mode, motion information of other restored blocks may be used to derive motion information of the target block. Other blocks may include neighboring blocks. In this specific inter-prediction mode, the motion information for the target block itself is not signaled individually, but other information used to derive motion information of the target block based on motion information of other restored blocks may be signaled. In this case, the other information may include information indicating which of the other restored blocks' motion information is used to derive motion information of the target block, such as a motion information candidate index.

[0197] - For example, these inter-prediction modes may include AMVP mode, merge mode, and skip mode. The motion information candidate index may be a merge index or an MV candidate index.

[0198] - In the embodiments, MV may be part of the motion information. In the embodiments, information about motion information, such as motion information candidates, a list of motion information candidates, and an index of motion information candidates, may be replaced with information about MV, such as MV candidates, a list of MV candidates, and an index of MV candidates, and descriptions of motion information may also be applied to MV.

[0199] Merge: Merge can refer to the merging of motion information for multiple blocks, or it can refer to applying the motion information of one block to a target block as well. In other words, merge mode can refer to a mode where the motion information of a target block is derived from the motion information of a neighboring block.

[0200] Merge Candidate: A merge candidate may refer to a specific (restored) block used for merging with a target block, or it may refer to movement information of a specific block. Alternatively, a merge candidate may include movement information of a specific block.

[0201] - Merge candidates for the target block may include spatial merge candidates, temporal merge candidates, history-based candidates, average candidates based on the average of two merge candidates, and zero merge candidates.

[0202] Merge candidate list: The merge candidate list may be a list composed of one or more merge candidates.

[0203] Merge Index: The merge index may be an indicator pointing to a merge candidate among the merge candidates in the merge candidate list that is used for prediction regarding the target block. Among the merge candidates in the merge candidate list, the movement information of the merge candidate indicated by the merge index may be used as movement information for the target block.

[0204] Neighbor block: A neighbor block may refer to a block adjacent to the target block. Neighbor blocks may include spatial and temporal neighbor blocks. A neighbor block may also refer to a reconstructed neighbor block within the reference image. A neighbor block does not necessarily have to be in direct contact with the target block.

[0205] Spatial neighbor blocks: Spatial neighbor blocks can be blocks that are spatially adjacent to the target block.

[0206] - The target block and spatial neighbor blocks can be included within the target image.

[0207] - Spatial neighbor blocks may include blocks whose boundaries, at least a portion of which abuts at least a portion of the target block's boundary. Alternatively, spatial neighbor blocks may include blocks whose distance from the target block is less than or equal to a specific value.

[0208] - Spatial neighbor blocks may include blocks diagonally adjacent to the vertices of the target block.

[0209] - Spatial neighbor blocks may include a top-left block adjacent to the top-left of the target block, a top block adjacent to the top of the target block, a top-right block entered at the top-right of the target block, a left block adjacent to the left of the target block, a right block adjacent to the right of the target block, a bottom-left block adjacent to the bottom of the target block, and a bottom-right block adjacent to the bottom-right of the target block.

[0210] Temporal neighbor blocks: Temporal neighbor blocks can be blocks that are temporally adjacent to the target block.

[0211] - Temporal neighbor blocks may include a collocated block (COL block). A collocated block may be a block within a restored image in a reference image buffer. A collocated picture (col picture) may refer to an image containing a collocated block. A collocated picture may be an image included in a reference image list.

[0212] - Call blocks can be determined based on the location of target blocks within the target image. Two blocks being 'temporarily adjacent' may mean that the locations of the two blocks satisfy certain conditions.

[0213] - The position of the call block within the call image may be the same as the position of the target block within the target image. Alternatively, the position of the call block within the call image may correspond to the position of the target block within the target image. Here, the correspondence of the block positions may mean that the regions of the blocks are identical, that the region of one block is included within the region of another block, or that one block occupies a specific location within another block.

[0214] - For example, the location of a call block within a call image may be the same as the location of a target block within a target image. Alternatively, the call block may be a block containing call samples within a call image. A call sample may be a sample having coordinates identical to the coordinates of a specific sample in the target block.

[0215] - Temporal neighbor blocks may be blocks that are temporally adjacent to the spatial neighbor blocks of the target block.

[0216] Neighbor sample: A neighbor sample may refer to a sample within a neighbor block. Neighbor samples may include prediction samples, reconstructed samples, residual samples, and decoding samples.

[0217] Search range: The search range may refer to a two-dimensional area where a search for an MV is performed during inter-prediction. For example, when an optimal MV needs to be derived for processing a target block, the optimal MV can be selected from among the MVs pointing inside the search range.

[0218] Transform coefficient: The transform coefficient may be a coefficient generated by performing a transformation on the residual block. Alternatively, the transform coefficient may be a coefficient value generated by performing inverse quantization on the quantized level.

[0219] Quantized level: A quantized level can be an integer quantity used as an input for inverse quantization.

[0220] Quantization: Quantization can be a process that generates quantized levels for transform coefficients. Quantized levels can be generated by applying quantization to transform coefficients. Transformation can also be considered as part of quantization.

[0221] Inverse Quantization: Inverse quantization can be a process of multiplying a quantized level by a factor. By applying inverse quantization to the quantized level, (restored) transformation coefficients can be generated.

[0222] Quantization Parameter (QP): QP may refer to the argument used to generate quantized levels for transform coefficients in quantization. Additionally, QP may refer to the argument used to generate (restored) transform coefficients for quantized levels in inverse quantization. Alternatively, QP may be a value mapped to the quantization step size.

[0223] Delta QP: Delta QP can be the difference between the QP predicted by a specific process and the QP of the target block. In other words, the QP of the target block can be the sum of the predicted QP and Delta QP.

[0224] Quantization matrix: A quantization matrix may be a matrix used in quantization or inverse quantization to improve the subjective or objective image quality.

[0225] Quantization matrix coefficients: Quantization matrix coefficients can be each element within the quantization matrix.

[0226] Scan: Scan can refer to a method of arranging values ​​within a block or matrix. The values ​​can be coefficients. For example, a scan can mean arranging values ​​arranged in a 2D form into a 1D form, or rearranging values ​​arranged in a 1D form into a 2D form. An inverse scan can be the opposite arrangement (or rearrangement) of the arrangement performed in a scan.

[0227] Non-zero transformation coefficients: Non-zero transformation coefficients may refer to transformation coefficients that have a non-zero value or quantized levels that have a non-zero value.

[0228] Bitstream: A bitstream may refer to a series or sequence of bits containing encoded information generated by encoding of an image. A bitstream may contain information according to specific syntax elements. For example, the information may include syntax elements. An encoding device may generate a bitstream containing information according to specific syntax elements. A decoder may obtain information from the bitstream according to specific syntax elements.

[0229] Signaling: Signaling of information may indicate that information is transmitted from an encoding device to a decoding device via a bitstream. For example, the information may include syntactic elements. Alternatively, signaling may mean that the encoding device includes information within the bitstream. Information signaled by the encoding device may be used by the decoding device. In signaling, the bitstream may be transmitted over a network and may be contained within a recording medium. In embodiments, the description that information is signaled may include: 1) the encoding device determining and generating information for signaling of information; 2) the encoding device performing encoding on the information to generate encoded information; 3) the (encoded) information being transmitted from the encoding device to the decoding device via a bitstream; 4) the decoding device performing decoding on the encoded information to obtain information; and 5) the decoding device determining and generating information through signaling of information.

[0230] - An encoding device can generate encoded information by performing encoding on the information. The encoded information can be signaled through a bitstream. A decoding device can obtain information by performing decoding on the encoded information.

[0231] - The fact that information is signaled to a specific target may mean that the information is used for each specific target, and that the processing represented by the information is applied to each specific target. For example, the fact that information is signaled at a specific unit level may indicate that the information is used or processed for each specific unit.

[0232] - The signaled information may include one or more sub-information. That specific information is signaled may mean that each piece of information of the one or more sub-information included in the specific information is signaled.

[0233] Optional Signaling: Signaling for information may be performed optionally. Optional signaling for information may mean that an encoding device optionally includes information within a bitstream (depending on specific conditions). Optional signaling for information may mean that a decoder optionally obtains information from a bitstream (depending on specific conditions).

[0234] Omission of Signaling: Signaling for information may be omitted. Omission of signaling for information may mean that the encoding device does not include information in the bitstream (depending on specific conditions). Omission of signaling for information may mean that the decoding device does not obtain information from the bitstream (depending on specific conditions). The decoding device may derive information with omitted signaling using other information of the embodiments.

[0235] Symbol: May represent at least one piece of information of a target unit, such as syntactic elements, coding parameters, quantized levels, and transform coefficients of a target unit or target block. Additionally, the symbol may represent the target of entropy encoding or the result of entropy decoding.

[0236] Entropy encoding: Entropy encoding can allocate a small number of bits to symbols with a high probability of occurrence and a large number of bits to symbols with a low probability of occurrence. Through this allocation, the size of the bitstream representing the symbols can be reduced.

[0237] Entropy coding can utilize methods such as Variable Length Coding (VLC) and Context-Adaptive Binary Arithmetic Coding (CABAC). For example, in Variable Length Coding, entropy coding can be performed using variable-length tables. For instance, in CABAC, a binaryization method for symbols and a probabilistic model of symbols / bins can be derived for entropy coding, and context-based arithmetic coding can be performed.

[0238] Entropy Decoding: In entropy decoding, the processes performed in entropy encoding can be performed in reverse. Symbols can be generated by entropy decoding of a bitstream.

[0239] Parsing: Parsing can refer to determining the values ​​of syntactic elements by performing entropy decoding on the encoded information of a bitstream. Alternatively, parsing can refer to entropy decoding itself.

[0240] Statistical Value: The values ​​of information related to specific entity(s) described in the embodiments may be used as inputs for specific operations. The statistical value may be a value derived by a specific operation on the values ​​related to these specific entity(s). For example, the statistical value for specific information may be one or more of the following: an average value, a weighted average value (weighted average), a weighted sum (weighted sum), a minimum value, a maximum value, a mode, a median value, an interpolated value, a sum of products, and a product of sums. Additionally, information of the embodiment having specific values ​​determined by operations, such as constants, variables, and coding parameters, may have a specific statistical value according to the embodiment.

[0241]

[0242] Coding parameters

[0243] In the embodiments, the coding parameters may be information required for coding. The coding parameters may include information signaled from an encoding device to a decoder, information calculated / derived during the processing of coding described in the embodiments, and information used for the processing of coding described in the embodiments.

[0244] In the embodiments, the coding parameters include the size of the CTU, the size of the unit, the form of the unit, the shape of the unit, the depth of the unit, the minimum unit size, the maximum unit size, the maximum unit depth, the minimum unit depth, the unit splitting information, QT splitting information, BT splitting information, the splitting direction of the BT splitting, the splitting form of the BT splitting, TT splitting information, the splitting direction of the TT splitting, the splitting form of the TT splitting, MTT splitting information, the combination of MTT splittings, the splitting direction of the MTT splitting, the splitting form of the MTT splitting, the prediction mode, the intra prediction mode, the luminance intra prediction mode, the chroma intra prediction mode, the intra prediction mode, the inter splitting information, the coding block splitting information, the prediction block splitting information, the transformation block splitting information, the reference sample line index, the reference sample filtering method, the reference sample filter tab, the reference sample filter coefficients, the prediction block filtering method, the prediction block filter tab, the prediction block filter coefficients, the prediction block boundary filtering method, the prediction block boundary filter tab, the prediction block boundary filter coefficients, the inter prediction mode, motion information, MV, and Motion Vector Difference; MVD), MVD resolution, MV size, MV representation accuracy, reference image list, reference image, reference image index, inter prediction direction, inter prediction indicator, prediction list utilization flag, POC, MV candidate, MV candidate index, MV candidate list, AMVP mode usage information, merge candidate, merge index, merge candidate list, merge mode usage information, motion information correction information, skip mode usage information, intra-block copy mode usage information, BV (Block Vector), Block Vector Difference (BVD), BVD resolution, BV size, BV representation accuracy, BV candidate, BV candidate index, BV candidate list, interpolation filter filter tab, interpolation filter filter coefficients, transform type, transform size, transform selection information, primary transform usage information,Secondary transform usage information, primary transform selection information, secondary transform selection information, residual block presence information, coded block pattern, coded block flag, QP, delta QP, quantization matrix, deblocking filter usage information, deblocking filter coefficients, deblocking filter filter tab, deblocking filter strength, deblocking filter shape / form, adaptive sample offset usage information, adaptive sample offset value, adaptive sample offset category, adaptive sample offset type, adaptive loop filter usage information, adaptive loop filter coefficients, adaptive loop filter filter tab, adaptive loop filter shape / form, binarization / debinarization method, context model, context model determination method, context model update method, regular mode usage information, bypass mode usage information, significant coefficient flag, last significant coefficient flag, coefficient group coding flag, last significant coefficient position, flag indicating whether the coefficient value is greater than 1, whether the coefficient value is greater than 2 Flag indicating presence, flag indicating whether the coefficient value is greater than 3, remaining coefficient value information, sign information, context bin, bypass bin, restored sample, restored luminance sample, restored chroma sample, residual sample, residual luminance sample, residual chroma sample, transform coefficient, luminance transform coefficient, chroma transform coefficient, transform coefficient level, luminance transform coefficient level, chroma transform coefficient level, transform coefficient level scanning method, quantized level, luminance quantized level, chroma quantized level, size of the MV seek area on the decoder side, shape of the MV seek area on the decoder side, number of MV seeks on the decoder side, picture type, slice identification information, slice type, slice splitting information, tile group identification information, tile group type, tile group splitting information, tile identification information, tile type, tile splitting information, bit depth,It may include one or more of input sample bit depth, restored sample bit depth, residual sample bit depth, transform factor bit depth, quantized level bit depth, mapping availability information, information about the luminance signal, information about the chroma signal, the color space of the target block, the color space of the residual block, and temporal layer information.

[0245] In addition, the coding parameter may further include 1) a value of information that may be included in the coding parameter, 2) a combination of multiple pieces of information that may be included in the coding parameter, 3) a statistical value of information that may be included in the coding parameter, 4) information related to the coding parameter, 5) information used to calculate / derive the coding parameter, and 6) information calculated / derived using the coding parameter.

[0246] In the embodiments, "X usage information" may be "information indicating whether X is used / applied / executed." Alternatively, "X usage information" may be "information indicating whether X is available." For example, "specific mode usage information" may be information indicating whether a specific mode is used. Mode information may indicate a mode used for a target block among the modes described in the embodiments. In the embodiments, specific mode usage information may be replaced with mode information, and the description of specific mode usage information may also apply to mode information. "X usage information" and "X indicator" may be used interchangeably.

[0247] In the embodiments, coding parameters and syntax elements may correspond to each other. For example, a syntax element of the embodiment may be used as a coding parameter, and a coding parameter may be signaled as a syntax element.

[0248] In the embodiments, "X existence information" may be considered as "information indicating whether X exists" or "information indicating whether information indicating X exists within the bitstream".

[0249] In the embodiments, "X selection information" may be information indicating one of the candidates or methods for X. "X selection information" may be considered as an "X index".

[0250] In the embodiments, the splitting form of a specific tree may represent one of symmetric splitting and asymmetric splitting, and may represent one of QT, BT, TT, and non-split. The splitting direction of a specific tree may represent one of horizontal direction and vertical direction.

[0251] In the embodiments, when the coding parameter has one of a plurality of values, "coding parameter" may be replaced with "whether the coding parameter has a specific value among the plurality of values ​​available to the coding parameter".

[0252] In the embodiments, when the coding parameter refers to one of a plurality of targets, the "coding parameter" may be replaced with "whether the coding parameter refers to a specific target among the plurality of targets."

[0253] In the embodiments, the coding parameter may include at least one of the type of target picture and the type of target slice. The type of target picture may be one of an I-picture, a B-picture, and a P-picture. The type of target slice may be one of an I-slice, a B-slice, and a P-slice.

[0254] - If the target image to be encoded is an I-slice, the target image can be encoded using data within the image itself without inter-predicting that references other images. For example, an I-slice can be encoded using only intra-predicting.

[0255] - If the target image is a P slice, the target image can be encoded through inter-prediction using only the reference slice existing in a unidirectional direction. Here, the unidirectional direction can be forward or reverse.

[0256] - If the target image is a B slice, the target image can be encoded through inter-prediction using reference slices existing in both directions or through inter-prediction using a reference slice existing in one of the forward and backward directions. Here, both directions can be the forward and backward directions.

[0257] - P slices and B slices encoded and / or decoded using a reference slice can be considered as images where inter-prediction is used.

[0258]

[0259] System for video coding

[0260] FIG. 1 shows a system for video coding according to one embodiment.

[0261] The system (100) may include at least one of an encoding device (110) and a decoding device (150).

[0262] Each of the encoding device (110) and the decoding device (150) may be a computer or an electronic apparatus.

[0263]

[0264] Structure of the encoding device

[0265] The encoding device (110) may include a processor (120), a storage (140), and a communicator (149).

[0266] The processor (120), storage (140), and communication device (149) can be connected via a bus.

[0267] The processor (120) may be a semiconductor device that executes instructions or computer-executable code, such as a Central Processing Unit (CPU). The processor (120) may be at least one hardware processor.

[0268] The processor (120) can perform generation and processing of information that is input to the encoding device (110) in the embodiments, output from the encoding device (110), or used inside the encoding device (110), and can perform comparison and judgment related to such information.

[0269] The processor (120) may include a plurality of components. The plurality of components may include a partitioner (122), a subtractor (124), a transformer (125), a quantizer (126), an inverse quantizer (127), an inverse transformer (128), an adder (129), a filter (130), and an entropy encoder (139).

[0270] At least some of the aforementioned multiple components may be program modules. Program modules may be included in the encoding device (110) in the form of an operating system, an application, and other program modules. Program modules may be instructions or computer-executable code stored in a storage (140) and executed by a processor (120).

[0271] The storage (140) may include various types of volatile storage media and non-volatile storage media. For example, the storage (140) may include memory such as ROM and RAM.

[0272] The storage (140) can store instructions and computer-executable code used for the operation of the encoding device (110), and can store information and bitstreams as described in the embodiments. The storage (140) may include a reference picture buffer (141).

[0273] The communication device (149) can perform functions related to the communication of information in the encoding device (110). For example, the communication device (149) can transmit a bitstream to the decoding device (150).

[0274] Among the names of the components of the encoding device (110), "-gi" ("-er" or "-or") may be replaced with "-bu" (- unit). The storage unit (140) may also be named a storage unit.

[0275]

[0276] Operation of the encoding device

[0277] The encoding device (110) can sequentially encode one or more images of the video.

[0278] The storage (140) can store the original image. In the encoding device (110), the original image can be used as the target image.

[0279] The processor (120) can generate a bitstream containing encoded information by performing encoding on the target image and can store the generated bitstream in a storage (140). The generated bitstream can be stored on a computer-readable recording medium and can be transmitted by the communication device (149) to the communication device (189) of the decoding device (150) via a wired and / or wireless transmission medium.

[0280] The splitter (122) can determine the target block by performing a split on the target image.

[0281] The predictor (123) can determine the prediction mode of the target block. The predictor (123) can generate a prediction block of the target block by performing a prediction according to the prediction mode.

[0282] The prediction mode of the target block may be one of the available prediction modes. For example, available prediction modes may include intra prediction, inter prediction, and IBC prediction.

[0283] For example, if the prediction mode is intra prediction, the predictor (123) can perform intra prediction on the target block to generate a prediction block of the target block.

[0284] For example, if the prediction mode is inter-prediction, the predictor (123) can perform inter-prediction on the target block to generate a prediction block of the target block.

[0285] For example, if the prediction mode is IBC, the predictor (123) can perform an IBC prediction for the target block to generate a prediction block of the target block.

[0286] The subtractor (124) can generate a residual block of the target block. The residual block may be the difference between the original block and the prediction block. The original block may be the region of the original image pointed to by the target block. Alternatively, the residual block may refer to a block generated by applying one or more of transformation and quantization to the difference between the original block and the prediction block.

[0287] The converter (125) can perform a conversion on the residual block to generate conversion coefficients.

[0288] The converter (125) can perform the conversion using one of a plurality of conversion methods.

[0289] For example, multiple transformation methods may include the Discrete Cosine Transform (DCT), Discrete Sine Transform (DST), Karhunen-Loeve Transform (KLT), and transformations based on each transformation.

[0290] The transform skip mode may be a mode that generates a restored block using the restored residual block and prediction block, for which transform and inverse transform have not been performed. When the transform skip mode is applied to a target block, the transform and inverse transform for the target block may be omitted, and only quantization and inverse quantization for the target block may be performed.

[0291] The quantizer (126) can generate quantized levels by applying quantization using quantization parameters to the conversion coefficients. In the embodiments, the quantized levels may also be referred to as conversion coefficients.

[0292] The entropy encoder (139) can generate encoded information by performing entropy encoding based on a probability distribution on information for decoding an image. The bitstream may contain encoded information.

[0293] Information for decoding the image may include quantized levels and syntax elements produced by the quantizer (126).

[0294] The probability distribution can be determined based on quantized levels and coding parameters.

[0295] The entropy encoder (139) can convert quantized levels, which have the form of a two-dimensional block, into the form of a one-dimensional vector by using scanning to perform encoding for the quantized levels. In the scanning, it can be determined which scan to use among an upper-right diagonal scan, a vertical scan, and a horizontal scan based on coding parameters such as the size of the block and the intra-prediction mode of the block.

[0296] When encoding is performed on a target image / block, the predictor (123) uses a reference image / block for prediction. The encoded target image / block can be used as a reference image / block for other images / blocks that are subsequently processed. Accordingly, the processor (120) can perform restoration on the encoded target block and can store the restored image containing the restored target block generated by the restoration in the reference picture buffer (141) as a reference image. Inverse quantization and inverse transform can be performed on the encoded target block for restoration.

[0297] The inverse quantizer (127) can generate inverse quantized conversion coefficients by performing inverse quantization on the quantized level.

[0298] The inverse converter (128) can generate inversely quantized and inversely converted coefficients by performing an inverse conversion on the inversely quantized conversion coefficients. In embodiments, the inversely quantized and / or inversely converted coefficients may refer to coefficients to which at least one of the inverse quantization and inverse conversion has been applied. The inversely quantized and inversely converted coefficients may be restored residual blocks.

[0299] The adder (129) can generate a recovery block by combining the prediction block and the recovered residual block.

[0300] The restoration block may pass through a filter (130). The filter (130) may apply one or more of a plurality of filters to the target. Each of the plurality of filters may be an in-loop filter. The target may be a restoration sample, a restoration block, or a restoration image.

[0301] The reference picture buffer (141) can store a restoration block / image provided from the filter (130). The restoration image may be an image containing the restoration block. Alternatively, the restoration image may be an image composed of restoration blocks.

[0302] The reference picture buffer (141) can provide the stored restored image to the predictor (123) as a reference image. In terms of storing the decoded (i.e., restored) picture, the reference picture buffer (141) may also be referred to as the Decoded Picture Buffer (DPB).

[0303]

[0304] Structure of the decoding device

[0305] The decoding device (150) may include a processor (160), a storage device (180), and a communication device (189).

[0306] The description of the processor (120), storage (140), and communication device (149) associated with the encoding device (110) may also apply to the processor (160), storage (180), and communication device (189) associated with the decoding device (150). Redundant descriptions are omitted.

[0307] The processor (160) may include a plurality of components. The plurality of components may include an entropy decoder (161), a splitter (162), a predictor (163), an inverse quantizer (167), an inverse converter (168), an adder (169), and a filter (170).

[0308] The storage (180) may include a reference picture buffer (181).

[0309] The communicator (189) can perform functions related to the communication of information in the decoding device (150). For example, the communicator (189) can receive a bitstream from the encoding device (110).

[0310] Among the names of the components of the decoding device (150), "-gi" ("-er" or "-or") may be replaced with "-bu" (- unit). The storage unit (180) may also be named a storage unit.

[0311]

[0312] Operation of the decoding device

[0313] The communication device (149) of the encoding device (110) can transmit the bitstream generated by the encoding device (100) to the decoding device (150). Alternatively, a computer-readable recording medium storing the bitstream can transmit the bitstream generated by the encoding device (100) to the decoding device (150).

[0314] The communication device (189) can receive a bitstream from the encoding device (110) via a wired and / or wireless transmission medium. The received bitstream can be stored in a storage device (180).

[0315] The processor (160) can obtain a bitstream from a storage (180) or a computer-readable recording medium.

[0316] A bitstream can contain encoded information.

[0317] The entropy decoder (161) can generate information for decoding an image by performing entropy decoding based on a probability distribution on the encoded information of the bitstream.

[0318] Information for decoding an image may include quantized levels and syntax elements, etc.

[0319] The entropy decoder (161) can convert quantized levels, which have the form of a one-dimensional vector, into the form of a two-dimensional block by using scanning to perform decoding on the quantized levels. In the scanning, it can be determined which scan to use among an upper-right diagonal scan, a vertical scan, and a horizontal scan based on coding parameters such as the size of the block and the intra-prediction mode of the block.

[0320] The entropy decoder (161) can provide syntax elements to other components of the processor (160), such as the splitter (162).

[0321]

[0322] Common explanation based on the relationship between the components of the encoding device and the components of the decoding device

[0323] The decoding device (150) performs decoding using the bitstream generated by the encoding device (110). The encoding device (110) may perform encoding for the target block using a restored image derived within the decoding device (150), rather than an original image that is not provided to the decoding device (150). Accordingly, the encoding device (110) and the decoding device (150) may need to generate the restored block / image in the same way. In this regard, the descriptions of the divider (122), predictor (123), inverse quantizer (127), inverse converter (128), adder (129), filter (130), and reference picture buffer (141) of the encoding device (110) disclosed in the embodiments may also be applied to the divider (162), predictor (163), inverse quantizer (167), inverse converter (168), adder (169), filter (170), and reference picture buffer (181) of the decoding device (150), respectively. Redundant descriptions are omitted.

[0324] Additionally, each of the divider (122), predictor (123), inverse quantizer (127), inverse converter (128), adder (129), and filter (130) of the encoding device (110) can generate syntactic element information that specifies processing for a target. Each of the divider (162), predictor (163), inverse quantizer (167), inverse converter (168), adder (169), and filter (170) of the decoding device (150) can perform processing for a target (such as that performed in the encoding device (110)) using the syntactic element information.

[0325] As described above, corresponding components of the encoding device (110) and the decoding device (150) may perform the same or corresponding functions. In embodiments, the processor may represent the processor (120) of the encoding device (110) and / or the processor (160) of the decoding device (150). For example, regarding the function of prediction, the processor may represent a predictor (123), a subtractor (124), and an adder (129), and may represent a predictor (163) and an adder (169). Regarding the function of conversion, the processor may represent a converter (125) and an inverse converter (128), and may represent an inverse converter (168). Regarding the function of quantization, the processor may represent a quantizer (126) and an inverse quantizer (127), and may represent an inverse quantizer (167). In terms of functions related to entropy encoding / decoding, the processing unit may represent an entropy encoder (139) and / or an entropy decoder (161). In terms of functions related to filtering, the processing unit may represent a filter (130) and / or a filter (170). The storage unit may represent a storage unit (140) of the encoding device (110) and / or a storage unit (180) of the decoding device (150). The reference picture buffer may represent a reference picture buffer (141) of the encoding device (110) and / or a reference picture buffer (181) of the decoding device (150). The communication unit may represent a communication unit (149) of the encoding device (110) and / or a communication unit (189) of the decoding device (150).

[0326]

[0327] Partitioning of the units that constitute the image

[0328] Figure 2 shows a segmentation structure of an image according to one embodiment.

[0329] Figure 2 schematically illustrates an example in which a single unit is divided into multiple sub-units.

[0330] CU can be used as a base unit for encoding and decoding of images. Additionally, CU can be a base unit for prediction, transformation, quantization, inverse quantization, inverse transformation, entropy encoding, and entropy decoding.

[0331] A CU can be used as a unit to which a prediction mode is applied. That is to say, in coding, it can be determined which of the available prediction modes will be applied to each CU. For example, available prediction modes may include intra prediction, inter prediction, and IBC intra block copy prediction.

[0332] The target image (200) can be sequentially divided into units of CTUs. A division structure can be determined for each CTU. The CTU can be divided into CUs according to the division structure. Alternatively, one CTU can be used as a CU. The size of the CTU can be the maximum size of the CU.

[0333] Each CU may have depth information. The depth information may represent the depth of the CU and the size of the CU. The depth of the CTU may be 0. The depth of the CU created by dividing the CTU may be 1. When a parent CU is divided into child CUs, the depth of the child CU may be 1 greater than the depth of the parent CU. The number of divided CUs may be a positive integer greater than or equal to 2, including 2, 4, 8, and 16. At least one of the width and height of the child CU created by dividing the parent CU may be smaller than at least one of the width and height of the parent CU, depending on the number of child CUs.

[0334] A partitioned CU can be recursively partitioned in the same way up to a predefined maximum depth or a predefined minimum size. The depth of a Smallest Coding Unit (SCU) can be the predefined maximum depth, and the size of an SCU can be the predefined minimum size. The size of an SCU can be the minimum CU size.

[0335] For example, the depth range of a CU can be values ​​from 0 to 3. Depending on the depth of the CU, the CU can have a size from 64x64 to 8x8. A CTU with a depth of 0 can be 64x64 blocks. 0 can be the minimum depth. An SCU with a depth of 3 can be 8x8 blocks. 3 can be the maximum depth. Depth 0 can represent a CTU that is 64x64 blocks. Depth 1 can represent a CU that is 32x32 blocks. Depth 2 can represent a CU that is 16x16 blocks. Depth 3 can represent an SCU that is 8x8 blocks.

[0336] The partition information of a CU may indicate whether the CU is partitioned. The partition information may be a 1-bit flag. All CUs except the SCU may include partition information. For example, the partition information of a CU that is not further partitioned may be a first value of '0', and the partition information of a CU that is partitioned may be a second value of '1'.

[0337] Quad Tree (QT) partitioning can mean that a single CU is partitioned into four CUs. When a parent CU is partitioned into four child CUs, the width and height of each child CU can be half the width and half the height of the parent CU, respectively.

[0338] A binary tree (BT) partition can mean that one CU is divided into two CUs. For example, if a parent CU is divided into two child CUs, the width or height of each child CU can be half the width or half the height of the parent CU.

[0339] Ternary tree (TT) partitioning can mean that a single CU is divided into three CUs. For example, when a parent CU is divided into three child CUs, the three child CUs can be created by dividing the width or height of the parent CU in a ratio of 1:2:1. The width or height of the child CUs can be 1 / 4, 1 / 2, and 1 / 4 of the width or height of the parent CU, respectively.

[0340] In FIG. 2, QT-type splitting was applied to the first CTU. QT splitting, BT splitting, and TT splitting were applied to the second CTU.

[0341] To split a CTU, at least one of different types of splits, such as QT splitting, BT splitting, and TT splitting, may be applied to the CTU. Different types of splits may be applied based on specific priorities.

[0342] For example, QT splitting may be applied preferentially to a CTU. A CU to which QT splitting can no longer be applied may correspond to a leaf node of QT. A CU that is a leaf node of QT may become a root node of BT and / or TT. A CU that is a leaf node of QT may be split into a BT form or a TT form, or may not be split further. In this case, QT splitting may not be applied again to a CU created by applying BT splitting or TT splitting to a CU that is a leaf node of QT.

[0343] The splitting of a CU corresponding to each node of QT can be signaled using QT splitting information. The QT splitting information may be a flag. The QT splitting information of a unit may be information indicating whether the unit is split into a QT form. A first value of the QT splitting information, '0', may indicate that the CU is not split into a QT form. QT splitting information having a first value may signify a Multi-Type Tree (MTT) split. MTT splitting may include BT splitting and TT splitting. A second value of the QT splitting information, '1', may indicate that the CU is split into a QT form.

[0344] There may be no priority between BT splitting and TT splitting. That is, CUs corresponding to the leaf nodes of QT can be split into BT form or TT form. Additionally, CUs generated by BT splitting or TT splitting can be split again into BT form or TT form, or they may not be split any further.

[0345] A CU corresponding to a leaf node of QT can be a root node of MTT. For a CU corresponding to each node of MTT, the CU may further include partition direction information and partition type information in the form of MTT.

[0346] The splitting direction information can indicate the splitting direction of the MTT split. The first value of the splitting direction information, '0', can indicate that the CU is split in the horizontal direction. The second value of the splitting direction information, '1', can indicate that the CU is split in the vertical direction.

[0347] The split type information may indicate the split type used for multi-type tree splitting. The first value of the split type information, '0', may indicate that CU is split into TT form. The second value of the split type information, '1', may indicate that CU is split into BT form.

[0348] Here, each of the aforementioned division direction information and division shape information may be a flag having a specified length (e.g., 1 bit).

[0349] The partitioning information of CU may also include QT partitioning information, partitioning direction information, and partitioning shape information.

[0350] A CU that is no longer divided by QT division, BT division, and TT division can be used as a unit for specific processing such as prediction, transformation, quantization, inverse quantization, inverse transformation, entropy encoding, and entropy decoding. That is, for a specific processing, the CU may no longer be divided. Therefore, division information for dividing such a CU into PU and / or TU, etc., may not exist within the bitstream.

[0351] On the other hand, if the size of a CU is larger than the maximum TU size, such a CU can be recursively partitioned until the size of the CU becomes less than or equal to the maximum TU size. For example, if the size of the CU is 64x64 and the maximum TU size is 32x32, the CU can be partitioned into 4 32x32 TUs for transformation. For example, if the size of the CU is 32x64 and the maximum TU size is 32x32, the CU can be partitioned into 2 32x32 TUs for transformation.

[0352] In such cases, information regarding whether the CU is split for transformation may not be signaled separately. Whether the CU is split may be determined without signaling by comparing the size of the CU (width / height) and the maximum TU size (width / height). For example, if the width of the CU is greater than the width of the maximum TU size, the CU may be split vertically into two. Additionally, if the height of the CU is greater than the height of the maximum TU size, the CU may be split horizontally into two.

[0353] For example, the minimum size of a CU can be 4x4. For example, the maximum size of a transformation block can be 64x64. For example, the minimum size of a transformation block can be 4x4. The minimum size of QT can be the minimum size of a CU corresponding to a leaf node of QT. The maximum depth of MTT can be the maximum depth of a path from the root node of MTT to a leaf node.

[0354] The BT maximum size may represent the maximum size of the CU corresponding to each node of the BT, and the TT maximum size may represent the maximum size of the CU corresponding to each node of the TT. The BT minimum size and / or the TT minimum size may be set as the minimum size of the CU.

[0355] If the depth of a CU within the MTT corresponding to a node of the MTT is equal to the maximum depth of the MTT, the CU may not be divided into BT form and / or TT form.

[0356] Based on the various sizes and depths of the aforementioned CU, each piece of information described in the embodiments may or may not be present in the bitstream.

[0357] Information regarding the maximum or minimum size described in the embodiments may be signaled at the upper level of the CU. In the embodiments, the upper level of the CU may include a video level, a sequence level, a picture level, a subpicture level, a tile group level, a tile level, and a slice level, etc.

[0358] The information described in the embodiments may be signaled separately for different types of slices. Different types of slices may include intra-slices and inter-slices.

[0359]

[0360] Processing of blocks based on block attributes

[0361] Whether a specific process described in the embodiments is applied or performed may be determined based on the attributes of the block associated with the specific process. Whether a specific process described in the embodiments is applied or performed may be determined based on whether the attributes of the block associated with the specific process satisfy specific conditions. For example, a block may include a target block, a neighbor block, and a reference block. A block may include other blocks described in the embodiments. A block may be one of the blocks and units described in the embodiments.

[0362] The block to which the specific treatment described in the embodiments is applied may have a square shape or a non-square shape.

[0363] In one embodiment, the attributes of the block may include the size of the block. The specific processing described in the embodiments may be applied / performed when specific conditions regarding the size of the block are met.

[0364] In one embodiment, specific conditions may include a minimum block size condition and a maximum block size condition. The block to which the minimum block size condition applies and the block to which the maximum block size condition applies may be different from each other.

[0365] In one embodiment, the minimum block size and / or maximum block size for a specific process may be predefined.

[0366] In one embodiment, the processing of the embodiment may be applied / performed when the block size is greater than or equal to the minimum block size and / or less than or equal to the maximum block size. Alternatively, in one embodiment, the processing of the embodiment may be applied / performed when the block size is greater than the minimum block size and / or less than the maximum block size.

[0367] In one embodiment, the processing of the embodiment may be applied / performed only when the block size is greater than or equal to the minimum block size and less than or equal to the maximum block size. Alternatively, the processing of the embodiment may be applied / performed only when the block size is greater than the minimum block size and less than or equal to the maximum block size. Alternatively, the processing of the embodiment may be applied / performed only when the block size is greater than the minimum block size and less than the maximum block size. The processing of the embodiment may be applied / performed only when the block size is greater than the minimum block size and less than the maximum block size.

[0368] In one embodiment, the processing of the embodiment may be applied / performed only when the block size is a predefined block size.

[0369] In the embodiments, the size of the block may be determined by various methods. For example, the size of the block may mean the width of the block or the height of the block. The size of the block may mean both the width and the height of the block. The size of the block may mean the area of ​​the block. The size of the block may mean 1) the result of a known formula using the width and height of the block, 2) the result of a formula of the embodiment, or 3) a statistical value.

[0370] Additionally, for the first size, the treatment of the first embodiment among the embodiments may be applied / performed, and for the second size, the treatment of the second embodiment among the embodiments may be applied / performed.

[0371] In the embodiments, the block size may be 2x2, 4x4, 8x8, 16x16, 32x32, 64x64, or 128x128, etc. Or, in the embodiments, the block size is (2*SIZE X )x(2*SIZE Y It may be ) etc. SIZE X is one of integers greater than or equal to 1. SIZE Y can be one of integers greater than or equal to 1.

[0372]

[0373] Predictive information for prediction

[0374] Predictive information can be used to generate a predicted block for a target block.

[0375] The encoding device (110) can generate prediction information required for prediction and can generate a bitstream containing the prediction information. The prediction information can be signaled from the encoding device (110) to the decoding device (150) through the bitstream. The decoding device (150) can obtain the prediction information from the bitstream and can generate a prediction block by performing a prediction on a target block using the prediction information.

[0376] Prediction information may include intra prediction information, inter prediction information, and IBC prediction information. In the embodiments, prediction information may be replaced with intra prediction information, inter prediction information, and / or IBC information. Intra prediction information may include information used for intra prediction as described in the embodiments. Inter prediction information may include information used for inter prediction as described in the embodiments. IBC information may include information used for IBC prediction as described in the embodiments.

[0377]

[0378] Intra prediction

[0379] Figure 3 shows the structure of an intra prediction according to one embodiment.

[0380] Intra-prediction can be performed using reference samples and coding parameters of the target block. The reference sample may be a (restored) sample within the (restored) reference block. Alternatively, an intermediate prediction sample may be generated using a sample described in an embodiment, such as the restored sample, and a reference sample may be generated again using the intermediate prediction sample. Processing described in an embodiment, such as filtering, may be applied when generating the reference sample.

[0381] The reference block may be a (spatial) neighbor block of the target block. The coding parameter may be a coding parameter for the target block and / or a coding parameter for the reference block. In intra-prediction, the reference sample may refer to a neighbor sample.

[0382] A prediction block can be generated by performing intra prediction on a target block according to an intra prediction mode, based on a reference sample within the target image and information related to the reference sample. The size of the target block and the size of the prediction block may be the same.

[0383] In the embodiments, the prediction block may be a PU. Alternatively, the prediction block may correspond to the CU or TU described in the embodiments. The prediction block may have a square or rectangular shape.

[0384] An intra prediction mode can be represented by at least one of a mode number, a mode value, a mode angle, and a mode direction. The prediction directions of a plurality of intra prediction modes for a target block are illustrated in the lower right corner of FIG. 3. Among the plurality of intra prediction modes, the remaining intra prediction modes, excluding DC and planar modes, may be directional modes. A directional mode may be an intra prediction mode having a specific direction or a specific angle. An intra prediction mode for a target block may be selected from directional modes and non-directional modes.

[0385] In the bottom-right rectangle representing the target block, the number '0' may represent Planner mode, which is a non-directional intra prediction mode. The number '1' may represent DC mode, which is a non-directional intra prediction mode. In the bottom-right rectangle representing the target block, arrows extending from the center of the rectangle outwards may represent the prediction directions of directional intra prediction modes. Additionally, the number displayed near the arrow may represent an example of a mode value assigned to an intra prediction mode or a prediction direction of an intra prediction mode.

[0386] Intra prediction can be performed according to the intra prediction mode for the target block. One of the intra prediction modes available for the target block can be used as the intra prediction mode for the target block.

[0387] The number of intra prediction modes available to the target block may be a predefined value. Alternatively, the number of intra prediction modes available to the target block may be determined based on the attributes of the prediction block. For example, the attributes of the prediction block may include coding parameters such as shape, size, and color components.

[0388] For example, in FIG. 3, the directional modes illustrated by dashed lines (i.e., directional modes with numbers from -14 to -1 or numbers from 67 to 80) can be applied only to predictions for non-square blocks. Therefore, the number of available intra-prediction modes for predictions for square blocks may be 67. (Planner mode, DC mode, and 65 directional modes)

[0389] For example, the number of available intra prediction modes may vary depending on whether the color component of a block is a luminance signal or a chroma signal. The number of available intra prediction modes for a block with a luminance component may be greater than the number of available intra prediction modes for a block with a chroma component.

[0390] Intra-prediction modes may include a horizontal-below mode, a horizontal mode, a vertical mode, and a vertical-right mode. The horizontal-below mode may be an intra-prediction mode located at the bottom of the horizontal mode. The vertical-right mode may be a mode located to the right of the vertical mode. For example, in FIG. 3, the mode value of the horizontal mode may be 18. The mode value of the vertical mode may be 50. Intra-prediction modes with a mode value of 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, and 66 may be vertical-right modes. Intra prediction modes with a mode value of 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16 and 17 may be horizontal bottom modes.

[0391] The number of the aforementioned intra-prediction modes and the mode number of each intra-prediction mode may be exemplary only. The number of the aforementioned intra-prediction modes and the mode number of each intra-prediction mode may be defined differently depending on the embodiment, implementation, and / or as necessary.

[0392] When the intra prediction mode is a planner mode, when generating a prediction block of a target block, the sample value of the prediction sample can be generated using a weighted sum (weighted sum) of the top reference sample of the target sample, the left reference sample of the target sample, the right top reference sample of the target block, and the left bottom reference sample of the target block, depending on the position of the prediction sample within the prediction block.

[0393] When the intra prediction mode is DC mode, a prediction block can be generated based on the average of the sample values ​​of multiple reference samples. The multiple reference samples may include top reference samples and left reference samples of the target block. The value of the prediction sample of the prediction block can be determined based on the average of the sample values ​​of the multiple reference samples. Additionally, filtering using the values ​​of the reference samples can be performed on specific rows and / or specific columns within the target block. The specific rows may be one or more top rows adjacent to the top reference samples. The specific columns may be one or more left columns adjacent to the left reference samples.

[0394] When the intra prediction mode is a directional mode, a prediction block can be generated using the top reference sample, left reference sample, right top reference sample, and / or left bottom reference sample of the target block.

[0395] The intra prediction mode of a target block can be determined based on the intra prediction mode of a neighboring block of the target block. Information for determining the intra prediction mode of a target block can be signaled.

[0396] For example, if the intra prediction modes of the target block and the neighbor block are the same, an indicator indicating that the intra prediction modes of the target block and the neighbor block are the same may be signaled.

[0397] For example, an indicator indicating an intra prediction mode such as the intra prediction mode of the target block among the intra prediction modes of multiple neighboring blocks may be signaled.

[0398] For example, if the intra prediction modes of the target block and neighboring blocks are different from each other, an indicator indicating the intra prediction mode of the target block may be signaled. Alternatively, information used to derive the intra prediction mode of the target block based on the intra prediction mode of the neighboring block may be signaled.

[0399] Reference samples used for intra-prediction of a target block may include bottom-left reference samples, left reference samples, top-left reference samples, top reference samples, and top-right reference samples, etc.

[0400] For example, left reference samples may be restored reference samples adjacent to the left side of the target block. Top reference samples may be restored reference samples adjacent to the top side of the target block. Top-left reference samples may be restored reference samples diagonally adjacent to the top-left side of the target block. Bottom-left reference samples may be reference samples located below the left reference samples among samples located on the same line as the left sample line composed of left reference samples. Top-right reference samples may be reference samples located to the right of the top reference samples among samples located on the same line as the top sample line composed of top reference samples.

[0401] Reference samples used for intra-prediction for a target block can be determined based on the intra-prediction mode of the target block. One or more reference samples may be used to determine the sample values ​​of the prediction samples of the prediction block. In FIG. 3, the direction of the intra-prediction mode indicated by the arrow may represent the direction from the prediction sample to the reference sample. The direction of the intra-prediction mode may represent the dependency relationship between the reference samples and the prediction samples. For example, depending on the intra-prediction mode, the sample value of a specific reference sample may be used as the sample value of at least one sample of the prediction block. Here, the specific reference sample and at least one sample of the prediction block may be samples designated by a straight line of the direction of the intra-prediction mode. That is to say, the sample value of the specific reference sample may be copied to the sample value of the prediction sample located in the reverse direction of the direction of the intra-prediction mode. Alternatively, the sample value of the prediction sample of the prediction block may be the sample value of the reference sample located in the direction of the intra-prediction mode relative to the location of the prediction sample.

[0402] Reference samples used for intra-prediction may not be limited to samples immediately adjacent to the target block. As illustrated in FIG. 3, at least one of reference sample line 0 to reference sample line 3 may be used for intra-prediction of the target block.

[0403] Each reference sample line in FIG. 3 may contain one or more reference samples. The smaller the number of the reference sample line, the closer the line of reference samples may be to the target block. Reference sample line 0 may be a line of reference samples immediately adjacent to the target block. When the top-left coordinates of the target block are (X, Y), the horizontal length is W, and the vertical length is H, the reference samples of reference sample line 0 may be samples with an x-coordinate of X-1 or a y-coordinate of Y-1. Here, the y-coordinates of the reference samples with an x-coordinate of X-1 may be Y-1 to Y+2H. The x-coordinates of the reference samples with a y-coordinate of Y-1 may be X-1 to X+2W. The reference samples of reference sample line A may be samples with an x-coordinate of XA-1 or a y-coordinate of YA-1. Here, the y-coordinates of the reference samples with an x-coordinate of XA-1 may be YA-1 to Y+2H+A. The x-coordinates of reference samples with y-coordinate YA-1 can be XA-1 to X+2W+A. A can be 1, 2, or 3.

[0404] Samples of segments A and F can be derived using padding that uses the nearest samples of segments B and E, respectively, instead of being obtained from restored neighbor blocks.

[0405] The reference sample line index may indicate a reference sample line among multiple reference sample lines used for intra-prediction of a target block. For example, the reference sample line index may have a value from 0 to 3. The reference sample line index may be signaled.

[0406] When intra-color component prediction is used for a target block, a prediction block for a second color component can be generated based on a reconstruction block of a first color component for the target block. For example, the first color component may be a luminance component, and the second color component may be a chroma component.

[0407] For intra-prediction between color components, parameters between the first and second color components can be derived based on a template. For example, the parameters can be parameters of a linear model.

[0408] For example, the template may include a top reference sample and / or a left reference sample of the target block, and may include a top reference sample and / or a left reference sample of the restoration block of the first color component corresponding to these reference samples.

[0409] Once the parameters are derived, a prediction block for a second color component for a target block can be generated by applying the reconstruction block of the first color component to a linear model. Depending on the image format or the type of intra-prediction between color components, subsampling or downsampling may be performed on the surrounding samples of the reconstruction block of the first color component and on the reconstruction block of the first color component. If subsampling is performed, the derivation of parameters and the intra-prediction between color components may be performed using corresponding samples derived by subsampling.

[0410] Intra Sub-Partitions (ISP) prediction may refer to sequential intra prediction for multiple subblocks generated by partitioning a target block. In ISP prediction, the target block may be partitioned into two or four subblocks in the horizontal and / or vertical directions. The partitioned subblocks may be restored sequentially. As intra prediction is performed on the subblocks, sub-prediction blocks for the subblocks may be generated. Additionally, as inverse quantization and / or inverse transformation is performed on the subblocks, sub-residual blocks for the subblocks may be generated. A restored subblock may be generated by adding the sub-prediction blocks to the sub-residual blocks. The restored subblocks may be used as reference samples for intra predictions for other subblocks to be processed next.

[0411] In performing a prediction for a target block, it may be determined whether samples included in a restored neighbor block can be used as reference samples for the target block. If there are non-available samples among the samples in the neighbor block that cannot be used as reference samples for the target block, a value generated by copying and / or interpolation using the sample value of at least one sample among the samples included in the restored neighbor block may replace the sample value of the non-available sample. If the value generated by copying and / or interpolation replaces the sample value of the sample, the sample may be used as a reference sample for the target block.

[0412] In intra-prediction, the sample value of a prediction sample in a prediction block can be determined by the sample value of a reference sample. The location of the reference sample can be specified by the location of the prediction sample and the direction of the intra-prediction mode. If the location specified by the location of the prediction sample and the direction of the intra-prediction mode is an integer location, the sample value of one reference sample pointed to by the integer location can be used to determine the sample value of the prediction sample in the prediction block. If the location specified by the location of the prediction sample and the direction of the intra-prediction mode is not an integer location, an interpolated reference sample can be generated based on the two reference samples closest to the specified location. The sample value of the interpolated reference sample can be used to determine the sample value of the prediction sample. That is to say, when the location specified by the location of the prediction sample and the direction of the intra-prediction mode represents the space between two reference samples, an interpolated sample value can be generated based on the sample values ​​of the two samples.

[0413]

[0414] Inter prediction

[0415] FIG. 4 shows the structure of an inter prediction to explain an inter prediction process according to one embodiment.

[0416] The rectangle shown in Fig. 4 can represent an image. Additionally, the arrow in Fig. 4 can represent the predicted direction.

[0417] Each image constituting a video can be classified into I-pictures (i.e., intra-pictures), P-pictures (i.e., uni-prediction pictures), and B-pictures (i.e., bi-prediction pictures) according to their coding type. Coding can be performed for each picture according to its coding type.

[0418] If the target picture is an I picture, coding for the target picture can be performed using information within the target picture without inter-prediction referencing other images. For example, coding for the I picture can be performed using intra-prediction and / or IBC prediction.

[0419] Coding for P picture and B picture can be performed by at least one of intra prediction, IBC prediction, and inter prediction using a reference image.

[0420] If the target picture is a P picture, coding for the target picture can be performed using unidirectional inter-prediction using a single reference image list.

[0421] When the target picture is picture B, coding for the target picture can be performed using unidirectional inter-prediction or bidirectional inter-prediction using two reference image lists.

[0422] Below, the inter prediction for the target block in the inter mode according to the embodiment is described in detail.

[0423] When the prediction mode of the target block is inter mode, inter prediction can be performed on the target block. The target block can be a prediction block or a partitioned prediction block.

[0424] Inter prediction can be performed using reference images and motion information. In inter prediction, a reference image can be selected using a reference image index, and a reference block corresponding to a target block within the reference image can be determined using motion information. A prediction block for the target block can be generated using the determined reference block.

[0425] Motion information can be derived using coding parameters, etc. For example, motion information can be derived using motion information of restored neighbor blocks, motion information of call blocks, and / or motion information of blocks adjacent to call blocks.

[0426] In the embodiments, a candidate list may be used for inter prediction. The candidate list may include multiple candidates. An index pointing to a candidate among the candidates in the candidate list that is used for inter prediction for a target block may be signaled. The candidate list may be derived in the same manner based on the same information in the encoding device (110) and the decoding device (150). Here, the same information may include a restored image and a restored block. Additionally, in order to specify a candidate by an index, the order of candidates within the candidate list may be constant.

[0427] In one embodiment, a prediction for a target block can be performed by using motion information of a spatial candidate or a temporal candidate as motion information of the target block. Motion information of a spatial candidate may be referred to as spatial motion information. Motion information of a temporal candidate may be referred to as temporal motion information.

[0428] Spatial candidates may be restored spatial neighbor blocks that are spatially adjacent to the target block.

[0429] Spatial candidates may be blocks that 1) exist within the target image, 2) have already been restored through decoding, and 3) are adjacent to the target block.

[0430] Spatial candidates may include the left block, top block, bottom-left block, top-right block, and top-left block of the target block.

[0431] Temporal candidates may be restored temporal neighbor blocks corresponding to the target block within the restored call (COL) image.

[0432] In the embodiments, the motion information of the spatial candidate may be the motion information of a block containing the spatial candidate. The motion information of the temporal candidate may be the motion information of a block containing the temporal candidate.

[0433] In inter prediction, a call block for a target block can be identified. The region of the target block within the target image and the region of the call block within the call image may be the same. That is to say, the call block may be a block that occupies a specific region within the call image. The specific region may be a region corresponding to the region of the target block within the call image.

[0434] Temporal candidates may be locations inside and / or outside the call block within the call image.

[0435] For example, a call block may include a first call block and a second call block. When the top-left coordinates of the call block are (xP, yP) and the size of the call block is (nPSW, nPSH), the first call block may be a block occupying the coordinates (xP + nPSW, yP + nPSH). The second call block may be a block occupying the coordinates (xP + (nPSW >> 1), yP + (nPSH >> 1)). The second call block may optionally be used as a call block when the first call block is unavailable.

[0436] The MV of the target block can be determined based on the MV of the call block. Scaling can be performed on the MV of the call block. The scaled MV of the call block can be used as the MV of the target block or the prediction MV. Alternatively, the MV of the temporal candidate stored in the candidate list associated with the inter-prediction can be the scaled MV.

[0437] The ratio of the scaled MV and the MV of the call block may be equal to the ratio of the first temporal distance and the second temporal distance. The first temporal distance may be the distance between the reference image of the target block and the target image. The second temporal distance may be the distance between the reference image of the call block and the call image.

[0438] The method by which motion information is derived can be determined by the inter prediction mode of the target block. For example, as an inter prediction mode, AMVP mode, merge mode, skip mode, merge mode with MVD, sub-block merge mode, GPM, Combined Inter Intra Prediction (CIIP) mode, and affine inter mode may be used. In the following embodiments, each of the inter prediction modes is described.

[0439]

[0440] AMVP mode

[0441] When AMVP mode is used as a prediction mode, a list of MV candidates including one or more MV candidates can be generated using spatial candidate MVs, temporal candidate MVs, history-based MV candidates, and zero vectors. At least one of the spatial candidate MVs, temporal candidate MVs, and zero vectors can be determined and used as an MV candidate.

[0442] Spatial candidates may include restored spatial neighbor blocks. The MV of a restored spatial neighbor block may be referred to as a spatial motion vector candidate. Temporal candidates may include a call block and a block adjacent to the call block. The MV of a call block or the MV of a block adjacent to the call block may be referred to as a temporal motion vector candidate. History-based MV candidates may be MVs in a list containing MVs of other blocks that were encoded / decoded before the encoding / decoding of the target block.

[0443] The encoding device (110) can determine the MV to be used for encoding a target block within a search range using an MV candidate list. The maximum number of MV candidates in the MV candidate list may be predefined. N may represent the predefined maximum number. For example, N may be 2. Alternatively, the maximum number of such candidates may be signaled from the encoding device to the decoding device or derived from the decoding device. The encoding device (110) can determine an MV candidate to be used as the predicted MV of the target block among the MV candidates in the MV candidate list. The MV to be used for encoding the target block may be an MV that can be encoded at the minimum cost. The encoding device (110) may determine whether to use the AMVP mode in encoding the target block and may generate AMVP mode usage information indicating whether the AMVP mode is used.

[0444] Inter prediction information may include 1) AMVP mode usage information, 2) MV candidate index, 3) MVD, 4) MVD resolution information, 5) reference direction and 6) reference image index, and may include residual blocks. Inter prediction information may be signaled from the encoding device (110) to the decoding device (150) in the form of a bitstream.

[0445] The decoding device (150) can obtain AMVP mode usage information from the bitstream. If the AMVP mode usage information indicates that the AMVP mode is being used, the decoding device (150) can obtain an MV candidate index, an MVD, MVD resolution information, a reference direction, and a reference image index from the bitstream. Among the MV candidates included in the MV candidate list, the MV candidate pointed to by the MV candidate index can be selected as the predicted MV of the target block.

[0446] The MVD may represent the difference between the MV that will actually be used for inter-prediction of the target block and the predicted MV. The encoding device (110) may derive a predicted MV that is close to the MV that will actually be used for inter-prediction of the target block in order to use an MVD of the smallest possible size. The decoding device (150) may derive the MV of the target block by summing the MVD and the predicted MV. That is to say, the MV of the target block derived by the decoding device (150) may be the sum of the MVD and the predicted MV candidates.

[0447] Additionally, the encoding device (110) can generate MVD resolution information. The MVD resolution information may be information used to adjust the resolution of the MVD. The decoding device (150) can adjust the resolution of the MVD using the MVD resolution information.

[0448] Meanwhile, the encoding device (110) can calculate the MVD based on an affine model. The affine control point MV of the target block can be derived based on the sum of the affine control point MV candidates and the MVD. Using the affine control point MV, the MV of each sub-block within the target block can be derived.

[0449]

[0450] Merge Mode

[0451] When merge mode is used, a merge candidate list containing multiple merge candidates can be generated using motion information of spatial candidates and motion information of temporal candidates, etc. Motion information may include 1) MV, 2) reference image index and 3) reference direction, etc. A merge candidate may be motion information.

[0452] Merge candidates may include 1) spatial merge candidates generated based on spatial candidates, 2) temporal merge candidates generated based on temporal candidates, 3) history-based merge candidates, 4) average merge candidates, and 5) zero merge candidates.

[0453] A history-based merge candidate may be movement information within a list containing movement information of other blocks that were encoded / decoded earlier than the encoding / decoding of the target block.

[0454] The average merge candidate may be a merge candidate generated based on the average of two merge candidates within the merge candidate list.

[0455] Zero merge candidates can be zero vector motion information. Zero vector motion information can be motion information where MV is a zero vector.

[0456] Merge candidates can be added to the merge candidate list according to a predefined method and a predefined order so that the merge candidate list has a set number of merge candidates. The same merge candidate list can be configured in the encoding device (110) and the decoding device (150) through the predefined method and a predefined order.

[0457] The encoding device (110) can select a merge candidate to be used for encoding a target block from among the merge candidates in the merge candidate list. The encoding device (110) can determine whether to use a merge mode in encoding the target block and can generate merge mode usage information indicating whether the merge mode is used.

[0458] Inter prediction information may include 1) merge mode usage information, 2) merge index and 3) correction information, etc., and may include residual blocks. Inter prediction information may be signaled in bitstream form from the encoding device (110) to the decoding device (150).

[0459] The decoding device (150) can obtain merge mode usage information from the bitstream. If the merge mode usage information indicates that the merge mode is being used, the decoding device (150) can obtain merge mode-related information, such as a merge index, from the bitstream.

[0460] The encoding device (110) can select the optimal merge candidate among the merge candidates included in the merge candidate list and can set the value of the merge index to point to the selected merge candidate.

[0461] Correction information may be information used for correcting the MV. The encoding device (110) may generate correction information. The decoding device (150) may derive a corrected MV by performing correction on the MV of a merge candidate selected by a merge index based on the correction information. The corrected MV may be used as the MV of the target block.

[0462] In one embodiment, the correction information may include an MVD. The correction information may include one or more of correction usage information, correction direction information, and correction magnitude information. The correction usage information may indicate whether to use correction for the MV. A merge mode that performs correction for the MV based on the correction information may be referred to as a merge mode having an MVD.

[0463] In merge mode, a prediction for the target block can be performed using the merge candidate pointed to by the merge index among the merge candidates included in the merge candidate list.

[0464] Movement information of the target block can be derived from 1) MV, 2) reference image index and 3) reference direction of the merge candidate pointed to by the merge index.

[0465] In one embodiment, the merge candidates in the merge candidate list may be specific modes that induce inter-prediction information. A merge candidate may be information pointing to a specific mode that induces inter-prediction information. Inter-prediction information of a target block may be induced according to the specific mode pointed to by the merge candidate. In this regard, the specific mode may be regarded as a specific inter-prediction information inducing mode or a specific movement information inducing mode. The specific mode may include a series of processes that induce inter-prediction information.

[0466] Inter-prediction information of the target block can be derived according to a specific mode pointed to by a merge candidate selected by a merge index among the merge candidates in the merge candidate list. For example, specific modes may include a mode for deriving motion information at the sub-block level and a mode for deriving motion information at the affine level, and may include other modes for deriving motion information as described in the embodiments.

[0467] Skip mode may be a mode that does not use residual blocks. That is to say, when skip mode is used, the restoration block may be identical to the prediction block. The description of the merge mode in the embodiments may also apply to skip mode. The difference between merge mode and skip mode may be whether or not residual blocks are signaled and used. That is to say, skip mode may be similar to merge mode except that residual blocks are not transmitted / used, and the description of merge mode may also apply to skip mode.

[0468] The subblock merge mode may be a mode in which motion information of a target subblock is induced for a target subblock within a target block. When the subblock merge mode is applied, a list of subblock merge candidates may be generated using affine control point motion vector merge candidates and / or subblock-based temporal merge candidates. The subblock-based temporal merge candidates may be motion information of the call subblock of the target subblock.

[0469] In GPM, a first prediction block and a second prediction block can be generated using two sets of motion information for a target block. For each coordinate of the target block, a final prediction sample of the final prediction block can be generated using the weighted sum of the first prediction sample of the first prediction block and the second prediction sample of the second prediction block.

[0470] Here, the first weight for the first prediction sample of the weighted consensus and the second weight for the second prediction sample can be determined based on the boundaries of the GPM. The boundaries may represent dividing lines that divide the target block. Depending on the boundaries, the target block may be divided into a first divided region and a second divided region.

[0471] If the distance between the final prediction sample and the boundary is less than or equal to a reference value, the value of the final prediction sample of the final prediction block may be determined using the weighted sum of the first prediction sample of the first prediction block and the second prediction sample of the second prediction block. If the distance between the final prediction sample and the boundary is greater than the reference value, one of the first weight and the second weight may be 1 and the other may be 0.

[0472] The Combined Inter-Intra Prediction (CIIP) mode may be a mode that derives a prediction sample of a target block using a weighted sum of a prediction sample generated by inter-prediction and a prediction sample generated by intra-prediction.

[0473] In the aforementioned modes, self-improvement of the derived motion information may be performed, and the improved motion information may be used as motion information for the target block. For example, blocks within a specific area determined based on the derived motion information may be searched, and the motion information of the block having the smallest Sum of Absolute Differences (SAD) value among the searched blocks may be used as the improved motion information for the target block. The specific area may be a square area within the reference image specified by the motion information. The point indicated by the motion information may be the center of the specific area.

[0474] In the aforementioned modes, compensation for prediction samples derived through inter-prediction can be performed using optical flow.

[0475]

[0476] FIG. 5 shows the order of addition of spatial candidates to the candidate list according to one embodiment.

[0477] In Fig. 5, the locations of the spatial candidates are shown.

[0478] The large block in the center can represent the target block. The five small blocks adjacent to the target block can represent spatial candidates.

[0479] The coordinates of the target block can be (xP, yP), and the size of the target block can be (nPSW, nPSH).

[0480] Spatial candidate A0 may be a block adjacent to the bottom-left of the target block. A0 may be a block occupying a sample of coordinates (xP - 1, yP + nPSH).

[0481] Spatial candidate A1 may be a block adjacent to the left of the target block. A1 may be the bottommost block among the blocks adjacent to the left of the target block. Or, A1 may be a block adjacent to the top of A0. A1 may be a block occupying a sample of coordinates (xP - 1, yP + nPSH - 1).

[0482] Spatial candidate B0 may be a block adjacent to the top right of the target block. B0 may be a block occupying a sample of coordinates (xP + nPSW, yP - 1).

[0483] Spatial candidate B1 may be a block adjacent to the top of the target block. B1 may be the rightmost block among the blocks adjacent to the top of the target block. Or, B1 may be a block adjacent to the left of B0. B1 may be a block occupying a sample of coordinates (xP + nPSW - 1, yP - 1).

[0484] Spatial candidate B2 may be a block adjacent to the top-left corner of the target block. B2 may be a block occupying a sample of coordinates (xP - 1, yP - 1).

[0485] As illustrated in Fig. 5, in adding spatial candidates to the candidate list, B1, A1, The order B0, A0, and B2 can be used. That is, B1, A1, Available spatial candidates can be added to the candidate list in the order of B0, A0, and B2. The order in which spatial candidates illustrated in FIG. 5 are added to the merge candidate list may be just one example.

[0486] The above candidate list may include a motion information candidate list, a merge candidate list, an MV candidate list, a BV candidate list, and an MPM list, etc.

[0487] To include a spatial or temporal candidate in the candidate list, it may be determined whether the spatial or temporal candidate is available. If a candidate block is outside the boundaries of an image, slice, or tile, the availability of the candidate block may be set to false. The description "availability is set to false" may mean "it is set to non-available."

[0488] The maximum number of candidates in the candidate list can be set. N can represent the set maximum number. The set maximum number can be signaled through a parameter set or header, etc. For example, the maximum number of candidates in the candidate list for a target block within a slice can be set by the slice header. For example, the value of N can be 5 by default.

[0489]

[0490] IBC mode

[0491] The IBC mode may be an intra-block copy prediction mode that generates a prediction block for a target block by referencing an already reconstructed region within the target image. In this respect, the IBC mode may also be referred to as a current image reference mode. A block vector (BV) may be used to identify the already reconstructed region.

[0492] Whether the target block is encoded / decoded in IBC mode can be determined using IBC mode usage information. The encoding device (110) can determine whether to use IBC mode in encoding the target block and can generate IBC mode usage information indicating whether IBC mode is used. The decoding device (150) can obtain IBC mode usage information from the bitstream.

[0493] In IBC mode, the predicted block of the target block can be generated based on the BV. The BV can specify the reference block. The BV can indicate the displacement between the target block and the reference block. The reference block can be a block within the target image. The description of the MV of the embodiments can also be applied to the BV.

[0494] The IBC mode may include a skip mode, a merge mode, and an AMVP mode, etc. The descriptions of the AMVP mode, merge mode, and skip mode of the embodiments may be similarly applied to the AMVP mode, merge mode, and skip mode of the IBC mode, respectively.

[0495] In skip mode or merge mode, a merge candidate list may be configured, and a merge index may specify one merge candidate from among the merge candidates in the merge candidate list. The BV of the specified merge candidate may be used as the BV of the target block.

[0496] In AMVP mode, BVD can be used. The description of MVD in the embodiments can also be applied to BVD.

[0497] The reference block in IBC mode may be limited to a block within an already restored region of the target image. Alternatively, the reference block may be contained within at least one of the target CTU or the left CTUs. For example, the value of BV may be restricted so that the reference block is located within a specific region. The specific region may be an area of ​​three blocks of a specific size that are encoded / decoded before the block of a specific size containing the target block. The specific size may be 64x64.

[0498]

[0499] Transformation and Quantization

[0500] Quantized levels can be generated by performing a transformation and / or quantization on the residual block. The residual block can represent the difference between the original block and the prediction block. A restored residual block can be generated by performing inverse quantization and / or inverse transformation on the quantized levels. The restored residual block can represent the difference between the restored block and the prediction block.

[0501] When a transformation or inverse transformation is performed, a separable transform or a 2D non-separable transform may be performed on the residual block. A separable transform may be a transformation that performs 1D transformations on the residual block in the horizontal and vertical directions, respectively.

[0502] The transformation kernels used for the transformation may include various DCT kernels such as DCT type 2 (DCT-II), 2) DST kernels, and 3) kernels derived by training. For 1D transformation, DCT type and DST type may include DCT-V, DCT-VIII, DST-I, and DST-VII in addition to DCT-II.

[0503] A transformation set may be used to determine the DCT type, DST type, or learning-derived kernel to be used for the transformation. Each transformation set may include multiple transformation candidates. Each transformation candidate may be a DCT type, a DST type, or a learning-derived kernel, etc.

[0504] The encoding device (110) can perform conversion and inverse conversion using conversion candidates included in the conversion set. The decoding device (150) can perform inverse conversion using conversion candidates included in the conversion set. Conversion selection information indicating which conversion candidate is used among the plurality of conversion candidates included in the conversion set applied to the residual block may be signaled. The conversion selection information may include vertical conversion selection information and horizontal conversion selection information. The vertical conversion selection information may indicate which conversion among the conversions belonging to the conversion set is used for the vertical conversion. The horizontal conversion selection information may indicate which conversion among the conversions belonging to the conversion set is used for the horizontal conversion.

[0505] The transformation may include at least one of a primary transformation and a secondary transformation. A primary transformation coefficient may be generated by performing a primary transformation on the residual block, and a secondary transformation coefficient may be generated by performing a secondary transformation on the transformation coefficient. Here, the transformation coefficient may include a primary transformation coefficient and a secondary transformation coefficient.

[0506] A first-order transformation may mean a Multiple Transform Selection (MTS) that applies different transformations to each of the 1D directions (i.e., vertical and horizontal directions).

[0507] A second-order transformation may be a transformation intended to improve the energy concentration of the transformation factors generated by a first-order transformation. A second-order transformation may be 1) a separable transformation like the first-order transformation, or 2) a 2D non-separable transformation. A 2D non-separable transformation may refer to a Low Frequency Non-Separable Transform (LFNST) or a Non-Separable Primary Transform (NSPT).

[0508] NSPT can be applied to specific block sizes such as 4x4, 4x8, 8x4, 4x16, 16x4, 8x8, 8x16, and 16x8 for intra-coding.

[0509] A first-order transformation may be performed using at least one of a plurality of predefined transformation methods. For example, the plurality of predefined transformation methods may include DCT, DST, and KLT, etc. Additionally, the first-order transformation may be a transformation having various transformation types according to transformation kernel functions that define DCT and DST. For example, the first-order transformation may include a plurality of transformations such as DCT-2, DCT-4, DCT-5, DCT-7, DCT-8, DST-1, DST-2, DST-4, DST-7, and DST-8 according to a plurality of transformation kernels.

[0510] In one embodiment, the transformation type may be determined based on coding parameters related to the target block. For example, the transformation type may be determined based on one or more of 1) the prediction mode of the target block (e.g., one of intra prediction and inter prediction), 2) the size of the target block, 3) the shape of the target block, 4) the intra prediction mode of the target block, 5) the components of the target block (e.g., one of luminance components and chroma components), and 6) the splitting type applied to the target block (e.g., one of QT, BT, TT, and non-split).

[0511] As in the first transformation, a set of transformations can be defined in the second transformation as well. Methods for deriving and / or determining the set of transformations of the embodiments can be applied to the second transformation as well as the first transformation.

[0512] In one embodiment, the first transformation and / or second transformation may be determined for a specific target. The transformation selection information may include transformation target information. The transformation target information may indicate the target to which the first transformation and / or second transformation is applied.

[0513] For example, first-order transformation and / or second-order transformation may be applied to one or more signal components among the luminance component and the chroma component.

[0514] In one embodiment, the transformation selection information may include first transformation usage information and second transformation usage information. The first transformation usage information may indicate whether a first transformation is applied to the residual block of the target block. The second transformation usage information may indicate whether a second transformation is applied to the residual block of the target block.

[0515] In one embodiment, whether a first transformation and / or a second transformation is applied may be determined based on coding parameters for the target / neighbor block, such as the size and shape of the target / neighbor block.

[0516] In one embodiment, the transformation selection information may include first transformation selection information and second transformation selection information. The first transformation selection information may indicate a transformation method applied to a residual block among a plurality of transformation methods that can be used in the first transformation. The first transformation selection information may be a first transformation index. The second transformation selection information may indicate a transformation method applied to a transformation coefficient among a plurality of transformation methods that can be used in the second transformation. The second transformation selection information may be a second transformation index.

[0517] In one embodiment, the transformation methods of the first transformation and the second transformation can each be derived based on specific information such as coding parameters. For example, the coding parameters may include coding parameters for target / neighbor blocks.

[0518] In the embodiments, information related to a transformation, such as transformation selection information, and sub-information of the transformation selection information may be signaled to a specific target. For example, the specific target may be a CU.

[0519] Information related to transformations, such as transformation selection information, and sub-information of transformation selection information can be derived for a specific target. For example, the specific target may be a CU.

[0520] Quantized levels can be generated by performing quantization on the result or residual block generated by performing a first-order transformation and / or a second-order transformation.

[0521] The description of the transformation described above may also be applied to the inverse transformation. In such application, the inverse processing of the processing described for the transformation may be performed in the inverse transformation. "Transformation" within the name related to the transformation may be changed to "inverse transformation." Additionally, the input of the transformation may be considered as the output of the inverse transformation. The output of the transformation may be considered as the input of the inverse transformation. The decoding device (150) may obtain information related to the transformation, such as transformation selection information, and may use the information related to the transformation to perform the inverse processing of the transformation related to the transformation indicated by the information related to the transformation.

[0522] The target block may include multiple subblocks. Each subblock may be defined according to a minimum block size or minimum block shape. The target block may be divided into multiple subblocks, and each subblock may include coefficients such as 4x4, 2x8, and 8x2. The target block may be a transformation block. Transform coefficients or quantized levels may be represented in the form of a block. Transform coefficients may be quantized transformation coefficients.

[0523] Transform coefficients or quantized levels may be scanned according to at least one scanning type among diagonal scanning, vertical scanning, and horizontal scanning. Diagonal scanning may be top-right diagonal scanning or bottom-left diagonal scanning.

[0524] For example, by scanning the coefficients of a block using diagonal scanning, the coefficients can be changed or arranged into a one-dimensional vector form. Vertical scanning may be scanning the coefficients in the form of a two-dimensional block in a column direction. Horizontal scanning may be scanning the coefficients in the form of a two-dimensional block in a row direction.

[0525] The scanning type for the coefficients can be determined based on coding parameters such as intra prediction mode, block size, and block shape. For example, based on coding parameters such as intra prediction mode, block size, and block shape, it can be determined which scanning method—diagonal scanning, vertical scanning, and horizontal scanning—will be used. A block may be a transformation unit.

[0526] Scanning according to each scanning type can start at a specific starting point and end at a specific ending point.

[0527] In scanning, the scanning order according to the scanning type can first be applied between subblocks. Next, the scanning order according to the scanning type can be applied to the transformation coefficients or quantized levels within the subblocks.

[0528] The encoding device (110) can perform entropy encoding on the conversion coefficients or quantized levels to generate a bitstream containing entropy-encoded conversion coefficients or entropy-encoded quantized levels.

[0529] The decoding device (150) can generate the transform coefficients or quantized levels by obtaining entropy-encoded transform coefficients or entropy-encoded quantized levels from the bitstream and performing entropy decoding. The coefficients can be arranged in the form of two-dimensional blocks through inverse scanning. The arrangement of inverse scanning may be a rearrangement opposite to the arrangement of scanning.

[0530] Backscanned transform coefficients or backscanned quantized levels can be generated through backscanning of the coefficients. In this case, the backscanning types of backscanning may include diagonal scans, vertical scans, and horizontal scans, and a backscanning type of the inverse transform corresponding to the scanning type of the transform may be selected.

[0531] In the decoding device (150), inverse quantization can be performed on the (backscanned) coefficients. Depending on whether a second inverse transform is performed, a second inverse transform can be performed on the result generated by the performance of inverse quantization. Also, depending on whether a first inverse transform is performed, a first inverse transform can be performed on the result generated by the performance of the second inverse transform. By selectively performing a second inverse transform and a first inverse transform on the coefficients, a restored residual block can be generated.

[0532]

[0533] Filtering

[0534] To improve the image quality, filtering may be performed on the blocks. The value of the target sample may be determined or updated by the filtering.

[0535] The target sample may be one of the samples described in the embodiments. For example, the target sample may be one or more of the samples described in the embodiments, such as a prediction sample, a reference sample, a residual sample, a reconstructed sample, and a reconstructed sample to which filtering has been applied.

[0536] The target sample may be a sample within one or more of the target picture, target slice, target CTB, target block, reference sample line, and template. The target block may be one of the blocks described in the embodiments. For example, the target block may be one or more of the blocks described in the embodiments, such as a transformation block, prediction block, reference block, residual block, and restoration block.

[0537] In the embodiments, the filtering process described as being applied to one target may also be applied to other targets. For example, the filtering process described in a specific in-loop filtering may also be applied to a transformation block, a prediction block, a reference block, and a residual block, etc.

[0538] For the filtering of the embodiments, a specific type of filtering may be used. The type of filtering may include a filter tap (or filter tap length), a filter shape, a filter strength, filter coefficients (or weights), and an offset.

[0539] The filter tab may indicate the number of input samples used for the filter. The input samples may include target samples. Alternatively, the input samples may include specific values ​​determined for the target samples. The input samples may include one or more reference samples. One or more reference samples may be determined based on the attributes of the target block described in the embodiments. The attributes may include coding parameters. For example, the attributes of the target sample may include the location of the target sample. One or more reference samples may be specified based on their relative position to the location of the target sample.

[0540] The filter shape can represent the shape formed by input samples. A specific value determined for a target sample can be considered as the target sample. In other words, if a specific value determined for a target sample is used as an input sample for a filter, the target sample can also be considered as constituting the filter shape.

[0541] There may be multiple samples whose values ​​are determined by filtering. Filter strength may represent the range of samples whose values ​​are determined by filtering. Filter strength may be either strong filtering strength or weak filtering strength. The number of samples whose values ​​are determined by strong filtering strength may be greater than the number of samples whose values ​​are determined by weak filtering strength. Alternatively, filter strength may represent the range of values ​​that are changed by filtering. The range of sample values ​​changed by strong filtering strength may be wider than the range of sample values ​​changed by weak filtering strength.

[0542] Filter coefficients can be coefficients or weights of the input samples.

[0543] The offset can be a specific value added to the result calculated using the values ​​and coefficients of the input samples, such as a weighted sum.

[0544] Filtering, interpolation, and sampling may be common in that they update the values ​​of samples. Accordingly, the description of any one of filtering, interpolation, and sampling in the embodiments may also apply to the other one of filtering, interpolation, and sampling. Here, sampling may include at least one of upsampling, downsampling, and subsampling.

[0545] Filtering may include filtering performed by a predictor (123) and a predictor (163), etc.

[0546] In encoding for a target block, a prediction error may exist between the original sample of the original block and the prediction sample of the prediction block. To reduce the prediction error, filtering may be performed on at least one of the prediction sample of the prediction block and the reference sample referenced for prediction.

[0547] For example, in intra-prediction, the reference samples may include one or more of the top-left reference sample, top reference sample, top-right reference sample, left reference sample, and bottom-left reference sample. Filtering of the prediction samples may be performed by applying specific weights to the prediction samples, left reference samples, top reference samples, and / or top-left reference samples, respectively.

[0548] Filtering for at least one of the prediction sample and the reference sample may be performed based on the attributes of the target block and the attributes of the prediction sample. For example, whether filtering is performed, the type of filter, the area to which filtering is applied, the weights of the filtering, the reference sample, the range of the reference sample, and the location of the reference sample may each be determined based on the attributes of the target block and the attributes of the prediction sample.

[0549] For example, the attributes of the target block may include information related to the target block described in the embodiments, such as 1) size, 2) prediction mode, 3) intra prediction mode, 4) reference sample line, 5) sample value, and 6) coding parameter.

[0550] For example, the attributes of the prediction sample may include information related to the prediction sample described in the embodiments, such as 1) the sample value and 2) the location within the target block of the prediction sample, and may include coding parameters regarding the prediction sample.

[0551] Filtering may include in-loop filtering performed by a filter (130) and a filter (170), etc.

[0552]

[0553] Figure 6 shows a plurality of in-loop filters according to one example.

[0554] Multiple in-loop filters of in-loop filtering may include one or more of Luma Mapping with Chroma Scaling (LMCS), deblocking filter, Sample Adaptive Offset (SAO), and Adaptive Loop Filter (ALF).

[0555] Multiple in-loop filters can be connected sequentially. For example, multiple in-loop filters can be connected in the order of LMCS, deblocking filter, SAO, and ALF. Additionally, multiple in-loop filters can be connected in any order of all available permutations of the multiple in-loop filters. The output from one of the multiple in-loop filters can be used as an input to the next filter.

[0556] As illustrated in FIG. 6, an input image may be input to the first filter. The input image may be a block as described in the embodiments. For example, the input image may be a restored block generated by an adder (129) or an adder (169). The output from one filter may be input to the next filter. An output image may be generated by the last filter. The output image may be a filtered block as described in the embodiments. For example, the output image may be a filtered restored image generated by a filter (130) or a filter (170).

[0557] The target block can represent the image input to the filter. The filtered target block can represent the image output from the filter.

[0558] LMCS may include luminance signal mapping for the luminance signal of the target block and chroma signal scaling for the chroma signal of the target block.

[0559] Luminous signal mapping can perform codeword redistribution for the luminous signal.

[0560] Luma signal mapping may include forward mapping and inverse mapping. In forward mapping, the existing dynamic range may be divided into multiple intervals. The mapped dynamic range can be determined by performing codeword redistribution on the input image using a linear model for each interval. In inverse mapping, inverse mapping from the mapped dynamic range to the existing dynamic range is performed.

[0561] Chroma scaling can correct the chroma signal based on the interrelationship between the luminance signal and the corresponding chroma signal.

[0562] Forward mapping can be performed between inter-prediction of the luminance signal and restoration of the luminance signal, and between inter-prediction of the luminance signal and chroma scaling. Inverse mapping can be performed between restoration of the luminance signal and in-loop filtering of the luminance signal. Chroma scaling can be performed between inverse transform and restoration of the chroma signal.

[0563] According to this structure, inverse quantizations for the luminance and chroma signals, inverse transforms for the luminance and chroma signals, prediction for the luminance signal, and restoration for the luminance signal can be performed within a mapped dynamic domain. In-loop filtering for the luminance and chroma signals, inter-predictions for the luminance and chroma signals, intra-predictions for the chroma signal, and restoration for the chroma signal can be performed within the existing dynamic domain.

[0564] A deblocking filter can remove block distortion occurring at the boundaries between blocks within the reconstructed image. For example, the blocks may be transformed blocks. Additionally, the blocks may be subblocks of a specific block described in the embodiments. Here, the boundaries between blocks may refer to samples adjacent to the boundaries between blocks.

[0565] A deblocking filter can be applied to the vertical and horizontal boundaries between blocks. After filtering is performed on the vertical boundaries of the blocks, filtering can be performed again on the horizontal boundaries of the filtered blocks.

[0566] A deblocking filter may be applied optionally. Whether to apply a deblocking filter to a target block may be determined based on at least one of sample(s) contained within a specific number of columns or rows within the target block and sample(s) contained within a specific number of columns or rows within a neighboring block adjacent to a specific boundary.

[0567] When a deblocking filter is applied to a target block, the filter to be applied may be determined according to the required strength of deblocking filtering. In other words, among a plurality of different filters, the filter determined according to the strength of deblocking filtering may be applied to the target block. The plurality of filters may include one of a long-tap filter, a strong filter, a weak filter, and a Gaussian filter.

[0568] The maximum length of the deblocking filter can be determined based on attributes of the target block, such as the size of the target block, the components of the target block, and coding parameters.

[0569] SAO can compensate for distortion between the original image and the reconstructed image on a sample basis. For compensation, SAO can apply an appropriate offset to the sample values. In other words, the offset can be added to the sample values.

[0570] An offset can be determined for the target block. For example, the offset can be determined for each component of the CTB. The determined offset can be applied to samples within a specific component of the CTB.

[0571] SAO may include an SAO using an edge offset (EO) and an SAO using a band offset (BO). Depending on the characteristics of samples within a specific block, such as a CTU, whether to perform an SAO using EO and whether to perform an SAO using BO may be determined, respectively.

[0572] In an SAO using EO, correction for distortion of samples can be performed based on the direction of edges within the target block. The pattern classes of the EO may include horizontal patterns, vertical patterns, 135-degree diagonal patterns, and 45-degree diagonal patterns. For the target block, information indicating the pattern class applied to the target block and multiple offsets of the said pattern class may be signaled. There may be four offsets. For a target sample within the target block, adjacent samples of the target sample may be determined according to the direction of the pattern class. An offset to be applied to the target sample may be determined by the pattern of the adjacent samples.

[0573] In an offset using BO, correction for sample distortion can be performed by classifying the brightness values ​​of samples within the target block into specific bands. The bit depth of the input image can be divided into m intervals. For example, m can be 32. The specific bands can be n consecutive intervals among the m intervals. For example, n can be 4. n offsets for the n intervals can be signaled. Additionally, information indicating the first interval selected as the n intervals among the m intervals can be signaled. The offset of the interval corresponding to the target sample can be added to the sample value of the target sample of the target unit.

[0574] ALF can compensate for distortion between the restored image and the original image.

[0575] The filter coefficients of the ALF can be signaled through the bitstream.

[0576] The filter shape of the ALF can be determined by the components of the target block. For example, a 7x7 diamond-shaped filter can be used for the luminance component. A 5x5 diamond-shaped filter can be used for the chroma component.

[0577] In ALF, the characteristics of a specific block can be determined for that block, and the class of that specific block can be determined based on those characteristics. In other words, the determination of characteristics and the determination of the class in ALF can be performed in units of 4x4 blocks. Filter coefficients can be calculated according to the class. A specific block can be a block with a size of 4x4.

[0578] One of 25 classes can be determined as the class of a specific block based on the direction and activity determined using the gradient of the specific block. Depending on the gradient of the specific block, a rotation transformation, a vertical reflection transformation, and / or a diagonal reflection transformation may be applied to the filter.

[0579] Information regarding whether ALF is applied can be signaled to specific units such as CTBs.

[0580] An index indicating a filter to be applied to a specific unit among the available filters may be signaled. Here, the available filters may include fixed filters and filters configured using a parameter set. For example, the parameter set may be an Adaptive Parameter Set (APS). The fixed filters may be predefined identically in the encoding device (110) and the decoder (150). The filter coefficients of the filters configured using the parameter set may be determined based on the coding parameters.

[0581]

[0582] Entropy Encoding and Entropy Decoding

[0583] Figure 7 shows entropy encoding and entropy decoding according to one example.

[0584] The processes of entropy encoding by the entropy encoder (139) are illustrated at the top of Fig. 7.

[0585] The entropy encoder (139) may include a context modeler, a binarization unit, and an entropy encoding unit. The context modeler may include a context selection unit and a context memory.

[0586] The binarization unit can generate binaries for syntactic elements by performing binarization on the syntactic elements of the target block. Binarization may be a process of converting syntactic elements into the form of binaries.

[0587] Information about syntax elements and beans can be provided from the binarization unit to the context selection unit.

[0588] The context modeler can perform context updates.

[0589] Context can refer to occurrence probability information for each bin regarding syntactic elements that have already been encoded.

[0590] The context modeler may perform a context update to apply current probability information to the entropy encoding of the bins of the syntactic elements of the target block. The updated context may be stored in context memory. At this time, the updated context corresponding to the syntactic elements of the target block (or the bins within the syntactic elements of the target block) may be derived by the context modeler.

[0591] The context selector can select a context corresponding to a bin of a syntactic element of a target block. The selected context can be loaded from context memory and used as an updated context for entropy encoding of the bins of the syntactic element of the target block.

[0592] The updated context can be used for entropy encoding of syntactic elements of the target block.

[0593] The entropy encoding unit can generate encoded information for syntactic elements of a target block by performing entropy encoding using generated bins and an updated context, and can generate a bitstream containing the encoded information. The entropy encoding unit may use at least one of an arithmetic encoding method and a bypass encoding method.

[0594] At the bottom of Fig. 7, the entropy decoding process by the entropy decoder (161) is illustrated.

[0595] The entropy decoder (161) may include a context modeler, an entropy decoder, and an inverse binary converter. The context modeler may include a context selection unit and a context memory.

[0596] The context modeler can perform context updates.

[0597] The context can refer to the probability information of occurrence for each bin regarding syntactic elements that have already been decoded.

[0598] The context modeler may perform a context update to apply the currently decoded probability information to the entropy decoding for the bins of the syntactic elements of the target block. The updated context may be stored in context memory. At this time, the updated context corresponding to the syntactic elements of the target block (or the bins within the syntactic elements of the target block) may be derived by the context modeler.

[0599] The context selection unit can select a context corresponding to a bin of a syntactic element of a target block. The selected context can be loaded from context memory and can be used as an updated context for entropy decoding of the syntactic element of the target block.

[0600] The updated context can be used for entropy decoding of the syntactic elements of the target block.

[0601] The entropy decoding unit can generate bins for the segmentation elements of the target block by performing entropy decoding on the encoded information of the bitstream based on the updated context. The entropy decoding unit may use at least one of an arithmetic decoding method and a bypass decoding method.

[0602] The debinaryization unit can obtain syntactic elements of a target block by performing debinaryization on at least one of the generated beans. Debinaryization may be a process of converting at least one of the beans into the form of a syntactic element.

[0603] Information about syntax elements and beans can be provided from the inverse binary unit to the context selector.

[0604] The syntax element may be one of the coding parameters described in the examples.

[0605]

[0606] Methods for binarization, inbinarization, entropy encoding, and entropy decoding

[0607] In the embodiments, to perform signaling for specific information, one or more of the binarization method, inverse binarization method, entropy encoding method and entropy decoding method listed below may be used.

[0608] - Signed 0-th order Exponential Golomb binarization / debinarization method (abbreviated as se(v))

[0609] - Signed k-order exponential-Golomb binarization / debinarization method (abbreviated as sek(v))

[0610] - 0-order exponentiation-Golomb binarization / debinarization method for unsigned positive integers (abbreviated as ue(v))

[0611] - k-order exponential-Golomb binarization / debinarization method for unsigned positive integers (abbreviated as uek(v))

[0612] - Fixed-length binarization / debinarization method (abbreviated as f(n))

[0613] - Truncated Rice binarization / debinarization method or truncated unary binarization / debinarization method (abbreviated as tu(v))

[0614] - Truncated binary binarization / debinarization method (abbreviated as tb(v))

[0615] - Context-adaptive arithmetic encoding / decoding method (abbreviated as ae(v))

[0616] - Bit string in bytes (abbreviated as b(8))

[0617] - Signed integer binary / debinary conversion method (abbreviated as i(n))

[0618] - Unsigned positive integer binarization / debinarization method (abbreviated as u(n)) ('u(n)' may also refer to a fixed-length binarization / debinarization method.)

[0619] - Unary Binary / Debinary Method

[0620]

[0621] Adaptive execution of the processing of the examples

[0622] The processing of the embodiments may be performed in the same and / or corresponding way in the encoding device (110) and the decoding device (150). Additionally, a combination of one or more of the above embodiments may be used for encoding and / or decoding of the image.

[0623] The order in which the embodiments are applied may differ from one another in the encoding device (110) and the decoding device (150). Alternatively, the order in which the above embodiments are applied may be the same (at least partially) in the encoding device (110) and the decoding device (150).

[0624] The processing of the embodiments may be performed for each of the specific targets. The processing of the embodiments may be performed identically for the specific targets. For example, the specific targets may include a luminance signal and a chroma signal.

[0625] The processes of the embodiments may be selectively applied / performed based on specific conditions or specific targets.

[0626] In one embodiment, the processing of the embodiment may be selectively applied / performed according to a temporal layer. Temporal layer information for a specific processing may be information indicating the temporal layer where the processing can be applied / performed. Temporal layer information may be signaled for a specific processing. Temporal layer information may indicate the lowest layer and / or the highest layer where the specific processing can be applied, and may indicate the specific layer where the specific processing is applied / performed. Alternatively, a fixed temporal layer where the processing of the embodiment is applied / performed may be defined.

[0627] In one embodiment, a type to which the processing of the embodiments is applied / performed may be defined, and whether the processing of the embodiments is applied / performed may be determined based on the defined type. The type may include a picture type, a slice type, and a tile group type, etc.

[0628] According to the description of the embodiments, when applying / performing a specific process on a specific target, specific conditions may be required, and the specific process may be performed under a specific decision. Where it is determined whether a specific condition is satisfied based on a specific coding parameter, or where a specific decision is made based on a specific coding parameter, such specific coding parameter may be interpreted as being replaceable with another coding parameter. That is to say, the coding parameter affecting the specific condition or specific decision described in the embodiments may be considered merely exemplary, and in addition to the specified coding parameter, one or more other coding parameters or a combination of one or more other coding parameters may be understood to perform the role of the specified coding parameter.

[0629] The processing of the embodiments may be applied / performed based on the size of at least one of the blocks described in the embodiments. For example, the blocks may include a coding block, a prediction block, a transformation block, a reference block, a current block, and a target block. Alternatively, the blocks may include adjacent blocks of the embodiments. Here, the size may be defined as a minimum size and / or a maximum size for the processing of the embodiments, or as a fixed size for the processing of the embodiments. Additionally, for the processing of the embodiments, a first embodiment may be applied at a first size, and a second embodiment may be applied at a second size. That is, the processing of the embodiments may be applied in combination depending on the size. Additionally, the processing of the embodiments may be applied only when the block size is greater than or equal to the minimum size and less than or equal to the maximum size. That is, the processing of the embodiments may be applied only when the block size falls within a specific range.

[0630] FIG. 8 is a flowchart illustrating an image encoding method or an image decoding method according to one embodiment of the present disclosure.

[0631] In FIG. 8, steps S810 to S820 can be performed in the aforementioned image encoding device or image decoding device.

[0632] In step S810, the video encoding / decoding device derives a prediction mode and a prediction signal for prediction encoding and decoding. A prediction mode of the current block is derived, and a prediction signal of the current block can be derived using the prediction mode of the current block.

[0633] In step S820, the video encoding / decoding device performs transformation and quantization using the derived prediction mode and signal. In terms of encoding, a residual signal of the current block is derived based on the original signal and the prediction signal of the current block, and quantized transformation coefficients can be derived by performing transformation and quantization on the residual signal of the current block using the derived prediction mode or prediction signal. In terms of decoding, a residual signal is derived by performing inverse quantization and inverse transformation on the quantized transformation coefficients of the current block obtained from the bitstream, and a restored signal can be derived based on the residual signal and the prediction signal. At this time, the prediction mode or prediction signal may be used to perform inverse quantization and inverse transformation on the quantized transformation coefficients.

[0634] Hereinafter, various embodiments of step S810 are described.

[0635] For example, to generate a prediction mode and prediction signal of the current block, a planar mode, a decoder-side intra-mode derivation (DIMD) method, an occurrence-based intra-mode coding (OBIC) method, a template-based intra-mode derivation (TIMD) method, an intra-sub-partitions (ISP) method, a method combining the TIMD method and the DIMD method, a merge prediction mode using prediction information derived from a neighboring block (or pixel), or a method of deriving intra-mode prediction information from a neighboring block / reference block may be used.

[0636] 1. Planner mode

[0637] To generate a prediction signal for the current block, a planar mode may be used.

[0638] In generating a prediction signal using a “planar” prediction, at least one prediction signal can be generated using multiple reference lines.

[0639] For example, in planar prediction, at least one prediction signal is generated through reference lines around the current block, the line that derives the prediction signal having the smallest cost in terms of encoding cost is determined as the optimal line, and the corresponding line information can be signaled from the encoder to the decoder.

[0640] For example, when determining the reference line to be used for planar prediction among multiple reference lines, the planar prediction can be performed using the reference line determined in MRL.

[0641] In generating a prediction signal through planar prediction, the generated prediction signal may differ depending on the direction of interpolation. In other words, directional planar prediction can be applied.

[0642] For example, as shown below, a prediction signal can be generated using only horizontal interpolation and only vertical interpolation, respectively.

[0643] When using only horizontal interpolation, the prediction signal can be generated using the equation pred(x, y) = ((W-1-x) * rec (-1, y) + (x+1) * rec(W, -1) + (W>>1)) >> log2(W), where W represents the width of the current block. In horizontal interpolation, to predict the target sample at position (x, y) within the current block, the left reference sample (rec (-1, y)) and the top-right reference sample (rec(W, 01)) located in the same row as the target sample can be used.

[0644] When using only vertical interpolation, the prediction signal can be generated using the equation pred(x, y) = ((H-1-y) * rec (x, -1) + (y+1) * rec(-1, H) + (H>>1)) >> log2(H). Here, H represents the height of the current block. In vertical interpolation, to predict the target sample at position (x, y) within the current block, the top reference sample (rec, (x, -1)) and the bottom-left reference sample (rec(-1, H)) located in the same column as the target sample can be used.

[0645] The transformation kernel can be applied differently depending on the interpolation direction (or planar prediction direction).

[0646] For example, in planar prediction based on horizontal interpolation, a transformation kernel used in the horizontal mode can be applied.

[0647] For example, when making planar predictions based on vertical interpolation, a transformation kernel used in the vertical mode can be applied.

[0648] In one embodiment, the planar mode information used for the final prediction among the horizontal planar and vertical planar can be signaled from the encoder to the decoder.

[0649] In one embodiment, when using a directional planar, the directional planar mode (horizontal planar, vertical planar) to be used for the final prediction can be determined by using encoding / prediction information derived from surrounding blocks.

[0650] For example, a directional planar mode can be determined based on directional prediction mode information derived from an adjacent block. Specifically, a directional planar mode can be determined depending on whether the direction of the directional prediction mode derived from the adjacent block is closer to the vertical direction or the horizontal direction.

[0651] For example, each directional planar mode can be determined depending on whether the directional prediction mode derived from an adjacent block is included in the near-vertical or near-horizontal defined below.

[0652] Below, an example is described in which a planar mode is determined according to the range of the directional prediction mode (best mode) derived from an adjacent block.

[0653] 1) Directional prediction mode range (angle range) First embodiment

[0654] · Near horizontal: 2<= best mode < 34 → horizontal planar

[0655] · Near vertical: 34<= best mode < 66 → vertical planar

[0656] 2) Directional prediction mode range (angle range) Second embodiment

[0657] · Near horizontal: 8<= best mode < 28 → horizontal planar

[0658] · Near vertical: 40<= best mode < 60 → vertical planar

[0659] · Others → Planar

[0660] 3) Directional prediction mode range (angle range) Third embodiment

[0661] · Near horizontal: 13<= best mode < 23 → horizontal planar

[0662] · Near vertical: 45<= best mode < 55 → vertical planar

[0663] · Others → Planar

[0664] In the above embodiments, a vertical planar may be used instead of a horizontal planar, or a horizontal planar may be used instead of a vertical planar.

[0665] In the above embodiments, directionality may be induced through a DIMD method using surrounding restoration samples of the current block.

[0666] In the case of the above embodiments, signaling for the planar mode used for the final prediction may be omitted.

[0667] In one embodiment, the directional planar mode can be determined based on the template cost. Specifically, a template including previously restored samples around the current block is constructed, and after predicting the template using each directional planar mode (horizontal planar, vertical planar), the template cost can be calculated by comparing the predicted samples and the restored samples within the template. The final planar mode can be determined to be the directional planar mode having the smaller template matching cost.

[0668] Although an embodiment has been described in which one of the horizontal planar and vertical planar is determined based on template costs, one of the general planar mode and two directional planar modes may also be determined based on template costs.

[0669] 2. DIMD mode

[0670] In generating (or deriving) a prediction signal for prediction encoding and decoding, a prediction signal can be generated through the decoder-side intra-mode derivation (DIMD) method.

[0671] The prediction mode and prediction signal induction through the above “decoder-side screen prediction mode induction method” can be performed by [STEP 1] inducing a directional prediction mode by inducing a gradient histogram (Histogram of Gradient: HoG) for the current block (or target block) and [STEP 2] generating a prediction block based on the determined prediction mode.

[0672] In the present disclosure, the HoG of DIMD can be replaced with the HoC of OBIC. In other words, embodiments related to the DIMD mode can also be applied to the OBIC mode.

[0673] [STEP 1] Inducing Directional Prediction Mode

[0674] FIG. 9 illustrates an example of deriving a gradient histogram.

[0675] Figure 10 illustrates an example of a template configuration.

[0676] Figure 11 illustrates an example of a juxtaposed block.

[0677] Gradient values ​​(slope values) for samples within a template area can be obtained by using at least one restored sample (or line, block, or region) adjacent to or around the current block as a template.

[0678] Gradient values ​​can be derived based on the position of the window. For example, in FIG. 9, one gradient can be derived by applying a gradient detection filter, such as a Sobel filter, to nine reference samples within the window. If the window moves by one sample, a total of 24 gradients can be derived.

[0679] In this case, the gradient value for the corresponding sample can be calculated for each of the horizontal (Gx) and vertical (Gy) directions.

[0680] The template region (or restored reference region) for calculating the above gradient histogram (HoG) can be configured as shown in FIG. 10.

[0681] For example, as shown in FIG. 10, a plurality of templates can be generated using at least one of an upper-left area, an upper area, or a left area of ​​a predetermined size around the current block (or target block).

[0682] At this time, the first template may be an example in which a template is generated using an upper-left area, an upper area, and a left area of ​​a predetermined size.

[0683] In this case, the second template may be an example where the template is generated using an upper-left area of ​​a predetermined size.

[0684] In this case, the third template may be an example in which a template is generated using an upper-left area and an upper area of ​​a predetermined size.

[0685] In this case, the fourth template may be an example in which a template is generated using an upper-left area and a left area of ​​a predetermined size.

[0686] At this time, the fifth template may be an example where the template is created using an upper area of ​​a predetermined size.

[0687] At this time, the 6th template may be an example where the template is created using a left area of ​​a predetermined size.

[0688] At this time, the 7th template may be an example in which a template is generated using an upper area and a left area of ​​a predetermined size.

[0689] Template matching may be performed using at least one of the above templates. In this case, the template information used for template matching may be a value pre-set in the encoder / decoder, or a value signaled from the encoder to the decoder.

[0690] The determination of the size and shape of the above template can be performed using the encoding and decoding information of the current block or surrounding blocks.

[0691] At this time, the size and shape of the template can be varied depending on the size of the current block or surrounding blocks.

[0692] In determining the template size based on the above encoding information, the template size can be different depending on the block partitioning information.

[0693] For example, the size of the template located on the side with the larger area in the split block can be made larger.

[0694] In constructing the above template, the template can be constructed using the motion vectors or block vectors of surrounding or collated blocks (Fig. 11).

[0695] For example, a template can be constructed based on the location indicated by the above motion vector or block vector.

[0696] An angular mode can be derived based on the orientation derived from the horizontal gradient and vertical gradient values.

[0697] Orient = Gy / Gx

[0698] The calculated direction or direction value above can be mapped to a direction prediction mode having the same or close direction, and the direction amplitude for the mapped direction prediction mode can be determined. By accumulating the direction amplitude values ​​for the same direction prediction mode, the histogram amplitude (i.e., HoG value) for the corresponding direction prediction mode can be derived.

[0699] Here, the accumulated amplitude of the mapped prediction mode can increase by a predefined value. For example, if the predefined value is 1, the accumulated amplitude of the directional prediction mode can increase by 1 in correspondence with one mapping of the directional prediction mode. If the predefined value is 1, the accumulated amplitude of the specific directional prediction mode is accumulated by 1 each time the specific directional prediction mode is mapped, so the accumulated amplitude of the specific directional prediction mode may represent the number of occurrences of the specific directional prediction mode.

[0700] Alternatively, the cumulative size of the mapped prediction mode can be increased by a variable value. For example, the increase in the cumulative size of a directional prediction mode can be set based on the amplitude of the gradient. For instance, the cumulative size of a specific directional prediction mode is (Gx2 +Gy 2 ) 1 / 2 It can increase by that amount. The variable value can be defined as the product of the number of occurrences of the directional prediction mode and the weight, and the weight can be determined according to the gradient magnitude.

[0701] Figure 21 illustrates examples of deriving gradient histograms in a restored reference region.

[0702] In calculating the gradient histogram (HoG), the gradient histogram (HoG) can be derived by configuring at least one reference sample group within a pre-specified template or a restored reference region as shown in FIG. 21. Here, the reference sample group may refer to a pre-restored reference sample, a reference sample line, or a reference sample region. The reference sample groups may be configured as the templates in FIG. 10.

[0703] For example, not only can gradient histograms be derived for each line within a template area or a restored reference area, but gradient histograms derived for each line can also be combined and used.

[0704] For example, a gradient histogram can be derived for each line, and then a weighted sum can be applied to derive a merged gradient histogram (Merged HoG).

[0705] As a first embodiment regarding the weights of a weighted sum, the weights of the weighted sum may vary depending on the distance from the current block. For example, a higher weight may be given to the gradient histogram obtained from a line closer to the current block. For example, a weight of 4 may be assigned to the first adjacent line, 3 to the second line, 2 to the third line, and 1 to the fourth line.

[0706] As a second embodiment regarding the weights of a weighted sum, after configuring a template including each line, the template matching cost can be calculated and the weights can be determined based on the size of the template matching cost.

[0707] · W_L1 (Weight for HoG derived from Line 1) = 1 - (Tmcost_L1 / (Tmcost_L1 + Tmcost_L2))

[0708] · W_L2 = 1- W1

[0709] In this case, Tmcost_L1 may represent the template matching cost of a template including the first line (line 1). Here, as a method for calculating the template matching cost, a HoG value is calculated from a reference template composed of surrounding restoration lines or restoration samples, at least one in-frame prediction mode is selected using the calculated HoG value (e.g., selected in order of increasing gradient amplitude), prediction samples of the reference template are derived using the surrounding samples of the reference template and the selected in-frame prediction mode, and the template matching cost may be calculated based on a comparison between the prediction samples of the reference template and the previously restored samples of the reference template. As another method for calculating the template matching cost, prediction samples of the surrounding area of ​​the current block (e.g., a template adjacent to the current block among the templates) are derived using the in-frame prediction mode derived from the HoG of the reference template, and the template matching cost may be calculated based on a comparison between the prediction samples of the surrounding area and the previously restored samples. As types of template matching costs, Sum of Absolute Differences (SAD), Sum of Squared Errors (SSE), Mean Absolute Difference (MAD), Sum of Absolute Transformed Differences (SATD), Hadamard Absolute Difference (HAAD), or Mean Reduced-SAD (MR-SAD) between predicted samples and reconstructed samples may be used.

[0710] For example, among the gradient histograms (HoG) calculated line by line or the prediction modes derived through the gradient histograms, a combined gradient histogram can be derived by comparing them line by line and collecting only the common gradient histograms (HoG) or common prediction modes.

[0711] When configuring gradient histograms for the above multiple lines or pixel groups, an optimal reference line or optimal pixel group can be determined based on the template matching cost.

[0712] For example, a prediction mode can be derived using a gradient histogram calculated for each line or each pixel group, and the derived prediction mode can be applied to a template area to calculate the template matching cost. Then, the optimal line or pixel group can be determined by comparing the calculated template matching sizes. In this case, the reference line or pixel group having the smallest template matching cost can be selected.

[0713] When deriving a gradient histogram for each line (or pixel / sample group) as described above, the same prediction mode can be selected from the prediction modes derived for each line, and at least one prediction mode can be selected among them to be used as a prediction mode for DIMD prediction.

[0714] The selection of a line or pixel group for calculating the above gradient histogram can be determined through MRL (multiple reference line) prediction.

[0715] For example, in MRL prediction, a gradient histogram can be derived from the optimally determined reference line and used for prediction.

[0716] For example, if the optimal reference line determined in the MRL prediction is L, a gradient histogram can be derived by selecting at least one of the surrounding reference lines, such as L-1, L-2, L-3, L-4, L+1, L+2, L+3, and L+4, based on this.

[0717] The size of the reference lines or pixel groups for deriving (calculating) the above gradient histogram can be varied depending on the block size or shape.

[0718] For example, as the block size increases, more lines or pixel groups containing more pixels can be assigned. If the number of samples in the block is less than the threshold, three reference lines may be used, and if the number of samples in the block is greater than the threshold, four reference lines may be used.

[0719] For example, by comparing the width and height of a block, you can assign pixel groups containing more lines or pixels to the longer side.

[0720] In deriving a gradient histogram representing the cumulative magnitudes of directional prediction modes, weights for the increase in the cumulative magnitude of the directional prediction modes may be different. Examples regarding weights may also be applied to OBIC modes, i.e., HoC (Histogram of Occurrence).

[0721] In this case, if weights are not considered, the cumulative size of the directional prediction mode derived from a single reference sample can be 1.

[0722] At this time, the weight applied to the increase in the cumulative size of the directional prediction mode can be determined based on the distance between the current block and the reference sample.

[0723] For example, a higher weight can be given to the increase in the cumulative size of the directional prediction mode derived from a reference sample located closer to the current block. The cumulative size of the directional prediction mode can increase by the weight value.

[0724] For example, the distance weight can be any one of 4, 2, 1, or 0.

[0725] In this case, different weights can be applied to the increase in the cumulative size of the directional prediction mode depending on the position of the reference sample.

[0726] For example, a higher weight can be given to the increase in the cumulative size of the directional prediction mode derived from samples located within the current block and adjacent blocks.

[0727] The number of reference samples / reference lines for calculating the gradient histogram (HoG) within the restored reference region can be determined according to the magnitude (value) of the directionality derived from the reference samples / lines.

[0728] In this case, the magnitude (value) of the directionality can be defined as the sum of the derived gradient histograms, the cumulative sum of the magnitudes per gradient, or the cumulative (or summation) of the number of occurrences per corresponding prediction mode.

[0729] Figure 25 illustrates an example of the accumulation of HoG by mode.

[0730] Figure 26 illustrates an example of a change in the reference sample group according to a comparison of HoG and thresholds by mode.

[0731] For example, referring to FIGS. 25 and 26, the number of surrounding reference samples / reference lines can be increased until the calculated directional magnitude (value) exceeds a certain threshold to induce an additional gradient (or gradient histogram). This may be because, when the directional magnitude is below a certain threshold, it is determined that there is insufficient dominant directionality information necessary to induce an in-frame prediction mode for the block from surrounding pixels, so increasing the number of surrounding reference samples or lines may allow for obtaining sufficient directional information.

[0732] When the cumulative size of the directional prediction mode is compared with a threshold, the threshold can be determined based on the size of the template, the size of the current block, or the size of surrounding blocks, etc.

[0733] The threshold can be the product of the template size and the scaling factor. The scaling factor can be determined based on the template size, the size of the current block, or the size of surrounding blocks, etc. For example, if the vertical length of the top template is longer than 8 samples and the horizontal length of the top template is equal to or longer than the horizontal length of the current block, the scaling factor can be set to 2200. Otherwise, the scaling factor can be set to 1700. For example, if the horizontal length of the left template is longer than 8 samples and the vertical length of the left template is equal to or longer than the vertical length of the current block, the scaling factor can be set to 2200. Otherwise, the scaling factor can be set to 1700.

[0734] In determining the number of reference samples / reference lines for deriving a gradient histogram, the number of reference samples (pixels) or lines required for gradient calculation can be increased and the amount of increase determined by separating (or distinguishing) them by template.

[0735] For example, as shown in Fig. 26, the directional size can be calculated for each template by distinguishing between the left template and the top template, and compared with a specific threshold to determine whether to increase the number of surrounding samples (pixels) or lines required for gradient calculation for each template and the amount of increase.

[0736] The length of an increased reference line can be equal to the length of an adjacent reference line or increase by 1. For example, if the reference line of the top template increases, the new reference line can be positioned above the existing reference line. The length of the new reference line can be equal to the length of the existing reference line or equal to the length of the existing reference line plus 1. If the reference line of the left template increases, the new reference line can be positioned to the left of the existing reference line. The length of the new reference line can be equal to the length of the existing reference line or equal to the length of the existing reference line plus 1.

[0737] In deriving a directional magnitude (value) for determining whether to increase the number of reference samples (pixels) or lines and the amount of increase, a directional magnitude (value) for comparison with a threshold can be derived by using only the cumulative magnitude of a specific gradient histogram or a specific prediction mode.

[0738] For example, based on the cumulative size of the directional prediction modes, only a specific number (e.g., 5) of gradient histograms or prediction modes from the top can be selected to obtain the magnitude (value) of the directionality for comparison with the threshold.

[0739] For example, the number of reference samples or lines can be determined by comparing the sum of the cumulative sizes of all directional prediction modes with a threshold.

[0740] When increasing the number of reference samples or reference lines, the number of samples or lines to be increased may be limited.

[0741] At this time, the number of samples and lines to be increased can be kept within a specific threshold.

[0742] For example, the threshold for reference samples may be up to 20 samples. Or, the threshold for reference lines may be up to 12 lines or up to 20 lines.

[0743] At this time, the number of reference samples or reference lines to be increased can be varied depending on the block size.

[0744] For example, the number of samples to increase can be determined using the width and height of the block. For example, the number of reference samples or reference lines to increase can be limited by multiplying the sum of the width and height or the product of the width and height by a certain weight.

[0745] · Maximum number of reference samples = weight * (width * height) or weight * (width + height) can be

[0746] For example, as the size of the block increases, the limit on the number of reference samples or reference lines can be increased.

[0747] If the number of samples in a block is 512 or more, the limit on reference lines can be increased up to 32 lines, and otherwise (if the number of samples in a block is less than 512), the maximum number of reference lines can be limited to 8 lines.

[0748] In the gradient calculation for deriving a gradient histogram, if the magnitude of the derived directionality is above a certain threshold, and it is determined that there is sufficient directionality information necessary to derive the prediction mode for the corresponding block from surrounding reference samples, the gradient calculation for the remaining reference samples / lines can be omitted.

[0749] In deriving a gradient histogram, the reference samples or reference lines referenced for gradient calculation may be filtered and used.

[0750] Figure 27 illustrates examples of a smoothing filter and a sharpening filter.

[0751] For example, a reference sample or reference line can be filtered using smoothing based on a Gaussian filter or a low-pass filter as shown in Fig. 27.

[0752] For example, a reference sample or reference line can be filtered using a sharpening filter such as the Laplacian or the Laplacian of Gaussian shown in Fig. 27.

[0753] For example, reference samples or reference lines can be filtered using edge-preserving filters such as bilateral filters.

[0754] At this time, when filtering a reference sample or reference line using the above filter, multiple filters may be applied sequentially. For example, after primary filtering is performed using a smoothing filter based on a Gaussian filter or a low-pass filter, additional filtering may be performed using an edge preserving filter such as a sharpening filter or a bilateral filter, such as a Laplacian or Laplacian of Gaussian.

[0755] For example, a reference sample or reference line can be filtered using at least one downsampling filter.

[0756] For example, reference samples or lines can be filtered using component-to-component prediction models derived from adjacent / non-adjacent blocks within a referenceable current / reference picture.

[0757] For the above filtering, at least one filter may be selected to perform the filtering. When multiple filters are used to filter a reference region or reference line, a separate gradient histogram may be derived and used for each filter. Alternatively, the derived gradient histograms may all be summed (or weighted summed) and used.

[0758] In the above example, whether filtering is applied and the filter information used (filter index or filter coefficient, offset) may be pre-set values ​​in the encoder / decoder, or may be signaled from the encoder to the decoder.

[0759] In determining whether to perform filtering, when the magnitude (value) of the directionality derived from the reference sample or reference line prior to filtering is below a specific threshold, it is determined that there is insufficient characteristic / dominant directionality information necessary to derive an in-frame prediction mode for the corresponding block from surrounding pixels, so filtering of the surrounding reference sample or reference line may be performed.

[0760] In deriving a gradient histogram, at least one edge detection filter can be used to derive the gradient (or gradient value).

[0761] At this time, gradients (values) can be derived using multiple filters, and gradient histograms can be derived using the derived gradients. In this case, the derived gradient histograms can be constructed separately for each filter used, or they can be combined to form a single integrated gradient histogram.

[0762] When constructing a gradient histogram using the aforementioned multiple filters, a prediction mode can be derived using the gradient histogram configured for each filter, and a prediction block can be generated using the derived prediction mode. At this time, after performing prediction coding using the prediction block generated for each filter, an optimal gradient detection filter among the filters used can be determined, and information regarding whether multiple filters are used and the optimal filter (filter index) can be signaled from the encoder to the decoder.

[0763] When using the above multiple filters, the optimal filter can be determined based on the magnitude (value) of the directionality induced for each filter.

[0764] For example, a filter that derives the largest directional magnitude (value) can be determined as the optimal filter, and a prediction signal can be generated using the prediction mode derived therefrom.

[0765] In determining the above gradient detection filter, the size of the filter (kernel size) can be varied depending on the size and division type of the current block.

[0766] Figure 28 illustrates examples of a 2x2 filter and a 3x3 filter.

[0767] For example, if the product of the width and height of the current block is less than 32, a 2x2 filter can be used, and if it is greater, a 3x3 filter can be used.

[0768] For example, if the product of the width and height of the current block is less than 128, a 2x2 filter can be used, and if it is greater than that, a 3x3 filter can be used.

[0769] In determining whether to use the above multiple filters, a gradient histogram may be derived first using one filter (e.g., a 3x3 filter), and when the sum of the derived gradient (or corresponding prediction mode) histograms (or occurrence counts) is below a certain threshold, it is determined that there is insufficient dominant directionality information required to derive an in-frame prediction mode for the block from surrounding pixels, so another filter may be used to derive an additional gradient histogram.

[0770] When using the above multiple filters, prediction modes can be derived using gradient histograms calculated for each filter, and at least two of the prediction modes derived for each filter can be selected and weighted summed to generate a new prediction signal.

[0771] For example, when using two filters, a prediction signal can be generated using the best mode and secondary mode among the prediction modes derived for each filter, and a new prediction signal can be generated by weighting and summing them.

[0772] Figure 11 illustrates an example of a juxtaposed block.

[0773] Figure 12 illustrates examples of surrounding blocks in the current image and surrounding blocks in the reference image.

[0774] Figure 13 illustrates an example of surrounding blocks adjusted (moved) using block vectors.

[0775] The gradient histogram (HoG) can be used by inheriting the gradient histogram (HoG) calculated from at least one surrounding block as well as the one calculated from the template of the current block. Related diagrams may be FIGS. 11 to 13.

[0776] In one example, a gradient histogram can be inherited from a block at the following location.

[0777] (1) Spatially adjacent location

[0778] (2) Spatial non-adjacent

[0779] (3) Temporal adjacent / non-adjacent + juxtaposed positions

[0780] At this time, positions adjusted by block vectors or motion vectors in sub-block units, such as sbTMVP, may also be included.

[0781] (4) History-based location

[0782] At this time, a FIFO list is defined, and gradient histograms (HOG) are stored in the list in the order of encoding / decoding, and history-based positions can be used in the order of entry.

[0783] (5) Temporal / spatial adjacent / non-adjacent positions adjusted using block vectors / motion vectors

[0784] At this time, the position of blocks within the same picture as the block from which the block vector / motion vector was derived can be adjusted.

[0785] At this time, the block vector / motion vector can adjust the position of blocks within a different picture from the derived block.

[0786] At this time, the position can be readjusted for all available candidate block positions regardless of the picture to which the block derived from the block vector / motion vector belongs.

[0787] In one embodiment, as shown in FIG. 29, a new candidate (vector) can be derived by using a block vector or motion vector stored in a block at a location indicated by a previously derived block vector or motion vector in a list.

[0788] At this time, as shown in FIG. 29, when a block vector is again present in the block at the position moved using the block vector in the list, a new candidate vector can be derived by adding (accumulating) the block vector of the previously derived block vector and the block vector of the position moved using it.

[0789] At this time, as shown in FIG. 29, BV m,n ul BLK m BLK n When defined as a block vector indicating the location to, BV 0,1 (Assuming the current block is BLK0) can be described as a block vector already derived within a list that indicates the location of reference block 1 within the current block. Here, BV 0,1 Let BLK1 be the block moved using [this], and BV pointing from BLK1 to BLK2 1,2 If BV exists 0,1 and BV 1,2 Using , a new block vector BV indicating BLK2 from the current block BLK0 0,1 It can induce.

[0790] At this time, if a block vector exists within the block at the position moved using the derived block vector, the trace depth can be increased. For example, if the block at the position moved using the block vector in the list has the block vector, the trace depth is 1, and if the block at the position moved again using the block vector of that block has the block vector, the trace depth can be increased by 1.

[0791] At this time, when the trace depth is 'd', the block vectors that can be derived for each trace depth based on the block vectors in the merge / AMVP list of the current block can be defined as follows.

[0792] BV 0,d+1 = BV0,d + BV d,d+1 = BV 0,1 + BV 1,2 + ... + BV d-1,d + BV d,d+1

[0793] At this time, when deriving the new block vector candidate, at least one of the block vectors derived according to tracking depth can be selected and added to the list.

[0794] A method for deriving a new motion vector or block vector by repeatedly accumulating the block vector or motion vector derived from the position moved using the above motion vector or block vector can be defined as follows.

[0795] Method for deriving block vectors: Auto-relocated block vector prediction (AR-BVP) Method for deriving motion vectors: Chained motion vector prediction (CMVP)

[0796] In one embodiment, a gradient histogram inherited from the surroundings can be used without calculating the gradient histogram (HoG) for the current block.

[0797] In one embodiment, the gradient histogram (HoG) derived from the current block and the gradient histogram inherited from the surroundings can be combined and used.

[0798] In one embodiment, a prediction signal can be generated by inducing at least one prediction mode through a gradient histogram (HoG) inherited from the defined locations ((1)-(5)).

[0799] In one embodiment, when inheriting and using gradient histogram information, gradient histogram information can be inherited and used from surrounding blocks determined by a specific prediction mode.

[0800] For example, gradient histogram information can be inherited and used only from reference blocks determined by DIMD.

[0801] For example, gradient histogram information can be inherited and used only from reference blocks determined by DIMD and TIMD.

[0802] For example, gradient histogram information can be inherited and used only in blocks determined by DIMD or DIMD merge.

[0803] The size of a merge list constructed by inheriting a gradient histogram can be limited to a specific size. For example, the size can be limited to a maximum of 12.

[0804] The calculation (or derivation) of the gradient histogram (HoG) can be performed to a limited extent.

[0805] For example, to reduce complexity, the gradient histogram (HoG) can be calculated only on blocks of a specific block size or larger. For example, the calculation for the gradient histogram (HoG) can be performed only on blocks of a size of 8x8 blocks or larger.

[0806] For example, gradient histograms can be calculated only in blocks where the number of samples within the block is 64 or more.

[0807] For example, calculations for deriving the gradient histogram (HoG) for the current block can be performed only when the prediction mode of the adjacent block is a prediction mode determined using DIMD or the gradient histogram (HoG).

[0808] At least one gradient histogram among the gradient histograms (HoG) inherited from the plurality of surrounding blocks can be selected and combined to derive a new merged gradient histogram (Merged HoG).

[0809] In this case, a new integrated gradient histogram can be derived by selecting only the surrounding blocks of a specific location, inheriting their gradient histograms, and combining them.

[0810] For example, at least one reference block can be selected from the surrounding block locations (1) to (5) above to inherit a gradient histogram and combine them to derive an integrated gradient histogram.

[0811] For example, a combined gradient histogram can be derived by inheriting only the gradient histograms of the current block and adjacent blocks.

[0812] For example, a combined gradient histogram can be derived by inheriting only the gradient histograms of the current block and non-adjacent blocks.

[0813] For example, a combined gradient histogram can be derived by inheriting only the gradient histograms of the blocks located on the left among adjacent blocks.

[0814] At this time, only the surrounding blocks determined by a specific prediction mode can be selected, and gradient histogram information can be inherited from those blocks to derive an integrated gradient histogram.

[0815] For example, an integrated gradient histogram can be derived using only the reference blocks determined by DIMD.

[0816] For example, an integrated gradient histogram can be derived using only the reference blocks determined by DIMD and TIMD.

[0817] For example, gradient histogram information can be inherited and used only in blocks determined by DIMD or DIMD merge.

[0818] At this time, the number of surrounding blocks used for constructing the integrated gradient histogram among all available surrounding blocks can be limited to a specific size. For example, the maximum number of surrounding blocks used for constructing the integrated gradient histogram can be limited to 12.

[0819] The above integrated gradient-based merge prediction can be used to a limited extent.

[0820] For example, integrated gradient-based merge prediction can be omitted for blocks of a specific size. For example, the merge prediction can be omitted for 4x4 blocks.

[0821] For example, integrated gradient-based merge prediction can be performed only when the surrounding blocks are in a prediction mode determined using DIMD or gradient histogram (Merged HoG).

[0822] The generation of the above Merged Gradient Histogram (Merged HoG) can be performed in a limited manner.

[0823] For example, to reduce complexity, a merged gradient histogram can be generated only in blocks of a specific block size or larger. For example, a merged gradient histogram (Merged HoG) can be generated only in blocks of a size of 8x8 blocks or larger.

[0824] For example, a prediction mode can be derived by generating a Merged Gradient Histogram (Merged HoG) only when the merge list of limited size is empty.

[0825] The gradient histogram (HoG) information calculated in the current block above can be stored and used.

[0826] At this time, the gradient histogram (HoG) information stored in the current block may include gradient histogram values ​​(cumulative occurrence or magnitude for each directional element or corresponding directional prediction mode), template information from which the gradient values ​​were obtained, etc.

[0827] At this time, gradient values ​​can be stored separately depending on the type or location of the template used to calculate the gradient.

[0828] For example, gradient histograms can be calculated using gradient values ​​derived from the left, top, and left and top (L-shape) respectively, and saved separately. (Save by template position / direction)

[0829] In this case, if there are directions for which no available restoration samples are available, the gradient histogram stored for each template position can be used.

[0830] For example, if there are no available restoration samples to the left of the current block or if a template cannot be constructed, gradient histogram (HOG) information can be inherited and used from neighboring blocks. In this case, only the gradient histogram information calculated from the left template can be inherited and used.

[0831] In storing the above-mentioned calculated gradient histogram (HoG) information, it can be stored by converting it into a directional prediction mode corresponding to the calculated gradient histogram (HoG) value. Additionally, as in the previous example, a specific prediction mode determined in the current block can be converted into a gradient histogram (HoG) and stored.

[0832] At this time, at least one of the orient elements or corresponding angular mode information on the gradient histogram can be selected and stored.

[0833] For example, the values ​​included in the orient element or corresponding angular mode information on the gradient histogram may be the angular mode value (index) and the histogram value (cumulative size) for each element (angular mode).

[0834] In storing the above-mentioned directional element information or corresponding directional mode information, the directional element or directional mode to be stored can be determined based on the cumulative size on the histogram.

[0835] For example, after sorting in descending order of accumulated count, N elements (or directional prediction mode) with the highest accumulated counts can be selected and stored. In this case, the number of elements N to be stored may be a pre-configured value in the encoder / decoder or a value signaled from the encoder to the decoder.

[0836] For example, only elements whose accumulated count is greater than a certain threshold can be selected and stored. In this case, the threshold may be a pre-set value in the encoder / decoder or a value signaled from the encoder to the decoder.

[0837] When storing directional element information on the above gradient histogram or the cumulative size for each corresponding directional mode, it can be stored in units of specific sizes such as upper size block units, CTU units, pictures, tiles, or slices.

[0838] For example, gradient histograms calculated in sub-partitions at the CTU level or the cumulative size for each directional element (by directional mode) can be summed and stored.

[0839] For example, the cumulative size for each element (by directional mode) can be summed and stored in MxN block units. In this case, the stored width and height, M and N, may be pre-set values ​​in the encoder / decoder or values ​​signaled from the encoder to the decoder.

[0840] For example, it can be stored in minimum block size units. For example, the minimum block unit can be 4x4.

[0841] For example, it can be stored in units of a maximum block size. For example, the minimum block unit can be 256x256.

[0842] When storing directional element information on the above gradient histogram or the cumulative size for each corresponding directional mode, the maximum number of cumulative times for each directional element (or directional mode) can be limited.

[0843] At this time, the maximum cumulative count for each prediction mode (directional element) can be limited to M. For example, it can be limited to a maximum of 32 / 64 / 128 / 256. At this time, the count M can be a pre-set value in the encoder / decoder or a value signaled from the encoder to the decoder.

[0844] In storing the above gradient histogram information, the size of the sum of the cumulative sizes on the entire gradient histogram can be limited.

[0845] When storing directional element information on the above gradient histogram or the cumulative size for each corresponding directional mode, the cumulative size for each directional element (for each directional mode) can be scaled and used.

[0846] In this case, scaling can be expressed as adding, subtracting, dividing, or multiplying a specific value to the accumulated size.

[0847] For example, the cumulative size for each directional element (direction mode) can be divided by N and stored. For example, N can be 2.

[0848] For example, the cumulative size for each directional element (direction mode) can be multiplied by M and then stored. For example, M can be 0.5.

[0849] For example, the average value of the entire histogram can be subtracted from the cumulative magnitude for each directional element (direction mode) and stored. In this case, the average value of the entire histogram can be the average of the cumulative magnitudes for each directional element (direction mode).

[0850] [STEP 2] Generate Predicted Blocks

[0851] At least one directional prediction mode can be derived through the gradient histogram (HOG). In this case, a prediction block can be generated using the derived prediction mode information, and prediction encoding and decoding can be performed.

[0852] When multiple reference sample groups are generated from a template area or a restored reference area, a prediction mode can be derived based on the gradient histograms of the reference sample groups. Multiple in-frame prediction modes corresponding to each reference sample group are derived based on the gradient histogram of each reference sample group, and a final prediction mode can be selected based on the template matching cost.

[0853] Multiple prediction modes can be derived through the gradient histogram (HoG), and a new prediction block (signal) can be generated through weighted sum, fusion, or blending thereof.

[0854] At this time, the prediction mode to be used for the weighted sum can be determined based on the cumulative magnitude of each directional mode on the histogram.

[0855] For example, after sorting in descending order of accumulated count, N prediction modes with the highest accumulated counts can be selected and used. In this case, the number of selected prediction modes N may be a pre-configured value in the encoder / decoder or a value signaled from the encoder to the decoder.

[0856] For example, only directional prediction modes (directional elements) with an accumulated count greater than a certain threshold can be selected and used. In this case, the threshold may be a pre-configured value in the encoder / decoder or a value signaled from the encoder to the decoder.

[0857] At this time, the prediction mode to be used for the weighted sum can be determined based on the encoding cost.

[0858] In this case, the encoding cost refers to the cost calculated to calculate the difference between the original block (signal) and the prediction block generated using the directional prediction mode derived from the gradient histogram (HoG), or to encode the residual signal generated through this. For example, there are SAD, SSE, SATD, which represent the error values ​​between two signals, and the rate-distortion cost, which is the cost required to encode the residual signal added to the error values.

[0859] In the prediction mode determination based on the above-mentioned encoding cost, after generating a prediction block for each prediction mode, the encoding cost is calculated through prediction encoding, and the prediction modes to be used for the weighted sum can be determined according to the encoding cost.

[0860] For example, after sorting in order of decreasing encoding cost, at least one prediction mode with a low encoding cost can be selected and used. In this case, the information of the selected prediction mode can be signaled from the encoder to the decoder.

[0861] For example, only prediction modes (directional elements) with an encoding cost smaller than a certain threshold can be selected and used. In this case, the information on the selected prediction modes can be signaled from the encoder to the decoder.

[0862] The generation of a prediction signal using the above weighted sum can be performed optionally.

[0863] For example, it may not be applied to color difference signals. In other words, the generation of a prediction signal using a weighted sum can be performed selectively depending on the components.

[0864] For example, from the perspective of rate-distortion optimization, the optimal method can be determined by comparing a method using a weighted sum with a method not using one, and information related to the optimal method can be signaled from the encoder to the decoder.

[0865] At this time, in the prediction mode determination based on coding cost, a prediction block is generated for each prediction mode, the coding cost is calculated through prediction coding, and the prediction modes to be used for the weighted sum can be determined according to the coding cost.

[0866] At this time, information regarding whether the weighted sum method is used and the prediction mode used in the weighting can be signaled from the encoder to the decoder.

[0867] At this time, the prediction mode to be used for the weighted sum can be determined based on the error values ​​between the prediction blocks generated using multiple prediction modes derived through the gradient histogram (HoG).

[0868] For example, if there are 5 prediction modes derived through the gradient histogram (HoG), a prediction block can be generated for each of the 5 prediction modes and the error values ​​(SAD, SSE, SATD) between each prediction block can be calculated.

[0869] Here, the error between prediction block 1 and each of prediction blocks 2 to 5, the error between prediction block 2 and each of prediction blocks 3 to 5, the error between prediction block 3 and each of prediction blocks 4 to 5, and the error between prediction block 4 and prediction block 5 can be composed of a total of 10 types.

[0870] Among the above prediction modes, at least one pair with a small error value can be selected to determine the prediction mode to be used for the weighted sum.

[0871] At this time, pairs with error values ​​smaller than a certain threshold can be selected to determine the prediction mode to be used for the weighted sum.

[0872] At this time, pairs with error values ​​greater than a certain threshold can be selected and excluded from the prediction mode to be used for the weighted sum.

[0873] In one embodiment, when there are multiple determined directional prediction modes, a prediction block can be generated for each determined directional prediction mode and a new prediction block can be generated by weighting the sums thereof.

[0874] In one embodiment, when generating a prediction block through a weighted sum, the prediction block can be generated through a weighted sum of the prediction block determined through the gradient histogram (HoG) and the prediction block generated through a different prediction method.

[0875] At this time, other prediction methods may include prediction methods other than “intra-frame prediction mode derivation (DIMD) on the decoder side,” such as inter-frame prediction, intra-frame template matching (Intra TMP), intra-block copy (IBC) prediction, matrix-weighted intra prediction (MIP), and non-directional prediction including planar and DC.

[0876] At this time, in generating a prediction block through the weighted sum above, not only the angular prediction mode determined through DIMD but also non-directional prediction modes such as DC, Planar, IntraTMP, and IBC can be used. In addition, for the color difference signal, a new prediction block can be generated through a weighted sum with a prediction block generated through component-to-component prediction (CCLM, GLM, CCCM, etc.).

[0877] For example, a new prediction block can be created by weighting a single prediction mode determined based on the template matching cost among N (e.g., 5) prediction modes derived in the DIMD process and non-directional prediction modes. In this case, a new prediction block can be created by weighting a two prediction blocks derived through a gradient histogram and a prediction block generated through planar prediction.

[0878] At this time, the non-directional prediction mode used in the weighted sum can be replaced with a prediction signal (block) generated by utilizing the prediction mode of the reference block at the moved position using the block vector (BV) derived through IBC or IntraTMP, or with a restored sample of the corresponding reference block position.

[0879] At this time, the non-directional prediction mode used in the weighted sum can be replaced with a prediction signal generated by utilizing the prediction mode of the prediction block at the position moved using a motion vector (MV) derived from the current block or surrounding blocks instead of the block vector, or with a restored sample of the corresponding reference block position.

[0880] The prediction signal derived using the above block vector or motion vector can be used in a weighted sum with a non-directional prediction mode. For example, the prediction signal derived using the block vector or motion vector can be used together in the weighted sum process without replacing the non-directional prediction mode.

[0881] At this time, among the prediction modes derived through the above process, the number of prediction modes to be used for the weighted sum may vary depending on the block size or the number of samples constituting the block. The number of derived prediction modes may be equal to the number of prediction modes to be used for the weighted sum. That is, the number of derived prediction modes may be determined according to the block size or the number of samples constituting the block.

[0882] For example, as the size of the block or the number of samples constituting it increases, the number of prediction modes used for the weighted sum can be increased.

[0883] For example, if the number of samples constituting the block is 128, up to 11 prediction modes can be used for the weighted sum. If the number of samples constituting the block is less than 128, up to 5 prediction modes can be used for the weighted sum.

[0884] In the generation of a prediction signal through the above weighted sum, when generating a prediction signal through planar prediction among non-directional prediction modes, a prediction signal can be generated using a directional planar mode instead of a general planar prediction according to the prediction direction of the directional prediction mode determined in DIMD.

[0885] For example, if the prediction mode determined by DIMD is near horizontal, a prediction signal can be generated through horizontal planar prediction. (e.g., 2 <= DIMD mode index < 34)

[0886] For example, if the prediction mode determined by DIMD is close to vertical (near vertical), a prediction signal can be generated through vertical planar prediction. (e.g., 34 <= DIMD mode index < 66)

[0887] In generating a prediction signal using the above directional planar mode, weights for each vertical planar and horizontal planar are determined according to the prediction direction of the directional prediction mode determined in DIMD, and the vertical planar and horizontal planar can be used by weighted sum.

[0888] [blended planar] At this time, depending on the directional range of the DIMD mode in the above example, it is determined whether the DIMD mode is close to horizontal (near horizontal) or vertical (near vertical), and a planar prediction signal can be generated by applying different weights accordingly.

[0889] For example, if it is determined that the directionality of the DIMD mode is close to horizontal, when generating a planar prediction signal by the weighted sum of directional planar modes, a higher weight can be given to the horizontal planar as follows.

[0890] blended planar = (w1 * vertical_planar + w2*horizontal_planar) >> 2, w1= 3, w2 = 1)

[0891] In determining the non-directional prediction mode during the process of generating a DIMD prediction signal through the above weighted sum, not only the prediction mode derived using planar, DC, and block vector (BV: block vector), but also directional planar (vertical planar, horizontal planar) and blended planar can be added to the prediction mode to determine the non-directional prediction mode to be used for generating the final DIMD prediction signal.

[0892] At this time, when determining the non-directional prediction mode, a plurality of prediction modes among the aforementioned prediction modes can be selected to calculate the template matching cost for each mode, and the prediction mode having the smallest template matching cost can be determined as the non-directional prediction mode to be used for the final weighted sum.

[0893] In determining the above candidate non-directional prediction mode, the candidate prediction mode to be used for template matching can be varied depending on the prediction direction of the prediction mode determined by DIMD.

[0894] For example, if the induced DIMD prediction mode is close to vertical (near vertical), the candidate prediction mode can be determined as follows.

[0895] · Candidate 1: DC

[0896] · Candidate 2: Block vector based prediction mode

[0897] · Candidate 3: Planar

[0898] · Candidate 4: vertical planar

[0899] · Candidate 5: blended planar = (w1 * vertical_planar + w2*horizontal_planar) >> 2, w1= 3, w2 = 1)

[0900] In the above example, if the DIMD prediction mode derived is close to horizontal (near horizontal), a candidate can be constructed by replacing the vertical planar with a horizontal planar, and giving greater weight to the horizontal planar in the blended planar.

[0901] The method of deriving a prediction signal through the directional planar mode in the above example can be applied to all methods using planar prediction.

[0902] For example, the above method can also be applied in the process of weighted sum of the prediction signal generated through the previously derived TIMD prediction mode and the prediction signal generated through planar prediction in TIMD fusion.

[0903] At this time, as in the example above, a directional planar mode or a blended planar mode to be used for the weighted sum can be determined based on the prediction direction of the best mode or secondary mode of TIMD.

[0904] In the above weighted sum process, the weights may be equal or differential.

[0905] The weights to be used in the above weighted sum process may be predefined weights.

[0906] In this case, multiple weight sets can be defined and used, and the optimal weight can be selected for each weight set in terms of encoding cost. At this time, information such as the set index of the selected weight set can be signaled from the encoder to the decoder.

[0907] For example, when the weights for two prediction blocks are denoted as w0 and w1 respectively, the weight sets can be determined as follows: 1st: {w0, w1}={1 / 4, 3 / 4}, 2nd: {w0, w1}={3 / 4, 1 / 4}, 3rd: {w0, w1}={1 / 2, 1 / 2}.

[0908] At this time, the above weight set can be distinguished and used according to the prediction mode information of the surrounding blocks.

[0909] For example, if both the left and top blocks are blocks encoded in component-to-component prediction mode, set 1 can be used.

[0910] For example, if both the left and top blocks are blocks encoded in a prediction mode other than the component-to-component prediction mode, set 2 can be used.

[0911] The determination of the above weights can be performed based on the cumulative histogram size for each directional element (for each directional prediction mode).

[0912] In this case, higher weights can be assigned to prediction blocks generated through a prediction mode with a high cumulative size.

[0913] For example, if the cumulative size of prediction mode 1 is amplitude(pred1), the cumulative size of prediction mode 2 is amplitude(pred2), and their respective weights are w1 and w2, then each weight can be defined by the following formula.

[0914] · W1(wDimd1) = amplitude(pred1) / (amplitude(pred1) + amplitude(pred2))

[0915] · W1(wDimd2) = amplitude(pred2) / (amplitude(pred1) + amplitude(pred2))

[0916] At this time, the weights between the prediction block determined through the gradient histogram (HoG) and the prediction block generated by a different prediction method can be made different.

[0917] At this time, the weights for prediction blocks generated by other prediction methods are fixed, and the weights for prediction modes (blocks) determined through the gradient histogram (HoG) can be determined according to the cumulative size.

[0918] For example, when performing weighting on a block determined by planar prediction and two prediction blocks derived by gradient histogram, the weight of the block determined by planar prediction can be fixed at 1 / 3, and the weight of the remaining prediction blocks can be determined based on the cumulative size of each prediction mode on the gradient histogram (HoG).

[0919] In determining the above weights, the weights may be varied according to the size (or ratio) of the gradient histogram (HoG) for each template.

[0920] Here, the weight Wi(x, y) for the sample at sample position (x, y) within the prediction block induced by prediction mode i can be defined as in Equation (1) or Equation (2) below. (where i may be the index of the prediction mode.)

[0921] · Formula (1) W i (x, y) = wDIMD i + a i - (2*a i * y / (H-1))

[0922] · Formula (2) W i (x, y) = wDIMD i + a i - (2*a i * x / (W-1))

[0923] "wDIMD i " represents a weight based on the cumulative magnitude of each prediction mode on the previously defined gradient histogram (HoG), which may refer to the weight based on the cumulative magnitude of prediction mode i. For example, the weights of two directional prediction modes and one non-directional prediction mode derived using the HoG of DIMD can be determined as follows.

[0924] Assuming the HoG value (amplitude) of Mode 1 = 20 and the HoG value (amplitude) of Mode 2 = 10,

[0925] · W mode1 (Weight for mode1) = 2 / 3 * amplitude(mode1) / (amplitude(mode1) + amplitude(mode2)) = 2 / 3 * 20 / (20 + 10) = 2 / 3 * 2 / 3 = 4 / 9

[0926] · W mode2 (Weight for mode2) = 2 / 3 * amplitude(mode2) / (amplitude(mode1) + amplitude(mode2)) = 2 / 3 * 10 / (20 + 10) = 2 / 3 * 1 / 3 = 2 / 9

[0927] · W non-angluar (Weights for non-angular mode) = 1 / 3

[0928] · W mode1 + W mode2 + W non-angular = 1

[0929] a i is a constant value, where, a i can be 10. In this case, the HoG value may represent the cumulative number of occurrences of a specific prediction mode induced through HoG. H may represent the height of the block, and W may represent the width of the block.

[0930] In this case, the size of the constant value ai can be varied depending on the block size or the number of samples included in the block. For example, if the number of samples included in the block is greater than 128, the constant value a i The size of can be made smaller than the original value (10).

[0931] In selecting Equation (1) and Equation (2) for determining the weight for a sample located at coordinates (x, y) within the prediction block induced by the above prediction mode i, the determination can be made based on the magnitude or ratio of the gradient histogram (HoG) values ​​between the surrounding templates of the current block.

[0932] For example, if the size of the gradient histogram (HoG) of the above template is N times larger than the size of the gradient histogram (HoG) of the left template, the weight for pixel location (x, y) can be determined using Equation (1). In this case, N can be 2.

[0933] At this time, in the above equation (1), as it moves further away from the upper side (i.e., as the y value increases), the effect of the weight Wi for the pixel becoming smaller can be observed.

[0934] Conversely, if the magnitude of the gradient histogram (HoG) of the left template is N times larger than the upper side, the weight for pixel location (x, y) can be determined using Equation (2). In this case, N can be 2.

[0935] At this time, in the above Equation 2), as it moves further to the left (i.e., as the x value increases), the effect of the weight Wi for the pixel becoming smaller can be observed.

[0936] In this case, if only the top template exists, the pixel-specific weight can be derived through Equation (1).

[0937] In other words, by using only the top template, the closer it is to the top template, the greater the weight can be given to the pixels (samples).

[0938] In this case, if only the left edge template exists, pixel-specific weights can be derived through Equation (2).

[0939] In other words, by using only the left edge template, the closer a pixel (sample) is to the left edge template, the greater the weight can be given to it.

[0940] In this case, the prediction mode required to generate the prediction signal can give greater weight to pixels closer to the derived template.

[0941] For example, when a gradient histogram is derived from a template and a prediction mode is derived through it, greater weight can be given to pixels that are closer to the template from which the gradient histogram derived the prediction mode was calculated.

[0942] When determining the weights above, the weights can be determined by considering the directionality in the directional prediction mode.

[0943] For example, in Vertical mode, you can give higher weight to pixels close to the top template.

[0944] For example, in Horizontal mode, higher weight can be given to pixels close to the left edge template.

[0945] In color difference signal prediction, as in luminance signals, a color difference prediction mode can be derived using a gradient histogram (HoG) derived from surrounding templates.

[0946] At this time, a prediction signal (block) can be generated using at least one prediction mode derived through the gradient histogram (HoG).

[0947] At this time, multiple prediction modes can be induced to generate at least one prediction signal (block).

[0948] If the above-derived predicted prediction mode is the same as the prediction mode determined as the Direct Mode (DM) in color difference prediction, the prediction mode having the next highest cumulative size on the gradient histogram (HoG) can be used as a candidate mode for color difference prediction.

[0949] A new prediction block can be generated by the weighted sum of the prediction block generated using the above-derived prediction mode and the prediction block generated through component-to-component prediction (CCLM, GLM, CCCM, etc.).

[0950] In prediction methods that generate prediction signals using multiple prediction modes, such as Geometric partitioning mode (GPM), combined inter-intra prediction (CIIP), and Intra fusion, at least one prediction mode can be derived and used for prediction by using a gradient histogram (HoG) derived from a surrounding template, as in the previous example.

[0951] For example, when making predictions based on GPM, a prediction mode can be derived using a gradient histogram from a restored reference region adjacent to each partitioned block and used for prediction.

[0952] For example, when making a prediction based on GPM, at least one of the prediction modes derived from the gradient histogram can be selected and used as a prediction mode for each segmented block.

[0953] For example, when making predictions based on GPM, the prediction mode with the largest gradient histogram size and the next-ranked prediction mode can be used as prediction modes for each segmented block.

[0954] In performing prediction mode derivation (DIMD) within the decoder-side screen, a gradient histogram (HoG) can be derived by configuring multiple templates.

[0955] At this time, prediction blocks can be generated for each template by going through the process of [STEP1] / [STEP2] above, and prediction coding can be performed using each prediction block.

[0956] In the above predictive coding process, the optimal template can be determined by comparing coding costs. At this time, the optimal template information can be signaled from the encoder to the decoder.

[0957] In performing the prediction mode derivation (DIMD) within the decoder-side screen, merge prediction (Merge) for the gradient histogram (HoG) can be performed.

[0958] A merge candidate list can be constructed by inheriting the gradient histogram (HoG) at block locations where the previously defined gradient histogram (HoG) can be inherited.

[0959] An angular prediction mode can be derived through the gradient histogram (HoG) within the above merge list, and a prediction block (signal) can be generated using the derived prediction mode.

[0960] At this time, the prediction block can be generated not only using a single prediction mode but also through a weighted sum of prediction blocks generated through multiple prediction modes.

[0961] In deriving a directional prediction mode using a gradient histogram (HoG) within the above merge list, at least one gradient histogram (HoG) within the merge list can be selected to derive a directional prediction mode.

[0962] At this time, multiple gradient histograms (HoG) within the list are selected and combined (or weighted) to derive a new combined gradient histogram (Merged HoG: MHoG), and a directional prediction mode can be derived using this.

[0963] At this time, the directional prediction mode derived through the above-derived integrated gradient histogram can be used as a merge prediction candidate.

[0964] After performing prediction coding using the prediction block generated through the above-derived prediction mode, the coding cost for each candidate in the merge list can be calculated.

[0965] The optimal candidate within the merge list can be determined by comparing the above-calculated encoding costs. At this time, index information for the corresponding gradient histogram (HoG) within the merge list (index of the optimal candidate within the list) can be signaled from the encoder to the decoder.

[0966] In the above DIMD merge, when adding a candidate in the merge list, if there is a block vector that can be referenced in a neighboring block, the merge can be performed by including information (HoG information) of the block moved by the block vector from the position of the reference block in the merge list.

[0967] The above method can be applied in the same way when using not only block vectors but also the motion vectors of surrounding reference blocks.

[0968] The above method can be applied not only to DIMD merging but also to all merging methods that utilize information from surrounding reference blocks.

[0969] In performing merge prediction for the Derived Prediction Mode Derivation (DIMD) within the decoder-side screen, not only the gradient histogram information of surrounding blocks but also the Derived Prediction Mode information derived from surrounding blocks can be used.

[0970] At this time, to inherit gradient histogram (HOG) information, a merge candidate list can be constructed by inheriting prediction mode information from surrounding block locations defined as follows.

[0971] · Spatial adjacent / non-adjacent

[0972] · Temporal adjacent / non-adjacent locations + juxtaposed locations

[0973] · History-based location

[0974] · Temporal / spatial adjacent / non-adjacent positions adjusted using block vectors / motion vectors

[0975] At this time, if the surrounding block to be referenced is encoded in IntraTMP / IBC, at least one prediction mode can be derived from the block at a position moved by the corresponding BV or through the HoG calculation used in DIMD on the block, and then used as a prediction mode for merge prediction.

[0976] If the surrounding blocks to be referenced above are encoded in IntraTMP / IBC, the prediction mode of the block at the moved position using the corresponding block vector can be referenced and used as the prediction mode for that block during merge prediction.

[0977] At this time, if the prediction mode of the neighboring block to be inherited is a mode that performs prediction coding using multiple prediction modes such as Intra fusion, DIMD, Adaptive Hog for DIMD, OBIC, TIMD, TIMD SAD, TIMD merge, CIIP, GPM, SGPM, TRML, Intra merge, or IntraTMP, at least one of the multiple prediction modes can be selected and used as the prediction mode for the block when performing merge prediction. Even if the neighboring block is predicted coded using a different mode, if it has parameter information regarding the exemplified modes, the parameter information can be inherited.

[0978] A merge candidate in the merge candidate list may include at least one of up to 8 intra-frame prediction modes, weights of intra-frame prediction modes, position-dependent information of intra-frame prediction modes, block vectors, or MTS transformation types.

[0979] In configuring the above merge list, not only a single merge list but also multiple merge lists can be configured.

[0980] At this point, a merge list can be configured for each prediction method as follows.

[0981] · Merge List 1: Block encoded in DIMD and the encoded information contained in that block

[0982] · Merge List 2: Blocks encoded with TIMD and the encoded information contained in those blocks

[0983] · Merge List 3: Blocks encoded with IntraTMP / IBC and the encoded information contained in those blocks

[0984] · Merge List 4: Other in-screen prediction modes

[0985] At this time, a merge list can be constructed by distinguishing between a prediction method having a single prediction mode and a prediction method having multiple prediction modes.

[0986] The candidates within the above merge list can be sorted using template matching. In this case, the template matching cost of each candidate may be the weighted sum of the template matching costs for each candidate's prediction modes, or the template matching cost of the first prediction mode. Then, after sorting each list by template matching cost, a new list can be constructed by selecting the top N candidates (positioned at the top in order of smallest template cost).

[0987] In configuring the above merge list, a single merge list can be constructed by including both the prediction mode derived using gradient histogram information and the prediction mode derived from surrounding blocks.

[0988] In configuring the above merge list, multiple merge lists can be configured by separately distinguishing between the prediction mode derived using gradient histogram information and the prediction mode derived (inherited) from surrounding blocks.

[0989] At this time, a merge list containing only prediction modes derived using gradient histogram information and a merge list containing only prediction modes derived from neighboring blocks can be constructed, respectively, and whether each merge list is used can be signaled from the encoder to the decoder. Index-related information pointing to the optimal candidate within the list can also be signaled.

[0990] In the merge prediction based on the above multiple lists, information regarding whether multiple lists are used, the list index, and candidate indices within the list can be signaled from the encoder to the decoder.

[0991] In the in-screen prediction based on the prediction mode information inherited from the surrounding blocks, the prediction mode information inherited from the surrounding blocks can be scaled for use or stored. That is, the scaled prediction mode information can be converted into the accumulated prediction mode size for the current block and used.

[0992] In this case, the inherited prediction mode information can be scaled or normalized based on the block size. For example, let's assume that the size of the surrounding block to be referenced is 32x32 and it has one prediction mode. If the prediction mode is stored or processed in 4x4 units, the number of occurrences of the inherited prediction mode can be set to 64 instead of 1, taking into account the block size. (A 32x32 block consists of 64 4x4 blocks.) In this case, when storing the gradient size of the prediction mode or the increment amount of the cumulative size of the prediction mode for the 32x32 block, it can be stored as 64 instead of 1.

[0993] In this case, the inherited prediction mode information can be scaled and used based on the number of pixels within the block. For example, let's assume that the size of the surrounding block to be referenced is 32x32 and it has one prediction mode. Then, considering the number of pixels included in the block, the increment amount of the prediction mode's cumulative size can be set to 1024 instead of 1. That is, the increment amount of the cumulative size can be determined in proportion to the block size (height, width) or the number of pixels determined by the prediction mode.

[0994] In the case where the surrounding blocks to be referenced in the calculation of the cumulative size by prediction mode above have multiple prediction modes, the cumulative size by prediction mode can be calculated by assigning the same weight or different weights to each prediction mode.

[0995] In this case, the increase in the cumulative size of the prediction mode can be determined based on the weights for each prediction mode used when generating a prediction signal using multiple prediction modes. For example, it can be assumed that the surrounding blocks to be referenced are encoded as follows.

[0996] · The block size is 32x32, and it is encoded using TIMD fusion with two prediction modes.

[0997] · The fusion weights for the two prediction modes used to generate the TIMD prediction block are 0.75:0.25 (= 3:1)

[0998] · In this case, the increase in cumulative size is calculated in pixel units (i.e., the sum of gradient sizes derived from a 32x32 block, i.e., the cumulative size is 1024)

[0999] If we calculate the increase in the cumulative size of the two prediction modes based on the example based on the block size above, the increase in the cumulative size of the best mode and secondary mode may be 768 (best mode) and 256 (secondary mode), respectively.

[1000] At this time, weights for each prediction mode can be determined using the gradient histogram (HoG) derived from surrounding blocks, and the increase in the cumulative size for each prediction mode can be calculated using the weights.

[1001] For example, the size of the surrounding blocks to be referenced is 32x32 (1024 samples), the prediction mode is DIMD, and a total of 5 prediction modes are used, and the ratio of HoGs by prediction mode may be as follows.

[1002] · Best mode: 2nd mode: 3rd mode: 4th mode: 5th mode = 12: 8: 6: 5: 1 (Total: 32)

[1003] · Based on the ratio of the above HoG, the increase in the cumulative size for each mode can be calculated as follows.

[1004] · Best mode: 2nd mode: 3rd mode: 4th mode: 5th mode = 384 (12x32) : 256 (8x32) : 192 (6x32) : 160 (5x32) : 32 (total 1024)

[1005] In the calculation of cumulative size by the above prediction mode, different weights can be assigned on a reference block basis depending on the position of the current block and surrounding reference blocks.

[1006] For example, when calculating the increase in cumulative size by prediction mode of a reference block, a higher weight can be given to adjacent blocks.

[1007] For example, when calculating the increase in cumulative size by prediction mode of a reference block, higher weights can be assigned to blocks belonging to the same picture.

[1008] For example, when calculating the increase in cumulative size by prediction mode of a reference block, a higher weight may be assigned to blocks having multiple prediction modes.

[1009] For example, when calculating the increase in the cumulative size for each prediction mode of a reference block, weights can be applied differently based on the distance between the current block and the reference block, and a higher weight can be assigned to the closer block.

[1010] At this time, different weights are assigned to each block to calculate the increase in cumulative size for each prediction mode and sum them to derive the Histogram of Occurrence Frequency (HoC) for each prediction mode.

[1011] 2-1. OBIC Mode

[1012] In a prediction based on prediction mode information (or prediction mode occurrence frequency) inherited from the surrounding blocks, the cumulative number of occurrences for the prediction modes inherited from the surrounding blocks can be calculated to generate a histogram of occurrences (HoC: Histograms of occurrence) for each prediction mode.

[1013] Based on the occurrence count histogram (or cumulative occurrence count or frequency) for each prediction mode generated above, at least one prediction mode to be used for intra-frame prediction or intra-frame prediction merge in the current block can be determined. (OBIC: occurrence based intra coding)

[1014] At this time, the frequency of occurrence of a prediction mode (area or number of samples) is calculated from at least one referenceable block in the vicinity, and a histogram of the number of occurrences can be derived by summing the number of samples for each prediction mode.

[1015] · HoC[IPM] += Width * Height;

[1016] At this time, weights based on the positions of the current block and the reference block may be applied. The occurrence count may be weighted by a scaling factor that depends on the distance between the current block and the reference block. When calculating the occurrence count for each prediction mode of the reference block, weights may be applied differently depending on the distance between the current block and the reference block, and a higher weight may be assigned to the closer block. For example, the distance weight can be any one of 4, 2, 1, or 0. For example, when calculating the increase in the cumulative size for each prediction mode of the reference block, a higher weight may be assigned to adjacent blocks.

[1017] This can be derived in the same form as the Merged Gradient Histogram (Merged HoG) in the previous example.

[1018] For example, the occurrence count of prediction modes inherited from the surroundings can be accumulated by prediction mode, and the prediction mode with the highest accumulated count can be used as the prediction mode for the current block.

[1019] For example, N candidates can be derived in order of highest frequency of occurrence of the above prediction modes and used as prediction modes for the current block.

[1020] For example, each prediction block can be created using N prediction modes, and a new prediction block can be generated by weighting and summing them for use in prediction.

[1021] At this time, up to 5 prediction modes, such as DIMD, can be selected and used by weighted sum.

[1022] At this time, up to two prediction modes, such as TIMD, can be selected and used by weighted summing them.

[1023] At this time, the weights used for each prediction mode for the above weighted sum can be applied in the same way as the method of calculating weights in the previous examples of TIMD and DIMD.

[1024] At this time, merge prediction can be performed using the occurrence frequency histogram (HoC) for each prediction mode inherited from the above surroundings.

[1025] For example, when configuring a merge list for the current block, the prediction mode with the highest occurrence frequency in the above HoC histogram can be added to the merge list as a candidate prediction mode for the current block.

[1026] For example, when configuring the merge list for the current block, N prediction modes can be added to the merge list as candidate prediction modes for the current block in order of highest frequency in the above occurrence frequency histogram (HoC).

[1027] For example, if the frequency of occurrence in the above frequency histogram (HoC) is greater than a certain size, at least one of the prediction modes can be selected and used for merge prediction for the current block.

[1028] At this time, gradient histogram information corresponding to each prediction mode can be derived using the occurrence frequency information for each prediction mode calculated through the inherited prediction modes.

[1029] The gradient histogram information derived from the above prediction mode occurrence frequency can be used to create an integrated gradient histogram.

[1030] Conversely, the occurrence frequencies for each prediction mode derived using gradient histogram information inherited from neighboring blocks can be summed and used to create an HoC.

[1031] By utilizing the above prediction mode information or prediction mode information inherited from surrounding blocks (or the frequency of occurrence per prediction mode derived therefrom), it is possible to derive the histogram per prediction mode or the number of occurrences of a prediction mode to be used like a gradient histogram without a separate operation (gradient filtering) to derive a gradient histogram. This can reduce the complexity caused by gradient histogram calculation.

[1032] At this time, when predicting the DIMD merge of the current block, storage space can be reduced by determining the candidate prediction mode to use by utilizing the in-screen prediction mode information of surrounding blocks instead of separately storing the gradient histogram (HOG).

[1033] In this case, if there is no information on intra-screen prediction modes, such as in inter-screen prediction modes, the frequency of occurrence by prediction mode can be derived by calculating the gradient histogram for the corresponding block.

[1034] The merge prediction method using the prediction mode information (frequency of occurrence of prediction modes) of the aforementioned surrounding blocks can operate as a sub-mode of the DIMD mode that is activated when in DIMD prediction mode. (i.e., when the DIMD flag is true)

[1035] At this time, when determining the candidate prediction mode to be used in the frequency-based in-frame prediction mode determination method (OBIC), at least one of the prediction modes used in DIMD may be excluded.

[1036] An in-frame prediction method (OBIC) based on the prediction mode information (prediction mode occurrence frequency) of the surrounding blocks mentioned above, or an in-frame merge prediction method using the same, can operate as a separate in-frame prediction mode. For example, it can select the optimal mode by competing with other in-frame prediction methods such as TIMD and DIMD.

[1037] Whether merge prediction is performed in the Decoder-side In-Face Prediction Mode Derivation (DIMD) method can be signaled from the encoder to the decoder.

[1038] Whether the prediction mode derivation (DIMD) method is performed within the above decoder-side screen can be signaled from the encoding to the decoder.

[1039] Instead of the prediction mode obtained using the above gradient histogram (HoG) information, merge prediction can be performed using a prediction mode derived through the frequency information of the prediction mode inherited from surrounding blocks.

[1040] At this time, the encoder can signal to the decoder whether to use the corresponding frequency-based in-frame prediction method (OBIC).

[1041] Using the prediction mode information inherited from the above surroundings, the frequency of occurrence for each prediction mode is calculated, and at least one of the prediction modes having a higher frequency of occurrence can be selected and used to generate a prediction signal.

[1042] For example, you can select up to 2 or up to 5 prediction modes to perform a weighted sum.

[1043] For example, the above weighted sum method (number of prediction modes and weights used in the weighted sum) may follow the DIMD method or the TIMD method.

[1044] The frequency-based prediction method used in the prediction method within the screen above can also be used for transformation prediction.

[1045] At this time, the frequency of occurrence for the transformation information (transformation type, DCT, DST, LFNST, NSPT) of surrounding blocks is derived and can be used when encoding the actual transformation information determined in the current block.

[1046] For example, this information can be used when context coding for a conversion type.

[1047] For example, this information can be used when context coding for kernel indices of LFNST or NSPT.

[1048] At this time, the probability table for the corresponding context can be rearranged using the above occurrence frequency.

[1049] At this time, the transformation candidate table (or index within the table) can be reordered using the above occurrence frequency.

[1050] In a prediction method having multiple in-frame prediction modes such as the above DIMD OBIC, TIMD, etc., one of the multiple prediction modes can be selected and used as the final prediction mode.

[1051] The final prediction mode among the above plurality of prediction modes can be selected based on the template matching cost.

[1052] For example, in TIMD, if the template cost of the best mode is below a certain threshold, the in-frame prediction can be performed using only the best mode.

[1053] For example, if the template matching costs of the best mode and the secondary mode are compared and the template matching cost of the secondary mode is N times or more than the template matching cost of the best mode, then in-screen prediction can be performed using only the best mode. In this case, N can be 1 or greater.

[1054] For example, if the difference in in-frame prediction mode index values ​​between best mode and secondary mode is below a certain threshold (i.e., judged to be prediction modes with similar directionality), in-frame prediction can be performed using only best mode.

[1055] The final prediction mode among the above plurality of prediction modes can be selected based on the gradient histogram (HoG).

[1056] For example, in DIMD, in-screen prediction can be performed using only the prediction mode with the largest HoG value.

[1057] For example, if the HoG value of the prediction mode with the largest HoG value exceeds a certain threshold, in-screen prediction can be performed using only that mode. (When that mode is dominant)

[1058] When selecting the final prediction mode among the above plurality of prediction modes, it can be determined based on the frequency of occurrence for each prediction mode.

[1059] For example, in-screen prediction can be performed using only the prediction mode with the highest frequency in OBIC.

[1060] For example, if the occurrence frequency of the prediction mode with the highest frequency exceeds a certain threshold, in-screen prediction can be performed using only the prediction mode with the highest frequency. (When the mode is dominant)

[1061] When selecting a final mode among multiple prediction modes, the decision can be made based on the number of pixels (area) encoded in the prediction mode.

[1062] For example, in SGPM, the area (number of pixels) encoded by each prediction mode (to which each prediction mode is applied) can be compared, and the mode with the larger area can be selected to perform in-frame prediction.

[1063] For example, if the area (or number of pixels) of a prediction mode having a larger area (number of pixels) is above a certain threshold, in-screen prediction can be performed using that mode.

[1064] Whether or not to use the method for determining the above-mentioned prediction mode can be signaled from the encoder to the decoder.

[1065] A method using multiple prediction modes and a method using a single prediction mode determined through the above method can be used together.

[1066] The method of selecting one final mode among the above multiple prediction modes can also be applied when configuring the merge list in merge prediction.

[1067] For example, if encoded as TIMD of a reference block, only one of the two prediction modes can be selected and added to the merge list by the above method, where the template matching cost is smaller than a specific threshold.

[1068] In performing merge prediction for the prediction mode induction (DIMD) within the decoder-side screen, a new candidate prediction mode can be derived using candidates within the merge list.

[1069] At this time, multiple candidates can be selected from the merge list and weighted summed to generate a new prediction mode, or a new prediction signal can be generated by weighted summing the prediction signals generated using the prediction mode.

[1070] For example, after selecting two candidates from the top of the merge list, a new prediction candidate can be derived by weighting these candidates.

[1071] For example, if candidate 1 has a prediction mode index of 8 and candidate 2 has a prediction mode index of 4, and the weights are 3:1, the new prediction mode can be (8*3 + 4*1) / 4 = 7 (prediction mode index).

[1072] For example, after selecting two candidates from the top of the merge list, a new prediction signal can be derived by weighting the prediction signals generated using these candidates.

[1073] At this time, new prediction candidates can be derived using the HoG or HoC (occurrence frequency) values ​​of the prediction candidates within the merge list.

[1074] Figures 23 and 24 illustrate an example of a merge list.

[1075] For example, when a merge list is configured as shown in FIG. 23, the HoG or HoC values ​​of the prediction modes within the merge list are summed for each mode, and S prediction modes are selected in order of increasing HoG or HoC value to generate new candidates as shown in FIG. 23. At this time, the previously selected S prediction modes may be included as a single candidate within the merge list as shown in FIG. 24, or each prediction mode may be included as a candidate within the merge list. That is, S new candidates may be included in the merge list.

[1076] In this case, in addition to using the new prediction mode as the new mode within the merge list, it may also be used as the final prediction mode for in-screen prediction. In this case, signaling for the existing final prediction mode index can be omitted.

[1077] 3. TIMD mode

[1078] In generating (or deriving) a prediction signal for prediction encoding and decoding, a prediction signal can be generated through a template-based intra-mode derivation (TIMD) method.

[1079] The derivation of a prediction mode and a prediction signal through the above “template-based intra-mode derivation (TIMD) method” can be performed by [STEP 1] configuring a list of most probable modes (MPM) for a current block (or target block), [STEP 2] generating a prediction template and determining a prediction mode using the prediction mode information within the configured list of most probable modes (MPM), and [STEP 3] generating a prediction block and performing prediction encoding and decoding using the determined prediction mode information.

[1080] [STEP 1] In the step of configuring the MPM (most probable mode) list for the current block (or target block),

[1081] The block location for inheriting intra prediction mode (IPM) information required to construct the MPM list can be defined as follows.

[1082] (1) Spatially adjacent location

[1083] (2) Spatial non-adjacent location

[1084] (3) Temporal adjacent / non-adjacent + collocated

[1085] (4) History-based

[1086] A FIFO-type list is defined, and prediction mode information is stored in the list in the order of encoding / decoding, and history-based positions can be used in the order they arrived.

[1087] (5) Temporal / spatial adjacent / non-adjacent positions adjusted using block vectors / motion vectors

[1088] At this time, the position of blocks within the same picture as the block from which the block vector / motion vector was derived can be adjusted.

[1089] At this time, the block vector / motion vector can adjust the position of blocks within a different picture from the derived block.

[1090] At this time, the position can be readjusted for all available candidate block positions regardless of the picture to which the block derived from the block vector / motion vector belongs.

[1091] Among the above positions, the block at the temporal position can be used as a block position to inherit the intra-frame prediction mode when performing intra-frame prediction in inter-frame pictures (B-picture, P-picture).

[1092] FIG. 14 illustrates an example where the upper right or lower left adjacent block is not available.

[1093] An MPM list including block locations for inheriting intra prediction mode (IPM) information can be configured through the following steps (1) to (12). (Figs. 11–13)

[1094] Step (1) Add MPM_LIST[0] = PLANAR_IDX

[1095] You can add planar in MPM mode. In this case, it can be added to the top of the list.

[1096] Step (2) Best mode stored in adjacent blocks: spatially adjacent locations

[1097] At this time, the order can be L (Left) → A (Above) → BL (Bottom Left) → AR (Above Right) → AL (Above Left).

[1098] Step (3) Add a prediction mode induced via DIMD in the current block

[1099] At this time, at least one prediction mode among the prediction modes derived through DIMD can be added to the MPM list.

[1100] For example, the best prediction mode (or first mode) and the secondary prediction mode can be added to the MPM list.

[1101] Step (4) Add a prediction mode derived (inherited) from spatial non-adjacent locations.

[1102] Step (5) Add a prediction mode derived (inherited) from temporal adjacent + collocated positions

[1103] Step (6) Add a prediction mode derived (inherited) from the history-based location

[1104] Step (7) Add temporal / spatial adjacent / non-adjacent positions adjusted using block vectors / motion vectors

[1105] Step (8) After adding candidates to the list in steps (1) through (7), perform list sorting using template matching costs.

[1106] Step (9) Add a prediction mode ranging from -4 to +4 to the prediction modes included in the above MPM list.

[1107] Step (10) Partial MPM reordering (Fig. 14)

[1108] The process of adding prediction modes from the reserved list to the MPM list can be performed.

[1109] Step (11) Select at least one of the prediction modes included in the MPM list and add a prediction mode from -4 to +4 to the MPM list.

[1110] For example, if the prediction mode number added to the 2nd MPM list is 6, the prediction modes added to the MPM list can be {2, 3, 4, 5, 7, 8, 9, 10}.

[1111] Step (12) Add { DC_IDX, VER_IDX, HOR_IDX, VER_IDX - 4, VER_IDX + 4, 14, 22, 42, 58, 10, 26, 38, 62, 6, 30, 34, 66, 2, 48, 52, 16}

[1112] Here, -4 to +4 may mean adding or subtracting -4, -3, -2, -1, +1, +2, +3, and +4 to the prediction mode.

[1113] In step (1) above, a prediction mode derived using block vectors can be used instead of the Planar mode.

[1114] In this case, the prediction mode of the block pointed to by the block vector inherited from neighboring blocks or juxtaposed blocks, etc., may be used.

[1115] The template matching cost-based list sorting process defined in step (8) above can be inserted between steps (1) and (12) above.

[1116] For example, after performing up to step (4), a template matching cost-based list sort can be performed.

[1117] The “Partial MPM reordering” process defined in step (10) above can be inserted between steps (1) and (12) above.

[1118] For example, after performing up to step (4), partial MPM reordering can be performed.

[1119] Here, partial MPM reordering is

[1120] A prediction mode pred_mode_idx is added to the MPM list when it satisfies the following threshold condition; otherwise, it is added to the Reserved list. Prediction modes in the Reserved list become candidate modes for addition to the MPM list.

[1121] · Lower bound < pred_mode_idx < upper bound

[1122] · Lower bound: DC_IDX(2) + nbRemovedFirest

[1123] · Upper bound: NUM_LUMA_MODE (Number of luminance prediction modes: 67) - nbRemovedLast

[1124] "nbRemovedFirst" and "nbRemovedLast" for determining the Upper bound and Lower bound can be determined based on the width and height or ratio of the current block.

[1125] For example, deltaSize is derived based on the width and height of the current block, and nbRemovedFirst and nbRemovedLast can be determined based on deltaSize.

[1126] deltaSize = abs(floorLog2(w) - abs(floorLog2(H))

[1127] As shown in Fig. 14, the nbRemovedFirst and nbRemovedLast values ​​may vary based on the location of unavailable adjacent blocks.

[1128] Up to N candidates can be selected and added to the MPM list through the above template matching cost-based list sorting.

[1129] For example, if 30 items are entered into the sorting list during steps (1) to (7), the list can be sorted in ascending order of template matching cost, and then 6 candidates from the top can be selected and added to the MPM list.

[1130] When adding the in-screen prediction modes derived in the steps (1) to (12) defined above to the MPM list, the template cost for the in-screen prediction modes derived for each step can be calculated, and then the in-screen prediction modes can be added by sorting them in ascending order of template cost.

[1131] At this time, template matching cost-based prediction mode alignment can be performed for at least one step among steps (1) to (12). This can be performed sequentially for each step.

[1132] Sort available prediction modes in Step (1) → Add MPM list → Sort available prediction modes in Step (2) → Add MPM list …

[1133] For example, sorting based on template matching costs can be performed only at each of steps (2), (3), and (4). At this time, the in-screen prediction modes derived for each step can be sorted in ascending order of template matching costs (sorted in order of smallest template matching cost) and N candidates can be selected from the top and added to the MPM list. At this time, N can be 12. At this time, N can be not defined and all available candidate modes at that step can be added to the MPM list.

[1134] In the above template matching cost-based MPM list sorting, sorting can be performed after filling the MPM list with all candidates.

[1135] For example, after all steps (1) through (12) are completed, sorting can be performed and up to N prediction modes can be selected and added to the MPM list.

[1136] For example, after setting the maximum number of candidates (or MPM list size) included in the MPM list, if the maximum number of candidates is reached in steps (1) to (12) above, the addition of candidates to the MPM list can be stopped and sorting can be performed.

[1137] In the above MPM process, a prediction mode for MPM configuration can be derived in the current block by utilizing gradient histogram information or prediction mode occurrence frequency information inherited from surrounding blocks.

[1138] For example, a combined gradient histogram can be derived using gradient histogram information inherited from neighboring blocks, and at least one of the prediction modes derived through the combined gradient histogram can be selected and added to the MPM list.

[1139] For example, you can select the prediction mode with the largest histogram size and add it to the MPM list. In this case, if the selected prediction mode is already included in the MPM list, you can add the next-highest prediction mode to the MPM list.

[1140] For example, N candidates can be selected in descending order of histogram size and added to the MPM list (e.g., N = 5). In this case, prediction candidates already included in the existing MPM list can be excluded before adding.

[1141] For example, the MPM list of the current block can be reconstructed using gradient histogram information inherited from neighboring blocks or prediction mode occurrence frequency information. As an example, candidates with low occurrence frequencies can be excluded from the MPM list of the current block.

[1142] The method for deriving MPM candidates using occurrence frequency information by prediction mode can also be applied in the same way as the method for deriving MPM through the gradient histogram above.

[1143] [STEP 2] A step of generating a prediction template and determining a prediction mode using the prediction mode information within the above-configured MPM (most probable mode) list.

[1144] FIGS. 15 and 16 illustrate an example of a reference template for the current block.

[1145] A prediction template can be generated using the prediction mode information within the configured MPM list.

[1146] At this time, when a template for the current block (or target block) is configured as in FIG. 14, a prediction template can be configured using a reference template or a sample within the reference template that is adjacent to or adjacent to the template of the current block, as in FIG. 15 or FIG. 16.

[1147] At this time, the reference template may consist of at least one sample or line.

[1148] In determining a sample or line for configuring the above reference template, a line determined by the Multiple Reference Line (MRL) method may be used.

[1149] In generating a prediction template using the above reference template, a prediction template can be generated using the prediction mode information within the MPM list and the samples within the reference template. At this time, the prediction samples within the prediction template can be derived using the same method as the intra-angular prediction method.

[1150] By utilizing the prediction modes within the above MPM list, a prediction template can be derived for each mode, and the template matching cost with the template of the current block can be calculated.

[1151] At this time, the prediction modes within the MPM list can be sorted in ascending order of template matching costs, and N prediction modes from the top of the list can be selected to be used as candidate prediction modes for TIMD prediction.

[1152] For example, the prediction mode with the smallest template matching cost can be selected and determined as the optimal prediction mode (best mode, first mode) for the template-based in-screen prediction mode induction method.

[1153] For example, when deriving multiple prediction modes, the prediction mode with the smallest template matching cost can be defined as the best mode or primary mode, and the prediction mode with the next-highest template matching cost can be designated as the secondary mode.

[1154] The above-derived prediction mode information can be stored to be utilized in the process of constructing the MPM list in blocks other than the current block.

[1155] The above-derived prediction mode information can be signaled in the encoder / decoder or derived using a peripheral template.

[1156] Template matching costs can be calculated using at least one of the SAD, SSE, SATD, and MR-SAD methods.

[1157] At this time, template matching can be performed using multiple template matching cost calculation methods. When multiple template matching cost calculation methods are used, template matching is performed using each calculation method to select at least one candidate having a small template matching cost for each calculation method, and the final encoding costs (e.g., rate-distortion costs) of the candidates determined for each calculation method are compared to determine the candidate with the smallest encoding cost. The template matching calculation method used to determine the candidate with the smallest encoding cost can be determined as the final template matching calculation method (metric, measure) for the corresponding block. At this time, information regarding the template matching cost calculation method to be used can be signaled from the encoder to the decoder.

[1158] When using multiple template matching cost calculation methods in the above TIMD, when calculating the template matching cost for prediction candidates, prediction candidates determined based on SATD used in the existing TIMD may be excluded.

[1159] The above-mentioned multiple template matching cost calculation methods can be applied to all methods using the template matching methods of the present disclosure.

[1160] Performing template matching using the above-mentioned plurality of template matching cost calculation methods may include, depending on the block size, applying a first template matching method to a block of a first size and applying a second template matching method to a block of a second size.

[1161] [STEP 3] A step of generating a prediction block using the above-determined prediction mode information and performing prediction encoding and decoding.

[1162] Using the prediction mode derived through [STEP 2] and the restored surrounding samples, a prediction block for the current block can be generated and prediction encoding and decoding can be performed.

[1163] At this time, whether to use the template-based intra-mode derivation (TIMD) method can be signaled from the encoder to the decoder.

[1164] If multiple prediction modes are derived in the above [STEP 2] process, multiple prediction blocks can be generated for each derived prediction mode.

[1165] At this time, prediction coding can be performed using multiple prediction blocks, and the prediction mode with the lowest coding cost can be determined as the optimal prediction mode.

[1166] The above optimal prediction mode information can be signaled from the encoder to the decoder.

[1167] If multiple prediction modes are derived in the above [STEP 2] process, multiple prediction blocks can be generated for each derived prediction mode, and a new prediction block can be generated by weighted sum / fusion / blending them.

[1168] Multiple prediction modes can be selected from the MPM list to generate a prediction block through the above weighted sum.

[1169] For example, after sorting the MPM list based on template matching costs, the best prediction mode and secondary prediction mode are derived to create prediction blocks, and a new prediction block can be generated by weighting and summing them.

[1170] At this time, whether to use a prediction block generation method through weighted sum, fusion, or blending can be signaled from the encoder to the decoder.

[1171] Alternatively, whether to use a prediction block generation method through weighted sum / fusion / blending can be derived in the encoder and decoder.

[1172] In this case, the prediction block generation method using weighted sum, fusion, or blending can be used only under specific conditions.

[1173] For example, when the template matching cost of the next-ranked prediction mode is less than twice the minimum template matching cost (template cost of the optimal prediction mode), a prediction block generation method through weighted sum / fusion / blending can be performed.

[1174] In generating prediction blocks through the above weighted sum, not only the angular prediction mode determined by TIMD but also non-directional prediction modes such as DC, Planar, IntraTMP, and IBC can be used.

[1175] For example, a new prediction block can be generated by weighting the prediction blocks generated from one prediction mode determined based on the template matching cost among the best mode, secondary mode, and non-directional prediction mode derived in the above TIMD process.

[1176] In this case, the non-directional prediction mode can be replaced with a prediction signal (block) generated by utilizing the prediction mode of the prediction block at the moved position using the block vector (BV) derived through IBC or IntraTMP.

[1177] Alternatively, the non-directional prediction mode can be replaced with a prediction signal generated by utilizing the prediction mode of the prediction block at the moved position, using motion vectors (MV) derived from the current block or surrounding blocks instead of the block vector.

[1178] The prediction signal derived using the above block vector or motion vector can be used in a weighted sum with a non-directional prediction mode. For example, the prediction signal derived using the block vector or motion vector can be used together in the weighted sum process without replacing the non-directional prediction mode.

[1179] In determining the reference samples (reference lines) required for generating the above prediction block, line information determined through the Multiple Reference Line (MRL) method can be used.

[1180] For example, if the line determined through the multiple reference line (MRL) method is called “Line L,” a prediction block can be generated using Line L.

[1181] At this time, prediction blocks can be generated using different lines depending on the prediction mode.

[1182] For example, a block created with the optimal prediction mode (primary, best) uses L line, and a block created through the secondary prediction mode (secondary) can use L + 1 line.

[1183] In this case, MRL information can be calculated in the current block or inherited from surrounding blocks.

[1184] In determining the prediction blocks to be used for generating the above weighted sum / fusion / blending-based prediction blocks, the determination can be made based on the error values ​​(SAD / SSE / SATD) between the prediction blocks.

[1185] For example, if four prediction modes are selected using template matching costs and four prediction blocks are generated using them, the error values ​​between each prediction block can be calculated, and the pair of prediction blocks with the smallest error values ​​can be determined as the prediction blocks to be used for the weighted sum.

[1186] For example, the prediction block with the smallest error value compared to the prediction block with the smallest template matching cost (the prediction block generated using the best mode) can be determined as the prediction block to be used for the weighted sum.

[1187] For example, multiple prediction blocks can be generated through extended lines such as L + 1 line and L + 2 line based on L line determined through the multiple reference line (MRL) method, and the error value between each prediction block can be calculated to determine the pair of prediction blocks with the smallest error value as the prediction blocks to be used for the weighted sum.

[1188] For example, if there are multiple prediction blocks, the block can be excluded from the weighted sum if the difference in error value with the prediction block having the smallest template matching cost exceeds a certain threshold.

[1189] When generating prediction blocks based on the above weighted sum / fusion / blending, the weights can be determined based on the template matching cost.

[1190] At this time, assuming that TMcost1 is the minimum template matching cost, TMcost2 is the next-highest template matching cost, W1 is the weight applied to the prediction block with the minimum template matching cost, and W2 is the weight applied to the prediction block with the next-highest template matching cost, the weights W1 / W2 can be derived through the following equation.

[1191] · W1 = TMcost2 / (TMcost1 + TMcost2)

[1192] · W2 = 1 - W1

[1193] In determining the weights for generating a prediction block (signal) based on the above weighted sum / fusion / blending, the weights can be determined on a sample-by-sample basis based on the template matching cost and pixel location.

[1194] In deriving the above pixel-specific weights, the method for calculating pixel-specific weights in DIMD described above can be applied as follows.

[1195] · Formula (3): w i (x, y) = wTimdi + △i - 2△i*y / (H-1)

[1196] · Formula (4): w i (x, y) = wTimdi + △i - 2△i*x / (W-1)

[1197] In this case, wTimdi can represent the weight for prediction candidate i. H can represent the height of the block, and W can represent the width of the block.

[1198] △i is a constant value, where, a i ≠ 10. In this case, the size of the constant value △i can be different depending on the block size or the number of samples included in the block. For example, if the number of samples included in the block is greater than 128, the size of the constant value △i can be made smaller than the original value (10).

[1199] In this case, weights per prediction mode or prediction candidate (wDimd in DIMD) i (meaning) wTimd i In determining the weights, DIMD calculates weights using the magnitude of the HoG value (the cumulative number of prediction candidates derived through HoG) or the ratio of the HoG value for each prediction mode (or candidate), whereas TIMD can calculate weights based on the template matching cost (SAD, SATD, SSE, etc.) of the corresponding prediction mode.

[1200] For example, the normalized template matching cost (or template cost) of prediction candidate i is “norSATD iIf we define the prediction candidates to be weighted summed as directional prediction mode mode1, mode2, and non-directional prediction mode mode3, the weights for each prediction mode can be derived as follows.

[1201] · wTimd1(mode 1) = 43 / 64 * norSATD1 / (norSATD1+ norSATD2)

[1202] · wTimd2(mode 2) = 43 / 64 * norSATD2 / (norSATD1+ norSATD2)

[1203] · Non-angular (mode 3) = 21 / 64

[1204] · In this case, 43 / 64 ≈ 2 / 3 and 21 / 64 ≈ 1 / 3 may be possible.

[1205] The amplitude of HoG in DIMD is replaced by norSATD in TIMD.

[1206] In selecting Equation (3) and Equation (4) for determining the weight for a sample located at coordinates (x, y) within a prediction block using the above-derived wDIMDi, the determination can be made based on the size or ratio of the template matching cost between the surrounding templates of the current block.

[1207] For example, let “norSATDA” be the normalized template matching cost (or template cost) of the template located at the top of the current block and “norSATDL” be the normalized template matching cost (or template cost) of the template located at the left. Then, an equation for calculating pixel-unit weights can be determined through the following method.

[1208] · If, norSATD A < N * norSATD L → Equation (3) → As it moves further from the top (as the y value increases), w i This shrinking effect

[1209] · Else if, norSATD L < N * norSATD A → Equation (4) → As it moves further from the left end (as the x value increases), w i This shrinking effect

[1210] · Otherwise, use equal weights based on the diagonal (x = y) within the block (apply only weights based on distance)

[1211] · In this case, N can be 1 / 2.

[1212] In this case, if only the top template exists, the pixel-specific weight can be derived through Equation (3).

[1213] In this case, if only the left edge template exists, the pixel-specific weights can be derived through Equation (4).

[1214] In performing the above template-based intra-mode derivation (TIMD), prediction coding can be performed using at least one template.

[1215] At this time, multiple templates can be used to derive the prediction mode and prediction block and perform prediction coding.

[1216] In inducing a template-based in-screen prediction mode based on the above multiple templates, a prediction block is generated for each configured template to perform prediction coding, and then the most optimal template in terms of coding cost ca...

Claims

1. In an image decoding method for decoding the current block, Step of determining the prediction mode of the current block; A step of deriving transformation information of the current block based on the prediction mode of the current block and the size of the current block; and A step of performing a transformation on the current block based on the transformation information of the current block. A video decoding method including 2. In Paragraph 1, The step of determining the prediction mode of the current block above is, A step of deriving reference samples by applying at least one of downsampling, subsampling, or filtering to at least one of the restoration samples of the surrounding blocks of the current block or the restoration samples of the reference block; and A step of deriving the prediction mode of the current block from the in-frame prediction modes derived by applying either a boundary detection filter or a HoG induction process to the hymn reference samples. A video decoding method including 3. In Paragraph 1, The step of deriving the transformation information of the current block above is, A step of deriving a first-order transformation type set corresponding to the prediction mode of the current block based on a mapping table between prediction modes within the screen and first-order transformation type sets. A video decoding method including 4. In Paragraph 3, The step of deriving the transformation information of the current block above is, If the prediction mode of the current block does not match the first transformation type sets, a step of inducing a prediction mode within a virtual screen by applying one of a boundary detection filter or a HoG induction process to at least one of the prediction samples of the current block or the restoration samples of the reference region; and Step of deriving a set of primary transformation types corresponding to the prediction mode within the virtual screen above A video decoding method including 5. In Paragraph 1, The step of deriving the transformation information of the current block above is, A step of inducing prediction modes within a plurality of virtual screens of the above current block; and A step of deriving a first transformation type set based on the difference between the prediction mode within the primary screen and the prediction modes within the secondary screen among the plurality of prediction modes within the virtual screens, the prediction mode within the primary screen, and the size of the current block. A video decoding method including 6. In Paragraph 1, The step of deriving the transformation information of the current block above is, A step of inducing multiple prediction modes within a virtual screen based on the prediction mode of the current block above; A step of deriving a plurality of secondary transformation kernel sets based on the prediction modes within the plurality of virtual screens; A step of obtaining secondary conversion kernel index information from a bitstream; and A step of deriving the secondary transformation kernel of the current block from the plurality of secondary transformation kernel sets based on the secondary transformation kernel index information. A video decoding method including 7. In Paragraph 6, The above plurality of prediction modes within the virtual screen are, An image decoding method comprising an in-frame prediction mode induced by applying one of a boundary detection filter or a HoG induction process to at least one of the prediction samples of the current block or the restoration samples of the reference region.

8. In Paragraph 6, A video decoding method in which, when the index difference between a primary screen prediction mode and a secondary screen prediction mode among the plurality of virtual screen prediction modes is less than or equal to a threshold, the secondary transformation kernel of the current block is derived using the primary screen prediction mode from a set of secondary transformation kernels corresponding to the primary screen prediction mode.

9. In Paragraph 6, The step of deriving the above plurality of secondary transformation kernel sets is, If the above plurality of secondary transformation kernel sets overlap, the step of replacing one of the overlapping secondary transformation kernel sets with a predefined secondary transformation kernel set. A video decoding method including 10. In Paragraph 6, The step of deriving the above plurality of secondary transformation kernel sets is, When the above plurality of secondary transformation kernel sets overlap, a step of deriving a non-overlapping secondary transformation kernel set from the remaining prediction modes within the virtual screen, excluding the prediction modes within the virtual screen corresponding to the overlapping secondary transformation kernel sets. A video decoding method including 11. In Paragraph 1, The step of deriving the transformation information of the current block above is, A step of generating a transformation merge list using transformation information of surrounding blocks of the current block; A step of obtaining conversion index information from a bitstream; and A step of deriving the transformation information of the current block from the transformation merge list using the transformation index information above. A video decoding method including 12. In a video encoding method for encoding a current block, Step of determining the prediction mode of the current block; A step of deriving transformation information of the current block based on the prediction mode of the current block and the size of the current block; A step of performing a transformation on the current block based on the transformation information of the current block; and Step of encoding the transformation information of the current block above A video encoding method including 13. A method for providing image data to an image decoding device, A step of generating a bitstream by encoding a target block using an affine transform mode in bidirectional prediction; and The method includes the step of transmitting the bitstream to the image decoder, and The step of generating the above bitstream is, Step of determining the prediction mode of the current block; A step of deriving transformation information of the current block based on the prediction mode of the current block and the size of the current block; A step of performing a transformation on the current block based on the transformation information of the current block; and Step of encoding the transformation information of the current block above A method including

Images