Method for encoding / decoding image, and recording medium for storing bitstream

Abstract

A method and a device for encoding / decoding an image, and a recording medium are disclosed. The image decoding method according to the present disclosure may comprise the steps of: determining whether combined intra prediction is applied to the current block; deriving, if the combined intra prediction is applied to the current block, a plurality of intra prediction modes for the current block; deriving reference samples of the current block; and acquiring a plurality of prediction blocks for the current block on the basis of the plurality of intra prediction modes and the reference samples. A final prediction block of the current block can be acquired by weighted-summing the plurality of prediction blocks.

Description

Method for video encoding / decoding and recording medium for storing a bitstream

[0001] The present invention relates to a method, apparatus, and recording medium for image encoding / decoding.

[0002] With the continuous development of the information and communication industry, services providing video through broadcasting and the Internet have spread globally.

[0003] Users demand videos with higher resolution and quality. To meet these user demands, video encoding and decoding technologies suitable for such videos are required. Video encoding technology can generate compressed video by compressing the video representing the images to have a smaller amount of data. Video decoding technology can generate reconstructed images using the compressed video.

[0004] Regarding video encoding and decoding technologies, various techniques exist, such as segmentation, prediction, transformation, quantization, filtering, and entropy encoding and decoding. By introducing, modifying, improving, and combining these various techniques, video and images can be compressed, transmitted, and stored more effectively.

[0005] The present disclosure aims to provide a method for determining the intra-prediction mode of a current block.

[0006] The present disclosure aims to provide a method for generating a candidate list to encode / decode the intra prediction mode of the current block.

[0007] The present disclosure aims to provide a method for applying combined intra-prediction to a current block.

[0008] A video decoding method according to the present disclosure may include: a step of determining whether combined intra prediction is applied to a current block; a step of deriving a plurality of intra prediction modes for the current block when the combined intra prediction is applied to the current block; a step of deriving reference samples of the current block; and a step of obtaining a plurality of prediction blocks for the current block based on the plurality of intra prediction modes and the reference samples. At this time, a final prediction block of the current block may be obtained by weighting the plurality of prediction blocks.

[0009] A video encoding method according to the present disclosure may include: determining whether combined intra prediction is applied to a current block; deriving a plurality of intra prediction modes for the current block when the combined intra prediction is applied to the current block; deriving reference samples of the current block; and obtaining a plurality of prediction blocks for the current block based on the plurality of intra prediction modes and the reference samples. At this time, a final prediction block of the current block may be obtained by weighting the plurality of prediction blocks.

[0010] In the image encoding / decoding method according to the present disclosure, the combined intra-mode prediction may include at least one of DIMD (Decoder side Intra Mode Derivation) or TIMD (Template-based Intra Mode Derivation).

[0011] In the image encoding / decoding method according to the present disclosure, when the combined intra prediction is applied to the current block, filtering of the reference samples may be omitted.

[0012] In the image encoding / decoding method according to the present disclosure, the plurality of intra prediction modes may be MPM candidates included in the MPM (Most Probable Mode) list.

[0013] In the image encoding / decoding method according to the present disclosure, the plurality of intra prediction modes can be selected from the histogram of the current block.

[0014] In the image encoding / decoding method according to the present disclosure, the plurality of intra prediction modes can be selected in order of increasing amplitude value or frequency of occurrence on the histogram.

[0015] In the image encoding / decoding method according to the present disclosure, an error cost is calculated for each of the intra prediction modes included in the histogram, and the plurality of intra prediction modes can be selected from the histogram in order of decreasing error cost.

[0016] In the image encoding / decoding method according to the present disclosure, the histogram can be generated using an intra-prediction mode of a spatial reference block of the current block.

[0017] In the image encoding / decoding method according to the present disclosure, when a plurality of intra prediction modes exist in the spatial reference block, a value to which a weight is applied to the occurrence frequency of each of the plurality of intra prediction modes of the spatial reference block can be accumulated in the histogram.

[0018] In the image encoding / decoding method according to the present disclosure, a weighted value applied to the occurrence frequency of the intra-prediction mode of the spatial reference block is accumulated in the histogram, and the weight may be determined according to the distance between the spatial reference block and the current block.

[0019] In the image encoding / decoding method according to the present disclosure, when the combined intra prediction is applied to the current block, only a predefined single interpolation filter may be available to generate the plurality of prediction blocks.

[0020] In the image encoding / decoding method according to the present disclosure, when the combined intra prediction is applied to the current block, even if the current block is capable of performing matrix-based intra prediction, none of the plurality of intra prediction modes may be switched to a matrix-based intra prediction mode.

[0021] In the image encoding / decoding method according to the present disclosure, when the combined intra prediction is applied to the current block, filtering for each of the plurality of prediction blocks may be omitted.

[0022] In the present disclosure, a recording medium for recording a bitstream generated by the image encoding method may be provided.

[0023] According to the present disclosure, the accuracy of intra prediction can be increased by providing an improved method for determining the intra prediction mode of the current block.

[0024] According to the present disclosure, encoding / decoding efficiency can be improved by providing an improved candidate list generation method for encoding / decoding the intra prediction mode of the current block.

[0025] According to the present disclosure, by providing a method for predicting the current block through combined intra prediction, the amount of data encoded / decoded can be reduced while improving intra prediction accuracy.

[0026] FIG. 1 shows a system for video coding according to one embodiment.

[0027] Figure 2 shows a segmentation structure of an image according to one embodiment.

[0028] Figure 3 shows the structure of an intra prediction according to one embodiment.

[0029] FIG. 4 shows the structure of an inter prediction to explain an inter prediction process according to one embodiment.

[0030] FIG. 5 shows the order of addition of spatial candidates to the candidate list according to one embodiment.

[0031] Figure 6 shows a plurality of in-loop filters according to one example.

[0032] Figure 7 shows the structure of entropy encoding and entropy decoding according to one example.

[0033] FIG. 8 is a flowchart of a method for performing intra prediction on a current block according to one embodiment of the present disclosure.

[0034] Figure 9 is a drawing illustrating spatial restoration blocks.

[0035] Figure 10 shows multiple candidates related to the configuration of the template.

[0036] FIG. 11 shows an example of deriving at least one directional prediction mode from reference samples of the current block.

[0037] Figure 12 shows an example where a filter is applied to some of the samples belonging to the luma block.

[0038] FIG. 13 is a diagram illustrating the process of exploring the intra-prediction mode of adjacent blocks.

[0039] Figure 14 shows an example of the configuration of a template.

[0040] Figure 15 shows an example in which a new reference block is specified to derive block vector candidates based on the enrichment parameters of the reference block.

[0041] Figure 16 shows an example of deriving the block vector of the current block through template-based search.

[0042] Figures 17 and 18 show examples of template configurations.

[0043] Figure 19 shows an example of calculating the error cost of a reference template based on the amount of change in sample values.

[0044] Various modifications may be applied to the present invention. Additionally, the present invention may have various embodiments. Specific embodiments are described by the drawings and the detailed description.

[0045] Specific embodiments are not intended to limit the invention to specific embodiments, and it should be understood that all modifications, equivalents, and substitutions that fall within the spirit and scope of the invention are included as embodiments of the invention.

[0046] The embodiments are described in sufficient detail to enable those skilled in the art to practice the embodiments. It should be understood that the various embodiments are different but need not be mutually exclusive. For example, it should be understood that the shapes, structures, and characteristics described in relation to one embodiment may be applied to or implemented in other embodiments without departing from the spirit and scope of the invention. It should also be understood that the location or arrangement of components within one embodiment may be changed without departing from the spirit and scope of the invention. Accordingly, the following detailed description is not intended to be limiting, and the scope of the exemplary embodiments is limited only by the appended claims and all equivalents to the scope claimed by such claims, provided that they are appropriately described.

[0047] The detailed description of the embodiments described below may refer to the drawings relating to the embodiments. Descriptions described in the drawings or descriptions represented by the drawings may be considered part of the detailed description. In the drawings, similar reference numerals may refer to the same or similar functions for various aspects. Dependencies between components may not be limited to those depicted in the drawings.

[0048] In the embodiments, singular expressions may include plural expressions and may be limited to and / or limited to plural expressions unless the context clearly excludes plural expressions. That is to say, in the embodiments, expressions such as 'at least one' and 'one or more' may be replaced with 'plural'. Terms such as ' / ', 'and / or', 'at least one of' and 'one or more of' described for plural items may mean 1) one of the plural items, 2) some of the plural items, 3) a combination of some of the plural items, or 4) a combination of the plural items. Additionally, plural expressions may be replaced with singular expressions. Plural may mean an integer of 1, 2, 3, 4, or 5 or more.

[0049] In the embodiments, numbered terms such as 'first' and 'second' may be used to describe various components. These terms are used solely for the purpose of distinguishing one component from another and do not limit the components. For example, without departing from the scope of the present invention, the first component may be named the second component, and similarly, the second component may be named the first component.

[0050] The statement that a first component transmits (or provides) information to a second component may mean that the first component directly transmits information to the second component, or it may mean that the first component transmits information to the second component through another third component. Here, the information received (or acquired) by the second component may be information transmitted by the first component, or information generated by applying a specific processing to information transmitted by the first component.

[0051] The components of the embodiments may be illustrated independently to represent different characteristic functions, and this does not imply that each component corresponds to a separate hardware or a single software unit. That is, the components of the embodiments may be classified and enumerated for convenience of description. Two or more components described in the embodiments may be regarded as a single component. Furthermore, a single component described in the embodiments may be separated into multiple components that perform the functions of the said component separately. Embodiments in which such components are integrated and embodiments in which components are separated are also included within the scope of the present invention, provided that they do not depart from the essence of the invention.

[0052] The terms used in the embodiments are used merely to describe specific embodiments and are not intended to limit the invention. In the embodiments, terms such as "comprising" or "having" indicate the presence of features, numbers, steps, actions, components, parts, or combinations thereof described in the embodiments. The existence or addition of other features, numbers, steps, actions, components, parts, or combinations thereof not explicitly described in the embodiments is not excluded by these terms. That is, the description of a specific component of an embodiment as "comprising" does not exclude components other than the specific component, and means that additional components may also be included within the scope of the embodiments or the technical concept of the invention.

[0053] Some of the components of the embodiments may be optional components that are not essential for performing the essential functions of the invention. Such optional components may be used to enhance performance. The embodiments may be implemented as a structure comprising only the essential components required to realize the essence of the embodiments, excluding the optional components. Such a structure is also included within the scope of the embodiments.

[0054] In the following, embodiments are described in detail with reference to the attached drawings so that a person skilled in the art can easily implement the embodiments. In describing the embodiments, if it is determined that a detailed description of related known configurations or known functions could obscure the gist of this specification, such detailed description is omitted. Additionally, the same reference numerals are used for identical components within the drawings, and redundant descriptions of identical components are omitted.

[0055]

[0056] Replacement of terms in the examples

[0057] Below, terms listed in a single line may be used with the same meaning in the examples and may be used interchangeably in the examples.

[0058] - 'one or more', 'at least one'

[0059] - 'two or more', 'a plurality of', 'multiple', 'multiple'. (In the examples, 'one or more' or 'at least one' may be further limited to 'two or more', 'multiple', or 'multiple'.)

[0060] - 'Information', 'Signal'

[0061] - 'value', 'predefined value', 'specific value', 'threshold', 'threshold value', 'baseline value', 'reference value'

[0062] - 'statistical value', 'statistics value'

[0063] - 'indicator', 'index', 'index', 'flag', 'information'

[0064] - 'encoder', 'encoding apparatus'

[0065] - 'decoder', 'decoding apparatus'

[0066] - 'Entropy encoding', 'encoding', 'encoding'

[0067] - 'Entropy decoding', 'decoding', 'decoding'

[0068] - 'Coding', 'Encoding and / or decoding'

[0069] - 'video', 'moving picture', 'image', 'picture', 'picture', 'frame', 'screen'

[0070] - 'Reference picture', 'Reference video'

[0071] - 'Reference Picture List (RPL), 'Reference Image List'

[0072] - 'original', 'input', 'source'

[0073] - 'Block', 'Unit', 'Signal'

[0074] - 'square', 'square shape'

[0075] - 'pixel', 'pixel', 'sample', 'pel'

[0076] - 'region', 'area', 'part', 'segment'

[0077] - 'partition', 'split', 'divide'

[0078] - 'quad', 'quadronary'

[0079] - 'Luma component', 'Luma', 'luminance component', 'luminance', 'Y'

[0080] - 'Chroma component', 'Chroma', 'chrominance', 'chrominance component', 'Cb and Cr', 'Cb or Cr', 'Cb', 'Cr', 'U and V', 'U or V', 'U', 'V'

[0081] - 'target', 'current' (e.g., target block and current block, or target image and current image)

[0082] - 'neighbor', 'neighboring', 'adjacent', 'neighbor / neighboring' (e.g., neighbor block, adjacent block, and neighboring block)

[0083] - 'collocated', 'COL'

[0084] - 'reconstruction', 'reconstruction', 'decoding'

[0085] - 'reconstructed', 'reconstructed', 'decoded'

[0086] - 'Difference', 'Difference', 'Difference', 'Error', 'Residual', 'Residual'

[0087] - Largest Coding Unit (LCU), Coding Tree Unit (CTU)

[0088] - 'inter', 'inter-screen'

[0089] - 'Inter prediction', 'inter prediction', 'motion compensation'

[0090] - 'Inter Mode', 'Inter Prediction Mode', 'Inter-frame Mode', 'Inter-frame Prediction Mode'

[0091] - 'Motion Vector', 'Predicted Motion Vector', 'Advanced Motion Vector Prediction (AMVP)'

[0092] - 'List', 'Candidate List'

[0093] - 'spatial candidate', 'spatial merge candidate'

[0094] - 'temporal candidate', 'temporal merge candidate'

[0095] - 'prediction motion vector candidate', 'motion vector predictor'

[0096] - 'Prediction method', 'Prediction mode'

[0097] - 'Intra', 'Inside the screen'

[0098] - 'Intra prediction', 'Intra prediction'

[0099] - 'Intra Mode', 'Intra Prediction Mode'

[0100] - 'Dequantization', 'Scaling'

[0101] - 'Quantization matrix', 'Scaling list'

[0102] - 'Quantization matrix coefficients', 'Matrix coefficients'

[0103] - 'transform coefficient level', 'quantized level', 'quantized coefficient', 'quantized transform coefficient', 'quantized transform coefficient level'

[0104] - 'dequantized coefficient', 'dequantized transform coefficient'

[0105] - 'Scanning type', 'Scanning direction'

[0106] - 'directional mode', 'angle mode', 'angular mode', 'intra-prediction mode'

[0107] - 'Intra-prediction mode (mode) number', 'Intra-prediction mode (mode) index', 'Intra-prediction mode (mode) value', 'Intra-prediction mode (mode) angle', 'Intra-prediction mode (mode) direction', 'Intra-prediction direction (mode) number', 'Intra-prediction direction (mode) index', 'Intra-prediction direction (mode) value', 'Intra-prediction direction (mode) angle'

[0108] - 'Merge Mode', 'Motion Merge Mode'

[0109] - 'Geometric Partitioning Mode (GPM)', 'Triangle Partitioning Mode'

[0110] In addition to the terms exemplified above, terms having the same meaning according to the ordinary knowledge of the technical field may be used interchangeably in the embodiments.

[0111]

[0112] Information and range of values ​​of information described in the embodiments

[0113] In the embodiments, information may include a constant, a flag, an index, a variable, a coding parameter, an element, a syntax element, motion information, an attribute, an entity, an object, and data, etc. That is to say, the term 'information' may be interchangeable with 'data', 'flag', 'index', 'variable', 'element', 'syntax element', 'motion information', 'attribute', or 'entity'.

[0114] Information can have one of multiple values. 'The nth value' can mean the nth value among multiple values.

[0115] For example, the first value can represent '0' or (logical) false. The second value can represent '1' or (logical) true. Or, the first value can represent '1' or (logical) true. The second value can represent '0' or (logical) false.

[0116] A flag may be information having a value of either '0' or '1'. In the embodiments, the values ​​'0' and '1' of the flag may be replaced with '1' and '0', respectively. For example, information indicating whether a specific process is performed or information indicating whether a specific process is applied may be considered as a flag.

[0117] When a variable such as i or j is used to represent a row, column, or index, the variable may be an integer between 0 and n - 1 inclusive. Or, the variable may be an integer between 1 and n inclusive. Here, n may be the number of rows, the number of columns, or the number of entities pointed to by the index.

[0118]

[0119] Concepts related to coding

[0120] Concepts related to coding are explained below. The descriptions disclosed below may be applied to embodiments.

[0121] Predefined value: A predefined value may refer to a value commonly used by the encoding device and the decoder. For example, a predefined value may be interpreted as being limited to a fixed value. Alternatively, a predefined value may be a value shared by the encoding device and the decoder through signaling. Alternatively, a predefined value may be a value derived through the same procedure in the encoding device and the decoder so that the encoding device and the decoder have a common value. Alternatively, a predefined value may be a common value possessed by the encoding device and the decoder. The above description of a predefined value may also apply to predefined information. In the above descriptions, 'value' may be replaced with 'information'.

[0122] Availability: The availability of specific modes for a specific target may mean that a selected mode among the specific modes is used for that specific target. Other modes belonging to the category of specific modes may be non-available modes. Non-available modes may not be used for a specific target. The above description of specific modes may also apply to other specific information. In the above descriptions, 'mode' may be replaced with 'information'.

[0123] Adjacency: 'Direction' for 'First Object'. 'Second Object' may refer to a 'Second Object' adjacent to the 'Direction' corner / face of the First Object. For example, the 'Top-left Block' for a 'Target Block' may be a block adjacent to the top-left of the Target Block. Here, the 'First Object' may be a Target Unit, Target Block, or Target Sample. 'Direction' may be one of left-above, above, right-above, left, right, left-below, below, and right-below. The 'Second Object' may be a Unit, Block, or Sample. For the directions of top-left, top-right, bottom-left, and bottom-right, the corner of the First Object and the corner of the Second Object may be diagonally adjacent. For the directions of top, left, right, and bottom, one face of the First Object and one face of the Second Object may be in contact with each other.

[0124] - For example, the block adjacent to the top-left of the target block may be the block adjacent to the top of the block adjacent to the left of the target block. The block adjacent to the top-right of the target block may be the block adjacent to the right of the block adjacent to the top of the target block. The block adjacent to the bottom-left of the target block may be the block adjacent to the bottom of the block adjacent to the left of the target block.

[0125] Coding: Coding can refer to encoding and / or decoding of an image.

[0126] Signal: A signal can represent information about an image, unit, or block. A specific signal can represent a specific image, a specific unit, or a specific block.

[0127] Image: An image can refer to a single picture constituting a video, or it can represent the video itself. For example, "encoding and / or decoding of an image" can mean "encoding and / or decoding of a video," or it can mean "encoding and / or decoding of one of the images constituting a video."

[0128] - An image can refer to the entirety of a picture, or it can refer to a part of a picture, such as a block.

[0129] Target image: The target image may be an encoding target image that is the subject of encoding and / or a decoding target image that is the subject of decoding. Additionally, the target image may be an input image processed by an encoding device and a restored image processed by a decoding device. The target image may be an image containing a target block.

[0130] Subpicture: A picture can be divided into one or more subpictures.

[0131] - A subpicture may be a square or rectangular area within the picture. A subpicture may include one or more CTUs.

[0132] - A subpicture may include one or more slices and / or one or more tiles. For example, a subpicture may consist of one or more slice rows and one or more slice columns. Alternatively, each subpicture may consist of one or more tile rows and one or more tile columns.

[0133] - A subpicture may include one or more slices that collectively cover a rectangular area within the picture. Accordingly, the boundary of each subpicture can always be the boundary of a slice. Additionally, each vertical subpicture boundary can always be the boundary of a vertical tile.

[0134] Slice: A slice may include one or more tiles within a picture. A slice may consist of one or more rows of tiles and one or more columns of tiles.

[0135] Tile: A tile can be a square or rectangular area within a picture. A tile can contain one or more CTUs. A picture can be divided into one or more tile rows and one or more tile columns.

[0136] CTU: An image can be divided into multiple Coding Tree Units (CTUs).

[0137] - A CTU may include one Y Coding Tree Block (CTB) and at least one of a Cb CTB and a Cr CTB associated with the Y CTB, and may include information for each CTB. The information may include syntax elements.

[0138] - Each CTU may be partitioned using one or more partitioning methods to form sub-units such as Coding Units (CU), Prediction Units (PU), and Transform Units (TU). One or more partitioning methods may include Quad Tree (QT) partitioning, Binary Tree (BT) partitioning, and Ternary Tree (TT) partitioning. Additionally, each CTU may be partitioned using Multi-Type Tree (MTT) partitioning, which uses a combination of multiple partitioning methods.

[0139] CTB: CTB can refer to one of Y CTB, Cb CTB, and Cr CTB.

[0140] Unit: A unit can be determined for specific processing in coding. A unit may be information about a specific region within an image. For specific processing in coding, the image may be recursively divided into multiple parts. A unit may represent the region to which the specific processing is applied and information about the aforementioned region.

[0141] - The unit type may represent a specific process applied to the unit. Depending on the unit type, a specific process may be applied to the unit. The 'specific' unit may be a unit for the process named 'specific' in the coding. For example, the unit may be at least one of the source unit, CTU, coding unit, prediction unit, residual unit, restored residual unit, transformation unit, and restored unit.

[0142] - A unit may include samples having a two-dimensional form or arrangement. In this respect, a 'unit' may mean a 'block'. For example, a block may be at least one of an original block, a CTB, a coding block (CB), a prediction block (PB), a residual block, a restored residual block, a transform block (TB), and a restored block. For example, a partition of a unit may mean a partition of a block corresponding to the unit.

[0143] - A unit may include syntactic elements. In other words, a block and the syntactic elements for the block can be combined and referred to as a unit.

[0144] - A block is an MxN array of samples. Here, M and N can represent positive integer values, and a block can commonly represent a two-dimensional array of samples. The current block can represent the encoding target block that is the subject of encoding during encoding, or the decoding target block that is the subject of decoding during decoding. Additionally, the current block can be at least one of a coding block, a prediction block, a residual block, a transformation block, or a restoration block. Blocks can have various sizes and shapes. For example, the shape of a block can be one or more of a tetragon, a rectangular, a square, a rectangle where the width differs from the height (i.e., an oblong), a trapezoid, a triangle, a right-angled triangle, and a pentagon. Here, the width and height of the rectangle can differ from each other. Additionally, the shape of a block may include other geometric figures that can be represented in two dimensions. For example, the shape of the block may be a square or a pentagon defined by subtracting the area of ​​a right triangle from the area of ​​a rectangle. Here, the right-angled vertex of the right triangle may be one of the vertices of the rectangle. Additionally, the shape of the block may be a combination of two or more of the aforementioned shapes. Additionally, the shape of the block may be the remainder of one of the aforementioned shapes after another shape has been subtracted.

[0145] - In the embodiments, the rectangle may be limited to a non-square rectangle. When the shape of a specific object in the embodiments is described as a rectangle, this description may additionally imply that the width and height of the specific object are different from each other.

[0146] - In the embodiments, the block may be limited to at least one of a vertically oriented block and a horizontally oriented block. A vertically oriented block may mean a block in which the vertical length is greater than the horizontal length. A horizontally oriented block may mean a block in which the horizontal length is greater than the vertical length.

[0147] - The unit may include a luma component block (i.e., a Y block) and two chroma component blocks (i.e., at least one of a Cb block and a Cr block), and may include information for each block. The information may include syntax elements.

[0148] - The unit information may include the unit type, unit size, unit depth, unit encoding order, and unit decoding order.

[0149] Target Unit: The target unit may be a block that is the target of encoding, an encoding target unit, and / or a block that is the target of decoding. The target unit may be a specific region within the target picture to which one or more specific processing steps of coding are applied. By applying a specific processing step to the target unit, a unit of a specific type may be generated. Alternatively, the target unit may represent a unit having a specific type for a specific processing step of coding.

[0150] Depth: A block can be hierarchically divided into multiple sub-blocks with depth according to a tree structure. The multiple sub-blocks created by the division of a block can be referred to as partitions.

[0151] - The block depth can represent the level of the node corresponding to the block when the blocks constituting the image are represented as a tree structure. Alternatively, the block depth can represent the number of divisions applied until the block is determined. The block depth can increase by 1 as the block is further divided.

[0152] - In a tree structure, the root node can be considered to have the smallest level, and the leaf node the largest level. The root node may be the top node of the tree structure and may correspond to the first undivided block. The level of the root node may be 0 or 1. When the level of the root node is 0, a node with level 1 may represent the block determined by the first block being divided once. A node with level n may represent the block determined by the first block being divided n times. A leaf node may be the lowest node of the tree structure. A leaf node may be a node that cannot be further divided. The depth of a leaf node may be a predefined maximum depth. For example, the maximum depth may be a positive integer such as 3. The root node may represent a CTU. A leaf node may represent at least one of a CU, PU, ​​or TU.

[0153] - Depth can have a type depending on the type of partition. QT depth can represent the depth for quadtree partitioning. BT depth can represent the depth for binary partitioning. TT depth can represent the depth for ternary partitioning.

[0154] Sample: A sample can be a base unit that constitutes a block. A sample can consist of one or more bits. Bit depth can be the number of bits that make up the sample. Samples range from 0 to 2 depending on the bit depth. Bd It can be expressed as values ​​up to -1.

[0155] PU: PU may refer to a base unit for processing related to prediction. For example, processing related to prediction may include inter-prediction, intra-prediction, intra-block copy (IBC) prediction, intra-compensation, and motion compensation.

[0156] A single PU can be divided into multiple sub-PUs that are smaller in size compared to the PU. These multiple sub-PUs can also serve as base units for processing related to prediction. In other words, a prediction unit partition generated by the division of the prediction unit can also be a prediction unit.

[0157] TU: A TU may be a base unit for processing related to a residual block. Processing related to a residual block may include at least one of transform, inverse transform, quantization, inverse quantization, transform coefficient encoding, transform coefficient decoding, entropy encoding, and entropy decoding. A single TU may be divided into a plurality of sub-transform units having a size smaller than that of the TU. The plurality of sub-TUs may also be base units for processing related to a residual block. That is to say, a transform unit partition generated by the division of the transform unit may also be a transform unit.

[0158] - The transformation may include one or more of a primary transformation and a secondary transformation, and the inverse transformation may include one or more of a primary inverse transformation and a secondary inverse transformation.

[0159] Parameter set: The parameter set can correspond to header information within the structure of the bitstream.

[0160] - The parameter set may include at least one of a Video Parameter Set (VPS), a Sequence Parameter Set (SPS), a Picture Parameter Set (PPS), an Adaptation Parameter Set (APS), and a Decoding Parameter Set (DPS).

[0161] Information signaled through a parameter set can be applied to pictures that reference the parameter set. For example, information within a VPS can be applied to pictures that reference the VPS. Information within an SPS can be applied to pictures that reference the SPS. Information within a PPS can be applied to pictures that reference the PPS. A parameter set can reference a higher-level parameter set. For example, a PPS can reference an SPS. An SPS can reference a VPS.

[0162] - Additionally, the parameter set may include tile group information, slice header information, and tile header information. A tile group may refer to a group or slice containing multiple tiles.

[0163] MPM (Most Probable Mode): MPM may represent an intra prediction mode that is likely to be used for intra prediction of a target block.

[0164] - One or more different MPMs can be determined based on coding parameters related to the target block and attributes of objects related to the target block.

[0165] - One or more MPMs may be determined based on the intra prediction mode of a reference block. There may be multiple reference blocks. One or more different MPMs may be determined depending on which intra prediction modes are used for one or more reference blocks. Reference blocks may include spatial neighbor blocks.

[0166] MPM List: An MPM list may be a list containing one or more MPMs. The number of one or more MPMs in an MPM list may be predefined.

[0167] MPM Index: The MPM index can indicate one or more MPMs in the MPM list that are used for intra prediction for the target block.

[0168] MPM Usage Indicator: The MPM Usage Indicator can indicate whether an MPM list is used for prediction regarding a target block.

[0169] Prediction mode: The prediction mode may be information indicating a prediction method for a target block, such as a mode used for intra-prediction or a mode used for inter-prediction. The prediction mode may refer to one of the prediction-related modes described in the embodiments. Additionally, the prediction mode may include at least one of an intra-mode, an inter-mode, and an intra-block copy mode.

[0170] Reference image list: The reference image list may be a list containing one or more reference images used for prediction of the target block.

[0171] - There may be multiple reference image lists. Multiple reference image lists may include List 0 (List 0; L0), List 1 (List 1; L1), etc.

[0172] - One or more reference image lists may be used for inter prediction for the target block. Parts such as 'L0' and 'L1' in the names of the information related to inter prediction may refer to the reference image lists associated with the information.

[0173] Reference picture: The reference picture may be an image referenced for prediction regarding the target block. Alternatively, the reference picture may be an image containing the reference block. The reference picture may include an image prior to the target image, the target image, and an image following the target image.

[0174] Reference image index: The reference image index may be an index indicating one reference image among one or more reference images in the reference image list that is used for prediction of the target block.

[0175] Reference Block: A reference block may be a block referenced for encoding / decoding of a target block, such as for prediction and filtering. For example, a reference block may include a reference sample referenced to derive a prediction sample, and may refer to a block that provides information used for decoding the target block.

[0176] Reference Sample: A reference sample may be a sample referenced for encoding / decoding of a target block, such as prediction and filtering.

[0177] Inter prediction indicator: The inter prediction indicator may indicate the direction of inter prediction for the target block. Inter prediction may be one of unidirectional prediction and bidirectional prediction. Alternatively, the inter prediction indicator may indicate the number of reference images used when generating prediction blocks for the target block. Alternatively, the inter prediction indicator may indicate the number of prediction blocks used for inter prediction for the target block. The reference direction may refer to the inter prediction indicator. For example, the inter prediction indicator may indicate either unidirectional or bidirectional. Alternatively, for an inter mode that uses only reference images within the L0 reference image list, the inter prediction indicator may have a first value of '0'; for an inter mode that uses only reference images within the L1 reference image list, the inter prediction indicator may have a second value of '1'; and for an inter mode that uses at least two of the reference images within the L0 reference image list and the L1 reference image list, the inter prediction indicator may have a third value of '2'.

[0178] Prediction List Utilization Flag: The prediction list utilization flag for a specific reference image list may indicate whether at least one reference image within that specific reference image list is used to generate the prediction block of the target block. For example, a value of '0' for the prediction list utilization flag for a specific reference image list may indicate that the prediction block is not generated using the reference images within that specific reference image list. A value of '1' for the prediction list utilization flag for a specific reference image list may indicate that the prediction block is generated using the reference images within that specific reference image list.

[0179] - An inter-prediction indicator can be derived using prediction list utilization flags. Conversely, an inter-prediction indicator can be derived using prediction list utilization flags. For example, an inter-prediction indicator can be derived using prediction list utilization flags for multiple reference image lists. If the inter-prediction indicator indicates that specific reference lists among the multiple reference image lists are being used, the prediction list utilization flags of the specific reference lists pointed to by the inter-prediction indicator among the prediction list utilization flags of the multiple reference image lists can be set to '1', and the prediction list utilization flags of the remaining reference image lists not pointed to by the inter-prediction indicator can be set to '0'.

[0180] Reference Direction: The reference direction may point to a list of reference images used for prediction of the target block. For example, the reference direction may point to one or more of reference image list L0 and reference image list L1.

[0181] - The reference direction merely refers to the list of reference images used for prediction of the target block, and does not indicate that the directions of the reference images within the list are restricted to a forward direction or a backward direction. That is to say, each of the reference image list L0 and the reference image list L1 may include forward images and backward images, respectively. Here, the forward direction may indicate a direction from the target image to the image preceding the target image. Forward inter-prediction may be an inter-prediction that uses the image preceding the target image as a reference image. The backward direction may indicate a direction from the target image to the image following the target image. Backward inter-prediction may be an inter-prediction that uses the image following the target image as a reference image.

[0182] - A unidirectional reference direction may mean that a single reference image list is used. A bidirectional reference direction may mean that two reference image lists are used. For example, the reference direction may indicate one of the following: that only reference image list L0 is used, that only reference image list L1 is used, or that two reference image lists are used. Additionally, the reference direction may be indicated by an inter-predictor.

[0183] Picture Order Count (POC): The POC of a picture can represent the display order or output order of the picture.

[0184] Motion information: Motion information may be information used to specify a reference block. Motion information may include information used for inter prediction, such as a motion vector (MV), reference image index, reference image, inter prediction indicator, prediction list utilization flag, etc. Additionally, motion information may include information used in a specific inter prediction mode, such as an MV candidate, MV candidate index, merge candidate, and merge index.

[0185] - Multiple motion information for multiple reference image lists may be used for inter-prediction of the target block. Motion information for a specific reference image list may be used for prediction using that specific reference image list. Multiple (intermediate) prediction blocks may be derived from the multiple motion information. A (final) prediction block for the target block may be generated using statistical values ​​for the multiple (intermediate) prediction blocks.

[0186] MV: MV can be a 2-dimensional vector used in inter-prediction. MV can represent the offset between the target block and the reference block. Alternatively, MV can represent the difference between the location of the target block and the location of the reference block.

[0187] - For example, MV is (mv x , mv y It can be expressed in the form of ). mv x can represent a horizontal component, and mv y It can represent a vertical component.

[0188] - The zero vector can be (0, 0) MV.

[0189] Block Vector (BV): A BV can be a two-dimensional vector used in intra-block copy prediction. A BV can represent the offset between a target block within a target image and a reference block within a target image. In other words, a BV can represent the displacement between a target block and a reference block within a target image.

[0190] - For example, BV is similar to MV (bv x , bv y It can be expressed in the form of ). bv x can represent a horizontal component, and bv y It can represent a vertical component.

[0191] - The zero vector can be (0, 0) BV.

[0192] Motion Information Candidates: In a specific prediction, motion information of the target block can be selected from motion information candidates determined by a specific method. A motion information candidate may refer to the motion information of a reference block, or it may refer to the reference block itself that possesses motion information. Here, the reference block may be a block determined by a specific method to select motion information candidates.

[0193] Candidate List: A candidate list may be a list containing one or more candidates. For example, a candidate list may include a motion information candidate list, a merge candidate list, an MV candidate list, an MPM list, etc. A candidate list may be generated in the same manner in both the encoding device and the decoder. That is to say, the candidate list used in the encoding device and the candidate list used in the decoder may be identical, and the same candidate list may be shared between the encoding device and the decoder. The encoding device may select a candidate from among the candidates in the candidate list to be used for processing the target block. An indicator pointing to the selected candidate may be signaled from the encoding device to the decoder. The decoder may use the indicator to identify the candidate from among the candidates in the candidate list to be used for processing the target block. Alternatively, the encoding device and the decoder may identify the candidate from among the candidates in the candidate list to be used for processing the target block by the same rule.

[0194] Motion Information Candidate List: A motion information candidate list may refer to a list constructed using one or more motion information candidates.

[0195] Motion Information Candidate Index: The motion information candidate index may be an identifier or indicator pointing to a motion information candidate used for prediction of a target block among the motion information candidates in the motion information candidate list.

[0196] - In a specific inter-prediction mode, motion information of other restored blocks may be used to derive motion information of the target block. Other blocks may include neighboring blocks. In this specific inter-prediction mode, the motion information for the target block itself is not signaled individually, but other information used to derive motion information of the target block based on motion information of other restored blocks may be signaled. In this case, the other information may include information indicating which of the other restored blocks' motion information is used to derive motion information of the target block, such as a motion information candidate index.

[0197] - For example, these inter-prediction modes may include AMVP mode, merge mode, and skip mode. The motion information candidate index may be a merge index or an MV candidate index.

[0198] - In the embodiments, MV may be part of the motion information. In the embodiments, information about motion information, such as motion information candidates, a list of motion information candidates, and an index of motion information candidates, may be replaced with information about MV, such as MV candidates, a list of MV candidates, and an index of MV candidates, and descriptions of motion information may also be applied to MV.

[0199] Merge: Merge can refer to the merging of motion information for multiple blocks, or it can refer to applying the motion information of one block to a target block as well. In other words, merge mode can refer to a mode where the motion information of a target block is derived from the motion information of a neighboring block.

[0200] Merge Candidate: A merge candidate may refer to a specific (restored) block used for merging with a target block, or it may refer to movement information of a specific block. Alternatively, a merge candidate may include movement information of a specific block.

[0201] - Merge candidates for the target block may include spatial merge candidates, temporal merge candidates, history-based candidates, average candidates based on the average of two merge candidates, and zero merge candidates.

[0202] Merge candidate list: The merge candidate list may be a list composed of one or more merge candidates.

[0203] Merge Index: The merge index may be an indicator pointing to a merge candidate among the merge candidates in the merge candidate list that is used for prediction regarding the target block. Among the merge candidates in the merge candidate list, the movement information of the merge candidate indicated by the merge index may be used as movement information for the target block.

[0204] Neighbor block: A neighbor block may refer to a block adjacent to the target block. Neighbor blocks may include spatial neighbor blocks and temporal neighbor blocks. A neighbor block may also refer to a reconstructed neighbor block within the reference image.

[0205] Spatial neighbor blocks: Spatial neighbor blocks can be blocks that are spatially adjacent to the target block.

[0206] - The target block and spatial neighbor blocks can be included within the target image.

[0207] - Spatial neighbor blocks may include blocks whose boundaries, at least a portion of which abuts at least a portion of the target block's boundary. Alternatively, spatial neighbor blocks may include blocks whose distance from the target block is less than or equal to a specific value.

[0208] - Spatial neighbor blocks may include blocks diagonally adjacent to the vertices of the target block.

[0209] - Spatial neighbor blocks may include a top-left block adjacent to the top-left of the target block, a top block adjacent to the top of the target block, a top-right block entered at the top-right of the target block, a left block adjacent to the left of the target block, a right block adjacent to the right of the target block, a bottom-left block adjacent to the bottom of the target block, and a bottom-right block adjacent to the bottom-right of the target block.

[0210] Temporal neighbor blocks: Temporal neighbor blocks can be blocks that are temporally adjacent to the target block.

[0211] - Temporal neighbor blocks may include a collocated block (COL block). A collocated block may be a block within a restored image in a reference image buffer. A collocated picture (col picture) may refer to an image containing a collocated block. A collocated picture may be an image included in a reference image list.

[0212] - Call blocks can be determined based on the location of target blocks within the target image. Two blocks being 'temporarily adjacent' may mean that the locations of the two blocks satisfy certain conditions.

[0213] - The position of the call block within the call image may be the same as the position of the target block within the target image. Alternatively, the position of the call block within the call image may correspond to the position of the target block within the target image. Here, the correspondence of the block positions may mean that the regions of the blocks are identical, that the region of one block is included within the region of another block, or that one block occupies a specific location within another block.

[0214] - For example, the location of a call block within a call image may be the same as the location of a target block within a target image. Alternatively, the call block may be a block containing call samples within a call image. A call sample may be a sample having coordinates identical to the coordinates of a specific sample in the target block.

[0215] - Temporal neighbor blocks may be blocks that are temporally adjacent to the spatial neighbor blocks of the target block.

[0216] Search range: The search range may refer to a two-dimensional area where a search for an MV is performed during inter-prediction. For example, when an optimal MV needs to be derived for processing a target block, the optimal MV can be selected from among the MVs pointing inside the search range.

[0217] Transform coefficient: The transform coefficient may be a coefficient generated by performing a transformation on the residual block. Alternatively, the transform coefficient may be a coefficient value generated by performing inverse quantization on the quantized level.

[0218] Quantized level: A quantized level can be an integer quantity used as an input for inverse quantization.

[0219] Quantization: Quantization can be a process that generates quantized levels for transformation coefficients. Quantized levels can be generated by applying quantization to transformation coefficients. Transformation can also be considered as part of quantization.

[0220] Inverse Quantization: Inverse quantization can be a process of multiplying a quantized level by a factor. By applying inverse quantization to the quantized level, (restored) transformation coefficients can be generated.

[0221] Quantization Parameter (QP): QP may refer to the argument used to generate quantized levels for transform coefficients in quantization. Additionally, QP may refer to the argument used to generate (restored) transform coefficients for quantized levels in inverse quantization. Alternatively, QP may be a value mapped to the quantization step size.

[0222] Delta QP: Delta QP can be the difference between the QP predicted by a specific process and the QP of the target block. In other words, the QP of the target block can be the sum of the predicted QP and Delta QP.

[0223] Quantization matrix: A quantization matrix may be a matrix used in quantization or inverse quantization to improve the subjective or objective image quality.

[0224] Quantization matrix coefficients: Quantization matrix coefficients can be each element within the quantization matrix.

[0225] Scan: Scan can refer to a method of arranging values ​​within a block or matrix. The values ​​can be coefficients. For example, a scan can mean arranging values ​​arranged in a 2D form into a 1D form, or rearranging values ​​arranged in a 1D form into a 2D form. An inverse scan can be the opposite arrangement (or rearrangement) of the arrangement performed in a scan.

[0226] Non-zero transformation coefficients: Non-zero transformation coefficients may refer to transformation coefficients that have a non-zero value or quantized levels that have a non-zero value.

[0227] Bitstream: A bitstream may refer to a series or sequence of bits containing encoded information generated by encoding of an image. A bitstream may contain information according to specific syntax elements. For example, the information may include syntax elements. An encoding device may generate a bitstream containing information according to specific syntax elements. A decoder may obtain information from the bitstream according to specific syntax elements.

[0228] Signaling: Signaling of information may indicate that information is transmitted from an encoding device to a decoding device via a bitstream. For example, the information may include syntactic elements. Alternatively, signaling may mean that the encoding device includes information within the bitstream. Information signaled by the encoding device may be used by the decoding device. In signaling, the bitstream may be transmitted over a network and may be contained within a recording medium. In embodiments, the description that information is signaled may include: 1) the encoding device determining and generating information for signaling of information; 2) the encoding device performing encoding on the information to generate encoded information; 3) the (encoded) information being transmitted from the encoding device to the decoding device via a bitstream; 4) the decoding device performing decoding on the encoded information to obtain information; and 5) the decoding device determining and generating information through signaling of information.

[0229] - An encoding device can generate encoded information by performing encoding on the information. The encoded information can be signaled through a bitstream. A decoding device can obtain information by performing decoding on the encoded information.

[0230] - The fact that information is signaled to a specific target may mean that the information is used for each specific target, and that the processing represented by the information is applied to each specific target. For example, the fact that information is signaled at a specific unit level may indicate that the information is used or processed for each specific unit.

[0231] - The signaled information may include one or more sub-information. That specific information is signaled may mean that each piece of information of the one or more sub-information included in the specific information is signaled.

[0232] Optional Signaling: Signaling for information may be performed optionally. Optional signaling for information may mean that an encoding device optionally includes information within a bitstream (depending on specific conditions). Optional signaling for information may mean that a decoder optionally obtains information from a bitstream (depending on specific conditions).

[0233] Omission of Signaling: Signaling for information may be omitted. Omission of signaling for information may mean that the encoding device does not include information in the bitstream (depending on specific conditions). Omission of signaling for information may mean that the decoding device does not obtain information from the bitstream (depending on specific conditions). The decoding device may derive information with omitted signaling using other information of the embodiments.

[0234] Symbol: May represent at least one piece of information of a target unit, such as syntactic elements, coding parameters, quantized levels, and transform coefficients of a target unit or target block. Additionally, the symbol may represent the target of entropy encoding or the result of entropy decoding.

[0235] Entropy encoding: Entropy encoding can allocate a small number of bits to symbols with a high probability of occurrence and a large number of bits to symbols with a low probability of occurrence. Through this allocation, the size of the bitstream representing the symbols can be reduced.

[0236] Entropy coding can utilize methods such as Variable Length Coding (VLC) and Context-Adaptive Binary Arithmetic Coding (CABAC). For example, in Variable Length Coding, entropy coding can be performed using variable-length tables. For instance, in CABAC, a binaryization method for symbols and a probabilistic model of symbols / bins can be derived for entropy coding, and context-based arithmetic coding can be performed.

[0237] Entropy Decoding: In entropy decoding, the processes performed in entropy encoding can be performed in reverse. Symbols can be generated by entropy decoding of a bitstream.

[0238] Parsing: Parsing can refer to determining the values ​​of syntactic elements by performing entropy decoding on the encoded information of a bitstream. Alternatively, parsing can refer to entropy decoding itself.

[0239] Statistical Value: The values ​​of information related to specific entity(s) described in the embodiments may be used as inputs for specific operations. The statistical value may be a value derived by a specific operation on the values ​​related to these specific entity(s). For example, the statistical value for specific information may be one or more of the following: an average value, a weighted average value (weighted average), a weighted sum (weighted sum), a minimum value, a maximum value, a mode, a median value, an interpolated value, a sum of products, and a product of sums. Additionally, information of the embodiment having specific values ​​determined by operations, such as constants, variables, and coding parameters, may have a specific statistical value according to the embodiment.

[0240]

[0241] Coding parameters

[0242] In the embodiments, the coding parameters may be information required for coding. The coding parameters may include information signaled from an encoding device to a decoder, information calculated / derived during the processing of coding described in the embodiments, and information used for the processing of coding described in the embodiments.

[0243] In the embodiments, the coding parameters include the size of the CTU, the size of the unit, the form of the unit, the shape of the unit, the depth of the unit, the minimum unit size, the maximum unit size, the maximum unit depth, the minimum unit depth, the unit splitting information, QT splitting information, BT splitting information, the splitting direction of the BT splitting, the splitting form of the BT splitting, TT splitting information, the splitting direction of the TT splitting, the splitting form of the TT splitting, MTT splitting information, the combination of MTT splittings, the splitting direction of the MTT splitting, the splitting form of the MTT splitting, the prediction mode, the intra prediction mode, the luminance intra prediction mode, the chroma intra prediction mode, the intra prediction mode, the inter splitting information, the coding block splitting information, the prediction block splitting information, the transformation block splitting information, the reference sample line index, the reference sample filtering method, the reference sample filter tab, the reference sample filter coefficients, the prediction block filtering method, the prediction block filter tab, the prediction block filter coefficients, the prediction block boundary filtering method, the prediction block boundary filter tab, the prediction block boundary filter coefficients, the inter prediction mode, motion information, MV, and Motion Vector Difference; MVD), MVD resolution, MV size, MV representation accuracy, reference image list, reference image, reference image index, inter prediction direction, inter prediction indicator, prediction list utilization flag, POC, MV candidate, MV candidate index, MV candidate list, AMVP mode usage information, merge candidate, merge index, merge candidate list, merge mode usage information, motion information correction information, skip mode usage information, intra-block copy mode usage information, BV (Block Vector), Block Vector Difference (BVD), BVD resolution, BV size, BV representation accuracy, BV candidate, BV candidate index, BV candidate list, interpolation filter filter tab, interpolation filter filter coefficients, transform type, transform size, transform selection information, primary transform usage information,Secondary transform usage information, primary transform selection information, secondary transform selection information, residual block presence information, coded block pattern, coded block flag, QP, delta QP, quantization matrix, deblocking filter usage information, deblocking filter coefficients, deblocking filter filter tab, deblocking filter strength, deblocking filter shape / form, adaptive sample offset usage information, adaptive sample offset value, adaptive sample offset category, adaptive sample offset type, adaptive loop filter usage information, adaptive loop filter coefficients, adaptive loop filter filter tab, adaptive loop filter shape / form, binarization / debinarization method, context model, context model determination method, context model update method, regular mode usage information, bypass mode usage information, significant coefficient flag, last significant coefficient flag, coefficient group coding flag, last significant coefficient position, flag indicating whether the coefficient value is greater than 1, whether the coefficient value is greater than 2 Flag indicating presence, flag indicating whether the coefficient value is greater than 3, remaining coefficient value information, sign information, context bin, bypass bin, restored sample, restored luminance sample, restored chroma sample, residual sample, residual luminance sample, residual chroma sample, transform coefficient, luminance transform coefficient, chroma transform coefficient, transform coefficient level, luminance transform coefficient level, chroma transform coefficient level, transform coefficient level scanning method, quantized level, luminance quantized level, chroma quantized level, size of the MV seek area on the decoder side, shape of the MV seek area on the decoder side, number of MV seeks on the decoder side, picture type, slice identification information, slice type, slice splitting information, tile group identification information, tile group type, tile group splitting information, tile identification information, tile type, tile splitting information, bit depth,It may include one or more of input sample bit depth, restored sample bit depth, residual sample bit depth, transform factor bit depth, quantized level bit depth, mapping availability information, information about the luminance signal, information about the chroma signal, the color space of the target block, the color space of the residual block, and temporal layer information.

[0244] In addition, the coding parameter may further include 1) a value of information that may be included in the coding parameter, 2) a combination of multiple pieces of information that may be included in the coding parameter, 3) a statistical value of information that may be included in the coding parameter, 4) information related to the coding parameter, 5) information used to calculate / derive the coding parameter, and 6) information calculated / derived using the coding parameter.

[0245] In the embodiments, "X usage information" may be "information indicating whether X is used / applied / executed." Alternatively, "X usage information" may be "information indicating whether X is available." For example, "specific mode usage information" may be information indicating whether a specific mode is used. Mode information may indicate a mode used for a target block among the modes described in the embodiments. In the embodiments, specific mode usage information may be replaced with mode information, and the description of specific mode usage information may also apply to mode information. "X usage information" and "X indicator" may be used interchangeably.

[0246] In the embodiments, coding parameters and syntax elements may correspond to each other. For example, a syntax element of the embodiment may be used as a coding parameter, and a coding parameter may be signaled as a syntax element.

[0247] In the embodiments, "X existence information" may be considered as "information indicating whether X exists" or "information indicating whether information indicating X exists within the bitstream".

[0248] In the embodiments, "X selection information" may be information indicating one of the candidates or methods for X. "X selection information" may be considered as an "X index".

[0249] In the embodiments, the splitting form of a specific tree may represent one of symmetric splitting and asymmetric splitting, and may represent one of QT, BT, TT, and non-split. The splitting direction of a specific tree may represent one of horizontal direction and vertical direction.

[0250] In the embodiments, when the coding parameter has one of a plurality of values, "coding parameter" may be replaced with "whether the coding parameter has a specific value among the plurality of values ​​available to the coding parameter".

[0251] In the embodiments, when the coding parameter refers to one of a plurality of targets, the "coding parameter" may be replaced with "whether the coding parameter refers to a specific target among the plurality of targets."

[0252]

[0253] System for video coding

[0254] FIG. 1 shows a system for video coding according to one embodiment.

[0255] The system (100) may include at least one of an encoding device (110) and a decoding device (150).

[0256] Each of the encoding device (110) and the decoding device (150) may be a computer or an electronic apparatus.

[0257]

[0258] Structure of the encoding device

[0259] The encoding device (110) may include a processor (120), a storage (140), and a communicator (149).

[0260] The processor (120), storage (140), and communication device (149) can be connected via a bus.

[0261] The processor (120) may be a semiconductor device that executes instructions or computer-executable code, such as a Central Processing Unit (CPU). The processor (120) may be at least one hardware processor.

[0262] The processor (120) can perform generation and processing of information that is input to the encoding device (110) in the embodiments, output from the encoding device (110), or used inside the encoding device (110), and can perform comparison and judgment related to such information.

[0263] The processor (120) may include a plurality of components. The plurality of components may include a partitioner (122), a subtractor (124), a transformer (125), a quantizer (126), an inverse quantizer (127), an inverse transformer (128), an adder (129), a filter (130), and an entropy encoder (139).

[0264] At least some of the aforementioned multiple components may be program modules. Program modules may be included in the encoding device (110) in the form of an operating system, an application, and other program modules. Program modules may be instructions or computer-executable code stored in a storage (140) and executed by a processor (120).

[0265] The storage (140) may include various types of volatile storage media and non-volatile storage media. For example, the storage (140) may include memory such as ROM and RAM.

[0266] The storage (140) can store instructions and computer-executable code used for the operation of the encoding device (110), and can store information and bitstreams as described in the embodiments. The storage (140) may include a reference picture buffer (141).

[0267] The communication device (149) can perform functions related to the communication of information in the encoding device (110). For example, the communication device (149) can transmit a bitstream to the decoding device (150).

[0268] Among the names of the components of the encoding device (110), "-gi" ("-er" or "-or") may be replaced with "-bu" (- unit). The storage unit (140) may also be named a storage unit.

[0269]

[0270] Operation of the encoding device

[0271] The encoding device (110) can sequentially encode one or more images of the video.

[0272] The storage (140) can store the original image. In the encoding device (110), the original image can be used as the target image.

[0273] The processor (120) can generate a bitstream containing encoded information by performing encoding on the target image and can store the generated bitstream in a storage (140). The generated bitstream can be stored on a computer-readable recording medium and can be transmitted by the communication device (149) to the communication device (189) of the decoding device (150) via a wired and / or wireless transmission medium.

[0274] The splitter (122) can determine the target block by performing a split on the target image.

[0275] The predictor (123) can determine the prediction mode of the target block. The predictor (123) can generate a prediction block of the target block by performing a prediction according to the prediction mode.

[0276] The prediction mode of the target block may be one of the available prediction modes. For example, available prediction modes may include intra prediction, inter prediction, and IBC prediction.

[0277] For example, if the prediction mode is intra prediction, the predictor (123) can perform intra prediction on the target block to generate a prediction block of the target block.

[0278] For example, if the prediction mode is inter-prediction, the predictor (123) can perform inter-prediction on the target block to generate a prediction block of the target block.

[0279] For example, if the prediction mode is IBC, the predictor (123) can perform an IBC prediction for the target block to generate a prediction block of the target block.

[0280] The subtractor (124) can generate a residual block of the target block. The residual block may be the difference between the original block and the prediction block. The original block may be the region of the original image pointed to by the target block. Alternatively, the residual block may refer to a block generated by applying one or more of transformation and quantization to the difference between the original block and the prediction block.

[0281] The converter (125) can perform a conversion on the residual block to generate conversion coefficients.

[0282] The converter (125) can perform the conversion using one of a plurality of conversion methods.

[0283] For example, multiple transformation methods may include the Discrete Cosine Transform (DCT), Discrete Sine Transform (DST), Karhunen-Loeve Transform (KLT), and transformations based on each transformation.

[0284] The transform skip mode may be a mode that generates a restored block using the restored residual block and prediction block, for which transform and inverse transform have not been performed. When the transform skip mode is applied to a target block, the transform and inverse transform for the target block may be omitted, and only quantization and inverse quantization for the target block may be performed.

[0285] The quantizer (126) can generate quantized levels by applying quantization using quantization parameters to the conversion coefficients. In the embodiments, the quantized levels may also be referred to as conversion coefficients.

[0286] The entropy encoder (139) can generate encoded information by performing entropy encoding based on a probability distribution on information for decoding an image. The bitstream may contain encoded information.

[0287] Information for decoding the image may include quantized levels and syntax elements produced by the quantizer (126).

[0288] The probability distribution can be determined based on quantized levels and coding parameters.

[0289] The entropy encoder (139) can convert quantized levels, which have the form of a two-dimensional block, into the form of a one-dimensional vector by using scanning to perform encoding for the quantized levels. In the scanning, it can be determined which scan to use among an upper-right diagonal scan, a vertical scan, and a horizontal scan based on coding parameters such as the size of the block and the intra-prediction mode of the block.

[0290] When encoding is performed on a target image / block, the predictor (123) uses a reference image / block for prediction. The encoded target image / block can be used as a reference image / block for other images / blocks that are subsequently processed. Accordingly, the processor (120) can perform restoration on the encoded target block and can store the restored image containing the restored target block generated by the restoration in the reference picture buffer (141) as a reference image. Inverse quantization and inverse transform can be performed on the encoded target block for restoration.

[0291] The inverse quantizer (127) can generate inverse quantized conversion coefficients by performing inverse quantization on the quantized level.

[0292] The inverse converter (128) can generate inversely quantized and inversely converted coefficients by performing an inverse conversion on the inversely quantized conversion coefficients. In embodiments, the inversely quantized and / or inversely converted coefficients may refer to coefficients to which at least one of the inverse quantization and inverse conversion has been applied. The inversely quantized and inversely converted coefficients may be restored residual blocks.

[0293] The adder (129) can generate a recovery block by combining the prediction block and the recovered residual block.

[0294] The restoration block may pass through a filter (130). The filter (130) may apply one or more of a plurality of filters to the target. Each of the plurality of filters may be an in-loop filter. The target may be a restoration sample, a restoration block, or a restoration image.

[0295] The reference picture buffer (141) can store a restoration block / image provided from the filter (130). The restoration image may be an image containing the restoration block. Alternatively, the restoration image may be an image composed of restoration blocks.

[0296] The reference picture buffer (141) can provide the stored restored image to the predictor (123) as a reference image. In terms of storing the decoded (i.e., restored) picture, the reference picture buffer (141) may also be referred to as the Decoded Picture Buffer (DPB).

[0297]

[0298] Structure of the decoding device

[0299] The decoding device (150) may include a processor (160), a storage device (180), and a communication device (189).

[0300] The description of the processor (120), storage (140), and communication device (149) associated with the encoding device (110) may also apply to the processor (160), storage (180), and communication device (189) associated with the decoding device (150). Redundant descriptions are omitted.

[0301] The processor (160) may include a plurality of components. The plurality of components may include an entropy decoder (161), a splitter (162), a predictor (163), an inverse quantizer (167), an inverse converter (168), an adder (169), and a filter (170).

[0302] The storage (180) may include a reference picture buffer (181).

[0303] The communicator (189) can perform functions related to the communication of information in the decoding device (150). For example, the communicator (189) can receive a bitstream from the encoding device (110).

[0304] Among the names of the components of the decoding device (150), "-gi" ("-er" or "-or") may be replaced with "-bu" (- unit). The storage unit (180) may also be named a storage unit.

[0305]

[0306] Operation of the decoding device

[0307] The communication device (149) of the encoding device (110) can transmit the bitstream generated by the encoding device (100) to the decoding device (150). Alternatively, a computer-readable recording medium storing the bitstream can transmit the bitstream generated by the encoding device (100) to the decoding device (150).

[0308] The communication device (189) can receive a bitstream from the encoding device (110) via a wired and / or wireless transmission medium. The received bitstream can be stored in a storage device (180).

[0309] The processor (160) can obtain a bitstream from a storage (180) or a computer-readable recording medium.

[0310] A bitstream can contain encoded information.

[0311] The entropy decoder (161) can generate information for decoding an image by performing entropy decoding based on a probability distribution on the encoded information of the bitstream.

[0312] Information for decoding an image may include quantized levels and syntax elements, etc.

[0313] The entropy decoder (161) can convert quantized levels, which have the form of a one-dimensional vector, into the form of a two-dimensional block by using scanning to perform decoding on the quantized levels. In the scanning, it can be determined which scan to use among an upper-right diagonal scan, a vertical scan, and a horizontal scan based on coding parameters such as the size of the block and the intra-prediction mode of the block.

[0314] The entropy decoder (161) can provide syntax elements to other components of the processor (160), such as the splitter (162).

[0315]

[0316] Common explanation based on the relationship between the components of the encoding device and the components of the decoding device

[0317] The decoding device (150) performs decoding using the bitstream generated by the encoding device (110). The encoding device (110) may perform encoding for the target block using a restored image derived within the decoding device (150), rather than an original image that is not provided to the decoding device (150). Accordingly, the encoding device (110) and the decoding device (150) may need to generate the restored block / image in the same way. In this regard, the descriptions of the divider (122), predictor (123), inverse quantizer (127), inverse converter (128), adder (129), filter (130), and reference picture buffer (141) of the encoding device (110) disclosed in the embodiments may also be applied to the divider (162), predictor (163), inverse quantizer (167), inverse converter (168), adder (169), filter (170), and reference picture buffer (181) of the decoding device (150), respectively. Redundant descriptions are omitted.

[0318] Additionally, each of the divider (122), predictor (123), inverse quantizer (127), inverse converter (128), adder (129), and filter (130) of the encoding device (110) can generate syntactic element information that specifies processing for a target. Each of the divider (162), predictor (163), inverse quantizer (167), inverse converter (168), adder (169), and filter (170) of the decoding device (150) can perform processing for a target (such as that performed in the encoding device (110)) using the syntactic element information.

[0319] As described above, corresponding components of the encoding device (110) and the decoding device (150) may perform the same or corresponding functions. In embodiments, the processor may represent the processor (120) of the encoding device (110) and / or the processor (160) of the decoding device (150). For example, regarding the function of prediction, the processor may represent a predictor (123), a subtractor (124), and an adder (129), and may represent a predictor (163) and an adder (169). Regarding the function of conversion, the processor may represent a converter (125) and an inverse converter (128), and may represent an inverse converter (168). Regarding the function of quantization, the processor may represent a quantizer (126) and an inverse quantizer (127), and may represent an inverse quantizer (167). In terms of functions related to entropy encoding / decoding, the processing unit may represent an entropy encoder (139) and / or an entropy decoder (161). In terms of functions related to filtering, the processing unit may represent a filter (130) and / or a filter (170). The storage unit may represent a storage unit (140) of the encoding device (110) and / or a storage unit (180) of the decoding device (150). The reference picture buffer may represent a reference picture buffer (141) of the encoding device (110) and / or a reference picture buffer (181) of the decoding device (150). The communication unit may represent a communication unit (149) of the encoding device (110) and / or a communication unit (189) of the decoding device (150).

[0320]

[0321] Partitioning of the units that constitute the image

[0322] Figure 2 shows a segmentation structure of an image according to one embodiment.

[0323] Figure 2 schematically illustrates an example in which a single unit is divided into multiple sub-units.

[0324] CU can be used as a base unit for encoding and decoding of images. Additionally, CU can be a base unit for prediction, transformation, quantization, inverse quantization, inverse transformation, entropy encoding, and entropy decoding.

[0325] A CU can be used as a unit to which a prediction mode is applied. That is to say, in coding, it can be determined which of the available prediction modes will be applied to each CU. For example, available prediction modes may include intra prediction, inter prediction, and IBC intra block copy prediction.

[0326] The target image (200) can be sequentially divided into units of CTUs. A division structure can be determined for each CTU. The CTU can be divided into CUs according to the division structure. Alternatively, one CTU can be used as a CU. The size of the CTU can be the maximum size of the CU.

[0327] Each CU may have depth information. The depth information may represent the depth of the CU and the size of the CU. The depth of the CTU may be 0. The depth of the CU created by dividing the CTU may be 1. When a parent CU is divided into child CUs, the depth of the child CU may be 1 greater than the depth of the parent CU. The number of divided CUs may be a positive integer greater than or equal to 2, including 2, 4, 8, and 16. At least one of the width and height of the child CU created by dividing the parent CU may be smaller than at least one of the width and height of the parent CU, depending on the number of child CUs.

[0328] A partitioned CU can be recursively partitioned in the same way up to a predefined maximum depth or a predefined minimum size. The depth of a Smallest Coding Unit (SCU) can be the predefined maximum depth, and the size of an SCU can be the predefined minimum size. The size of an SCU can be the minimum CU size.

[0329] For example, the depth range of a CU can be values ​​from 0 to 3. Depending on the depth of the CU, the CU can have a size from 64x64 to 8x8. A CTU with a depth of 0 can be 64x64 blocks. 0 can be the minimum depth. An SCU with a depth of 3 can be 8x8 blocks. 3 can be the maximum depth. Depth 0 can represent a CTU that is 64x64 blocks. Depth 1 can represent a CU that is 32x32 blocks. Depth 2 can represent a CU that is 16x16 blocks. Depth 3 can represent an SCU that is 8x8 blocks.

[0330] The partition information of a CU may indicate whether the CU is partitioned. The partition information may be a 1-bit flag. All CUs except the SCU may include partition information. For example, the partition information of a CU that is not further partitioned may be a first value of '0', and the partition information of a CU that is partitioned may be a second value of '1'.

[0331] Quad Tree (QT) partitioning can mean that a single CU is partitioned into four CUs. When a parent CU is partitioned into four child CUs, the width and height of each child CU can be half the width and half the height of the parent CU, respectively.

[0332] A binary tree (BT) partition can mean that one CU is divided into two CUs. For example, if a parent CU is divided into two child CUs, the width or height of each child CU can be half the width or half the height of the parent CU.

[0333] Ternary tree (TT) partitioning can mean that a single CU is divided into three CUs. For example, when a parent CU is divided into three child CUs, the three child CUs can be created by dividing the width or height of the parent CU in a ratio of 1:2:1. The width or height of the child CUs can be 1 / 4, 1 / 2, and 1 / 4 of the width or height of the parent CU, respectively.

[0334] In FIG. 2, QT-type splitting was applied to the first CTU. QT splitting, BT splitting, and TT splitting were applied to the second CTU.

[0335] To split a CTU, at least one of different types of splits, such as QT splitting, BT splitting, and TT splitting, may be applied to the CTU. Different types of splits may be applied based on specific priorities.

[0336] For example, QT splitting may be applied preferentially to a CTU. A CU to which QT splitting can no longer be applied may correspond to a leaf node of QT. A CU that is a leaf node of QT may become a root node of BT and / or TT. A CU that is a leaf node of QT may be split into a BT form or a TT form, or may not be split further. In this case, QT splitting may not be applied again to a CU created by applying BT splitting or TT splitting to a CU that is a leaf node of QT.

[0337] The splitting of a CU corresponding to each node of QT can be signaled using QT splitting information. The QT splitting information may be a flag. The QT splitting information of a unit may be information indicating whether the unit is split into a QT form. A first value of the QT splitting information, '0', may indicate that the CU is not split into a QT form. QT splitting information having a first value may signify a Multi-Type Tree (MTT) split. MTT splitting may include BT splitting and TT splitting. A second value of the QT splitting information, '1', may indicate that the CU is split into a QT form.

[0338] There may be no priority between BT splitting and TT splitting. That is, CUs corresponding to the leaf nodes of QT can be split into BT form or TT form. Additionally, CUs generated by BT splitting or TT splitting can be split again into BT form or TT form, or they may not be split any further.

[0339] A CU corresponding to a leaf node of QT can be a root node of MTT. For a CU corresponding to each node of MTT, the CU may further include partition direction information and partition type information in the form of MTT.

[0340] The splitting direction information can indicate the splitting direction of the MTT split. The first value of the splitting direction information, '0', can indicate that the CU is split in the horizontal direction. The second value of the splitting direction information, '1', can indicate that the CU is split in the vertical direction.

[0341] The split type information may indicate the split type used for multi-type tree splitting. The first value of the split type information, '0', may indicate that CU is split into TT form. The second value of the split type information, '1', may indicate that CU is split into BT form.

[0342] Here, each of the aforementioned division direction information and division shape information may be a flag having a specified length (e.g., 1 bit).

[0343] The partitioning information of CU may also include QT partitioning information, partitioning direction information, and partitioning shape information.

[0344] A CU that is no longer divided by QT division, BT division, and TT division can be used as a unit for specific processing such as prediction, transformation, quantization, inverse quantization, inverse transformation, entropy encoding, and entropy decoding. That is, for a specific processing, the CU may no longer be divided. Therefore, division information for dividing such a CU into PU and / or TU, etc., may not exist within the bitstream.

[0345] On the other hand, if the size of a CU is larger than the maximum TU size, such a CU can be recursively partitioned until the size of the CU becomes less than or equal to the maximum TU size. For example, if the size of the CU is 64x64 and the maximum TU size is 32x32, the CU can be partitioned into 4 32x32 TUs for transformation. For example, if the size of the CU is 32x64 and the maximum TU size is 32x32, the CU can be partitioned into 2 32x32 TUs for transformation.

[0346] In such cases, information regarding whether the CU is split for transformation may not be signaled separately. Whether the CU is split may be determined without signaling by comparing the size of the CU (width / height) and the maximum TU size (width / height). For example, if the width of the CU is greater than the width of the maximum TU size, the CU may be split vertically into two. Additionally, if the height of the CU is greater than the height of the maximum TU size, the CU may be split horizontally into two.

[0347] For example, the minimum size of a CU can be 4x4. For example, the maximum size of a transformation block can be 64x64. For example, the minimum size of a transformation block can be 4x4. The minimum size of QT can be the minimum size of a CU corresponding to a leaf node of QT. The maximum depth of MTT can be the maximum depth of a path from the root node of MTT to a leaf node.

[0348] The BT maximum size may represent the maximum size of the CU corresponding to each node of the BT, and the TT maximum size may represent the maximum size of the CU corresponding to each node of the TT. The BT minimum size and / or the TT minimum size may be set as the minimum size of the CU.

[0349] If the depth of a CU within the MTT corresponding to a node of the MTT is equal to the maximum depth of the MTT, the CU may not be divided into BT form and / or TT form.

[0350] Based on the various sizes and depths of the aforementioned CU, each piece of information described in the embodiments may or may not be present in the bitstream.

[0351] Information regarding the maximum or minimum size described in the embodiments may be signaled at the upper level of the CU. In the embodiments, the upper level of the CU may include a video level, a sequence level, a picture level, a subpicture level, a tile group level, a tile level, and a slice level, etc.

[0352] The information described in the embodiments may be signaled separately for different types of slices. Different types of slices may include intra-slices and inter-slices.

[0353]

[0354] Processing of blocks based on block attributes

[0355] Whether a specific process described in the embodiments is applied or performed may be determined based on the attributes of the block associated with the specific process. Whether a specific process described in the embodiments is applied or performed may be determined based on whether the attributes of the block associated with the specific process satisfy specific conditions. For example, a block may include a target block, a neighbor block, and a reference block. A block may include other blocks described in the embodiments. A block may be one of the blocks and units described in the embodiments.

[0356] The block to which the specific treatment described in the embodiments is applied may have a square shape or a non-square shape.

[0357] In one embodiment, the attributes of the block may include the size of the block. The specific processing described in the embodiments may be applied / performed when specific conditions regarding the size of the block are met.

[0358] In one embodiment, specific conditions may include a minimum block size condition and a maximum block size condition. The block to which the minimum block size condition applies and the block to which the maximum block size condition applies may be different from each other.

[0359] In one embodiment, the minimum block size and / or maximum block size for a specific process may be predefined.

[0360] In one embodiment, the processing of the embodiment may be applied / performed when the block size is greater than or equal to the minimum block size and / or less than or equal to the maximum block size. Alternatively, in one embodiment, the processing of the embodiment may be applied / performed when the block size is greater than the minimum block size and / or less than the maximum block size.

[0361] In one embodiment, the processing of the embodiment may be applied / performed only when the block size is greater than or equal to the minimum block size and less than or equal to the maximum block size. Alternatively, the processing of the embodiment may be applied / performed only when the block size is greater than the minimum block size and less than or equal to the maximum block size. Alternatively, the processing of the embodiment may be applied / performed only when the block size is greater than the minimum block size and less than the maximum block size. The processing of the embodiment may be applied / performed only when the block size is greater than the minimum block size and less than the maximum block size.

[0362] In one embodiment, the processing of the embodiment may be applied / performed only when the block size is a predefined block size.

[0363] In the embodiments, the size of the block may be determined by various methods. For example, the size of the block may mean the width of the block or the height of the block. The size of the block may mean both the width and the height of the block. The size of the block may mean the area of ​​the block. The size of the block may mean 1) the result of a known formula using the width and height of the block, 2) the result of a formula of the embodiment, or 3) a statistical value.

[0364] Additionally, for the first size, the processing of the first embodiment among the embodiments may be applied / performed, and for the second size, the processing of the second embodiment among the embodiments may be applied / performed.

[0365] In the embodiments, the block size may be 2x2, 4x4, 8x8, 16x16, 32x32, 64x64, or 128x128, etc. Or, in the embodiments, the block size is (2*SIZE X )x(2*SIZE Y It may be ) etc. SIZE X is one of integers greater than or equal to 1. SIZE Y can be one of integers greater than or equal to 1.

[0366]

[0367] Predictive information for prediction

[0368] Predictive information can be used to generate a predicted block for a target block.

[0369] The encoding device (110) can generate prediction information required for prediction and can generate a bitstream containing the prediction information. The prediction information can be signaled from the encoding device (110) to the decoding device (150) through the bitstream. The decoding device (150) can obtain the prediction information from the bitstream and can generate a prediction block by performing a prediction on a target block using the prediction information.

[0370] Prediction information may include intra prediction information, inter prediction information, and IBC prediction information. In the embodiments, prediction information may be replaced with intra prediction information, inter prediction information, and / or IBC information. Intra prediction information may include information used for intra prediction as described in the embodiments. Inter prediction information may include information used for inter prediction as described in the embodiments. IBC information may include information used for IBC prediction as described in the embodiments.

[0371]

[0372] Intra prediction

[0373] Figure 3 shows the structure of an intra prediction according to one embodiment.

[0374] Intra-prediction can be performed using reference samples and coding parameters of the target block. The reference sample may be a (restored) sample within the (restored) reference block. Alternatively, an intermediate prediction sample may be generated using a sample described in an embodiment, such as the restored sample, and a reference sample may be generated again using the intermediate prediction sample. Processing described in an embodiment, such as filtering, may be applied when generating the reference sample.

[0375] The reference block may be a (spatial) neighbor block of the target block. The coding parameter may be a coding parameter for the target block and / or a coding parameter for the reference block. In intra-prediction, the reference sample may refer to a neighbor sample.

[0376] A prediction block can be generated by performing intra prediction on a target block according to an intra prediction mode, based on a reference sample within the target image and information related to the reference sample. The size of the target block and the size of the prediction block may be the same.

[0377] In the embodiments, the prediction block may be a PU. Alternatively, the prediction block may correspond to the CU or TU described in the embodiments. The prediction block may have a square or rectangular shape.

[0378] An intra prediction mode can be represented by at least one of a mode number, a mode value, a mode angle, and a mode direction. The prediction directions of a plurality of intra prediction modes for a target block are illustrated in the lower right corner of FIG. 3. Among the plurality of intra prediction modes, the remaining intra prediction modes, excluding DC and planar modes, may be directional modes. A directional mode may be an intra prediction mode having a specific direction or a specific angle. An intra prediction mode for a target block may be selected from directional modes and non-directional modes.

[0379] In the bottom-right rectangle representing the target block, the number '0' may represent Planner mode, which is a non-directional intra prediction mode. The number '1' may represent DC mode, which is a non-directional intra prediction mode. In the bottom-right rectangle representing the target block, arrows extending from the center of the rectangle outwards may represent the prediction directions of directional intra prediction modes. Additionally, the number displayed near the arrow may represent an example of a mode value assigned to an intra prediction mode or a prediction direction of an intra prediction mode.

[0380] Intra prediction can be performed according to the intra prediction mode for the target block. One of the intra prediction modes available for the target block can be used as the intra prediction mode for the target block.

[0381] The number of intra prediction modes available to the target block may be a predefined value. Alternatively, the number of intra prediction modes available to the target block may be determined based on the attributes of the prediction block. For example, the attributes of the prediction block may include coding parameters such as shape, size, and color components.

[0382] For example, in FIG. 3, the directional modes illustrated by dashed lines (i.e., directional modes with numbers from -14 to -1 or numbers from 67 to 80) can be applied only to predictions for non-square blocks. Therefore, the number of available intra-prediction modes for predictions for square blocks may be 67. (Planner mode, DC mode, and 65 directional modes)

[0383] For example, the number of available intra prediction modes may vary depending on whether the color component of a block is a luminance signal or a chroma signal. The number of available intra prediction modes for a block with a luminance component may be greater than the number of available intra prediction modes for a block with a chroma component.

[0384] Intra-prediction modes may include a horizontal-below mode, a horizontal mode, a vertical mode, and a vertical-right mode. The horizontal-below mode may be an intra-prediction mode located at the bottom of the horizontal mode. The vertical-right mode may be a mode located to the right of the vertical mode. For example, in FIG. 3, the mode value of the horizontal mode may be 18. The mode value of the vertical mode may be 50. Intra-prediction modes with a mode value of 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, and 66 may be vertical-right modes. Intra prediction modes with a mode value of 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16 and 17 may be horizontal bottom modes.

[0385] The number of the aforementioned intra-prediction modes and the mode number of each intra-prediction mode may be exemplary only. The number of the aforementioned intra-prediction modes and the mode number of each intra-prediction mode may be defined differently depending on the embodiment, implementation, and / or as necessary.

[0386] When the intra prediction mode is a planner mode, when generating a prediction block of a target block, the sample value of the prediction sample can be generated using a weighted sum (weighted sum) of the top reference sample of the target sample, the left reference sample of the target sample, the right top reference sample of the target block, and the left bottom reference sample of the target block, depending on the position of the prediction sample within the prediction block.

[0387] When the intra prediction mode is DC mode, a prediction block can be generated based on the average of the sample values ​​of multiple reference samples. The multiple reference samples may include top reference samples and left reference samples of the target block. The value of the prediction sample of the prediction block can be determined based on the average of the sample values ​​of the multiple reference samples. Additionally, filtering using the values ​​of the reference samples can be performed on specific rows and / or specific columns within the target block. The specific rows may be one or more top rows adjacent to the top reference samples. The specific columns may be one or more left columns adjacent to the left reference samples.

[0388] When the intra prediction mode is a directional mode, a prediction block can be generated using the top reference sample, left reference sample, right top reference sample, and / or left bottom reference sample of the target block.

[0389] The intra prediction mode of a target block can be determined based on the intra prediction mode of a neighboring block of the target block. Information for determining the intra prediction mode of a target block can be signaled.

[0390] For example, if the intra prediction modes of the target block and the neighbor block are the same, an indicator indicating that the intra prediction modes of the target block and the neighbor block are the same may be signaled.

[0391] For example, an indicator indicating an intra prediction mode such as the intra prediction mode of the target block among the intra prediction modes of multiple neighboring blocks may be signaled.

[0392] For example, if the intra prediction modes of the target block and neighboring blocks are different from each other, an indicator indicating the intra prediction mode of the target block may be signaled. Alternatively, information used to derive the intra prediction mode of the target block based on the intra prediction mode of the neighboring block may be signaled.

[0393] Reference samples used for intra-prediction of a target block may include bottom-left reference samples, left reference samples, top-left reference samples, top reference samples, and top-right reference samples, etc.

[0394] For example, left reference samples may be restored reference samples adjacent to the left side of the target block. Top reference samples may be restored reference samples adjacent to the top side of the target block. Top-left reference samples may be restored reference samples diagonally adjacent to the top-left side of the target block. Bottom-left reference samples may be reference samples located below the left reference samples among samples located on the same line as the left sample line composed of left reference samples. Top-right reference samples may be reference samples located to the right of the top reference samples among samples located on the same line as the top sample line composed of top reference samples.

[0395] Reference samples used for intra-prediction for a target block can be determined based on the intra-prediction mode of the target block. One or more reference samples may be used to determine the sample values ​​of the prediction samples of the prediction block. In FIG. 3, the direction of the intra-prediction mode indicated by the arrow may represent the direction from the prediction sample to the reference sample. The direction of the intra-prediction mode may represent the dependency relationship between the reference samples and the prediction samples. For example, depending on the intra-prediction mode, the sample value of a specific reference sample may be used as the sample value of at least one sample of the prediction block. Here, the specific reference sample and at least one sample of the prediction block may be samples designated by a straight line of the direction of the intra-prediction mode. That is to say, the sample value of the specific reference sample may be copied to the sample value of the prediction sample located in the reverse direction of the direction of the intra-prediction mode. Alternatively, the sample value of the prediction sample of the prediction block may be the sample value of the reference sample located in the direction of the intra-prediction mode relative to the location of the prediction sample.

[0396] Reference samples used for intra-prediction may not be limited to samples immediately adjacent to the target block. As illustrated in FIG. 3, at least one of reference sample line 0 to reference sample line 3 may be used for intra-prediction of the target block.

[0397] Each reference sample line in FIG. 3 may contain one or more reference samples. The smaller the number of the reference sample line, the closer the line of reference samples may be to the target block. Reference sample line 0 may be a line of reference samples immediately adjacent to the target block. When the top-left coordinates of the target block are (X, Y), the horizontal length is W, and the vertical length is H, the reference samples of reference sample line 0 may be samples with an x-coordinate of X-1 or a y-coordinate of Y-1. Here, the y-coordinates of the reference samples with an x-coordinate of X-1 may be Y-1 to Y+2H. The x-coordinates of the reference samples with a y-coordinate of Y-1 may be X-1 to X+2W. The reference samples of reference sample line A may be samples with an x-coordinate of XA-1 or a y-coordinate of YA-1. Here, the y-coordinates of the reference samples with an x-coordinate of XA-1 may be YA-1 to Y+2H+A. The x-coordinates of reference samples with y-coordinate YA-1 can be XA-1 to X+2W+A. A can be 1, 2, or 3.

[0398] Samples of segments A and F can be derived using padding that uses the nearest samples of segments B and E, respectively, instead of being obtained from restored neighbor blocks.

[0399] The reference sample line index may indicate a reference sample line among multiple reference sample lines used for intra-prediction of a target block. For example, the reference sample line index may have a value from 0 to 3. The reference sample line index may be signaled.

[0400] When intra-color component prediction is used for a target block, a prediction block for a second color component can be generated based on a reconstruction block of a first color component for the target block. For example, the first color component may be a luminance component, and the second color component may be a chroma component.

[0401] For intra-prediction between color components, parameters between the first and second color components can be derived based on a template. For example, the parameters can be parameters of a linear model.

[0402] For example, the template may include a top reference sample and / or a left reference sample of the target block, and may include a top reference sample and / or a left reference sample of the restoration block of the first color component corresponding to these reference samples.

[0403] Once the parameters are derived, a prediction block for a second color component for a target block can be generated by applying the reconstruction block of the first color component to a linear model. Depending on the image format or the type of intra-prediction between color components, subsampling or downsampling may be performed on the surrounding samples of the reconstruction block of the first color component and on the reconstruction block of the first color component. If subsampling is performed, the derivation of parameters and the intra-prediction between color components may be performed using corresponding samples derived by subsampling.

[0404] Intra Sub-Partitions (ISP) prediction may refer to sequential intra prediction for multiple subblocks generated by partitioning a target block. In ISP prediction, the target block may be partitioned into two or four subblocks in the horizontal and / or vertical directions. The partitioned subblocks may be restored sequentially. As intra prediction is performed on the subblocks, sub-prediction blocks for the subblocks may be generated. Additionally, as inverse quantization and / or inverse transformation is performed on the subblocks, sub-residual blocks for the subblocks may be generated. A restored subblock may be generated by adding the sub-prediction blocks to the sub-residual blocks. The restored subblocks may be used as reference samples for intra predictions for other subblocks to be processed next.

[0405] In performing a prediction for a target block, it may be determined whether samples included in a restored neighbor block can be used as reference samples for the target block. If there are non-available samples among the samples in the neighbor block that cannot be used as reference samples for the target block, a value generated by copying and / or interpolation using the sample value of at least one sample among the samples included in the restored neighbor block may replace the sample value of the non-available sample. If the value generated by copying and / or interpolation replaces the sample value of the sample, the sample may be used as a reference sample for the target block.

[0406] In intra-prediction, the sample value of a prediction sample in a prediction block can be determined by the sample value of a reference sample. The location of the reference sample can be specified by the location of the prediction sample and the direction of the intra-prediction mode. If the location specified by the location of the prediction sample and the direction of the intra-prediction mode is an integer location, the sample value of one reference sample pointed to by the integer location can be used to determine the sample value of the prediction sample in the prediction block. If the location specified by the location of the prediction sample and the direction of the intra-prediction mode is not an integer location, an interpolated reference sample can be generated based on the two reference samples closest to the specified location. The sample value of the interpolated reference sample can be used to determine the sample value of the prediction sample. That is to say, when the location specified by the location of the prediction sample and the direction of the intra-prediction mode represents the space between two reference samples, an interpolated sample value can be generated based on the sample values ​​of the two samples.

[0407]

[0408]

[0409] Inter prediction

[0410] FIG. 4 shows the structure of an inter prediction to explain an inter prediction process according to one embodiment.

[0411] The rectangle shown in Fig. 4 can represent an image. Additionally, the arrow in Fig. 4 can represent the predicted direction.

[0412] Each image constituting a video can be classified into I-pictures (i.e., intra-pictures), P-pictures (i.e., uni-prediction pictures), and B-pictures (i.e., bi-prediction pictures) according to their coding type. Coding can be performed for each picture according to its coding type.

[0413] If the target picture is an I picture, coding for the target picture can be performed using information within the target picture without inter-prediction referencing other images. For example, coding for the I picture can be performed using intra-prediction and / or IBC prediction.

[0414] Coding for P picture and B picture can be performed by at least one of intra prediction, IBC prediction, and inter prediction using a reference image.

[0415] If the target picture is a P picture, coding for the target picture can be performed using unidirectional inter-prediction using a single reference image list.

[0416] When the target picture is picture B, coding for the target picture can be performed using unidirectional inter-prediction or bidirectional inter-prediction using two reference image lists.

[0417] Below, the inter prediction for the target block in the inter mode according to the embodiment is described in detail.

[0418] When the prediction mode of the target block is inter mode, inter prediction can be performed on the target block. The target block can be a prediction block or a partitioned prediction block.

[0419] Inter prediction can be performed using reference images and motion information. In inter prediction, a reference image can be selected using a reference image index, and a reference block corresponding to a target block within the reference image can be determined using motion information. A prediction block for the target block can be generated using the determined reference block.

[0420] Motion information can be derived using coding parameters, etc. For example, motion information can be derived using motion information of restored neighbor blocks, motion information of call blocks, and / or motion information of blocks adjacent to call blocks.

[0421] In the embodiments, a candidate list may be used for inter prediction. The candidate list may include multiple candidates. An index pointing to a candidate among the candidates in the candidate list that is used for inter prediction for a target block may be signaled. The candidate list may be derived in the same manner based on the same information in the encoding device (110) and the decoding device (150). Here, the same information may include a restored image and a restored block. Additionally, in order to specify a candidate by an index, the order of candidates within the candidate list may be constant.

[0422] In one embodiment, a prediction for a target block can be performed by using motion information of a spatial candidate or a temporal candidate as motion information of the target block. Motion information of a spatial candidate may be referred to as spatial motion information. Motion information of a temporal candidate may be referred to as temporal motion information.

[0423] Spatial candidates may be restored spatial neighbor blocks that are spatially adjacent to the target block.

[0424] Spatial candidates may be blocks that 1) exist within the target image, 2) have already been restored through decoding, and 3) are adjacent to the target block.

[0425] Spatial candidates may include the left block, top block, bottom-left block, top-right block, and top-left block of the target block.

[0426] Temporal candidates may be restored temporal neighbor blocks corresponding to the target block within the restored call (COL) image.

[0427] In the embodiments, the motion information of the spatial candidate may be the motion information of a block containing the spatial candidate. The motion information of the temporal candidate may be the motion information of a block containing the temporal candidate.

[0428] In inter prediction, a call block for a target block can be identified. The region of the target block within the target image and the region of the call block within the call image may be the same. That is to say, the call block may be a block that occupies a specific region within the call image. The specific region may be a region corresponding to the region of the target block within the call image.

[0429] Temporal candidates may be locations inside and / or outside the call block within the call image.

[0430] For example, a call block may include a first call block and a second call block. When the top-left coordinates of the call block are (xP, yP) and the size of the call block is (nPSW, nPSH), the first call block may be a block occupying the coordinates (xP + nPSW, yP + nPSH). The second call block may be a block occupying the coordinates (xP + (nPSW >> 1), yP + (nPSH >> 1)). The second call block may optionally be used as a call block when the first call block is unavailable.

[0431] The MV of the target block can be determined based on the MV of the call block. Scaling can be performed on the MV of the call block. The scaled MV of the call block can be used as the MV of the target block or the prediction MV. Alternatively, the MV of the temporal candidate stored in the candidate list associated with the inter-prediction can be the scaled MV.

[0432] The ratio of the scaled MV and the MV of the call block may be equal to the ratio of the first temporal distance and the second temporal distance. The first temporal distance may be the distance between the reference image of the target block and the target image. The second temporal distance may be the distance between the reference image of the call block and the call image.

[0433] The method by which motion information is derived can be determined by the inter prediction mode of the target block. For example, as an inter prediction mode, AMVP mode, merge mode, skip mode, merge mode with MVD, sub-block merge mode, GPM, Combined Inter Intra Prediction (CIIP) mode, and affine inter mode may be used. In the following embodiments, each of the inter prediction modes is described.

[0434]

[0435] AMVP mode

[0436] When AMVP mode is used as a prediction mode, a list of MV candidates including one or more MV candidates can be generated using spatial candidate MVs, temporal candidate MVs, history-based MV candidates, and zero vectors. At least one of the spatial candidate MVs, temporal candidate MVs, and zero vectors can be determined and used as an MV candidate.

[0437] Spatial candidates may include restored spatial neighbor blocks. The MV of a restored spatial neighbor block may be referred to as a spatial motion vector candidate. Temporal candidates may include a call block and a block adjacent to the call block. The MV of a call block or the MV of a block adjacent to the call block may be referred to as a temporal motion vector candidate. History-based MV candidates may be MVs in a list containing MVs of other blocks that were encoded / decoded before the encoding / decoding of the target block.

[0438] The encoding device (110) can determine the MV to be used for encoding a target block within a search range using an MV candidate list. The maximum number of MV candidates in the MV candidate list may be predefined. N may represent the predefined maximum number. For example, N may be 2. Alternatively, the maximum number of such candidates may be signaled from the encoding device to the decoding device or derived from the decoding device. The encoding device (110) can determine an MV candidate to be used as the predicted MV of the target block among the MV candidates in the MV candidate list. The MV to be used for encoding the target block may be an MV that can be encoded at the minimum cost. The encoding device (110) may determine whether to use the AMVP mode in encoding the target block and may generate AMVP mode usage information indicating whether the AMVP mode is used.

[0439] Inter prediction information may include 1) AMVP mode usage information, 2) MV candidate index, 3) MVD, 4) MVD resolution information, 5) reference direction and 6) reference image index, and may include residual blocks. Inter prediction information may be signaled from the encoding device (110) to the decoding device (150) in the form of a bitstream.

[0440] The decoding device (150) can obtain AMVP mode usage information from the bitstream. If the AMVP mode usage information indicates that the AMVP mode is being used, the decoding device (150) can obtain an MV candidate index, an MVD, MVD resolution information, a reference direction, and a reference image index from the bitstream. Among the MV candidates included in the MV candidate list, the MV candidate pointed to by the MV candidate index can be selected as the predicted MV of the target block.

[0441] The MVD may represent the difference between the MV that will actually be used for inter-prediction of the target block and the predicted MV. The encoding device (110) may derive a predicted MV that is close to the MV that will actually be used for inter-prediction of the target block in order to use an MVD of the smallest possible size. The decoding device (150) may derive the MV of the target block by summing the MVD and the predicted MV. That is to say, the MV of the target block derived by the decoding device (150) may be the sum of the MVD and the predicted MV candidates.

[0442] Additionally, the encoding device (110) can generate MVD resolution information. The MVD resolution information may be information used to adjust the resolution of the MVD. The decoding device (150) can adjust the resolution of the MVD using the MVD resolution information.

[0443] Meanwhile, the encoding device (110) can calculate the MVD based on an affine model. The affine control point MV of the target block can be derived based on the sum of the affine control point MV candidates and the MVD. Using the affine control point MV, the MV of each sub-block within the target block can be derived.

[0444]

[0445] Merge Mode

[0446] When merge mode is used, a merge candidate list containing multiple merge candidates can be generated using motion information of spatial candidates and motion information of temporal candidates, etc. Motion information may include 1) MV, 2) reference image index and 3) reference direction, etc. A merge candidate may be motion information.

[0447] Merge candidates may include 1) spatial merge candidates generated based on spatial candidates, 2) temporal merge candidates generated based on temporal candidates, 3) history-based merge candidates, 4) average merge candidates, and 5) zero merge candidates.

[0448] A history-based merge candidate may be movement information within a list containing movement information of other blocks that were encoded / decoded earlier than the encoding / decoding of the target block.

[0449] The average merge candidate may be a merge candidate generated based on the average of two merge candidates within the merge candidate list.

[0450] Zero merge candidates can be zero vector motion information. Zero vector motion information can be motion information where MV is a zero vector.

[0451] Merge candidates can be added to the merge candidate list according to a predefined method and a predefined order so that the merge candidate list has a set number of merge candidates. The same merge candidate list can be configured in the encoding device (110) and the decoding device (150) through the predefined method and a predefined order.

[0452] The encoding device (110) can select a merge candidate to be used for encoding a target block from among the merge candidates in the merge candidate list. The encoding device (110) can determine whether to use a merge mode in encoding the target block and can generate merge mode usage information indicating whether the merge mode is used.

[0453] Inter prediction information may include 1) merge mode usage information, 2) merge index and 3) correction information, etc., and may include residual blocks. Inter prediction information may be signaled in bitstream form from the encoding device (110) to the decoding device (150).

[0454] The decoding device (150) can obtain merge mode usage information from the bitstream. If the merge mode usage information indicates that the merge mode is being used, the decoding device (150) can obtain merge mode-related information, such as a merge index, from the bitstream.

[0455] The encoding device (110) can select the optimal merge candidate among the merge candidates included in the merge candidate list and can set the value of the merge index to point to the selected merge candidate.

[0456] Correction information may be information used for correcting the MV. The encoding device (110) may generate correction information. The decoding device (150) may derive a corrected MV by performing correction on the MV of a merge candidate selected by a merge index based on the correction information. The corrected MV may be used as the MV of the target block.

[0457] In one embodiment, the correction information may include an MVD. The correction information may include one or more of correction usage information, correction direction information, and correction magnitude information. The correction usage information may indicate whether to use correction for the MV. A merge mode that performs correction for the MV based on the correction information may be referred to as a merge mode having an MVD.

[0458] In merge mode, a prediction for the target block can be performed using the merge candidate pointed to by the merge index among the merge candidates included in the merge candidate list.

[0459] Movement information of the target block can be derived from 1) MV, 2) reference image index and 3) reference direction of the merge candidate pointed to by the merge index.

[0460] In one embodiment, the merge candidates in the merge candidate list may be specific modes that induce inter-prediction information. A merge candidate may be information pointing to a specific mode that induces inter-prediction information. Inter-prediction information of a target block may be induced according to the specific mode pointed to by the merge candidate. In this regard, the specific mode may be regarded as a specific inter-prediction information inducing mode or a specific movement information inducing mode. The specific mode may include a series of processes that induce inter-prediction information.

[0461] Inter-prediction information of the target block can be derived according to a specific mode pointed to by a merge candidate selected by a merge index among the merge candidates in the merge candidate list. For example, specific modes may include a mode for deriving motion information at the sub-block level and a mode for deriving motion information at the affine level, and may include other modes for deriving motion information as described in the embodiments.

[0462] Skip mode may be a mode that does not use residual blocks. That is to say, when skip mode is used, the restoration block may be identical to the prediction block. The description of the merge mode in the embodiments may also apply to skip mode. The difference between merge mode and skip mode may be whether or not residual blocks are signaled and used. That is to say, skip mode may be similar to merge mode except that residual blocks are not transmitted / used, and the description of merge mode may also apply to skip mode.

[0463] The subblock merge mode may be a mode in which motion information of a target subblock is derived for a target subblock within a target block. When the subblock merge mode is applied, a list of subblock merge candidates may be generated using affine control point motion vector merge candidates and / or subblock-based temporal merge candidates. The subblock-based temporal merge candidates may be motion information of the call subblock of the target subblock.

[0464] In GPM, a first prediction block and a second prediction block can be generated using two sets of motion information for a target block. For each coordinate of the target block, a final prediction sample of the final prediction block can be generated using the weighted sum of the first prediction sample of the first prediction block and the second prediction sample of the second prediction block.

[0465] Here, the first weight for the first prediction sample of the weighted consensus and the second weight for the second prediction sample can be determined based on the boundaries of the GPM. The boundaries may represent dividing lines that divide the target block. Depending on the boundaries, the target block may be divided into a first divided region and a second divided region.

[0466] If the distance between the final prediction sample and the boundary is less than or equal to a reference value, the value of the final prediction sample of the final prediction block may be determined using the weighted sum of the first prediction sample of the first prediction block and the second prediction sample of the second prediction block. If the distance between the final prediction sample and the boundary is greater than the reference value, one of the first weight and the second weight may be 1 and the other may be 0.

[0467] The Combined Inter-Intra Prediction (CIIP) mode may be a mode that derives a prediction sample of a target block using a weighted sum of a prediction sample generated by inter-prediction and a prediction sample generated by intra-prediction.

[0468] In the aforementioned modes, self-improvement of the derived motion information may be performed, and the improved motion information may be used as motion information for the target block. For example, blocks within a specific area determined based on the derived motion information may be searched, and the motion information of the block having the smallest Sum of Absolute Differences (SAD) value among the searched blocks may be used as the improved motion information for the target block. The specific area may be a square area within the reference image specified by the motion information. The point indicated by the motion information may be the center of the specific area.

[0469] In the aforementioned modes, compensation for prediction samples derived through inter-prediction can be performed using optical flow.

[0470]

[0471] FIG. 5 shows the order of addition of spatial candidates to the candidate list according to one embodiment.

[0472] In Fig. 5, the locations of the spatial candidates are shown.

[0473] The large block in the center can represent the target block. The five small blocks adjacent to the target block can represent spatial candidates.

[0474] The coordinates of the target block can be (xP, yP), and the size of the target block can be (nPSW, nPSH).

[0475] Spatial candidate A0 may be a block adjacent to the bottom-left of the target block. A0 may be a block occupying a sample of coordinates (xP - 1, yP + nPSH).

[0476] Spatial candidate A1 may be a block adjacent to the left of the target block. A1 may be the bottommost block among the blocks adjacent to the left of the target block. Or, A1 may be a block adjacent to the top of A0. A1 may be a block occupying a sample of coordinates (xP - 1, yP + nPSH - 1).

[0477] Spatial candidate B0 may be a block adjacent to the top right of the target block. B0 may be a block occupying a sample of coordinates (xP + nPSW, yP - 1).

[0478] Spatial candidate B1 may be a block adjacent to the top of the target block. B1 may be the rightmost block among the blocks adjacent to the top of the target block. Or, B1 may be a block adjacent to the left of B0. B1 may be a block occupying a sample of coordinates (xP + nPSW - 1, yP - 1).

[0479] Spatial candidate B2 may be a block adjacent to the top-left corner of the target block. B2 may be a block occupying a sample of coordinates (xP - 1, yP - 1).

[0480] As illustrated in Fig. 5, in adding spatial candidates to the candidate list, B1, A1, The order B0, A0, and B2 can be used. That is, B1, A1, Available spatial candidates can be added to the candidate list in the order of B0, A0, and B2. The order in which spatial candidates illustrated in FIG. 5 are added to the merge candidate list may be just one example.

[0481] The above candidate list may include a motion information candidate list, a merge candidate list, an MV candidate list, a BV candidate list, and an MPM list, etc.

[0482] To include a spatial or temporal candidate in the candidate list, it may be determined whether the spatial or temporal candidate is available. If a candidate block is outside the boundaries of an image, slice, or tile, the availability of the candidate block may be set to false. The description "availability is set to false" may mean "it is set to non-available."

[0483] The maximum number of candidates in the candidate list can be set. N can represent the set maximum number. The set maximum number can be signaled through a parameter set or header, etc. For example, the maximum number of candidates in the candidate list for a target block within a slice can be set by the slice header. For example, the value of N can be 5 by default.

[0484]

[0485] IBC mode

[0486] The IBC mode may be an intra-block copy prediction mode that generates a prediction block for a target block by referencing an already reconstructed region within the target image. In this respect, the IBC mode may also be referred to as a current image reference mode. A block vector (BV) may be used to identify the already reconstructed region.

[0487] Whether the target block is encoded / decoded in IBC mode can be determined using IBC mode usage information. The encoding device (110) can determine whether to use IBC mode in encoding the target block and can generate IBC mode usage information indicating whether IBC mode is used. The decoding device (150) can obtain IBC mode usage information from the bitstream.

[0488] In IBC mode, the predicted block of the target block can be generated based on the BV. The BV can specify the reference block. The BV can indicate the displacement between the target block and the reference block. The reference block can be a block within the target image. The description of the MV of the embodiments can also be applied to the BV.

[0489] The IBC mode may include a skip mode, a merge mode, and an AMVP mode, etc. The descriptions of the AMVP mode, merge mode, and skip mode of the embodiments may be similarly applied to the AMVP mode, merge mode, and skip mode of the IBC mode, respectively.

[0490] In skip mode or merge mode, a merge candidate list may be configured, and a merge index may specify one merge candidate from among the merge candidates in the merge candidate list. The BV of the specified merge candidate may be used as the BV of the target block.

[0491] In AMVP mode, BVD can be used. The description of MVD in the embodiments can also be applied to BVD.

[0492] The reference block in IBC mode may be limited to a block within an already restored region of the target image. Alternatively, the reference block may be contained within at least one of the target CTU or the left CTUs. For example, the value of BV may be restricted so that the reference block is located within a specific region. The specific region may be an area of ​​three blocks of a specific size that are encoded / decoded before the block of a specific size containing the target block. The specific size may be 64x64.

[0493]

[0494] Transformation and Quantization

[0495] Quantized levels can be generated by performing a transformation and / or quantization on the residual block. The residual block can represent the difference between the original block and the prediction block. A restored residual block can be generated by performing inverse quantization and / or inverse transformation on the quantized levels. The restored residual block can represent the difference between the restored block and the prediction block.

[0496] When a transformation or inverse transformation is performed, a separable transform or a 2D non-separable transform may be performed on the residual block. A separable transform may be a transformation that performs 1D transformations on the residual block in the horizontal and vertical directions, respectively.

[0497] The transformation kernels used for the transformation may include various DCT kernels such as DCT type 2 (DCT-II), 2) DST kernels, and 3) kernels derived by training. For 1D transformation, DCT type and DST type may include DCT-V, DCT-VIII, DST-I, and DST-VII in addition to DCT-II.

[0498] A transformation set may be used to determine the DCT type, DST type, or learning-derived kernel to be used for the transformation. Each transformation set may include multiple transformation candidates. Each transformation candidate may be a DCT type, a DST type, or a learning-derived kernel, etc.

[0499] The encoding device (110) can perform conversion and inverse conversion using conversion candidates included in the conversion set. The decoding device (150) can perform inverse conversion using conversion candidates included in the conversion set. Conversion selection information indicating which conversion candidate is used among the plurality of conversion candidates included in the conversion set applied to the residual block may be signaled. The conversion selection information may include vertical conversion selection information and horizontal conversion selection information. The vertical conversion selection information may indicate which conversion among the conversions belonging to the conversion set is used for the vertical conversion. The horizontal conversion selection information may indicate which conversion among the conversions belonging to the conversion set is used for the horizontal conversion.

[0500] The transformation may include at least one of a primary transformation and a secondary transformation. A primary transformation coefficient may be generated by performing a primary transformation on a residual block, and a secondary transformation coefficient may be generated by performing a secondary transformation on the transformation coefficient. Here, the transformation coefficient may include a primary transformation coefficient and a secondary transformation coefficient.

[0501] A first-order transformation may mean a Multiple Transform Selection (MTS) that applies different transformations to each of the 1D directions (i.e., vertical and horizontal directions).

[0502] A second-order transformation may be a transformation intended to improve the energy concentration of the transformation factors generated by a first-order transformation. A second-order transformation may be 1) a separable transformation like the first-order transformation, or 2) a 2D non-separable transformation. A 2D non-separable transformation may refer to a Low Frequency Non-Separable Transform (LFNST) or a Non-Separable Primary Transform (NSPT).

[0503] NSPT can be applied to specific block sizes such as 4x4, 4x8, 8x4, 4x16, 16x4, 8x8, 8x16, and 16x8 for intra-coding.

[0504] A first-order transformation may be performed using at least one of a plurality of predefined transformation methods. For example, the plurality of predefined transformation methods may include DCT, DST, and KLT, etc. Additionally, the first-order transformation may be a transformation having various transformation types according to transformation kernel functions that define DCT and DST. For example, the first-order transformation may include a plurality of transformations such as DCT-2, DCT-4, DCT-5, DCT-7, DCT-8, DST-1, DST-2, DST-4, DST-7, and DST-8 according to a plurality of transformation kernels.

[0505] In one embodiment, the transformation type may be determined based on coding parameters related to the target block. For example, the transformation type may be determined based on one or more of 1) the prediction mode of the target block (e.g., one of intra prediction and inter prediction), 2) the size of the target block, 3) the shape of the target block, 4) the intra prediction mode of the target block, 5) the components of the target block (e.g., one of luminance components and chroma components), and 6) the splitting type applied to the target block (e.g., one of QT, BT, TT, and non-split).

[0506] As in the first transformation, a transformation set can be defined in the second transformation as well. Methods for deriving and / or determining the transformation set of the embodiments can be applied to the second transformation as well as the first transformation.

[0507] In one embodiment, the first transformation and / or second transformation may be determined for a specific target. The transformation selection information may include transformation target information. The transformation target information may indicate the target to which the first transformation and / or second transformation is applied.

[0508] For example, first-order transformation and / or second-order transformation may be applied to one or more signal components among the luminance component and the chroma component.

[0509] In one embodiment, the transformation selection information may include first transformation usage information and second transformation usage information. The first transformation usage information may indicate whether a first transformation is applied to the residual block of the target block. The second transformation usage information may indicate whether a second transformation is applied to the residual block of the target block.

[0510] In one embodiment, whether a first transformation and / or a second transformation is applied may be determined based on coding parameters for the target / neighbor block, such as the size and shape of the target / neighbor block.

[0511] In one embodiment, the transformation selection information may include first transformation selection information and second transformation selection information. The first transformation selection information may indicate a transformation method applied to a residual block among a plurality of transformation methods that can be used in the first transformation. The first transformation selection information may be a first transformation index. The second transformation selection information may indicate a transformation method applied to a transformation coefficient among a plurality of transformation methods that can be used in the second transformation. The second transformation selection information may be a second transformation index.

[0512] In one embodiment, the transformation methods of the first transformation and the second transformation can each be derived based on specific information such as coding parameters. For example, the coding parameters may include coding parameters for target / neighbor blocks.

[0513] In the embodiments, information related to a transformation, such as transformation selection information, and sub-information of the transformation selection information may be signaled to a specific target. For example, the specific target may be a CU.

[0514] Information related to transformations, such as transformation selection information, and sub-information of transformation selection information can be derived for a specific target. For example, the specific target may be a CU.

[0515] Quantized levels can be generated by performing quantization on the result or residual block generated by performing a first-order transformation and / or a second-order transformation.

[0516] The description of the transformation described above may also be applied to the inverse transformation. In such application, the inverse processing of the processing described for the transformation may be performed in the inverse transformation. "Transformation" within the name related to the transformation may be changed to "inverse transformation." Additionally, the input of the transformation may be considered as the output of the inverse transformation. The output of the transformation may be considered as the input of the inverse transformation. The decoding device (150) may obtain information related to the transformation, such as transformation selection information, and may use the information related to the transformation to perform the inverse processing of the transformation related to the transformation indicated by the information related to the transformation.

[0517] The target block may include multiple subblocks. Each subblock may be defined according to a minimum block size or minimum block shape. The target block may be divided into multiple subblocks, and each subblock may include coefficients such as 4x4, 2x8, and 8x2. The target block may be a transformation block. Transform coefficients or quantized levels may be represented in the form of a block. Transform coefficients may be quantized transformation coefficients.

[0518] Transform coefficients or quantized levels may be scanned according to at least one scanning type among diagonal scanning, vertical scanning, and horizontal scanning. Diagonal scanning may be top-right diagonal scanning or bottom-left diagonal scanning.

[0519] For example, by scanning the coefficients of a block using diagonal scanning, the coefficients can be changed or arranged into a one-dimensional vector form. Vertical scanning may be scanning the coefficients in the form of a two-dimensional block in a column direction. Horizontal scanning may be scanning the coefficients in the form of a two-dimensional block in a row direction.

[0520] The scanning type for the coefficients can be determined based on coding parameters such as intra prediction mode, block size, and block shape. For example, based on coding parameters such as intra prediction mode, block size, and block shape, it can be determined which scanning method—diagonal scanning, vertical scanning, and horizontal scanning—will be used. A block may be a transformation unit.

[0521] Scanning according to each scanning type can start at a specific starting point and end at a specific ending point.

[0522] In scanning, the scanning order according to the scanning type can first be applied between subblocks. Next, the scanning order according to the scanning type can be applied to the transformation coefficients or quantized levels within the subblocks.

[0523] The encoding device (110) can perform entropy encoding on the conversion coefficients or quantized levels to generate a bitstream containing entropy-encoded conversion coefficients or entropy-encoded quantized levels.

[0524] The decoding device (150) can generate the transform coefficients or quantized levels by obtaining entropy-encoded transform coefficients or entropy-encoded quantized levels from the bitstream and performing entropy decoding. The coefficients can be arranged in the form of two-dimensional blocks through inverse scanning. The arrangement of inverse scanning may be a rearrangement opposite to the arrangement of scanning.

[0525] Backscanned transform coefficients or backscanned quantized levels can be generated through backscanning of the coefficients. In this case, the backscanning types of backscanning may include diagonal scans, vertical scans, and horizontal scans, and a backscanning type of the inverse transform corresponding to the scanning type of the transform may be selected.

[0526] In the decoding device (150), inverse quantization can be performed on the (backscanned) coefficients. Depending on whether a second inverse transform is performed, a second inverse transform can be performed on the result generated by the performance of inverse quantization. Also, depending on whether a first inverse transform is performed, a first inverse transform can be performed on the result generated by the performance of the second inverse transform. By selectively performing a second inverse transform and a first inverse transform on the coefficients, a restored residual block can be generated.

[0527]

[0528] Filtering

[0529] To improve the image quality, filtering may be performed on the blocks. The value of the target sample may be determined or updated by the filtering.

[0530] The target sample may be one of the samples described in the embodiments. For example, the target sample may be one or more of the samples described in the embodiments, such as a prediction sample, a reference sample, a residual sample, a reconstructed sample, and a reconstructed sample to which filtering has been applied.

[0531] The target sample may be a sample within one or more of the target picture, target slice, target CTB, target block, reference sample line, and template. The target block may be one of the blocks described in the embodiments. For example, the target block may be one or more of the blocks described in the embodiments, such as a transformation block, prediction block, reference block, residual block, and restoration block.

[0532] In the embodiments, the filtering process described as being applied to one target may also be applied to other targets. For example, the filtering process described in a specific in-loop filtering may also be applied to a transformation block, a prediction block, a reference block, and a residual block, etc.

[0533] For the filtering of the embodiments, a specific type of filtering may be used. The type of filtering may include a filter tap (or filter tap length), a filter shape, a filter strength, filter coefficients (or weights), and an offset.

[0534] The filter tab may indicate the number of input samples used for the filter. The input samples may include target samples. Alternatively, the input samples may include specific values ​​determined for the target samples. The input samples may include one or more reference samples. One or more reference samples may be determined based on the attributes of the target block described in the embodiments. The attributes may include coding parameters. For example, the attributes of the target sample may include the location of the target sample. One or more reference samples may be specified based on their relative position to the location of the target sample.

[0535] The filter shape can represent the shape formed by input samples. A specific value determined for a target sample can be considered as the target sample. In other words, if a specific value determined for a target sample is used as an input sample for a filter, the target sample can also be considered as constituting the filter shape.

[0536] There may be multiple samples whose values ​​are determined by filtering. Filter strength may represent the range of samples whose values ​​are determined by filtering. Filter strength may be either strong filtering strength or weak filtering strength. The number of samples whose values ​​are determined by strong filtering strength may be greater than the number of samples whose values ​​are determined by weak filtering strength. Alternatively, filter strength may represent the range of values ​​that are changed by filtering. The range of sample values ​​changed by strong filtering strength may be wider than the range of sample values ​​changed by weak filtering strength.

[0537] Filter coefficients can be coefficients or weights of the input samples.

[0538] The offset can be a specific value added to the result calculated using the values ​​and coefficients of the input samples, such as a weighted sum.

[0539] Filtering, interpolation, and sampling may be common in that they update the values ​​of samples. Accordingly, the description of any one of filtering, interpolation, and sampling in the embodiments may also apply to the other one of filtering, interpolation, and sampling. Here, sampling may include at least one of upsampling, downsampling, and subsampling.

[0540] Filtering may include filtering performed by a predictor (123) and a predictor (163), etc.

[0541] In encoding for a target block, a prediction error may exist between the original sample of the original block and the prediction sample of the prediction block. To reduce the prediction error, filtering may be performed on at least one of the prediction sample of the prediction block and the reference sample referenced for prediction.

[0542] For example, in intra-prediction, the reference samples may include one or more of the top-left reference sample, top reference sample, top-right reference sample, left reference sample, and bottom-left reference sample. Filtering of the prediction samples may be performed by applying specific weights to the prediction samples, left reference samples, top reference samples, and / or top-left reference samples, respectively.

[0543] Filtering for at least one of the prediction sample and the reference sample may be performed based on the attributes of the target block and the attributes of the prediction sample. For example, whether filtering is performed, the type of filter, the area to which filtering is applied, the weights of the filtering, the reference sample, the range of the reference sample, and the location of the reference sample may each be determined based on the attributes of the target block and the attributes of the prediction sample.

[0544] For example, the attributes of the target block may include information related to the target block described in the embodiments, such as 1) size, 2) prediction mode, 3) intra prediction mode, 4) reference sample line, 5) sample value, and 6) coding parameter.

[0545] For example, the attributes of the prediction sample may include information related to the prediction sample described in the embodiments, such as 1) the sample value and 2) the location within the target block of the prediction sample, and may include coding parameters regarding the prediction sample.

[0546] Filtering may include in-loop filtering performed by a filter (130) and a filter (170), etc.

[0547]

[0548] Figure 6 shows a plurality of in-loop filters according to one example.

[0549] Multiple in-loop filters of in-loop filtering may include one or more of Luma Mapping with Chroma Scaling (LMCS), deblocking filter, Sample Adaptive Offset (SAO), and Adaptive Loop Filter (ALF).

[0550] Multiple in-loop filters can be connected sequentially. For example, multiple in-loop filters can be connected in the order of LMCS, deblocking filter, SAO, and ALF. Additionally, multiple in-loop filters can be connected in any order of all available permutations of the multiple in-loop filters. The output from one of the multiple in-loop filters can be used as an input to the next filter.

[0551] As illustrated in FIG. 6, an input image may be input to the first filter. The input image may be a block as described in the embodiments. For example, the input image may be a restored block generated by an adder (129) or an adder (169). The output from one filter may be input to the next filter. An output image may be generated by the last filter. The output image may be a filtered block as described in the embodiments. For example, the output image may be a filtered restored image generated by a filter (130) or a filter (170).

[0552] The target block can represent the image input to the filter. The filtered target block can represent the image output from the filter.

[0553] LMCS may include luminance signal mapping for the luminance signal of the target block and chroma signal scaling for the chroma signal of the target block.

[0554] Luminous signal mapping can perform codeword redistribution for the luminous signal.

[0555] Luma signal mapping may include forward mapping and inverse mapping. In forward mapping, the existing dynamic range may be divided into multiple intervals. The mapped dynamic range can be determined by performing codeword redistribution on the input image using a linear model for each interval. In inverse mapping, inverse mapping from the mapped dynamic range to the existing dynamic range is performed.

[0556] Chroma scaling can correct the chroma signal based on the interrelationship between the luminance signal and the corresponding chroma signal.

[0557] Forward mapping can be performed between inter-prediction of the luminance signal and restoration of the luminance signal, and between inter-prediction of the luminance signal and chroma scaling. Inverse mapping can be performed between restoration of the luminance signal and in-loop filtering of the luminance signal. Chroma scaling can be performed between inverse transform and restoration of the chroma signal.

[0558] According to this structure, inverse quantizations for the luminance and chroma signals, inverse transforms for the luminance and chroma signals, prediction for the luminance signal, and restoration for the luminance signal can be performed within a mapped dynamic domain. In-loop filtering for the luminance and chroma signals, inter-predictions for the luminance and chroma signals, intra-predictions for the chroma signal, and restoration for the chroma signal can be performed within the existing dynamic domain.

[0559] A deblocking filter can remove block distortion occurring at the boundaries between blocks within the reconstructed image. For example, the blocks may be transformed blocks. Additionally, the blocks may be subblocks of a specific block described in the embodiments. Here, the boundaries between blocks may refer to samples adjacent to the boundaries between blocks.

[0560] A deblocking filter can be applied to the vertical and horizontal boundaries between blocks. After filtering is performed on the vertical boundaries of the blocks, filtering can be performed again on the horizontal boundaries of the filtered blocks.

[0561] A deblocking filter may be applied optionally. Whether to apply a deblocking filter to a target block may be determined based on at least one of sample(s) contained within a specific number of columns or rows within the target block and sample(s) contained within a specific number of columns or rows within a neighboring block adjacent to a specific boundary.

[0562] When a deblocking filter is applied to a target block, the filter to be applied may be determined according to the required strength of deblocking filtering. In other words, among a plurality of different filters, the filter determined according to the strength of deblocking filtering may be applied to the target block. The plurality of filters may include one of a long-tap filter, a strong filter, a weak filter, and a Gaussian filter.

[0563] The maximum length of the deblocking filter can be determined based on attributes of the target block, such as the size of the target block, the components of the target block, and coding parameters.

[0564] SAO can compensate for distortion between the original image and the reconstructed image on a sample basis. For compensation, SAO can apply an appropriate offset to the sample values. In other words, the offset can be added to the sample values.

[0565] An offset can be determined for the target block. For example, the offset can be determined for each component of the CTB. The determined offset can be applied to samples within a specific component of the CTB.

[0566] SAO may include an SAO using an edge offset (EO) and an SAO using a band offset (BO). Depending on the characteristics of samples within a specific block, such as a CTU, whether to perform an SAO using EO and whether to perform an SAO using BO may be determined, respectively.

[0567] In an SAO using EO, correction for distortion of samples can be performed based on the direction of edges within the target block. The pattern classes of the EO may include horizontal patterns, vertical patterns, 135-degree diagonal patterns, and 45-degree diagonal patterns. For the target block, information indicating the pattern class applied to the target block and multiple offsets of the said pattern class may be signaled. There may be four offsets. For a target sample within the target block, adjacent samples of the target sample may be determined according to the direction of the pattern class. An offset to be applied to the target sample may be determined by the pattern of the adjacent samples.

[0568] In an offset using BO, correction for sample distortion can be performed by classifying the brightness values ​​of samples within the target block into specific bands. The bit depth of the input image can be divided into m intervals. For example, m can be 32. The specific bands can be n consecutive intervals among the m intervals. For example, n can be 4. n offsets for the n intervals can be signaled. Additionally, information indicating the first interval selected as the n intervals among the m intervals can be signaled. The offset of the interval corresponding to the target sample can be added to the sample value of the target sample of the target unit.

[0569] ALF can compensate for distortion between the restored image and the original image.

[0570] The filter coefficients of the ALF can be signaled through the bitstream.

[0571] The filter shape of the ALF can be determined by the components of the target block. For example, a 7x7 diamond-shaped filter can be used for the luminance component. A 5x5 diamond-shaped filter can be used for the chroma component.

[0572] In ALF, the characteristics of a specific block can be determined for that block, and the class of that specific block can be determined based on those characteristics. In other words, the determination of characteristics and the determination of the class in ALF can be performed in units of 4x4 blocks. Filter coefficients can be calculated according to the class. A specific block can be a block with a size of 4x4.

[0573] One of 25 classes can be determined as the class of a specific block based on the direction and activity determined using the gradient of the specific block. Depending on the gradient of the specific block, a rotation transformation, a vertical reflection transformation, and / or a diagonal reflection transformation may be applied to the filter.

[0574] Information regarding whether ALF is applied can be signaled to specific units such as CTBs.

[0575] An index indicating a filter to be applied to a specific unit among the available filters may be signaled. Here, the available filters may include fixed filters and filters configured using a parameter set. For example, the parameter set may be an Adaptive Parameter Set (APS). The fixed filters may be predefined identically in the encoding device (110) and the decoder (150). The filter coefficients of the filters configured using the parameter set may be determined based on the coding parameters.

[0576]

[0577] Entropy Encoding and Entropy Decoding

[0578] Figure 7 shows entropy encoding and entropy decoding according to one example.

[0579] The processes of entropy encoding by the entropy encoder (139) are illustrated at the top of Fig. 7.

[0580] The entropy encoder (139) may include a context modeler, a binarization unit, and an entropy encoding unit. The context modeler may include a context selection unit and a context memory.

[0581] The binarization unit can generate binaries for syntactic elements by performing binarization on the syntactic elements of the target block. Binarization may be a process of converting syntactic elements into the form of binaries.

[0582] Information about syntax elements and beans can be provided from the binarization unit to the context selection unit.

[0583] The context modeler can perform context updates.

[0584] Context can refer to occurrence probability information for each bin regarding syntactic elements that have already been encoded.

[0585] The context modeler may perform a context update to apply current probability information to the entropy encoding of the bins of the syntactic elements of the target block. The updated context may be stored in context memory. At this time, the updated context corresponding to the syntactic elements of the target block (or the bins within the syntactic elements of the target block) may be derived by the context modeler.

[0586] The context selector can select a context corresponding to a bin of a syntactic element of a target block. The selected context can be loaded from context memory and used as an updated context for entropy encoding of the bins of the syntactic element of the target block.

[0587] The updated context can be used for entropy encoding of syntactic elements of the target block.

[0588] The entropy encoding unit can generate encoded information for syntactic elements of a target block by performing entropy encoding using generated bins and an updated context, and can generate a bitstream containing the encoded information. The entropy encoding unit may use at least one of an arithmetic encoding method and a bypass encoding method.

[0589] At the bottom of Fig. 7, the entropy decoding process by the entropy decoder (161) is illustrated.

[0590] The entropy decoder (161) may include a context modeler, an entropy decoder, and an inverse binary converter. The context modeler may include a context selection unit and a context memory.

[0591] The context modeler can perform context updates.

[0592] The context can refer to the probability information of occurrence for each bin regarding syntactic elements that have already been decoded.

[0593] The context modeler may perform a context update to apply the currently decoded probability information to the entropy decoding for the bins of the syntactic elements of the target block. The updated context may be stored in context memory. At this time, the updated context corresponding to the syntactic elements of the target block (or the bins within the syntactic elements of the target block) may be derived by the context modeler.

[0594] The context selection unit can select a context corresponding to a bin of a syntactic element of a target block. The selected context can be loaded from context memory and can be used as an updated context for entropy decoding of the syntactic element of the target block.

[0595] The updated context can be used for entropy decoding of the syntactic elements of the target block.

[0596] The entropy decoding unit can generate bins for the segmentation elements of the target block by performing entropy decoding on the encoded information of the bitstream based on the updated context. The entropy decoding unit may use at least one of an arithmetic decoding method and a bypass decoding method.

[0597] The debinaryization unit can obtain syntactic elements of a target block by performing debinaryization on at least one of the generated beans. Debinaryization may be a process of converting at least one of the beans into the form of a syntactic element.

[0598] Information about syntax elements and beans can be provided from the inverse binary unit to the context selector.

[0599] The syntax element may be one of the coding parameters described in the examples.

[0600]

[0601] FIG. 8 is a flowchart of a method for performing intra prediction on a current block according to one embodiment of the present disclosure.

[0602] Referring to FIG. 8, in order to perform intra prediction for the current block, an intra prediction mode of the current block can be induced (S810). The number of induced intra prediction modes may be one or multiple.

[0603] In the present disclosure, an intra prediction mode may mean a general intra prediction mode. A general intra prediction mode may represent a non-directional prediction mode (e.g., DC or Planar) or a directional (or angular) prediction mode.

[0604] Alternatively, in the present disclosure, the intra prediction mode may represent at least one of a method for deriving a general intra prediction mode or a method for performing intra prediction. As an example, the intra prediction mode may mean DIMD (Decoder-side Intra Mode Derivation), OBIC (Occurrence based Intra Coding), TIMD (Template-based Intra Mode Derivation), MIP (Matrix-based Intra Prediction), IntraTMP (Intra Template Matching Prediction), IBC (Intra Block Copy), TMRL (Template-based multiple reference line intra prediction), EIP (Extrapolation Intra Prediction), CCCM (Convolutional Cross-Component Model), CCLM (Cross-Component Linear Model), GLM (Gradient Linear Model), SGPM (Spatial Geometry Partition Mode), Intra merge, Intra fusion, PDP (matrix-based position-dependent intra prediction), or MPMB (MPM-based Blend).

[0605] The intra prediction mode for the current block can be derived based on at least one of a derivation method using an intra prediction mode of at least one of a spatial reconstruction block or a temporal reconstruction block, a derivation method using samples within the reconstruction block, or a derivation method based on encoding parameters.

[0606] A temporally restored block may represent a block included in a picture restored prior to the current picture, or a block restored prior to the current block within the current picture. Here, the current picture refers to the picture containing the current block.

[0607] A spatially restored block may represent a block belonging to the same picture as the current block (i.e., the current picture). A spatially restored block may be a neighbor block adjacent to the current block or a non-neighbor block not adjacent to the current block.

[0608] Figure 9 is a drawing illustrating spatial restoration blocks.

[0609] Neighbor blocks represent blocks containing samples adjacent to the current block. Non-neighbor blocks may represent blocks excluding adjacent blocks.

[0610] Meanwhile, when inducing the intra prediction mode of the current block, it may be configured to make available only non-neighbor blocks belonging to a predetermined area among the non-neighbor blocks. Here, information indicating the predetermined area may be encoded and signaled.

[0611] Alternatively, a predetermined area may be predefined in the encoder and decoder. For example, the predetermined area may be determined based on at least one of the size of the CTU, the width and / or height of the current block.

[0612] The encoding parameters may include at least one of a flag related to the intra prediction mode, a block vector (BV), a motion vector, a merge flag, an induced intra prediction mode (e.g., an intra prediction mode induced by DIMD or an intra prediction mode induced by TIMD), intra prediction mode amplitude information (e.g., HoG: Histogram of Gradients) or intra prediction mode occurrence frequency information (e.g., HoC: Histogram of oCcurrences).

[0613] Here, the flags associated with the intra prediction mode may include at least one of DIMD_flag, OBIC_flag, TIMD_flag, MIP_flag, TMRL_flag, IntraTMP_flag, IBC_flag, EIP_flag, MPM_flag, Fusion_flag, SGPM_flag, CCCM_flag, CCLM_flag, GLM_flag, or MPMB_flag.

[0614] Below, the method for inducing the intra-prediction mode will be explained in detail.

[0615] Derivation method using intra-prediction mode of temporal reconstruction block and / or spatial reconstruction block

[0616] When deriving the intra prediction mode of the current block, the intra prediction mode of a restoration block spatially adjacent to the current block (i.e., a neighbor block) or a restoration block not spatially adjacent to the current block (i.e., a non-neighbor block) may be used.

[0617] Alternatively, when deriving the intra prediction mode of the current block, the intra prediction mode of the temporal reconstruction block of the current block may be used. For example, the temporal reconstruction block may be a reconstruction block included in a reference picture.

[0618] Alternatively, the intra prediction mode of the current block can be derived based on an accumulated list of intra prediction modes of blocks restored prior to the current block. The intra prediction modes accumulated in the above list may be referred to as history-based intra prediction modes.

[0619] If the current block does not have an intra prediction mode in the restoration block it intends to reference, the intra prediction mode of the current block can be derived by assuming that the intra prediction mode of the restoration block is a predetermined intra prediction mode.

[0620] Alternatively, if the restoration block that the current block intends to reference is not available, the intra prediction mode of the current block can be derived based on the intra prediction mode of an available block adjacent to the restoration block.

[0621] Alternatively, if the intra prediction mode of the recovery block corresponds to a predefined mode, the intra prediction mode of the recovery block may be replaced with another intra prediction mode to induce the intra prediction mode of the current block. For example, if the intra prediction mode of the recovery block is IntraTMP, IBC, EIP, PDP, or MIP, the Planar mode may be set as the intra prediction mode of the recovery block.

[0622] Alternatively, if the intra prediction mode of the recovery block is IntraTMP, IBC, PDP, MIP, or EIP, the intra prediction mode induced by applying DIMD or TIMD to the recovery block can be set as the intra prediction mode of the recovery block.

[0623] Alternatively, if the recovery block has a block vector (i.e., the recovery block is encoded / decoded in intra-block copy mode), the intra-prediction mode of the reference block indicated by the recovery block's block vector can be set to the intra-prediction mode of the recovery block.

[0624] If the intra prediction mode of the restoration block refers to a mode that performs combination prediction, at least one of the intra prediction modes used for combination prediction can be set as the intra prediction mode of the restoration block. Here, the mode that performs combination prediction may include at least one of DIMD, OBIC, TIMD, MPMB, SGPM, Intra Fusion, or IntraTMP. In this case, weights corresponding to each mode may be used. For example, one or more intra prediction modes may be set as the intra prediction mode of the restoration block in order of decreasing weight.

[0625] Induction method for intra-prediction mode using samples within a restoration block

[0626] When the DIMD method is applied, at least one intra prediction mode can be derived by applying a predetermined filter to the reconstructed samples. For example, at least one intra prediction mode can be derived by applying a Sobel filter to three reference sample lines around the current block.

[0627] When the TIMD method is applied, one or more intra predictions are performed on the reconstructed sample, and based on the difference between the reconstructed sample and the predicted sample, the error costs for one or more intra prediction modes can be calculated. Subsequently, at least one intra prediction mode can be derived by comparing the error costs of each intra prediction mode.

[0628] One or more restoration samples adjacent to the current block can be set as templates. Subsequently, a template similar to the template adjacent to the current block (hereinafter referred to as the current template) can be searched in a search area not adjacent to the current block. Based on the intra prediction mode of the reference block specified by the searched template, the intra prediction mode of the current block can be derived.

[0629] At least one intra prediction mode can be induced by the method described above. For example, the number of intra prediction modes induced by the DIMD method or the TIMD method may be 1, 2, 3, 4, 5, or 6.

[0630] At least one restoration sample used when the DIMD method or TIMD method is applied can be defined as a template.

[0631] The composition of the template can be selected from multiple candidates.

[0632] Figure 10 shows multiple candidates related to the configuration of the template.

[0633] In FIG. 10, a first candidate including an upper restoration area, a top-left restoration area, and a left restoration area, a second candidate including an upper restoration area and a left restoration area, a third candidate including only an upper restoration area, and a fourth candidate including only a left restoration area are shown.

[0634] The size of the template can be fixed in the encoder and decoder.

[0635] Alternatively, the size of the top restoration area and the left restoration area constituting the template can be adaptively determined based on the width and height of the current block. For example, the size of the top restoration area may be determined by the width of the current block, and the size of the left restoration area may be determined by the height of the current block. Accordingly, if the current block is non-square, the sizes of the top restoration area and the left restoration area within the template may differ.

[0636] Method for deriving intra-prediction mode based on encoding parameters

[0637] Depending on what the flags regarding the intra prediction mode of the current block indicate, the intra prediction mode of the current block can be induced. For example, if the value of DIMD_flag, OBIC_flag, TIMD_flag, MIP_flag, TMRL_flag, IntraTMP_flag, IBC_flag, MPM_flag, Fusion_flag, CCCM_flag, CCLM_flag, GLM_flag, IntraMerge_flag, BVIP_flag (Block Vector Intra Prediction) or MPMB_flag is 1, the mode corresponding to the flag can be induced as the intra prediction mode of the current block.

[0638] If the intra prediction mode of the current block is DIMD, OBIC, TIMD, TMRL, MPM, Fusion, or MPMB, the intra prediction mode of the current block can be re-derived. For example, if the intra prediction mode of the current block is DIMD mode or TIMD mode, the intra prediction mode of the current block can be derived using a reconstructed sample.

[0639] An intra prediction mode can be derived using the block vector of the current block or the restored block. For example, if the IBC_flag or IntraTMP_flag of the current block is 1, the intra prediction mode of the reference block indicated by the block vector can be set to the intra prediction mode of the current block.

[0640] Alternatively, a BVIP_flag indicating whether the intra prediction mode of the current block is a block vector-based prediction mode may be encoded and signaled. If BVIP_flag is 1, an IBC_flag indicating whether to apply an intra block copy mode may be additionally encoded and signaled.

[0641] If IBC_flag is 1, the intra prediction mode of the current block can be set to IBC mode. On the other hand, if IBC_flag is 0, the intra prediction mode of the current block can be set to IntraTMP mode.

[0642] When inter prediction is applied to the current block, the intra prediction mode of the reference block indicated by the motion information (e.g., motion vector) of the current block can be set to the intra prediction mode of the current block.

[0643] Alternatively, if the recovery block is encoded / decoded using inter-prediction, the intra-prediction mode of the reference block indicated by the movement information of the recovery block can be set to the intra-prediction mode of the current block.

[0644] A list of Most Probable Modes (MPMs) can be constructed based on at least one intra-prediction mode derived by the methods described above. Meanwhile, the number of MPM lists may be one or more.

[0645] For example, at least one of a previously defined intra prediction mode, an intra prediction mode derived from neighboring blocks, an intra prediction mode derived from non-neighboring blocks, an intra prediction mode derived based on DIMD, an intra prediction mode derived based on OBIC, an intra prediction mode derived based on TIMD, or an intra prediction mode derived based on block vectors may be added to the MPM list. In this case, the intra prediction modes may be added to the MPM list in a predetermined order.

[0646] The first MPM in the MPM list may be a planner. Alternatively, whether to set the first MPM in the MPM list as a planner may be adaptively determined based on whether a neural network-based intra prediction mode is applicable to the current block. For example, the first MPM in the MPM list may be set as a planner only when a neural network-based intra prediction mode is not applicable to the current block.

[0647] The MPM candidates included in the MPM list can be reordered according to a predetermined method. For example, the error for each MPM candidate can be calculated, and then the MPM candidates can be reordered in order of smallest error. In this case, the error of an MPM candidate may represent the difference between the predicted sample obtained by applying intra-prediction to the reconstructed sample location using the MPM candidate and the reconstructed sample.

[0648] If matrix-based intra prediction is applicable to the current block, a general intra prediction mode other than the matrix-based intra prediction mode may be replaced with an adjacent matrix-based intra prediction mode. Here, whether matrix-based intra prediction is applicable to the current block may be determined based on at least one of the availability of left and top reference samples and whether the size of the current block satisfies the conditions for performing matrix-based intra prediction.

[0649] Specifically, a general intra prediction mode can be replaced with a matrix-based intra prediction mode with a larger value than the general intra prediction mode. Depending on the size of the current block, the replacement matrix-based intra prediction mode may be set differently.

[0650] Some of the MPMs included in the MPM list can be inserted into the first (Primary) MPM list, and the remainder can be inserted into the second (Secondary) MPM list. For example, if there were 22 MPMs inserted in the initial MPM list, 6 of them can be inserted into the first MPM list, and the remaining 16 can be inserted into the second MPM list.

[0651] The initial MPM list used to generate the first MPM list and the second MPM list may also be referred to as the General MPM list.

[0652] If the intra prediction mode of the current block is DIMD, OBIC, TIMD, MPMB, SGPM, IntraTMP, IBC, CCCM, CCLM, or GLM, the merge list of the current block can be configured.

[0653] The merge candidates in the merge list may include at least one of HoG (Histogram of Gradient), HoC (Histogram of Occurrence), SAD (Sum of Absolute Difference), IntraTMP Index, IntraTMP Fusion Index, at least one intra prediction mode derived based on DIMD, or at least one intra prediction mode derived based on TIMD.

[0654] Merger candidates can be derived from neighboring blocks adjacent to the current block. For example, at least one intra prediction mode derived by applying DIMD to neighboring blocks or at least one intra prediction mode derived by applying TIMD can be set as a merge candidate.

[0655] When inserting merge candidates into the merge list, a duplicate check may be performed. Accordingly, merge candidates that duplicate merge candidates already inserted into the merge list may not be inserted into the merge list.

[0656] Merge candidates can be reordered based on error costs. The error costs of the merge candidates may be template matching costs. That is, for a template adjacent to the current block, prediction samples can be generated by performing intra prediction based on at least one intra prediction mode included in the merge candidate, and then the error costs of the merge candidate can be calculated based on the difference between the reconstructed samples within the template and the prediction samples.

[0657] If a merge candidate includes multiple intra-prediction modes, the error cost of the merge candidate can be calculated by weighting the error costs of each of the intra-prediction modes.

[0658] The intra prediction modes of the merge candidate can be replaced with block vectors. Specifically, if the error cost of the block vector derived through template matching is smaller than the cost of the intra prediction modes included in the merge candidate, the intra prediction modes of the merge candidate can be replaced with block vectors.

[0659] A merge list can be derived based on a predetermined block size. Multiple blocks (e.g., multiple sub-blocks) belonging to a predetermined block size can share a merge list derived based on the predetermined block size.

[0660] Next, referring to FIG. 8, reference samples can be configured to perform intra prediction for the current block (S820). One of a plurality of reference sample line candidates can be selected, and reference samples can be derived based on the reconstructed samples belonging to the selected reference sample line.

[0661] Reference samples may be constructed to perform intra-prediction on the current block, or on sub-blocks having a size smaller than the current block or a size different from the current block. Here, a sub-block may represent a block generated by dividing a coding block or a prediction block.

[0662] Reference samples can be derived through reconstructed samples around the current block or a combination of reconstructed samples. Additionally, filtering may be applied when constructing the samples.

[0663] For example, reconstructed samples belonging to multiple reconstructed sample lines can be set as reference samples as they are. Alternatively, reference samples can be derived by filtering reconstructed samples belonging to the same reconstructed sample line or by filtering reconstructed samples belonging to different reconstructed sample lines.

[0664] You can configure reference samples by selecting one or more restoration sample lines adjacent to the current block.

[0665] For example, reference samples can be constructed by selecting one of multiple restoration sample lines.

[0666] For example, reference samples can be constructed by combining multiple restored sample lines.

[0667] For example, multiple restored sample lines can be selected to form reference samples. The number of restored sample lines may be two or more, for example, two, three, or four.

[0668] Depending on the intra prediction mode, one of multiple reference sample lines may be fixedly selected. In this case, information indicating the selected reference sample line may not be encoded / decoded.

[0669] Multiple sets containing one or more reference sample lines may be defined. For example, a first set (set1) may consist only of the first reference sample line, and a second set (set2) may consist of the first and second reference sample lines. A third set (set3) may consist of the first through third reference sample lines, and a fourth set (set4) may consist of the first through fourth reference sample lines. A fifth set (set5) may consist of the first through fifth reference sample lines. As described above, after defining multiple sets, information indicating one of the multiple sets can be encoded and signaled.

[0670] The number or location of reference sample lines can be determined based on at least one of the intra-prediction mode of the current block, the width and / or height of the current block, or the encoding parameters of the current block.

[0671] For example, if the intra prediction mode of the current block is DIMD, TIMD, or IntraTMP, the number of reference sample lines may be 2, 3, or 4.

[0672] For example, if the width or height of the current block is 4, the number of reference sample lines is 1, whereas if the width or height of the current block is 8, the number of reference sample lines may be 2. Also, if the width or height of the current block is 16, the number of reference sample lines is 3, and if the width or height of the current block is 32, the number of reference sample lines may be 4.

[0673] Meanwhile, the number of reference sample lines located at the top of the current block and the number of reference sample lines located to the left of the current block can be set differently.

[0674] You can construct the reference samples of the current block by selecting one of multiple reference sample lines. For example, you can select one of four reference sample lines and derive reference samples based on the restored samples belonging to the selected reference sample line.

[0675] Alternatively, depending on at least one of the distance between the current block and the reference sample or the intra-prediction mode of the current block, the average, maximum, minimum, or median value of a plurality of restored samples can be derived as the reference sample.

[0676] At least one of the number, location, or configuration of the restoration sample lines used to derive reference samples can be adaptively determined based on whether the top boundary or left boundary of the current block corresponds to the boundary of the parent unit. Here, the parent unit may represent a picture, slice, tile, or encoding tree block (CTB).

[0677] For example, if the top boundary of the current block corresponds to the boundary of a picture, tile slice, or encoding tree block, the number of reference sample lines located at the top of the current block can be set to 1.

[0678] The configured reference samples can also be used as templates.

[0679] In constructing reference samples, a padding process may be performed.

[0680] Whether padding is performed may be determined based on the availability of the block containing the reference sample. For example, if the block containing the reference sample is available, the reference sample may be derived based on the restored sample at that location. Conversely, if the block containing the reference sample is unavailable, the reference sample may be derived through padding. Specifically, the reference sample may be derived by padding one or more available reference samples at the location of the reference sample contained in the unavailable block.

[0681] If a reference sample exists outside the boundaries of a predetermined unit, the reference sample may be determined to be unavailable. Here, the predetermined unit may include at least one of a picture, a tile, a slice, a encoding tree block (CTB), or an area of ​​a predefined size.

[0682] For example, a reference sample located outside of one or two coding tree blocks located in the upper direction of a coding tree block containing the current block may be determined to be unavailable.

[0683] For example, a reference sample located outside a template of a predetermined size may be determined to be unavailable.

[0684] If a block containing a reference sample is encoded / decoded in Inter Mode, IBC, or IntraTMP, the reference sample may be determined to be unavailable. In this case, padding for the unavailable reference sample may be performed using a sample at a location indicated by the motion vector or block vector of the block containing the unavailable reference sample.

[0685] When performing padding on a reference sample, the availability of the sample used for padding may not be determined. That is, without determining whether the adjacent reference sample of an unavailable reference sample is available, the adjacent reference sample may be padded to the location of the unavailable reference sample. Whether to omit availability may be determined based on at least one of the intra-prediction mode or encoding parameters of the current block.

[0686] Filtering can be applied to reference samples.

[0687] Based on at least one of the intra-prediction mode of the current block or the size and / or shape of the current block, whether to apply filtering to reference samples or at least one of the filter type may be determined.

[0688] For example, if the intra prediction mode of the current block is DIMD, OBIC, TIMD, IntraTMP, MIP, IBC, PDP, or MPMB, reference samples may not be filtered.

[0689] For example, if the intra prediction mode of the current block is DIMD, unfiltered reference samples may be used when deriving the intra prediction mode used for combination prediction. Additionally, unfiltered reference samples may be used when performing intra prediction using the intra prediction mode to be used for combination prediction. Furthermore, unfiltered reference samples may be used when performing filtering (e.g., PDPC) on the prediction samples. Since filtered reference samples are not used, one buffer is sufficient for storing reference samples.

[0690] For example, if the intra-frame prediction mode of the current block is DIMD, TIMD, IntraTMP, MIP, or IBC, a low-pass filter can be applied to the reference samples.

[0691] For example, the application of filtering to multiple reference sample lines may differ. For instance, filtering may be applied to reference samples belonging to the first reference sample line adjacent to the current block, while filtering may not be applied to reference samples belonging to the second reference sample line.

[0692] For example, you can use a reference sample with filtering applied and a reference sample without filtering applied together.

[0693] For example, the type of filter to be used for filtering may be set differently based on at least one of the intra prediction mode of the current block or the size and / or shape of the current block. For example, one of a 3-tap filter, a 5-tap filter, a 7-tap filter, or an N-tap filter may be selected. Here, N may be an integer greater than 7.

[0694] For example, the filter shape can be adaptively determined based on at least one of the intra-prediction mode of the current block or the size and / or shape of the current block.

[0695]

[0696] Based on a reference sample of the current block, at least one intra prediction mode can be derived. For example, if a DIMD mode is applied to the current block, at least one directional prediction mode or non-directional prediction mode can be derived by analyzing a reference sample of the current block.

[0697] FIG. 11 shows an example of deriving at least one directional prediction mode from reference samples of the current block.

[0698] At least one directional prediction mode can be derived using N reference sample lines adjacent to the current block. N can be a natural number such as 1, 2, 3, or 4. In FIG. 11, it was assumed that N is 4.

[0699] A specific filter may be applied to the reference samples of the current block. For example, a 3x3 Sobel filter of the following Equation 1 or a 3x3 Prewitt filter of Equation 2 may be applied to the reference samples.

[0700]

[0701]

[0702] By applying the filter of Equation 1 or Equation 2, the horizontal and vertical gradients of the reference sample corresponding to the center position of the filter can be derived. The gradients are values ​​obtained through a dot product, and can be derived by multiplying each of the filter coefficients by the reference sample at the same position and then summing the obtained values.

[0703] Directionality can be derived based on the horizontal and vertical gradients of the reference sample.

[0704] By applying a filter to multiple reference samples, the directionality of each of the multiple reference samples can be induced.

[0705] Meanwhile, the reference samples to which the filter is applied can be determined adaptively.

[0706] For example, the reference samples to which the filter is applied may be determined differently based on at least one of the width or height of the current block. For example, if the current block is a non-square block where the width is greater than the height, the filter may be applied only to reference samples located above the top boundary of the current block. Conversely, if the current block is a non-square block where the height is greater than the width, the filter may be applied only to reference samples located to the left of the left boundary of the current block.

[0707] For example, a reference sample that is the target of direction induction can be determined by moving the filter window in predetermined units. For example, instead of applying a filter to all reference samples, a filter can be applied only to even-numbered or odd-numbered reference samples. In this case, the filter applied to even-numbered reference samples and the filter applied to odd-numbered reference samples may be different. Meanwhile, whether a reference sample is located at an even position or an odd position can be determined based on at least one of the x-coordinate or y-coordinate of the reference sample.

[0708] For example, at least one of multiple reference sample lines can be selected, and a filter can be applied only to the reference samples belonging to the selected reference sample line. For example, after selecting a second reference sample line and a third reference sample line, a filter can be applied only to the reference samples belonging to the second reference sample line and the third reference sample line.

[0709] By combining the above embodiments, the positions of reference samples to which a filter is applied can be set differently for each reference sample line. For example, a filter may be applied only to reference samples at even positions among the reference samples belonging to the second reference sample line, and a filter may be applied only to reference samples at odd positions among the reference samples belonging to the third reference sample line.

[0710] For example, a filter may not be applied to the reference sample(s) located at the top-left of the current block. That is, while a filter is applied to the reference samples located at the top, top-right, left, and bottom-left of the current block, a filter may not be applied to the reference sample(s) located at the top-left of the current block.

[0711] For example, if the current block corresponds to a chroma component, samples belonging to the lumina block corresponding to the current block (i.e., collocated lumina block) can be used as reference samples. That is, by applying a filter to samples belonging to the lumina block, the directionality of the samples belonging to the lumina block can be obtained.

[0712] Figure 12 shows an example where a filter is applied to some of the samples belonging to the luma block.

[0713] In the example illustrated in FIG. 12, the size of the luma block is exemplified as being at an 8x8 position. In the example illustrated in FIG. 12, by applying a filter to the sample at the shaded position, the orientation of the sample at the shaded position can be induced.

[0714] The directionality of a reference sample can be mapped to a directionality prediction mode. That is, based on the directionality of the reference sample, the directionality prediction mode of the reference sample can be determined.

[0715] After determining the directional prediction mode for each of the reference samples to which the filter is applied, a histogram can be generated by accumulating the amplitude values ​​for each intra-prediction mode. Here, the amplitude value can be derived by summing the horizontal gradient and the vertical gradient as shown in Equation 3.

[0716]

[0717] The method of deriving the amplitude value may be determined differently based on at least one of a quantization parameter (QP) or a change in sample value.

[0718] Alternatively, a histogram can be generated based on the frequency of occurrence in the intra-prediction mode.

[0719] Meanwhile, if the gradient value is smaller than the threshold value, the gradient value can be determined as an outlier and the gradient value can be set to 0.

[0720] Alternatively, if the accumulated amplitude value of the intra prediction mode on the histogram is smaller than the threshold value, the value can be determined as an outlier and the accumulated amplitude value of the intra prediction mode can be set to 0.

[0721] Even though the number of inductions of the first intra prediction mode is greater than that of the second intra prediction mode, there may be cases where the sum of the amplitude values ​​of multiple reference samples having the first intra prediction mode is smaller than the sum of the amplitude values ​​of reference samples having the second intra prediction mode. To address this problem, when a single intra prediction mode occurs multiple times, a weight can be applied to the amplitude value of the corresponding intra prediction mode. Here, the weight may be based on the frequency of occurrence of the intra prediction mode.

[0722] For example, the ratio of the number of occurrences of the intra prediction mode (e.g., #mode) to the number of reference samples to which the filter is applied (e.g., #total) can be reflected in the amplitude value. For example, Equation 4 or Equation 5 shows an example of adjusting the amplitude value by reflecting the above ratio.

[0723]

[0724]

[0725] Instead of setting the number of reference samples to which the filter is applied as the variable #total, you can also set the total number of derived intra-prediction modes as the variable #total.

[0726] Meanwhile, in mathematical formula 5, the item log2(#total) may be replaced with (log2(#total)+1) depending on the value of the variable #total.

[0727] The amplitude value of the intra prediction mode derived from the reference sample can be modified based on at least one of the number of occurrences of the intra prediction mode or the size of the current block.

[0728] For example, if the number of occurrences of an intra prediction mode derived based on the gradient of a reference sample is 1, the intra prediction mode can be determined as an outlier and the amplitude value of the intra prediction mode can be set to 0.

[0729] Alternatively, the amplitude value of an intra prediction mode can be modified based on the total number of intra prediction modes derived from the reference samples. For example, if the occurrence count of an intra prediction mode is less than or equal to a threshold value, the amplitude value of that intra prediction mode can be set to 0. In this case, the threshold value can be set to (#total >> N). Here, the variable #total can represent the total number of reference samples or the total number of intra prediction modes derived from the reference samples. Also, N can be an integer greater than or equal to 1. For example, if the variable #total is 32 and N is 5, the amplitude value of an intra prediction mode that occurs once can be determined to be 0 (threshold value: 32 >> 5 = 1).

[0730] Alternatively, threshold values ​​may be predefined in the encoder and decoder.

[0731] Alternatively, information indicating a threshold value may be encoded and signaled.

[0732] Alternatively, the threshold value may be set differently depending on the size of the current block. For example, if the current block size is small, such as 4x4, 4x8, or 8x4, all intra prediction modes derived from reference samples can be used. On the other hand, if the current block size is large, such as 64x64, only intra prediction modes with occurrences greater than or equal to the threshold value can be used. In this case, the amplitude value of intra prediction modes with occurrences less than the threshold value can be set to 0.

[0733] Alternatively, the amplitude values ​​of the intra prediction mode may be modified based on the results of comparing the width and height of the current block. For example, if the current block is a non-square block where the width is greater than the height, a greater weight may be assigned to the amplitude values ​​of the intra prediction mode derived from reference samples located at the top of the current block than to the amplitude values ​​of the intra prediction mode derived from reference samples to the left of the current block. Conversely, if the current block is a non-square block where the height is greater than the width, a greater weight may be assigned to the amplitude values ​​of the intra prediction mode derived from reference samples located to the left of the current block than to the amplitude values ​​of the intra prediction mode derived from reference samples to the top of the current block.

[0734] Alternatively, the amplitude value may be modified based on the directionality of the intra prediction mode. For example, if the intra prediction mode derived from a reference sample located at the top of the current block has a vertical direction or a mode similar to the vertical direction, a weight greater than 1 may be applied to the amplitude value of the intra prediction mode. That is, by applying the weight, the amplitude value of the intra prediction mode may be modified to a larger value. Additionally, if the intra prediction mode derived from a reference sample located at the left of the current block has a horizontal direction or a mode similar to the horizontal direction, a weight greater than 1 may be applied to the amplitude value of the intra prediction mode.

[0735] Here, similar modes may represent intra-prediction modes in which the index difference with respect to the reference mode (i.e., vertical mode or horizontal mode) is below a threshold value.

[0736] The amplitude value of a non-directional intra prediction mode can be derived based on at least one of a histogram, the amplitude value of intra prediction modes, or the frequency of occurrence of derived modes.

[0737] For example, when the horizontal and vertical gradients obtained by applying a filter to a reference sample are both zero, the directionality of the reference sample cannot be derived. In this case, the intra prediction mode of the reference sample can be derived into a non-directional intra prediction mode. Here, the non-directional intra prediction mode may represent a Planar mode.

[0738] Meanwhile, when both the horizontal gradient and the vertical gradient are zero, the amplitude value is also set to zero. Accordingly, when the intra prediction mode of the reference sample is a non-directional intra prediction mode, the amplitude value can be derived in a different way.

[0739] Mathematical Equation 6 shows an example of deriving the amplitude value of a non-directional intra-prediction mode.

[0740]

[0741] In Equation 6, the variable AMP_SUM represents the sum of the amplitude values ​​of all intra prediction modes (i.e., all directional intra prediction modes). The variable #total represents the total number of occurrences of all intra prediction modes, and the variable #planar represents the number of occurrences of non-directional intra prediction modes. That is, as in the example of Equation 6, the amplitude value AMP_Planar of the non-directional intra prediction mode can be derived by multiplying the sum of the amplitude values ​​of all intra prediction modes (AMS_SUM) by the ratio of the number of occurrences of non-directional intra prediction modes (#planar) to the total number of occurrences of all intra prediction modes (#total).

[0742] Alternatively, the amplitude value of the non-directional intra prediction mode can be derived by multiplying the number of occurrences of the non-directional intra prediction mode by a predefined value.

[0743] Histograms can be generated by group. Specifically, histograms can be generated for each of the group containing the top reference samples of the current block, the group containing the left reference samples of the current block, and the group containing the top-left reference samples of the current block.

[0744] Dependencies between individually generated histograms can be determined. These dependencies can be used to determine the weights used for joint prediction.

[0745] For example, the top histogram (H_Top) can be derived based on the top and top-right reference samples of the current block, the left histogram (H_Left) based on the left and bottom-left reference samples of the current block, and the top-left histogram (H_Top-Left) based on the top-left reference samples of the current block. Additionally, all of the above histograms can be combined to derive a combined histogram (H_Total).

[0746] It is possible to determine whether the dependency is high based on whether the amplitude value of a predefined intra prediction mode has a relatively large value. For example, if the value obtained by multiplying the amplitude value of the first intra prediction mode in the combined histogram (H_Total) by 1 / 3 is greater than the amplitude value of the first intra prediction mode in the top histogram (H_Top), the first intra prediction mode may be determined to be more dependent on the left reference sample.

[0747] If the value obtained by multiplying the amplitude value of the second intra prediction mode in the combined histogram (H_Total) by 1 / 3 is greater than the amplitude value of the second intra prediction mode in the left histogram (H_Left), the second intra prediction mode may be determined to be more dependent on the upper reference sample.

[0748] Alternatively, dependency can be determined based on the directionality of the intra prediction mode. For example, if the intra prediction mode has a vertical direction, the intra prediction mode may be determined to be dependent on the top reference sample. On the other hand, if the intra prediction mode has a horizontal direction, the intra prediction mode may be determined to be dependent on the left reference sample.

[0749] Alternatively, dependencies can be determined based on the size and / or shape of the current block. For example, dependency determination can be performed after normalizing the top histogram (H_Top) and left histogram (H_Left) based on the width and height of the current block.

[0750] Mathematical Equation 7 shows an example where the upper histogram and the left histogram are adjusted.

[0751]

[0752] As illustrated in Equation 7, the amplitude value H_Top[ipm] of the intra-prediction mode ipm included in the top histogram (H_Top) can be modified based on the width of the current block, and the amplitude value H_Left[ipm] of the intra-prediction mode ipm included in the left histogram (H_Left) can be modified based on the height of the current block.

[0753] Subsequently, the dependency of an intra prediction mode can be determined by comparing the adjusted amplitude value of a specific intra prediction mode on the upper histogram with the adjusted amplitude value of a specific intra prediction mode on the left histogram. For example, if the amplitude value of a first intra prediction mode on the upper histogram is greater than the amplitude value of a first intra prediction mode on the left histogram, the first intra prediction mode can be determined to be dependent on the upper reference sample. Conversely, if the amplitude value of a second intra prediction mode on the left histogram is greater than the amplitude value of a second intra prediction mode on the upper histogram, the second intra prediction mode can be determined to be dependent on the left reference sample.

[0754] Meanwhile, if it is determined that a specific intra prediction mode is not dependent on both the upper reference sample and the left reference sample, the said intra prediction mode may be determined to have no dependency. The specific intra prediction mode may be a mode whose dependency is not determined by the amplitude-based comparison or a non-directional mode.

[0755] Dependency determination can be applied in the same way in TIMD mode. Specifically, under TIMD mode, dependency of the intra prediction mode can be determined using at least one of SAD, SATD (Sum of Absolute Transformed Differences), or MRSAD (Mean Removed SAD).

[0756] At least one intra prediction mode can be selected from a histogram, and intra prediction can be performed based on the selected at least one intra prediction mode. The intra prediction mode selected from the histogram can be referred to as the derived intra prediction mode.

[0757] Meanwhile, intra prediction modes on the histogram can be sorted based on amplitude values. For example, intra prediction modes on the histogram can be reordered in descending order of amplitude values.

[0758] On the histogram, you can perform intra prediction for the current block by selecting the first intra prediction mode. Here, the first intra prediction mode may represent the intra prediction mode with the largest amplitude value.

[0759] Alternatively, the top N intra prediction modes on the histogram can be selected, and intra prediction can be performed for the current block based on each intra prediction mode. That is, combined prediction using N intra prediction modes can be performed. When multiple prediction blocks are obtained, the prediction block of the current block can be obtained by weighting the multiple prediction blocks. N can be a natural number greater than or equal to 1. For example, N can be 6.

[0760] If the number of intra prediction modes included in the histogram is less than N, all intra prediction modes included in the histogram can be used.

[0761] Meanwhile, all N intra prediction modes can be directional prediction modes.

[0762] Alternatively, if the number of intra prediction modes included in the histogram is less than N, an additional non-directed intra prediction mode may be used in addition to the intra prediction modes included in the histogram.

[0763] Alternatively, the number of intra prediction modes used for combination prediction can be adaptively determined based on the size and / or shape of the current block. Equation 8 shows an example where the number of intra prediction modes used for combination prediction is adaptively determined according to the size of the current block.

[0764]

[0765] As in the example of mathematical formula 8, the smaller value between the current block size Block_size and the constant N can be determined as the number of intra prediction modes M used for combined prediction.

[0766] Meanwhile, the size of the current block may represent a value obtained by taking the logarithm of the width (W) and height (H) of the current block, or at least one of these. For example, the size of the current block may represent at least one of W, H, (W+H)>>2, log2(W)+log2(H), (log2(W)+log2(H)-2), or log2(W*H).

[0767] When selecting an intra prediction mode, the amplitude value of the intra prediction mode may be considered. For example, if the amplitude value of the first intra prediction mode is smaller than or equal to a threshold value compared to the amplitude value of the second intra prediction mode, the first intra prediction mode may be excluded from the combined prediction.

[0768] Here, the threshold value may be a value predefined in the encoder and decoder. Alternatively, the threshold value may be set to 1 / N of the amplitude value of a second intra prediction mode, which has an amplitude value one step higher than the first intra prediction, or 1 / N of the amplitude value of a third intra prediction mode, which has the largest amplitude value. Here, N may be 2 or 3.

[0769] In performing combined prediction, non-directed intra prediction modes may be additionally utilized. For example, if all intra prediction modes derived from the histogram are directional modes, at least one of DC, planar, or block vector-based prediction may be additionally utilized. That is, the prediction blocks of the current block may be derived by weighting the prediction blocks derived from the directional modes and at least one prediction block derived based on at least one of DC, planar, or block vector.

[0770]

[0771] Based on the derived intra prediction modes, weights assigned to the intra prediction modes can be determined when performing combined intra prediction.

[0772] For example, weights assigned to prediction blocks derived based on each intra prediction mode can be determined based on the amplitude values ​​of each intra prediction mode on the histogram. Specifically, weights assigned to prediction blocks can be determined by converting the weights corresponding to the amplitude values ​​of the intra prediction modes on the histogram into 1 / 64 units. Alternatively, weights assigned to prediction blocks derived from non-directional intra prediction modes can be determined by converting the ratio of the number of non-directional intra prediction modes to the number of derived intra prediction modes into a 1 / 64 unit ratio.

[0773] Alternatively, weights assigned to prediction blocks derived based on intra prediction modes can be determined based on weights derived based on the amplitude of intra prediction modes and weights derived based on the frequency of occurrence of intra prediction modes.

[0774] In determining the weights in units of 1 / 64, the weight applied to the non-directional intra prediction mode may be fixed at 16 (i.e., 16 / 64), and the weight applied to the directional intra prediction mode may be fixed at 48 (i.e., 48 / 64). Meanwhile, when multiple directional intra prediction modes are used, the weight for each directional prediction mode may be determined in units of 1 / 48 based on the amplitude value or frequency of occurrence.

[0775] In determining amplitude-based weights, a lookup table may be used to eliminate complexity caused by division operations. If the sum of the determined weights is less than or greater than 64, the weight of the mode with the smallest amplitude value among the N intra prediction modes may be adjusted. The adjustment may be to apply an offset (e.g., +1, -1, +2, -2).

[0776] When using the induced intra-prediction mode, the relationship between the location where the intra-prediction mode was induced and the reference sample may be considered.

[0777] For example, intra prediction for the current block can be performed using a first intra prediction mode derived based on the top reference samples of the current block and a second intra prediction mode derived based on the left reference samples of the current block. In this case, the weighted sum method may differ depending on the degree of similarity or difference between the first intra prediction mode and the second intra prediction mode.

[0778] Template-based intra prediction can be performed based on intra prediction modes derived from DIMD modes. Subsequently, N intra prediction modes with small error costs can be selected. For example, three reference sample lines can be set as templates, and intra prediction can be performed on the templates based on each of the intra prediction modes derived from DIMD modes.

[0779] Specifically, intra-prediction can be performed on the template based on reference samples belonging to the fourth reference sample line.

[0780] By performing intra prediction on a template based on each of the derived intra prediction modes, the error cost of each derived intra prediction mode can be calculated. Subsequently, each derived intra prediction mode can be sorted based on the error cost.

[0781] Based on the error cost, at least one intra prediction mode can be selected, and an intra prediction for the current block can be performed. If there are multiple selected intra prediction modes, the prediction block of the current block can be obtained by weighting the multiple prediction blocks.

[0782] An intra prediction mode derived based on the DIMD mode can be used when performing a transformation of the current block. For example, a first transformation kernel or a second transformation kernel of the current block can be selected based on one or more intra prediction modes derived based on the DIMD mode.

[0783] When performing intra prediction for the current block based on an intra prediction mode derived from DIMD mode, MIP or PDP prediction can be configured not to be used.

[0784] For example, if the intra prediction mode satisfies certain conditions, the intra prediction mode can be replaced with an MIP or PDP prediction method. However, if the intra prediction mode is derived through a DIMD mode, the intra prediction mode may not be replaced with an MIP or PDP prediction method regardless of whether it satisfies certain conditions.

[0785] Here, the predetermined conditions may include at least one of the size, shape, or index of the current block or intra prediction mode. For example, if the index of the intra prediction mode is 16 and satisfies the predetermined conditions, MIP or PDP prediction may be performed. However, if the intra prediction mode with an index of 16 is derived through the DIMD mode, the existing directional prediction method may be used instead of replacing the intra prediction mode with MIP or PDP prediction.

[0786]

[0787] When the OBIC mode is applied, an intra prediction mode of adjacent blocks of the current block can be derived, and then the derived intra prediction modes of adjacent blocks can be accumulated. Here, the adjacent blocks may include at least one of spatially adjacent blocks or temporally adjacent blocks. Subsequently, at least one of the accumulated intra prediction modes can be selected to perform an intra prediction for the current block.

[0788] FIG. 13 is a diagram illustrating the process of exploring the intra-prediction mode of adjacent blocks.

[0789] As shown in the example illustrated in FIG. 13, 4x4 areas can be sequentially explored in a rightward direction from the top-left position of the current block. During the above exploration process, the first intra prediction mode explored can be set as the intra prediction mode of the upper adjacent block.

[0790] Likewise, 4x4 areas can be sequentially searched downwards from the top-left position of the current block. In the above search process, the first intra prediction mode discovered can be set as the intra prediction mode of the left adjacent block.

[0791] The intra prediction modes of adjacent blocks derived by the above method may be included in a predetermined list. In this case, the number of intra prediction modes included in the list may be predefined in the encoder and decoder.

[0792] In addition to the intra prediction modes of adjacent blocks adjacent to the current block, the intra prediction modes of non-adjacent blocks not adjacent to the current block can also be added to the list.

[0793] The list may include information on the intra prediction mode and the encoding block from which the intra prediction mode was derived.

[0794] Meanwhile, the list may contain duplicate identical intra prediction modes.

[0795] The intra prediction modes included in the list can be sorted according to a predetermined criterion. For example, the intra prediction modes derived from adjacent blocks can be sorted in order of the distance between the current block and adjacent blocks.

[0796] For each intra prediction mode included in the list, the size of the derived encoding block can be accumulated. Here, the size of the encoding block can represent the product of the width and the height. That is, the number of samples encoded / decoded using the corresponding intra prediction mode can be accumulated.

[0797] When accumulating intra prediction modes by the size of the encoding block, the intra prediction modes can be accumulated by the value obtained by applying a weight to the size of the encoding block. That is, the larger the weight assigned to the intra prediction mode, the larger the value accumulated for that intra prediction mode becomes. For example, the weight of the intra prediction mode accumulated in the histogram can be determined as shown in the following mathematical formula 9.

[0798]

[0799] In accumulating the size of the encoding block, weights based on the distance from the current block may be used. For example, the weight applied to the intra-prediction mode of an adjacent block may have a larger value than the weight applied to the intra-prediction mode of a non-adjacent block. For example, a weight (K) of {4, 2, 1, 0} i ) can be applied, and weights can be adaptively determined based on the distance from the current block.

[0800] Alternatively, when accumulating the size of the encoding block, if the intra prediction mode of the encoding block is a mode that performs combined prediction (e.g., at least one of DIMD, OBIC, TIMD, or IntraTMP), weights used for combined prediction may be used. For example, it is assumed that combined prediction using the first to sixth intra prediction modes is performed on the encoding block. In addition, it is assumed that the weights assigned to the first to sixth intra prediction modes are the first to sixth weights.

[0801] When accumulating the size of a coding block, if an SGPM mode is applied to the coding block, the accumulated value of the intra prediction modes can be determined based on the partition information of the coding block. For example, the first intra prediction mode used for the first partition within the coding block can be accumulated by the size of the first partition, and the second intra prediction mode used for the second partition within the coding block can be accumulated by the size of the second partition.

[0802] Intra-prediction modes can be sorted in order of accumulated size. A list or sorted list can be referred to as a HoC (Holy Odds) histogram.

[0803] The top N intra prediction modes among the sorted intra prediction modes in the list can be selected. Here, N can be an integer greater than or equal to 1.

[0804] When selecting an intra prediction mode, a non-directional intra prediction mode may be excluded. Here, the non-directional intra prediction mode may include at least one of a planar, DC, or block vector-based prediction.

[0805] The weights for the intra prediction modes selected from the list can be determined based on the accumulated block size values. For example, when a weight of 1 / 64 units is applied, the weights for each intra prediction mode can be determined by scaling the accumulated block size values ​​of each intra prediction mode to 1 / 64 units.

[0806] When performing combined intra prediction, it can be configured so that an undirected intra prediction mode is always included. In this case, the weight for the undirected intra prediction mode can be set to a fixed value. For example, a weight of 16 (i.e., 16 / 64) can be applied to a prediction block derived based on an undirected intra prediction mode. Accordingly, the sum of the weights assigned to the intra prediction modes derived from the list can be 48 (i.e., 48 / 64).

[0807] If the number of intra prediction modes selected from the list is 1, the weights for the non-directional intra prediction mode and the weights for the directional intra prediction mode can be determined as fixed values. For example, the two weights can be determined as (21, 43) or (16, 48).

[0808] The weights applied to non-directional intra prediction modes can be adjusted based on a list. For example, if the frequency of non-directional intra prediction modes is high among the intra prediction modes included in the list, the weight for non-directional intra prediction modes can be set to a larger value.

[0809]

[0810] When a TIMD mode is applied to the current block, at least one intra prediction mode can be induced by applying multiple intra prediction modes to reference samples.

[0811] Intra prediction can be performed on the current block to generate a first prediction block (i.e., first prediction samples), and intra prediction can be performed on the template of the current block to generate a second prediction block (i.e., second prediction samples). The template may be a region composed of reference samples around the current block.

[0812] Figure 14 shows an example of the configuration of a template.

[0813] As shown in the example illustrated in FIG. 14, a region including multiple reference sample lines (e.g., two, three, or four reference sample lines) can be set as a template. In this case, as shown in the example on the left side of FIG. 14, the template may be configured to include a region adjacent to the top-left corner of the current block, or, as shown in the example on the right side of FIG. 14, the template may be configured to include only the region adjacent to the top and the region adjacent to the left, excluding the region adjacent to the top-left corner of the current block.

[0814] Meanwhile, the reference sample used to perform intra prediction for the current block may be referred to as the first reference sample, and the reference sample used to perform intra prediction for the template may be referred to as the second reference sample.

[0815] The configuration of the template can be adaptively determined based on the width and / or height of the current block.

[0816] For example, if the width or height of the current block is greater than the first threshold value, the template can be configured to include four reference sample lines. On the other hand, if the width or height of the current block is less than or equal to the first threshold value, the template can be configured to include two reference sample lines.

[0817] Alternatively, the size of the template can be adjusted based on the width and / or height of the current block. When the size of the current block is MxN, if the width M of the current block is greater than the first threshold value, the number of left reference sample lines can be determined to be 4. Conversely, if the width M of the current block is less than or equal to the first threshold value, the number of left reference sample lines can be determined to be 2. Similarly, if the height N of the current block is greater than the first threshold value, the number of top reference sample lines can be determined to be 4. Conversely, if the height N of the current block is less than or equal to the first threshold value, the number of top reference sample lines can be determined to be 2.

[0818] Intra-prediction can be performed on the template to generate a second prediction block (i.e., second prediction samples). For example, reconstructed samples belonging to a reconstructed sample line adjacent to the template can be set as reference samples (i.e., second reference samples) for performing intra-prediction on the template.

[0819] For example, if the template consists of two reference sample lines, the reference samples belonging to the third reference sample line are set as the second reference samples, and if the template consists of four reference sample lines, the reference samples belonging to the fifth reference sample line can be set as the second reference samples.

[0820] Alternatively, if the number of top reference sample lines included in the template is 2 and the number of left reference sample lines included in the template is 4, the reference samples belonging to the third top reference sample line and the reference samples belonging to the fifth left reference sample line can be set as the second reference samples.

[0821] At least one of the MPM candidates (i.e., intra prediction modes included in the MPM list), intra prediction modes derived based on DIMD modes, planners, DCs, intra merge modes, block vector-based modes, or intra prediction modes of adjacent encoding blocks may be used for intra prediction of the template.

[0822] At this time, the intra prediction modes derived from adjacent encoding blocks can be sorted based on the distance between the current block and the adjacent encoding blocks. For example, the distance from the top-left position of the current block to the top-left position of the adjacent encoding block can be calculated, and the intra prediction modes derived from the adjacent encoding blocks can be sorted in order of shortest distance. At this time, the distance between the current block and the adjacent encoding block can be set as the sum of the absolute value of the x-coordinate difference and the absolute value of the y-coordinate difference between the two blocks.

[0823] Alternatively, intra prediction modes may be aligned based on the distance between the top-left position of the current block and the bottom-right position of an adjacent encoding block.

[0824] The angles of directional intra-prediction modes can be refined further. Additionally, wide-angle intra-prediction modes can be derived based on intra-prediction modes. Such extended angles or wide angles can be applied not only to TIMD modes but also to at least one of DIMD modes, OBIC modes, SGPM modes, or MPMB modes. Meanwhile, to ensure compatibility between blocks that do not use extended directional modes, a mapping process between extended directional modes and general directional modes can be performed.

[0825] The error cost of the intra-prediction mode can be calculated based on the difference between the reconstructed samples and the predicted samples within the template. Specifically, the error cost can be derived based on at least one of the methods SAD, SATD, or MRSAD.

[0826] A flag and an index representing one of the error cost methods can be transmitted. Alternatively, an error cost can be derived by combining one or more of the above error cost methods.

[0827] Intra prediction modes can be sorted in order of lowest error cost.

[0828] Among the sorted intra prediction modes, N intra prediction modes can be selected in order of smallest error cost. Here, N can be 1, 2, or 3.

[0829] Based on the selected intra prediction modes, intra prediction for the current block can be performed.

[0830] For example, an intra prediction can be performed on the current block based on an intra prediction mode with the minimum error cost to derive a first prediction block. Meanwhile, the intra prediction for the current block can be performed based on first reference samples.

[0831] When multiple intra prediction modes are selected, intra prediction can be performed on the current block based on each of the multiple intra prediction modes. Subsequently, the prediction block of the current block can be obtained by weighting the multiple prediction blocks.

[0832] Meanwhile, a prediction block for the current block can be derived by including a non-directional intra prediction mode. For example, if N intra prediction modes derived based on error costs are all directional intra prediction modes, an intra prediction for the current block can be performed by additionally using a non-directional intra prediction mode. Here, the non-directional mode may include at least one of DC, planar, or block vector-based prediction.

[0833] Meanwhile, at least one of the intra prediction modes of the current block can be replaced with a block vector-based prediction. Here, the intra prediction modes of the current block may include N directional intra prediction modes derived based on error costs and a predefined non-directional intra prediction mode.

[0834] For example, if the error cost for the block vector-based prediction is smaller than the error cost for the directional intra prediction mode, the directional intra prediction mode can be replaced with the block vector-based prediction.

[0835] Meanwhile, the block vector may be obtained from a merge list. In this case, at least one of the intra prediction modes of the current block may be replaced with a block vector-based prediction only if the error cost of the block vector is smaller than the error cost of all the intra prediction modes of the current block.

[0836] Alternatively, if the error cost of the block vector is smaller than the largest error cost among the intra prediction modes of the current block, the intra prediction mode with the largest error cost can be replaced with the block vector-based prediction.

[0837] When performing a weighted sum, the weights applied to each intra-prediction mode can be determined based on the error cost. For example, weights corresponding to the magnitude of the error cost may be applied to the intra-prediction mode. In this case, the fact that the weights correspond to the magnitude of the error cost may mean that a large weight is assigned to a mode with a small error cost, and a small weight is assigned to a mode with a large error cost.

[0838] Intra-prediction mode may also be derived from the top restoration area and the left restoration area of ​​the current block, respectively.

[0839]

[0840] Referring to FIG. 8, an intra prediction for the current block can be performed based on an intra prediction mode for the current block and a reference sample (S830).

[0841] For example, if the intra prediction mode is a non-directional intra prediction mode, non-directional intra prediction can be performed on the current block. Here, the non-directional intra prediction mode may be a DC or planner mode.

[0842] For example, when the intra prediction mode is DC mode, the average value of the reference samples can be set as the prediction value of the current block. In this case, the reference samples used to derive the average value can be derived from multiple reference sample lines. For example, the average value can be derived by averaging the reference samples belonging to two, three, or four reference sample lines.

[0843] Alternatively, the number of reference sample lines can be adaptively determined based on the width and / or height of the current block. For example, if the current block is a non-square block where the width is greater than the height, the number of top reference sample lines can be set greater than the number of left reference sample lines, and if the current block is a non-square block where the height is greater than the width, the number of left reference sample lines can be set greater than the number of top reference sample lines. For example, if the size of the current block is 16x4, DC prediction can be performed using reference samples belonging to the three top reference sample lines located at the top of the current block and the one left reference sample line located at the left of the current block.

[0844] When the intra prediction mode is the planner mode, a prediction value can be obtained through a weighted sum of at least one reference sample selected according to the position of the sample to be predicted within the current block. Specifically, the prediction value of the sample to be predicted can be derived by weighting the reference samples based on the distance between the reference sample and the prediction sample.

[0845] The directional intra-prediction mode may be a horizontal direction mode, a vertical direction mode, or a mode having a predetermined angle.

[0846] When the intra prediction mode is a horizontal direction mode or a vertical direction mode, the predicted value of the prediction target sample can be derived by using at least one reference sample located on a horizontal or vertical line from the prediction target sample within the current block.

[0847] When the intra prediction mode is a mode having a predetermined angle, a predicted value of a sample to be predicted can be obtained based on a reference sample on a predetermined angle line and at least one reference sample existing around said reference sample from a sample to be predicted within the current block. Consequently, the number of reference samples used to derive the predicted value of a sample to be predicted can be N. To obtain a predicted value from N reference samples, an N-tap filter may be used. Here, N can be 2, 3, 4, 5, or 6.

[0848] Different directional intra-prediction modes can be applied for each predetermined unit.

[0849] For example, a directional intra prediction mode can be determined individually for each sample line to be predicted within the current block. Accordingly, different directional intra prediction modes can be applied to each column or row within the current block.

[0850] Alternatively, the prediction target samples within the current block may be divided into multiple groups, and different directional intra-prediction modes may be applied to each group. Each group may include N prediction target samples. Alternatively, the current block may be divided into two groups based on a predetermined boundary.

[0851] After dividing the current block into multiple groups, different directional intra-prediction modes can be applied to each group.

[0852] Information regarding the boundary dividing the current block can be encoded and signaled. Blending (i.e., weighted sum) can be applied to prediction samples adjacent to the boundary. The weights used for the weighted sum can be determined based on two directional intra prediction modes.

[0853]

[0854] Block vector-based prediction can be performed on the current block. Here, block vector-based prediction may represent Intra-Block Copying (IBC) or Intra-Template Matching (IntraTMP). Meanwhile, block vector-based prediction may be included in an undirected intra prediction mode.

[0855] Information for restoring the block vector can be encoded and signaled.

[0856] In the encoder, the block most similar to the current block is searched, and the position difference between the searched block and the current block can be determined as a block vector. A search area can be set for searching for the block most similar to the current block.

[0857] In this case, the search area can be determined based on the size of the CTB. For example, the current CTB containing the current block and the top CTB adjacent to the top of the current CTB can be set as the search area.

[0858] Alternatively, you can set a search area with a fixed size. If the fixed value is 128, the search area can be set to the top 128 sample positions of the CTB containing the current block.

[0859] After creating a list containing at least one block vector candidate, you can signal by encoding an index that points to one of the block vector candidates in the list.

[0860] Candidate block vectors can be derived from the reference blocks of the current block. The reference blocks may include at least one of spatially adjacent blocks, temporally adjacent blocks, spatially non-adjacent blocks, or temporally non-adjacent blocks.

[0861] If Intra-block copy or IntraTMP is applied to a reference block, block vector candidates can be derived based on the block vector of the reference block.

[0862] When GPM or SGPM is applied to a reference block, a block vector candidate can be derived based on at least one of the intra prediction mode, inter prediction mode, segmentation information, block vector, or motion vector of the reference block.

[0863] Candidate block vectors can be derived based on the block vector and encoding parameters of a reference block. After deriving the candidate block vectors, the block vectors can be encoded / decoded through merging or error-based prediction (i.e., prediction based on Block Vector Difference (BVD)).

[0864] Block Vector Difference (BVD) can represent the difference between the block vector of the current block and the block vector prediction value, BVP (Block Vector Predictor).

[0865] Block vectors that occurred prior to the current block can be accumulated and stored, and the stored block vectors can be used as block vector candidates. Block vector candidates derived from the accumulated stored block vectors can refer to history-based block vector candidates.

[0866] A block vector candidate may include a block vector and at least one encoding parameter. The encoding parameter may include at least one of IBC_flag, IntraTMP_flag, IntraTMP index, fusion_flag, fusion index, lic_flag, filter_flag, sub-pel mode, sub-pel direction, flip_flag, GPM (Geometric Partition Mode), SGPM (Spatial Geometric partitioning mode), inter-prediction mode, motion vector, or partition information.

[0867] A block vector candidate may include information for combining two or more predictions. For example, a block vector candidate may include information for performing a weight-based bi-prediction (i.e., BCW, Bi-Prediction with CU-level Weights).

[0868] After specifying a new reference block based on the encoding parameters of the reference block, block vector candidates can be derived using the newly specified reference block.

[0869] Figure 15 shows an example in which a new reference block is specified to derive block vector candidates based on the enrichment parameters of the reference block.

[0870] A new reference block can be identified based on the encoding parameters of the initial reference block. The initial reference block may be a block spatially adjacent to the current block.

[0871] For example, as shown in the example illustrated in FIG. 15, if an initial reference block adjacent to the top of the current block has a first block vector, a first block at a position spaced apart by the first block vector from the position of the initial reference block can be identified. If a block vector exists in the first block, the block vector of the first block (i.e., the second block vector) and the encoding parameter can be set as block vector candidates.

[0872] Alternatively, instead of setting the block vector of the additionally specified block as a block vector candidate, the position difference of the block specified by the block vector of the additionally specified block from the initial reference block can be set as a block vector candidate. For example, if the block vector of the initial reference block (i.e., the first block vector) points to the first block and the block vector of the first block (i.e., the second block vector) points to the second block, the difference between the initial reference block and the second block (i.e., the third block vector) can be set as a block vector candidate.

[0873] Additional reference blocks can be searched sequentially. For example, if the first block has a block vector, the second block can be additionally identified based on the block vector of the first block (i.e., the second block vector). Subsequently, the block vector and encoding parameters of the second block can be set as block vector candidates. Alternatively, the position difference between the third block indicated by the block vector of the second block vector and the initial reference block can be set as a block vector candidate.

[0874] The number of seeks may be predefined in the encoder and decoder.

[0875] Alternatively, the search for additional blocks can be performed repeatedly until no additional specified block has a block vector.

[0876] Alternatively, one may consider whether the block specified by the block vector of the reference block exists within the search range. For example, if the block specified by the block vector of the reference block is out of the search range, additional reference blocks may not be searched further.

[0877] Whether a block vector exists within a block can be determined based on whether the block vector is stored in a sample at a predefined location within the block. For example, whether the block vector is stored can be searched for by targeting at least one of the center location sample, top-left location sample, top-right location sample, bottom-left location sample, or bottom-right location sample within the block.

[0878] In addition to the block vector of an additionally specified reference block, encoding parameters such as motion information or intra-prediction mode may also be used.

[0879] After inserting block vector candidates derived from reference blocks adjacent to the current block into the list, block vector candidates derived using additional specified reference blocks can be inserted into the list.

[0880] One or more lists may be constructed based on block vector candidates. The lists may include at least one of a merge list or a block vector prediction list.

[0881] Block vector candidates can be added to a list in a predetermined order.

[0882] Additionally, the block vector candidates included in the list can be reordered. For example, the block vector candidates can be reordered according to their error cost. The error cost of a block vector candidate can be calculated based on the template of the current block. That is, the error cost of a block vector candidate can be calculated based on the difference between the prediction samples obtained by performing a prediction based on the block vector candidate against the template of the current block and the reconstructed samples within the template.

[0883] An index indicating at least one of the block vector candidates included in the list can be encoded and signaled.

[0884] If a block vector candidate indicates that at least one of the LIC, CCCM, or IntraTMP filters is applied, LIC may be applied during block vector-based prediction for the current block.

[0885] If Intra-block copying or IntraTMP is applied to the reference block indicated by the block vector, additional reference blocks can be indicated based on the block vector of the reference block. Subsequently, the predicted block of the current block can be derived by averaging or weighting the multiple reference blocks.

[0886] Alternatively, if GPM or SGPM is applied to a reference block indicated by a block vector candidate, an additional reference block can be derived based on at least one of the intra prediction mode, inter prediction mode, segmentation information, block vector, or motion vector of the reference block. A prediction block of the current block can be derived by averaging or weighting multiple reference blocks.

[0887]

[0888] Block vectors can also be derived by performing template-based search in the encoder and decoder.

[0889] Figure 16 shows an example of deriving the block vector of the current block through template-based search.

[0890] As shown in the example illustrated in FIG. 16, one or more reference sample lines adjacent to the current block can be set as the template of the current block. The template adjacent to the current block may be referred to as the 'current template'. Subsequently, a template similar to the current template within the search area may be searched. A template that is the subject of similarity determination with the current template within the search area may be referred to as a reference template. In the embodiments described below, 'template' may refer to at least one of the current template or the reference template.

[0891] When a reference template similar to the current template is determined, the positional difference between the current template and the searched reference template can be set as the block vector of the current block. Consequently, a predicted block of the current block can be generated using a corresponding block adjacent to the reference template.

[0892] When determining the block vector of the current block through template search, signaling of information for restoring the block vector may be omitted.

[0893] Alternatively, based on template matching, a list containing multiple candidates may be constructed, and an index indicating one of the candidates included in the list may be encoded and signaled. In the decoder, the block vector of the current block can be derived using the candidate indicated by the index.

[0894] The search area can be determined based on the size of the CTB. For example, the current CTB containing the current block and the top CTB adjacent to the top of the current CTB can be set as the search area.

[0895] Alternatively, the size of the search area can be determined based on the width and / or height of the current block. For example, a search area located at the top of the current block may be determined based on the height of the current block, and a search area located to the left of the current block may be determined based on the width of the current block.

[0896] Alternatively, the search area can be set to a fixed size. For example, the maximum width or maximum height of the search area can have a fixed value, such as 64 or 128.

[0897] Alternatively, the size of the search area can be determined by combining two or more of the listed embodiments. For example, the size of the search area can be determined based on a combination of the size of the current block (i.e., width and / or height) and a fixed size. For example, the larger of 5 times the width or height of the current block and 64 can be determined as the width or height of the search area.

[0898] The search area may be divided into multiple sub-regions. The number of sub-regions to be divided may be N. Here, N may be an integer such as 3, 4, 5, or 6.

[0899] Sub-regions can be searched sequentially according to a predetermined order. For example, the search can be performed starting from the sub-region closest to the current block.

[0900] The seek area may be determined to be the same as the seek area set by the encoder to seek the block vector of the current block when applying intra-block copying.

[0901] The configuration of the template can be determined adaptively.

[0902] Figures 17 and 18 show examples of template configurations.

[0903] As shown in the example illustrated in FIG. 17, the template may have an L-shape including the left restoration area, the top restoration area, and the top-left restoration area of ​​the current block. Alternatively, the template may be configured in an L-shape excluding the top-left restoration area of ​​the current block.

[0904] Alternatively, you can configure the template using only the top restoration area of ​​the current block, or using only the left restoration area of ​​the current block.

[0905] The template may be configured to include N reference sample lines adjacent to the current block. Here, N can be a natural number such as 1, 2, 3, or 4.

[0906] The number of reference sample lines constituting the template can be adaptively determined based on the width and / or height of the current block. Table 1 shows an example where the number of reference sample lines constituting the template is adaptively determined based on the width and / or height of the current block.

[0907] W or HTemplate lineCase1Case2411822163232446444

[0908] As shown in the example in Table 1, as the width and / or height of the current block increases, the number of reference sample lines constituting the template may also increase.

[0909] The size of the top restoration area constituting the template and the size of the left restoration area may differ from each other. For example, it is assumed that the current block size is 16x4 and the template has a shape excluding the top-left restoration area from an L-shaped template. In this case, as shown in the example illustrated in FIG. 18, the top restoration area within the template may consist of two reference sample lines, while the left restoration area within the template may consist of one reference sample line.

[0910] The template configuration can be adaptively determined by comparing the width and height of the current block. For example, if the width of the current block is greater than its height, the template can be configured to include only the top restoration area. Conversely, if the height of the current block is greater than its width, the template can be configured to include only the left restoration area.

[0911] The size or shape of the template can be adaptively determined based on the values ​​of the samples included in the template and / or the corresponding block. For example, if the change in the values ​​of the samples included in the reference template and / or the corresponding block is large, the template can be configured to include one reference sample line. On the other hand, if the change in the values ​​of the samples included in the template and / or the corresponding block is small, the template can be configured to include four reference sample lines.

[0912] Template matching can be performed within a search area. Specifically, template matching can be performed for a search area and / or a sub-search area. If multiple sub-search areas exist, template matching can be performed for each sub-search area sequentially in order of proximity to the current block.

[0913] Template matching can be performed by moving by a predetermined sample unit. Here, the predetermined sample unit can be a natural number such as 3, 4, 5, or 6.

[0914] Template matching may involve calculating the error cost between the current template and the reference template. The error cost may be derived based on at least one of SAD, SATD, or MRSAD.

[0915] Alternatively, error costs can be calculated based on the variation of samples within the template or the trend of the gradient.

[0916] In calculating the error cost, the amount of variation in sample values ​​between the template and the corresponding block may be additionally utilized. For example, the difference in sample values ​​at the boundary between the corresponding block and the template may be reflected. That is, if the sample values ​​between the corresponding block and the template are similar, the error cost will have a low value. On the other hand, if the difference in sample values ​​between the corresponding block and the template is large, the error cost will have a high value. Here, the template may represent the current template or the reference template.

[0917] Figure 19 shows an example of calculating the error cost of a reference template based on the amount of change in sample values.

[0918] Based on at least one of the following mathematical formulas 10 to 14, the amount of change in sample values ​​at the boundary between the template and the corresponding block can be measured.

[0919]

[0920]

[0921]

[0922]

[0923]

[0924] In the above mathematical formulas 10 to 14, abs(A) represents the absolute value of A. REF_TEM1(x) represents a sample at position x belonging to the first reference sample line within the reference template adjacent to the corresponding block, and REF_TEM2(x) represents a sample at position x belonging to the second reference sample line within the reference template.

[0925] CUR_TEM1(x) represents the sample at position x belonging to the first reference sample line within the current template, and CUR_TEM2(x) represents the sample at position x belonging to the second reference sample line within the current template.

[0926] PRED1(x) represents a sample at position x belonging to the first row from the top boundary within the corresponding block, and PRED2(x) represents a sample at position x belonging to the second row from the top boundary within the corresponding block.

[0927] Shift and round can be variables for division and rounding, respectively. Each variable can be set to a predetermined non-zero value or 0.

[0928] Based on at least one of mathematical formulas 10 to 14, the amount of sample value change at the upper boundary of the corresponding block can be calculated, and the calculated amount of sample value change can be reflected in the error cost.

[0929] In the example illustrated in FIG. 19, it was assumed that the template is composed only of the top restoration area, but the described embodiment can also be applied even if the configuration of the template is different from FIG. 19.

[0930] For example, if the template consists only of the left restoration region, the amount of sample value change at the left boundary of the corresponding block can be calculated, and the calculated amount of sample value change can be reflected in the error cost.

[0931] The variables shift and round may be determined based on at least one of the width or height of the current block. For example, shift may be a value obtained by taking Log2 of the width or height value of the current block (i.e., log2(W) and log2(H)). When calculating the amount of change at the top boundary of the corresponding block, the variable shift may be derived using the width (or height) of the current block, and when calculating the amount of change at the left boundary of the corresponding block, the variable shift may be derived using the height (or width) of the current block.

[0932] round can be the value of shift divided by 2.

[0933] Alternatively, the variable shift can be derived based on the number of reference sample lines constituting the template. For example, if the number of reference sample lines constituting the template is 2, the variable shift can be determined as 2, and if the number of reference sample lines constituting the template is 4, the variable shift can be determined as 4.

[0934] The value derived based on the average value of the samples included in the template and the average value of the samples included in the corresponding block can also be used when calculating the error cost.

[0935] Whether to reflect the change in sample values ​​(or, the average of the samples) at the top boundary of the corresponding block when calculating the error cost can be determined based on the width and / or height of the current block. For example, if the width and / or height of the current block is greater than 4, the change in sample values ​​(or, the average of the samples) at the top boundary of the corresponding block can be reflected when calculating the error cost.

[0936] In calculating the error cost of a reference template, the distance from the current block may be reflected. Specifically, the distance between the current block and the corresponding block may be reflected in the error cost of the reference template.

[0937] In calculating the error cost of a reference template, a high weight may be applied to samples located at block boundaries. For example, a weight equal to the variable shift may be applied to the SAD values ​​of samples located at the template boundaries. Here, the template boundary may be tangent to the top or left boundary of the current block or the corresponding block.

[0938] For example, the error between the current template and the reference template for samples located at the block and template boundary can be derived by the following mathematical formula 15.

[0939]

[0940] The value of the variable shift can be 0, 1, 2, 3, or 4. The value of the variable shift can be determined based on the number of reference sample lines constituting the template. For example, the variable shift can be derived by differing 1 from the number of reference sample lines constituting the template. In this case, the weight shift applied to samples touching the upper boundary of the current block or corresponding block can be set to the value obtained by differing 1 from the number of reference sample lines constituting the upper restoration area of ​​the template, and the weight shift applied to samples touching the left boundary of the current block or corresponding block can be set to the value obtained by differing 1 from the number of reference sample lines constituting the left restoration area of ​​the template.

[0941] Alternatively, the weight shift can be set to a different value for each reference sample line constituting the template.

[0942] A candidate list can be derived based on the error cost. Specifically, by moving the search area in predetermined units, a first list containing N candidates in order of lowest error cost can be derived. Each candidate may have a block vector representing the positional difference between the current template and the reference template.

[0943] Candidates can be generated by varying the template shape. For example, 30 candidates can be generated based on an L-shaped template, 6 candidates based on a template containing only the top restoration area, and 6 candidates based on a template containing only the left restoration area.

[0944] You can determine whether to add a candidate to the list by comparing the candidate's error cost with a threshold value. For example, if the candidate's error cost is greater than the threshold value, the candidate is not added to the list; otherwise, the candidate can be added to the list.

[0945] Meanwhile, a unit may be set for comparing the candidate's error cost and threshold value. For example, the sum of the errors of each sample within the template may be compared with the threshold value. Alternatively, the sum of the errors of each line (e.g., each row or each column) within the template may be compared with the threshold value. Or, if the template consists of a single line, the sum of the errors of four consecutive samples in the vertical direction among the samples belonging to the left restoration area within the template may be compared with the threshold value.

[0946] The threshold value may be determined based on at least one of the bit depth or the number of samples within the template. For example, (1< <bit_depth)에 템플릿 내 샘플들의 개수를 곱한 값을 문턱값으로 설정할 수 있다. 비트 뎁스(bit_depth)가 10이고, 템플릿이 포함하는 샘플들의 개수가 32인 경우, 문턱값은 32768 (1024*32)로 설정될 수 있다.

[0947] Merge candidates can be derived, and error costs for the merge candidates can be calculated. Merge candidates can be derived from the reference blocks of the current block. The reference blocks may include at least one of spatially adjacent blocks, temporally adjacent blocks, spatially non-adjacent blocks, or temporally non-adjacent blocks.

[0948] A merge candidate may include at least one of a block vector, an IntraTMP index, a fusion_flag, a fusion index, a lic_flag, a filter_flag, a sub-pel mode, or a sub-pel direction.

[0949] If the error cost of a merge candidate satisfies a predetermined condition, the merge candidate can be inserted into the list. Here, the predetermined condition may include at least one of the following: the error cost of the merge candidate is smaller than a threshold value or the error cost is smaller than that of the candidate with the maximum error cost among the candidates included in the list.

[0950] Meanwhile, when inserting a merge candidate into a list, a duplicate check with candidates already inserted into the list may be performed. That is, whether to insert the merge candidate into the list may be determined based on whether a candidate with the same value as the merge candidate's block vector exists. If a candidate with the same value as the merge candidate's block vector exists, the merge candidate may not be added to the list.

[0951] New block vector candidates can be derived based on the candidates included in the list. Specifically, new candidates can be derived by correcting the block vectors of the candidates included in the list or by exploring the sub-search area indicated by the candidates. The newly derived candidates can be inserted into a new list. For convenience of explanation, the list containing the existing candidates will be referred to as the first list, and the list to which the newly derived candidates are added will be referred to as the second list.

[0952] Error costs can be calculated for samples belonging to a sub-search area specified within the range (±x, ±y) centered on the location indicated by the block vector of a candidate included in the first list. If the error cost calculated at a specific location is smaller than the error cost of a candidate included in the first list, the block vector pointing to that location can be added to the second list.

[0953] For example, if a candidate is derived by performing template matching in units of 4 samples, an area of ​​(±4, ±4) based on the location indicated by the candidate inserted in the first list can be designated as a sub-search area, and an error cost can be calculated for the samples belonging to the sub-search area. If the error cost calculated at a specific location is smaller than the error cost of the candidate inserted in the first list, the block vector indicating that location can be inserted into the second list.

[0954] The second list may contain N candidates. The number of candidates included in the second list may differ from the number of candidates included in the first list. For example, the first list may contain 30 candidates, while the second list may contain 19 candidates.

[0955] Depending on the form of the template, the positions of the candidates added to the second list may be predefined. For example, candidates derived using an L-shaped template may be inserted into the second list first. That is, for the M candidates derived using an L-shaped template, indices 0 to (M-1) may be assigned.

[0956] Candidates derived using a template consisting only of the top restoration area or a template consisting only of the left restoration area may be assigned an index of 13 or higher. That is, after inserting candidates derived using an L-shaped template, candidates derived using a template consisting only of the top restoration area or a template consisting only of the left restoration area can be inserted.

[0957] When adding a candidate to a list, duplicates with a candidate already inserted in the list can be checked. Whether two candidates are duplicates can be determined based on a block vector and at least one encoding parameter. If a candidate to be inserted duplicates a candidate already inserted in the list, that candidate may not be inserted into the list. Additionally, the candidate that duplicates the candidate to be inserted may be moved to a higher position within the list. That is, the index value assigned to the candidate that duplicates the candidate to be inserted may be reduced.

[0958] Candidates can be inserted into a list in order of error cost. In this case, if there are multiple candidates with the same error cost, the insertion order or sorting order of the candidates can be determined according to a specified condition.

[0959] For example, among two candidates with the same error cost, the candidate with the smaller block vector size can be inserted into the list first.

[0960] If certain conditions are met, candidates may no longer be inserted into the list.

[0961] There may be multiple lists. Specifically, multiple lists can be generated by varying the composition of the template. For example, a first list can be derived based on a template composed of four reference sample lines, and a second list can be derived based on a template composed of one reference sample line.

[0962]

[0963] Whether a block vector can be used as a candidate can be determined by considering whether the location indicated by the block vector exists outside the search area. For example, if the location indicated by the block vector exists outside the search area, the block vector may be set as unavailable.

[0964] Alternatively, if the location indicated by the block vector exists outside the search area, the corrected block vector derived by clipping the block vector can be used as a candidate. The corrected block vector indicates a location inside the search area.

[0965] You can create a list using the candidates.

[0966] For example, a list can be generated using at least one of a merge candidate derived from the reference block of the current block, a block vector difference candidate, a candidate derived using a template, or a history-based candidate.

[0967] Alternatively, a LIC-based merge list can be constructed. Candidates using filter-based encoding parameters can be added to the LIC-based merge list.

[0968] Based on the list, the block vector of the current block can be derived. Once the block vector of the current block is derived, the predicted block of the current block can be derived using the corresponding block indicated by the block vector.

[0969] For example, the corresponding block can be set as the predicted block of the current block.

[0970] Alternatively, at least one of LMCS, a deblocking filter, SAO (Sample Adaptive Offset), ALF (Adaptive Loop Filter), a Low-pass filter, or PDPC can be applied to the corresponding block and set as the prediction block of the current block.

[0971] Alternatively, a template-based correction performed on a corresponding block can be set as the prediction block of the current block. For example, at least one adjacent reference sample line of a corresponding block indicated by a merge candidate can be determined as a template. In this case, the number of reference samples constituting the template may be a natural number such as 1, 2, 3, or 4. The template of the corresponding block may include a right restoration area and / or a bottom restoration area of ​​the corresponding block.

[0972] The error cost between the template adjacent to the current block (i.e., the current template) and the template adjacent to the corresponding block (i.e., the reference template) can be calculated. The error cost can be derived based on at least one of SAD, SATD, or MRSAD.

[0973] Based on the error cost, the position of the corresponding block (i.e., the block vector) can be corrected. The corrected position of the corresponding block may be a position moved by N samples from the initial position of the corresponding block. For example, the error cost can be calculated at positions that are 1 sample away from the initial position of the corresponding block in at least one of the upward, downward, left, or right directions. If the cost calculated at a position different from the initial position is smaller than the error cost at the initial position, the position of the corresponding block can be corrected to the position where the error cost is minimized. For example, if the top-left position of the corresponding block is (x, y), the corrected top-left position of the corresponding block may be (x-1, y-1), (x, y-1), (x+1, y-1), (x-1, y), (x+1, y), (x-1, y+1), (x, y+1), or (x+1, y+1).

[0974] The above correction can be performed on the decoder side in the same way as on the encoder. Accordingly, the signaling of information indicating the corrected position of the corresponding block (i.e., the corrected block vector) can be omitted.

[0975] By using the corresponding block of the corrected position, the predicted block of the current block can be derived.

[0976] By applying Local Illumination Compensation (LIC) to the corresponding block, the predicted block of the current block can be derived. At this time, the LIC parameters can be derived by analyzing the association between reference samples adjacent to the current block and reference samples adjacent to the corresponding block. Alternatively, the LIC parameters can be derived from merge candidates.

[0977] By applying LIC parameters to the corresponding block, the predicted block of the current block can be derived. When LIC is applied, the flag lic_flag, which indicates whether LIC is applied, can be encoded and signaled to have a value of 1.

[0978] Alternatively, the application of LIC can be determined based on reference samples without encoding / decoding a flag indicating whether to apply LIC. For example, the error cost between the reference samples of the current block and the reference samples of the corresponding block can be calculated, and the error cost can be compared with a threshold value to determine whether to apply LIC.

[0979] Whether to apply LIC or at least one of the LIC parameters can be determined by referring to an adjacent block. For example, if LIC parameters are stored in an adjacent block, LIC can be applied to the corresponding block based on the LIC parameters of the adjacent block.

[0980] Meanwhile, when the restoration of a block is completed, LIC parameters can be derived based on the restored block and the predicted block. Subsequently, the derived LIC parameters can be stored at the location of the block. Accordingly, LIC parameters may also be stored in adjacent blocks of the current block.

[0981] LIC parameters stored in adjacent blocks may be included in the merge candidates.

[0982] The prediction block of the current block may be derived by modifying the corresponding block. For example, the prediction block of the current block may be set by rotating the corresponding block by a predetermined angle or flipping it in a horizontal or vertical direction. The predetermined angle may be 90 degrees, 180 degrees, or 270 degrees.

[0983] Flipping the corresponding block in a vertical or horizontal direction may involve flipping the corresponding block vertically or horizontally based on the center point of the corresponding block.

[0984] Alternatively, a corresponding block flipped along a diagonal may be set as the prediction block of the current block. Here, the diagonal may be a line in the upper right direction connecting the upper right vertex and the lower left vertex of the corresponding block, or a line in the upper left direction connecting the upper left vertex and the lower right vertex of the corresponding block.

[0985] The flipping direction can be determined based on the error cost of the template.

[0986]

[0987] By applying a filter to the corresponding block, the predicted block of the current block can be derived.

[0988] Filter coefficients or parameters can be derived using a template adjacent to the current block (i.e., the current template) and a template adjacent to the corresponding block (i.e., the reference template). For example, convolution filter coefficients can be derived such that the Mean Square Error (MSE) between the current template and the reference template is minimized. The filter shape may be a cross shape (e.g., center sample, top sample, left sample, bottom sample, and right sample). The filter coefficients may include five coefficients applied to the five samples included in the shape, and two coefficients for linear and / or bias.

[0989] When a convolution filter is applied, the value of the flag filter_flag, which indicates whether the filter is applied, can be set to 1 to signal. In the decoder, a convolution filter can be applied to the corresponding block based on filter coefficients and / or parameters derived based on the current template and the reference template.

[0990] In deriving filter coefficients and / or parameters, reference samples included in the template may be used. For example, filter coefficients can be derived based on reference samples excluding the top-left restoration region in an L-shaped template.

[0991] Alternatively, an interpolation filter can be applied to the corresponding block to derive a prediction block corresponding to sub-pixel precision.

[0992] The sub-pixel precision can be 1 / 2, 1 / 4, or 3 / 4. The interpolation filter can have a length of 4-tab or 6-tab.

[0993] The sub-pixel may be located in the upper-left direction, upper direction, upper-right direction, left direction, right direction, lower-left direction, lower direction, or lower-right direction based on integer position samples within the corresponding block.

[0994] At least one of information regarding sub-pixel precision or information regarding the orientation in which the sub-pixel is located can be encoded and signaled.

[0995] Alternatively, at least one encoding / decoding of the information regarding sub-pixel precision or the direction in which the sub-pixel is located may be omitted, and the above information may be derived based on a template. For example, an interpolation filter may be applied to a template adjacent to a corresponding block (i.e., a reference template), and then the error cost between the reference template with the applied interpolation filter and the template adjacent to the current block (i.e., the current template) may be calculated. By performing the above procedure for each sub-pixel precision, the sub-pixel precision or the direction in which the sub-pixel is located that minimizes the error cost can be determined.

[0996] Information regarding subpixel precision and information regarding the direction in which the subpixel is located may not be signaled at all, or only one of them may not be signaled. For example, information regarding subpixel precision may be encoded and signaled, while information regarding the direction in which the subpixel is located may be derived in the decoder based on the current template and the reference template. Specifically, the decoder may determine the position where the error cost is minimized by adjusting the position of the reference template within the range of (±1, ±1). The direction from the initial reference template to the position where the error cost is minimized can be determined as the direction in which the subpixel is located.

[0997] Alternatively, depending on the sub-pixel precision, at least one reference template composed of sub-pixel location samples can be derived, and then the error cost of each of the reference template composed of integer location samples and the at least one reference template composed of sub-pixel location samples can be calculated. Subsequently, information indicating one of the top N with the lowest error cost can be encoded and signaled. By applying an interpolation filter to the corresponding block according to the sub-pixel precision of the selected reference template, the prediction block of the current block can be derived.

[0998]

[0999] The predicted block of the current block can be derived by performing a weighted sum of multiple corresponding blocks. For example, N candidates can be selected from a list, and N corresponding blocks can be derived based on each of the N candidates. Subsequently, the predicted block of the current block can be derived by performing a weighted sum of the N corresponding blocks.

[1000] Candidates included in a list can be divided into N groups, and different indices can be assigned to each group. For example, if one group contains 5 candidates, the group with index 0 may contain candidates from index 0 to 4, the group with index 1 may contain candidates from index 5 to 9, and the group with index 3 may contain candidates from index 10 to 14.

[1001] An index specifying one of multiple groups can be encoded and signaled. When one of the multiple groups is selected, a corresponding block can be derived based on each candidate included in the selected group. For example, if the group with an index of 3 is selected, five corresponding blocks can be derived based on candidates with indices from 10 to 14. Subsequently, the predicted block of the current block can be derived by weighting the five corresponding blocks.

[1002] Alternatively, an error cost can be calculated for each candidate included in the list, and N candidates can be selected based on the error cost. The error cost can be calculated based on the template adjacent to the current block.

[1003] A template may consist of one reference sample line adjacent to the current block. Based on the error between the current template and the reference template indicated by the candidate, the candidate's error cost can be calculated.

[1004] Once the error cost of each candidate is calculated, N candidates can be selected in order of lowest error cost.

[1005] Among multiple candidates, only some candidates that satisfy certain conditions may be re-selected. For example, when N candidates are selected, a candidate with an error cost greater than twice the error cost of the candidate with the smallest error cost among the N candidates may be excluded. That is, after re-selecting M candidates from among the N candidates, M corresponding blocks can be derived using only the re-selected M candidates. Subsequently, the predicted block of the current block can be derived by performing a weighted sum of the M corresponding blocks.

[1006] In a weighted sum of corresponding blocks, the weights applied to the weighted sum can be determined based on the error costs of the candidates. That is, the weight assigned to a corresponding block derived from a candidate with a small error cost can have a larger value than the weight assigned to a corresponding block derived from a candidate with a large error cost. Accordingly, the encoding / decoding of information for the weighted sum (e.g., at least one of an index for determining weights or information indicating weight types) can be omitted.

[1007] Error cost can be determined based on a template. In this case, the template may be a first template consisting of four reference sample lines adjacent to the current block or a second template consisting of one reference sample line.

[1008] A weight can be determined based on one of a plurality of error cost derivation methods. For example, one of a first weight obtained based on an error cost calculated using a first template or a second weight obtained based on an error cost calculated using a second template can be selected. The selection of one of the first weight and the second weight may be based on the amount of change in error cost between N intra prediction modes.

[1009] For example, for each of N intra-prediction modes, an error cost is calculated based on a first template. After sorting the N intra-prediction modes according to their error costs, an error cost change amount based on the first template is calculated. Here, the error cost change amount can represent the error cost difference between the intra-prediction modes. That is, when the intra-prediction modes are sorted according to their error costs, a large difference between the error costs indicates a large error cost change amount, and a small difference between the error costs indicates a small error cost change amount.

[1010] In addition, for each of the N intra prediction modes, an error cost is calculated based on the second template. After sorting the N intra prediction modes according to their error costs, the change in error cost based on the second template is calculated.

[1011] Among the error costs based on the first template and the second template, the weight obtained based on the template with a relatively larger change in error cost can be selected.

[1012] Alternatively, a weight combining the first weight and the second weight may be used for the weighted sum.

[1013] If the prediction block of the current block is derived by a weighted sum of multiple corresponding blocks, at least one of the block vectors used to derive the multiple corresponding blocks can be stored at the current block location. For example, the block vector of the candidate with the smallest error cost among the multiple candidates used to derive the multiple corresponding blocks can be stored at the current block location. Alternatively, the block vectors of multiple candidates selected in order of lowest error cost (e.g., two candidates) can be stored at the current block location. Alternatively, a block vector formed by combining multiple candidates can be stored at the current block location.

[1014] After applying LIC to corresponding blocks, the corresponding blocks to which LIC has been applied can also be used in the weighted sum.

[1015] Alternatively, a weighted sum of reference templates adjacent to corresponding blocks can be derived, and then LIC parameters can be derived based on the weighted sum of the reference template and the current template. Afterward, the predicted block of the current block can be derived by weighting the corresponding blocks and applying the LIC parameters to the weighted sum result.

[1016] Predicted blocks may also be derived by combining one or more of the methods described above.

[1017] For example, a first prediction block of the current block can be obtained by applying LIC, and a second prediction block of the current block can be obtained by applying a filter. Then, the first prediction block and the second prediction block can be weighted to derive the final prediction block of the current block.

[1018]

[1019] When block vector-based prediction is applied to the current block, a predefined intra prediction mode may be stored at the current block location. Here, the predefined intra prediction mode may be a non-directional mode (e.g., DC or planar).

[1020] Alternatively, the intra prediction mode stored in the corresponding block indicated by the block vector can be stored at the current block location.

[1021] Alternatively, the intra prediction mode of a corresponding block or a block adjacent to the current block can be stored at the current block location.

[1022] Alternatively, an intra prediction mode derived by applying a DIMD mode to the prediction block of the current block (i.e., the corresponding block) can be stored at the current block location. That is, an intra prediction mode having the largest amplitude value on the gradient histogram derived based on the prediction block of the current block can be stored at the current block location.

[1023] The intra prediction mode stored at the current block location can be used when constructing the MPM list of adjacent blocks to be encoded / decoded later.

[1024] After deriving the intra prediction mode of the current block, the prediction block can be corrected based on the derived intra prediction mode. Specifically, the prediction block can be corrected by applying PDPC (Position Dependent Prediction Combination) based on the derived intra prediction mode.

[1025] To apply block vector-based prediction to the current block, at least one syntax can be encoded and signaled. For example, a flag indicating whether block vector-based intra prediction (BVIP) is performed can be encoded and signaled. If the flag is 1, information for determining at least one of whether the block vector is derived based on merge mode, derived using block vector difference, or derived based on template matching can be additionally encoded and signaled.

[1026] For example, if the flag indicating whether to perform block vector-based intra prediction is 1, the merge flag may be encoded and signaled. A merge flag of 1 indicates that the block vector is derived based on the merge mode.

[1027] When the merge flag is 0, the template matching flag may be encoded and signaled. A template matching flag of 1 indicates that the block vector is derived based on template matching. A template matching flag of 0 indicates that the block vector is derived using the block vector difference. In this case, information for decoding the block vector difference may be additionally encoded and signaled.

[1028] Alternatively, if the flag indicating whether to perform block vector-based intra prediction is 1, information indicating whether to apply either the intra-block copy (IBC) mode or the IntraTMP mode may be encoded. For example, if the flag indicating whether to perform block vector-based intra prediction is 1, at least one of the IBC_flag indicating whether to apply the IBC mode or the IntraTMP_flag indicating whether to apply IntraTMP may be additionally encoded and signaled.

[1029] An IBC_flag of 1 indicates that IBC mode is applied to the current block, and an IBC_flag of 0 indicates that IntraTMP mode is applied to the current block.

[1030] IntraTMP_flag being 1 indicates that IntraTMP mode is applied to the current block, and IntraTMP_flag being 0 indicates that IBC mode is applied to the current block.

[1031] If only IBC mode is applicable to the current block or only IntraTMP mode is applicable, encoding / decoding of IBC_flag or IntraTMP_flag may be omitted.

[1032] The encoding / decoding of the flag indicating whether block vector-based intra prediction is performed can be omitted, and IBC_flag or IntraTMP_flag can be encoded and signaled. That is, through IBC_flag or IntraTMP_flag, it is possible to determine whether block vector-based intra prediction is performed and the block vector-based mode (i.e., IBC mode or IntraTMP mode).

[1033] In this case, if only the IBC mode is applicable to the current block, the IBC_flag can be encoded / decoded to determine whether block vector-based intra prediction is performed on the current block. On the other hand, if only the IntraTMP mode is applicable to the current block, the IntraTMP_flag can be encoded / decoded to determine whether block vector-based intra prediction is performed on the current block. If both the IBC mode and the IntraTMP mode are applicable to the current block, the IBC_flag and IntraTMP_flag can be encoded and signaled, respectively.

[1034]

[1035] Intra prediction for the current block can be performed based on a matrix. For example, a predicted block of the current block can be generated by applying a matrix filter, determined based on the size of the current block, to reference samples.

[1036] Whether to perform matrix-based intra prediction may be determined based on at least one of the current block size, the availability of reference samples, the intra prediction mode, or the index of the intra prediction mode. For example, if the intra prediction mode of the current block is a directional prediction mode and has an odd or even index, matrix-based intra prediction (MIP or PDP) may be performed.

[1037] In performing matrix-based intra-prediction, a matrix derived by weighted summing multiple matrices may be used. For example, when performing matrix-based intra-prediction based on a predetermined directional prediction mode, matrices corresponding to directional prediction modes adjacent to the directional prediction mode may be weighted summed. For example, when the index of a directional prediction mode is N, a matrix of a directional prediction mode with index N can be derived by weighted summing the matrix of a directional prediction mode with index (N-1) and the matrix of a directional prediction mode with index (N+1).

[1038]

[1039] Based on filters, intra prediction for the current block can be performed.

[1040] For example, after deriving the coefficients of an extrapolation filter from the restored reference samples, the extrapolation filter can be applied to reference samples adjacent to the current block to generate predicted samples. Here, the restored reference samples may be those adjacent to the current block.

[1041] Alternatively, the reconstructed samples belonging to the reference region indicated by the block vector can be set as reference samples for deriving the coefficients of the extrapolation filter. Specifically, the reconstructed samples included in the corresponding block indicated by the block vector and the template of the corresponding block can be set as reference samples for deriving the coefficients of the extrapolation filter.

[1042] Deriving the coefficients of an extrapolation filter based on reconstructed samples belonging to the reference region indicated by the block vector may be a submode of prediction based on the extrapolation filter. That is, if it is determined that prediction based on the extrapolation filter is performed on the current block, a flag indicating whether the coefficients of the extrapolation filter are derived based on reconstructed samples belonging to the reference region indicated by the block vector may be additionally encoded / decoded.

[1043] Depending on the above flag, extrapolation filter coefficients can be derived based on reference samples adjacent to the current block, or based on reference samples within the reference region indicated by the block vector.

[1044] Alternatively, if the current block is a chroma component, filter coefficients can be derived based on the restored luminance signal and the restored color difference signal, and the derived filter coefficients can be applied to the restored luminance block (i.e., the collocated luminance block) to derive the predicted block of the current block.

[1045] The filter may include at least one of a linear filter, a convolutional filter, or a gradient filter.

[1046] In the process of performing intra prediction of the current block, a filtering process using reconstructed samples around the current block may be performed. At this time, whether to perform the filtering process may be determined based on at least one of the intra prediction mode of ...

Claims

1. A step of determining whether combined intra prediction is applied to the current block; When the combined intra prediction is applied to the current block, a step of inducing a plurality of intra prediction modes for the current block; A step of deriving reference samples of the current block above; and The method includes the step of obtaining a plurality of prediction blocks for the current block based on the plurality of intra prediction modes and the reference samples, wherein An image decoding method characterized by obtaining a final prediction block of the current block by weighting the plurality of prediction blocks.

2. In Paragraph 1, An image decoding method characterized in that the combined intra-mode prediction includes at least one of DIMD (Decoder-side Intra-mode Derivation) or TIMD (Template-based Intra-mode Derivation).

3. In Paragraph 1, An image decoding method characterized in that, when the combined intra prediction is applied to the current block, filtering for the reference samples is omitted.

4. In Paragraph 1, A video decoding method characterized in that the above-mentioned plurality of intra prediction modes are MPM candidates included in the MPM (Most Probable Mode) list.

5. In Paragraph 1, An image decoding method characterized in that the plurality of intra prediction modes are selected from the histogram of the current block.

6. In Paragraph 5, An image decoding method characterized by selecting the plurality of intra prediction modes in order of increasing amplitude value or frequency of occurrence on the above histogram.

7. In Paragraph 5, Calculate the error cost for each of the intra prediction modes included in the above histogram, and An image decoding method characterized by selecting a plurality of intra-prediction modes from the histogram in order of decreasing error cost.

8. In Paragraph 5, An image decoding method characterized in that the above histogram is generated using an intra-prediction mode of the spatial reference block of the above current block.

9. In Paragraph 8, A video decoding method characterized by the fact that when multiple intra prediction modes exist in the spatial reference block, a value with weights applied to the occurrence frequency of each of the multiple intra prediction modes of the spatial reference block is accumulated in the histogram.

10. In Paragraph 8, The values ​​to which weights are applied to the occurrence frequency of the intra prediction mode of the above spatial reference block are accumulated in the above histogram, and An image decoding method characterized in that the above weight is determined according to the distance between the spatial reference block and the current block.

11. In Paragraph 1, An image decoding method characterized in that when the combined intra prediction is applied to the current block, only a predefined single interpolation filter is available for generating the plurality of prediction blocks.

12. In Paragraph 1, When the above combined intra prediction is applied to the above current block, A video decoding method characterized in that, even if the current block is capable of performing matrix-based intra prediction, none of the plurality of intra prediction modes is switched to a matrix-based intra prediction mode.

13. In Paragraph 1, A video decoding method characterized in that, when the combined intra prediction is applied to the current block, filtering for each of the plurality of prediction blocks is omitted.

14. A step of determining whether combined intra prediction is applied to the current block; When the combined intra prediction is applied to the current block, a step of inducing a plurality of intra prediction modes for the current block; A step of deriving reference samples of the current block above; and The method includes the step of obtaining a plurality of prediction blocks for the current block based on the plurality of intra prediction modes and the reference samples, wherein An image encoding method characterized by obtaining a final prediction block of the current block by weighting the plurality of prediction blocks.

15. A step of determining whether combined intra prediction is applied to the current block; When the combined intra prediction is applied to the current block, a step of inducing a plurality of intra prediction modes for the current block; A step of deriving reference samples of the current block above; and The method includes the step of obtaining a plurality of prediction blocks for the current block based on the plurality of intra prediction modes and the reference samples, wherein A computer-readable recording medium storing a bitstream generated by an image encoding method, characterized in that the final prediction block of the current block is obtained by weighting the plurality of prediction blocks.

Images