Image encoding / decoding method and apparatus using mapping model adjustment-offset, and bitstream storage medium

Abstract

The image encoding / decoding method and the method for transmitting a bitstream according to the disclosure can include the steps of generating a candidate list of a current block, rearranging the candidate list based on a matching cost, obtaining prediction information about the current block from the rearranged candidate list, and predicting the current block based on the prediction information about the current block, wherein the candidate list can include at least one of a spatial candidate, a temporal candidate, a history-based candidate, or a default candidate of the current block.

Description

Technical Field

[0001] This disclosure relates to a method and apparatus for encoding / decoding images, and more specifically, to a method and apparatus for encoding / decoding images including a weighted sum of prediction blocks. Background Technology

[0002] Recently, the demand for high-resolution and high-quality images, such as HD (High Definition) and UHD (Ultra High Definition) images, has increased across various application areas. As image data becomes higher resolution and higher quality, the data volume increases relatively compared to existing image data, thus increasing transmission and storage costs when transmitting or storing image data using existing wired and wireless broadband circuits or storage media. Efficient image encoding / decoding technologies are needed for images with higher resolution and image quality to address these problems arising from the increasing resolution and quality of image data.

[0003] Various techniques exist, such as inter-frame prediction techniques that use image compression technology to predict the pixel values ​​included in the current frame from previous or subsequent frames, intra-frame prediction techniques that use pixel information in the current frame to predict the pixel values ​​included in the current frame, energy transformation and quantization techniques for compressing residual signals, and entropy coding techniques that assign short codewords to values ​​with high frequency and long codewords to values ​​with low frequency. These image compression techniques can be used to effectively compress, transmit, or store image data. Summary of the Invention

[0004] Technical issues In traditional image encoding / decoding methods and devices, there is a limitation on improving encoding efficiency because of the redundancy of information between luminance component samples and chrominance component samples, as well as between samples in the neighboring blocks and samples within the target block.

[0005] Therefore, this disclosure can provide a method and apparatus for improving image encoding / decoding efficiency by utilizing information redundancy between samples through model-based prediction to reduce the number of bits transmitted in the signal.

[0006] Technical solution An embodiment of this disclosure may be an image encoding method and apparatus, characterized by configuring a sample set and deriving a mapping model; applying the mapping model to a target block; and performing entropy encoding / entropy decoding on the encoded information.

[0007] The image encoding / decoding method and the method for transmitting a bit stream disclosed herein include: generating a candidate list for the current block; reordering the candidate list based on a matching cost; obtaining prediction information for the current block from the reordered candidate list; and performing a prediction for the current block based on the prediction information for the current block, wherein the candidate list may include at least one of spatial candidates, temporal candidates, history-based candidates, and default candidates for the current block.

[0008] In the image encoding / decoding method and the method for transmitting bit stream disclosed herein, the spatial candidate may include adjacent spatial candidates that are adjacent to the current block and non-adjacent spatial candidates that are not adjacent to the current block.

[0009] In the image encoding / decoding method and the method for transmitting bit stream disclosed herein, the non-adjacent space candidates can be determined based on the width and height of the current block.

[0010] In the image encoding / decoding method and the method for transmitting bit stream disclosed herein, the non-adjacent spatial candidate can be a block that is separated from a specific location by N times the unit distance in the horizontal or vertical direction.

[0011] In the image encoding / decoding method and the method for transmitting bit stream disclosed herein, the time candidate can be obtained from the block at the shift position of the time block of the current block.

[0012] In the image encoding / decoding method and the method for transmitting bit stream disclosed herein, the time block can be determined by the motion vector of the spatial neighboring block of the current block.

[0013] In the image encoding / decoding method and the method for transmitting bit stream disclosed herein, the history-based candidates can be obtained by using a table stored by accumulating prediction information prior to the current block.

[0014] In the image encoding / decoding method and the method for transmitting bit stream disclosed herein, the prediction of the current block can be performed by performing a weighted sum of a first prediction block obtained based on a first mapping model and a second prediction block obtained based on a second mapping model.

[0015] In the image encoding / decoding method and the method for transmitting bit stream disclosed herein, a first mapping model may be included in the prediction information of the current block obtained from the candidate list of the current block.

[0016] Technical effect A method and apparatus may be provided for reducing signaling bits to improve image encoding / decoding efficiency by using the redundancy of information between samples through model-based prediction. Attached Figure Description

[0017] Figure 1A system for video encoding according to an embodiment is shown.

[0018] Figure 2 An image partitioning structure according to an embodiment is shown.

[0019] Figure 3 The structure of intra-frame prediction according to an embodiment is shown.

[0020] Figure 4 The structure for explaining the inter-frame prediction process according to the embodiment is shown.

[0021] Figure 5 The order in which spatial candidates are added to the candidate list is shown according to an embodiment.

[0022] Figure 6 Several loop filters are shown based on the example.

[0023] Figure 7 The structure of entropy encoding and entropy decoding is shown based on the example.

[0024] Figure 8 This is an example illustrating the partition boundaries in a geometric partitioning pattern.

[0025] Figure 9 It is a diagram used to describe the partition boundaries, partition offsets, and partition angles in a geometric partitioning mode.

[0026] Figure 10 This is an example illustrating a weighted graph for each prediction block based on specific partition boundaries.

[0027] Figure 11 This is an illustration of an embodiment used for template matching.

[0028] Figure 12 Various examples of subsampling methods are shown.

[0029] Figure 13 Various examples of subsampling methods are shown.

[0030] Figures 14 to 19 The search methods in template matching based on the examples are shown respectively.

[0031] Figure 20 This illustrates the first template configuration method under affine mode, based on the example.

[0032] Figure 21 This illustrates a second template configuration method under affine mode, based on an example.

[0033] Figure 22 Examples of neighboring blocks of the target block are shown.

[0034] Figure 23This shows a bilateral match based on the example.

[0035] Figure 24 A to Figure 24 B is a flowchart of an encoding method for a target block according to an embodiment.

[0036] Figure 25 A to Figure 25 B is a flowchart of a decoding method for a target block according to an embodiment.

[0037] Figure 26 This is a diagram illustrating the reference region in a model-based prediction method based on an example.

[0038] Figure 27 It is a diagram used to describe a specific location.

[0039] Figure 28 This is an example used to describe neighboring samples within BOUND_LINE_NUM rows / columns adjacent to the target block and samples within BOUND_LINE_NUM rows / columns within the target block.

[0040] Figure 29 This is an embodiment of a configuration for determining whether to use neighboring samples in the top direction for at least one of the independent sample set, dependent sample set, and subject sample set based on the sample values ​​of at least one neighboring sample in two adjacent rows / columns and / or at least one sample in two rows / columns within the target block, for the top boundary surface of the target block.

[0041] Figure 30 This shows the positions of the luminance component and neighboring components corresponding to CHROMA_PIX.

[0042] Figure 31 An example of a 3×3 low-pass filter used for predicting sample modifications is shown.

[0043] Figure 32 The brightness component sample points and neighboring sample points are shown.

[0044] Figure 33 This is a block diagram illustrating the configuration of an encoding apparatus according to an embodiment of the present invention.

[0045] Figure 34 This is a block diagram illustrating the configuration of an embodiment of the decoding apparatus according to which the present invention is applied.

[0046] Figure 35 It is a schematic diagram illustrating the partitioning structure of an image during the encoding and decoding process.

[0047] Figure 36 This is a diagram illustrating an embodiment of intra-frame prediction processing.

[0048] Figure 37 This is a diagram illustrating an embodiment of inter-frame prediction processing.

[0049] Figure 38 It is a diagram used to describe the transformation and quantization processes.

[0050] Figure 39 An example of partition boundaries in a geometric partitioning mode is shown.

[0051] Figure 40 It is a diagram showing the partition boundaries, partition offsets, and partition angles in the geometric partitioning mode.

[0052] Figure 41 An example of a weighted graph for each prediction block based on geometric partition boundaries is shown.

[0053] Figure 42 This is an illustration of an embodiment used for template matching.

[0054] Figure 43 This illustrates the subsampling method for the search range of template matching.

[0055] Figure 44 This illustrates the subsampling method for template matching.

[0056] Figures 45 to 50 The search methods in template matching according to the embodiments are shown respectively.

[0057] Figure 51 A first template configuration method in affine mode is shown according to an embodiment.

[0058] Figure 52 This illustrates a second template configuration method under affine mode, based on an example.

[0059] Figure 53 Various examples of subsampling methods in bilateral matching are shown.

[0060] Figure 54 A bilateral matching according to an embodiment is shown.

[0061] Figure 55 This is a flowchart of an encoding method for a target block according to an embodiment.

[0062] Figure 56 This is a flowchart of a decoding method for a target block according to an embodiment.

[0063] Figure 57 This is a flowchart of a decoding method for a target block according to an embodiment.

[0064] Figure 58 Examples of the first input structure and the second input structure are shown.

[0065] Figure 59 It is a diagram used to describe a specific location.

[0066] Figure 60 The reference area used to derive the mapping model is shown.

[0067] Figure 61 The sample points of the computational mapping model output are shown.

[0068] Figure 62 This is an example used to describe neighboring samples within BOUND_LINE_NUM rows / columns adjacent to the target block and samples within BOUND_LINE_NUM rows / columns of the target block.

[0069] Figure 63 An embodiment is shown for determining whether to use top neighbor samples when configuring various sample sets, based on at least one of the neighboring samples in two top rows / columns of the target block or samples in two rows / columns within the target block.

[0070] Figure 64 This is a diagram showing the non-adjacent blocks of the current block.

[0071] Figure 65 The filter is shown based on the position of the luminance component.

[0072] Figure 66 An example of a 3×3 low-pass filter used for predicting sample modifications is shown.

[0073] Figure 67 This is a reference diagram for easy description of the mapping model function.

[0074] Figure 68 The first luminance component sample and its neighboring sample are shown.

[0075] Figure 69 A first embodiment is shown for generating a reference block when a motion vector (or block vector) indicates an integer pixel position.

[0076] Figure 70 A second embodiment is shown for generating a reference block when a motion vector (or block vector) indicates the position of a sub-pixel. Detailed Implementation

[0077] Various modifications can be applied to this disclosure. Furthermore, this disclosure can have various embodiments. Specific embodiments will be described with reference to the accompanying drawings and detailed description.

[0078] The specific embodiments are not intended to limit this disclosure to a particular mode of practice, and it should be understood that all changes, equivalents, and alternatives are included as embodiments in this disclosure without departing from the spirit or technical scope of this disclosure.

[0079] These embodiments are described to enable those skilled in the art to readily practice them. It should be noted that the various embodiments differ from one another but are not necessarily mutually exclusive. For example, shapes, structures, and characteristics associated with the embodiments may be applied or implemented in other embodiments without departing from the spirit and scope of this disclosure. Furthermore, it should be understood that the position or arrangement of various components in the embodiments may be changed without departing from the spirit and scope of this disclosure. Therefore, the appended detailed description is not intended to limit the scope of this disclosure, and the scope of the exemplary embodiments is limited only by the appended claims and their equivalents, provided they are properly described.

[0080] A detailed description of the embodiments, which will be described later, can be referenced in the accompanying drawings. The descriptions made in or shown in the drawings are to be considered part of the detailed description of this disclosure. In the drawings, similar reference numerals may be used to designate functions that are identical or similar in various respects. Dependencies between components are not limited to those shown in the drawings.

[0081] In embodiments, singular expressions may include, and may be limited to, plural expressions unless specifically indicated in the context. In other words, in embodiments, expressions such as “at least one” and “one or more” can be replaced by the term “multiple”. Terms such as “ / ”, “and / or”, “at least one of…” and “one or more of…” describing multiple items can refer to 1) one of the multiple items, 2) some of the multiple items, 3) a combination of some of the multiple items, or 4) a combination of the multiple items. Furthermore, plural expressions can be replaced by singular expressions. Here, a plural number can represent an integer of 1, 2, 3, 4, or 5 or greater.

[0082] In embodiments, terms associated with numbers such as "first" and "second" may be used to describe various components. These terms are used only to distinguish one component from another and are not intended to limit the components. For example, a first component may be referred to as a second component without departing from the scope of this disclosure. Similarly, a second component may also be referred to as a first component.

[0083] The fact that the first component sends (or provides) information to the second component can mean that the first component sends the information directly to the second component, or it can mean that the first component sends the information to the second component through a third component. Here, the information received (or acquired) by the second component can be information sent by the first component, or information generated by applying specific processing to information sent by the first component.

[0084] The components in the embodiments may be shown independently to indicate different functional characteristics, and the illustration does not imply that each component corresponds to a separate hardware or software element. That is, for ease of description, the components in the embodiments may be divided and enumerated. Two or more components described in an embodiment may be considered as one component. Furthermore, a component described in an embodiment may be divided into multiple components, which are divided and perform the function of the component individually. Embodiments in which these components are integrated and embodiments in which these components are separate may be included within the scope of this disclosure without departing from its spirit.

[0085] The terminology used in the embodiments is for describing specific embodiments only and is not intended to limit this disclosure. In the embodiments, terms such as "comprising" or "having" are intended to indicate the presence of features, numbers, steps, operations, components, portions, or combinations thereof described in the embodiments. These terms do not exclude the possibility of the presence or addition of other features, numbers, steps, operations, components, portions, or combinations thereof not explicitly described in the embodiments. That is, the description of a component as "comprising" a specific component in the embodiments does not exclude additional components beyond the specific component, and means that additional components may be included within the scope of the embodiments of this disclosure or the technical spirit of this disclosure.

[0086] Some components in the embodiments may be optional components in addition to the basic components of this disclosure used to perform basic functions. Such optional components can be used to improve performance. Embodiments may be implemented as structures that include only the basic components necessary to implement the essential points of the embodiments, excluding optional components. Such structures are included within the scope of the embodiments.

[0087] Embodiments of this disclosure will now be described in detail with reference to the accompanying drawings to enable those skilled in the art to readily practice this disclosure. In the description of this disclosure, detailed descriptions of known functions and configurations that are considered to obscure the main points of the disclosure will be omitted. Throughout the following description of this disclosure, the same reference numerals will be used to denote the same or similar components, and repeated descriptions of the same components will be omitted.

[0088] Substitution of terms in the embodiments In the following description, terms listed in a single line may be used in the embodiments to have the same meaning and may be used interchangeably with each other in the embodiments.

[0089] - "one or more" and "at least one" - "two or more", "multiple", "more than one" and "more" (in embodiments, "one or more" or "at least one" may be further limited to "two or more", "multiple" or "more") - "Information" and "Signal" - "Value", "Predefined Value", "Specific Value", "Threshold", "Baseline Value", and "Reference Value" - "Statistical Values" and "Statistical Values" - "Indicator", "Index", "Flag", and "Information" - "Encoder" and "Encoding Device" - "Decoder" and "Decoding Device" - "Entropy coding", "encoding" and "encoding" - "Entropy decoding", "decoding" and "decoding" - "Encoding / Decoding" and "Encoding and / or Decoding" - "video", "moving image", "image", "picture", "frame", and "screen" - "Reference Screenshot" and "Reference Image" - "Reference Screen List (RPL)" and "Reference Image List" - "Original", "Input", and "Source" - "block", "unit", and "signal" - "Square" and "Square Shape" - "pixel", "sample", and "pel" - "Region", "District", "Part", and "Fragment" - "Partition", "Segment", and "Divide" - "quaternary" and "quaternary" - "luma component", "luma", "luminance component", "luminance", and "Y" - "chroma component", "chroma", "chromaticity", "chromaticity component", "Cb and Cr", "Cb or Cr", "Cb", "Cr", "U and V", "U or V", "U or "V" - "target", "current" (e.g., target block and current block, or target image and current image) - "Neighbor", "Nearby", "Adjacent", "Neighbor / Nearby" (e.g., neighbor block, adjacent block, and neighbor / nearby block) - "Paired" and "COL" - "Reconstruction", "Reconstruction", and "Decoding" - "Reconstructed", "Reconstructed", and "Decoded" - "difference", "error", "residual" - Maximum coding unit (LCU) and coding tree unit (CTU) - "Inter-frame" and "inter" - "Inter-frame prediction", "inter-prediction", and "motion compensation" - "Inter-frame mode", "Inter-frame prediction mode", "Inter-mode", and "Inter-prediction mode" - "Motion Vectors", "Predicted Motion Vectors", and "Advanced Motion Vector Prediction (AMVP)" - "List" and "Candidate List" - "Spatial Candidates" and "Spatial Merge Candidates" - "Time Candidates" and "Time Merged Candidates" - "Predicting Motion Vector Candidates" and "Motion Vector Predictors" - "Forecasting Methods" and "Forecasting Models" - "intra" and "withinframe" - "Intra-prediction" and "Intra-frame prediction" - "Intra-frame mode" and "Intra-frame prediction mode" - "Inverse quantization" and "scaling" - "Quantization Matrix" and "Scaling List" - "Quantization matrix coefficients" and "Matrix coefficients" - "Transformation Coefficient Level", "Quantization Level", "Quantization Coefficient", "Quantization Transformation Coefficient", and "Quantization Transformation Coefficient Level" - "Inverse quantization coefficients" and "Inverse quantization transform coefficients" - "Scan Type" and "Scan Direction" - "Directional Mode", "Angle Mode", "Angled Mode", and "Intra-Frame Prediction Mode" - "Intra-prediction mode (mode number)," "Intra-prediction mode (mode index)," "Intra-prediction mode (mode value)," "Intra-prediction mode (mode angle)," "Intra-prediction mode (mode direction)," "Intra-prediction direction (mode number)," "Intra-prediction direction (mode index)," "Intra-prediction direction (mode value)," and "Intra-prediction direction (mode angle)." - "Merge Mode" and "Motion Merge Mode" - "Geometric Partitioning (GPM)" and "Triangle Partitioning" In addition to the terms exemplified above, terms that have the same meaning based on common knowledge in the art may be used interchangeably with each other in the embodiments.

[0090] The information and range of information values ​​described in the embodiments In embodiments, information may include constants, flags, indexes, variables, encoding parameters, elements, syntax elements, motion information, attributes, entities, objects, data, etc. In other words, the term "information" may be used interchangeably with "data," "flags," "indexes," "variables," "elements," "syntax elements," "motion information," "attributes," or "entities."

[0091] This information can have one of multiple values. The term "nth value" can refer to the nth value among multiple values.

[0092] For example, the first value can represent "0" or logical false. The second value can represent "1" or (logical) true. Alternatively, the first value can represent "1" or (logical) true. The second value can represent "0" or (logical) false.

[0093] A flag can be information with either a value of "0" or "1". In embodiments, the values ​​of flags, i.e., "0" and "1", can be replaced with "1" and "0", respectively. For example, information indicating whether to perform a specific process or whether to apply a specific process can be considered a flag.

[0094] When variables such as i or j are used to indicate rows, columns, or indices, the variables can be integers equal to or greater than 0 and less than or equal to n-1. Optionally, the variables can be integers equal to or greater than 1 and less than or equal to n. Here, n can be the number of rows, the number of columns, or the number of entities indicated by the index.

[0095] Concepts related to encoding and decoding The concepts related to encoding and decoding will be described below. These descriptions can be applied to embodiments.

[0096] Predefined value: A predefined value can refer to a value shared by both the encoding and decoding devices. For example, a predefined value can be restricted to and interpreted as a fixed value. Optionally, a predefined value can be a value shared between the encoding and decoding devices via signaling. Optionally, a predefined value can be a value derived by both the encoding and decoding devices through the same process, such that the encoding and decoding devices have a common value. Optionally, a predefined value can be a common value possessed by both the encoding and decoding devices. The description of predefined values ​​can also be applied to predefined information. In the above description, "information" can be used instead of "value".

[0097] Availability: The fact that a particular pattern is available for a particular objective means that the pattern selected from the particular patterns is used for the particular objective. Other patterns belonging to the category of a particular pattern may be unavailable patterns. Unavailable patterns may not be used for a particular objective. The description of a particular pattern may also apply to other specific information. In the above description, "information" can be used instead of "pattern".

[0098] Adjacent: The terms "direction" and "second entity" for "first entity" can refer to the "direction" of the first entity and the "second entity" adjacent to a corner / surface of the first entity. For example, the term "top-left block" for "target block" can be the block adjacent to the top-left corner of the target block. Here, "first entity" can be a target cell, target block, or target sample. The term "direction" can refer to one of the directions corresponding to top-left, top, top-right, top-right, left, right, bottom-left, bottom, and bottom-right. The term "second entity" can be a cell, block, or sample. Here, regarding the directions corresponding to top-left, top-right, bottom-left, and bottom-right, the corners of the first entity and the corners of the second entity can be diagonally adjacent to each other. Regarding the directions corresponding to top, left, right, and bottom, a surface of the first entity and a surface of the second entity can be in contact with each other.

[0099] - For example, the block adjacent to the top left of the target block can be the top adjacent block of the block adjacent to the left of the target block. - The block adjacent to the top right of the target block can be the right adjacent block of the block adjacent to the top of the target block. - The block adjacent to the bottom left of the target block can be the bottom adjacent block of the block adjacent to the left of the target block.

[0100] Encoding and decoding: Encoding and decoding can refer to the encoding and / or decoding of images.

[0101] Signals: Signals can refer to information about an image, unit, or block. A specific signal can refer to a specific image, unit, or block.

[0102] Image: An image can refer to one of the frames that make up a video, or it can refer to the video itself. For example, "image encoding and / or decoding" can refer to "video encoding and / or decoding," and it can also refer to "encoding and / or decoding of one of the images that make up a video."

[0103] - An image can refer to the entirety of a picture, or it can refer to a part of a picture, such as a block.

[0104] Target image: The target image can be an encoded target image that serves as the encoding target and / or a decoded target image that serves as the decoding target. Furthermore, the target image can be an input image processed by an encoding device or a reconstructed image processed by a decoding device. The target image can be an image that includes target blocks.

[0105] Sub-screen: A screen can be divided into one or more sub-screens.

[0106] - A sub-screen can be a square or rectangular area within the screen. Each sub-screen can have one or more CTUs.

[0107] A subscreen may include one or more stripes and / or one or more parallel blocks. For example, a subscreen may include one or more stripe rows and one or more stripe columns. Optionally, each subscreen may include one or more parallel block rows and one or more parallel block columns.

[0108] - A sub-picture may include one or more strips that collectively cover a rectangular area within the picture. Therefore, the boundary of each sub-picture can always be the boundary of a strip. Furthermore, the boundary of each vertical sub-picture can always be the boundary of a vertical parallel block.

[0109] Strip: A strip may include one or more parallel blocks in the frame. A strip may include one or more rows of parallel blocks and one or more columns of parallel blocks.

[0110] Parallel Blocks: Parallel blocks can be square or rectangular areas on the screen. A parallel block can have one or more CTUs. The screen can be divided into one or more parallel block rows and one or more parallel block columns.

[0111] CTU: An image can be partitioned into multiple coding tree units (CTUs).

[0112] - A CTU may include a Y-coded tree block (CTB) and at least one of a Cb CTB and a Cr CTB associated with the Y CTB, and may include information about each CTB. This information may include syntax elements.

[0113] - One or more partition types can be used for each CTU partition to form sub-units such as coding units (CUs), prediction units (PUs), and transform units (TUs). One or more partition types can include quadtree (QT) partitions, binary tree (BT) partitions, and ternary tree (TT) partitions. Furthermore, multi-type tree (MTT) partitions can be used for each CTU partition, employing a combination of multiple partition types.

[0114] CTB: CTB can refer to one of Y CTB, Cb CTB, and Cr CTB.

[0115] Unit: A unit can be defined for a specific process in encoding and decoding. A unit can be information about a specific region in an image. An image can be recursively divided into multiple parts to perform specific encoding and decoding processes. A unit can refer to the region to which a specific process is applied and information about that region.

[0116] - Unit type can refer to the specific processing applied to a unit. A specific processing can be applied to a unit based on its unit type. A "specific" unit can be a unit that is named as a "specific" processing in the encoding / decoding process. For example, a unit can be at least one of the following: original unit, CTU, coding unit, prediction unit, residual unit, reconstructed residual unit, transform unit, and reconstruction unit.

[0117] - A cell may include samples having a two-dimensional (2D) form or arrangement. In this respect, the term "cell" may refer to a "block". For example, a block may be at least one of a raw block, a CTB, a coded block (CB), a prediction block (PB), a residual block, a reconstructed residual block, a transform block (TB), and a reconstructed block. For example, a partition of a cell may refer to a partition of the block corresponding to the cell.

[0118] - A unit can include syntax elements. In other words, a block and its syntax elements can be collectively referred to as a "unit".

[0119] - A block can be an M×N sample array. Here, M and N can refer to positive integer values, and a block can typically refer to a sample array in 2D form. The current block can refer to an encoded target block as the target to be encoded in the encoding process and a decoded target block as the target to be decoded. Furthermore, the current block can be at least one of an encoded block, a prediction block, a residual block, a transform block, and a reconstruction block. Blocks can have various sizes and shapes. For example, block shapes can include one or more of quadrilaterals, rectangles, squares, rectangles with a horizontal length different from their vertical length (i.e., elongated rectangles), trapezoids, triangles, right triangles, and pentagons. Here, the horizontal and vertical lengths of a rectangle can be different from each other. Furthermore, block shapes can include other geometric shapes that can be represented in two dimensions. For example, a block shape can be a rectangle or a pentagon defined by excluding right triangle regions from a rectangular region. Here, the right-angled vertex of the right triangle can be one of the vertices of the rectangle. Furthermore, a block shape can be a combination of two or more of the shapes described above. Additionally, a block shape can be the remaining shape obtained by excluding one shape from the aforementioned shapes.

[0120] - In embodiments, the rectangle may be limited to non-square rectangles. In embodiments, when the shape of a particular target is described as a rectangular shape, the description may further imply that the horizontal length and vertical length of the particular target are different from each other.

[0121] - In an embodiment, the block may be limited to at least one of a vertically oriented block and a horizontally oriented block. A vertically oriented block may be a block with a vertical length greater than its horizontal length. A horizontally oriented block may be a block with a horizontal length greater than its vertical length.

[0122] - A unit may include a luminance component block (i.e., a Y block) and two chrominance component blocks (i.e., at least one of a Cb block, a Cr block, or a combination thereof), and may include information about each block. The information may contain syntax elements.

[0123] Information about a cell may include the cell type, cell size, cell depth, cell encoding order, cell decoding order, etc.

[0124] Target unit: A target unit can be an encoding target unit that is to be encoded in encoding and a decoding target unit that is to be decoded. A target unit can be a specific region in a target image to which one or more specific encoding / decoding processes will be applied. A specific type of unit can be generated by applying specific processes to the target unit. Optionally, a target unit can refer to a unit having a specific type for a specific encoding / decoding process.

[0125] Depth: A block can be hierarchically partitioned into multiple sub-blocks while having a depth that depends on the tree structure. The multiple sub-blocks generated from the partitioning operation of a block can be called "partitions".

[0126] - When the blocks that make up an image are represented in a tree structure, the block depth can represent the level of the node corresponding to each block. Optionally, the block depth can indicate the number of partitions applied until the block is determined. As the block is further partitioned, the block depth can increase by 1.

[0127] In a tree structure, the root node can be considered to have the lowest level, and leaf nodes have the highest level. The root node can be the topmost node of the tree structure and can correspond to the unpartitioned initial block. The level of the root node can be 0 or 1. When the root node's level is 0, a node with level 1 can refer to the block determined as the initial block is partitioned once. A node with level n can refer to the block determined when the initial block is partitioned n times. Leaf nodes can be the bottommost nodes of the tree structure. Leaf nodes can be nodes that cannot be further partitioned. The depth of a leaf node can be a predefined maximum depth. For example, the maximum depth can be a positive integer such as 3. The root node can refer to CTU. Leaf nodes can refer to at least one of CU, PU, ​​and TU.

[0128] - Depth can have a type that depends on the partition type. QT depth can represent the depth in a quadtree partition. BT depth can represent the depth in a binary tree partition. TT depth can represent the depth in a ternary tree partition.

[0129] Sample: A sample can be the basic unit that makes up a block. A sample can consist of one or more bits. Bit depth refers to the number of bits that make up a sample. Samples can range from 0 to 2. Bd The value of -1 indicates that this depends on the bit depth.

[0130] PU: PU can represent the basic unit used for prediction-related processing. For example, prediction-related processing may include inter-frame prediction, intra-frame prediction, intra-block copy (IBC) prediction, intra-frame compensation, and motion compensation.

[0131] A prediction unit (PU) can be partitioned into multiple sub-PUs, each with a size smaller than the PU's size. Each of the multiple sub-PUs can also be a basic unit for prediction-related processing. In other words, prediction unit partitions generated from the partitioning operation of prediction units can also be prediction units.

[0132] TU: A TU can be a basic unit for processing associated with a residual block. Processing associated with a residual block can include at least one of transform, inverse transform, quantization, dequantization, transform coefficient encoding, transform coefficient decoding, entropy encoding, or entropy decoding, or combinations thereof. A TU can be partitioned into multiple sub-transform units, each sub-transform unit having a size smaller than the TU. Each of the multiple sub-TUs can also be a basic unit for processing associated with a residual block. In other words, a transform unit partition generated from a partitioning operation of a transform unit can also be a transform unit.

[0133] - The transformation may include one or more of the primary transformation and the secondary transformation, and the inverse transformation may include one or more of the primary inverse transformation and the secondary inverse transformation.

[0134] Parameter set: The parameter set can correspond to the header information in the structure of the bit stream.

[0135] - The parameter set may include at least one of the following: video parameter set (VPS), sequence parameter set (SPS), picture parameter set (PPS), adaptive parameter set (APS), or decoding parameter set (DPS), or a combination thereof.

[0136] Information transmitted via signals using a parameter set can be applied to a screen referencing that parameter set. For example, information in a VPS can be applied to a screen referencing a VPS. Information in an SPS can be applied to a screen referencing an SPS. Information in a PPS can be applied to a screen referencing a PPS. A parameter set can reference a higher-level parameter set. For example, a PPS can reference an SPS. An SPS can reference a VPS.

[0137] - In addition, the parameter set may include parallel block group information, stripe header information, and parallel block header information. A parallel block group can refer to a group or stripe that includes multiple parallel blocks.

[0138] Most Probable Mode (MPM): MPM can refer to an intra-prediction mode that has a high probability of intra-prediction for the target block.

[0139] - One or more different MPMs can be determined based on the encoding parameters associated with the target block and the attributes of the entities associated with the target block.

[0140] - One or more MPMs can be determined based on the intra-prediction modes of a reference block. A reference block can include multiple reference blocks. One or more different MPMs can be determined based on which intra-prediction modes have been used for one or more reference blocks. Reference blocks can include spatially neighboring blocks.

[0141] MPM List: An MPM list can be a list that includes one or more MPMs. The number of MPMs in the MPM list can be predefined.

[0142] MPM Index: The MPM index indicates the MPM used for intra-frame prediction of the target block among one or more MPMs in the MPM list.

[0143] MPM Usage Indicator: The MPM usage indicator indicates whether the MPM list has been used for the prediction of the target block.

[0144] Prediction mode: The prediction mode can be information indicating the prediction method used for the target block, such as a mode to be used for intra-frame prediction or a mode to be used for inter-frame prediction. The prediction mode can indicate one of the prediction-related modes, as will be described in the embodiments. Furthermore, the prediction mode can include at least one of intra-frame mode, inter-frame mode, intra-frame block copy mode, or a combination thereof.

[0145] Reference image list: The reference image list can be a list that includes one or more reference images that will be used for prediction of the target block.

[0146] - The list of reference images can include multiple lists. Multiple lists of reference images can include list 0 (List0; L0), list 1 (List1; L1), etc.

[0147] - For inter-frame prediction of a target block, one or more lists of reference images can be used. In the names of information related to inter-frame prediction, parts such as "L0" and "L1" may refer to the list of reference images associated with that information.

[0148] Reference image (reference frame): The reference image (frame) can be an image referenced for predicting the target block. Optionally, the reference frame can be an image that includes the reference block. The reference frame can include images before the target image, the target image, and images after the target image.

[0149] Reference Image Index: A reference image index can be an index that indicates a reference image (picture) among one or more reference images in a list of reference images used for predicting the target block.

[0150] Reference block: A reference block can be a block that is referenced for encoding / decoding the target block (such as prediction and filtering). For example, a reference block can include reference samples that are referenced to derive prediction samples, and can refer to a block that provides information for decoding the target block.

[0151] Reference sample: A reference sample can be a sample used for encoding / decoding a target block (such as prediction and filtering).

[0152] Inter-frame prediction indicator: The inter-frame prediction indicator can indicate the direction of inter-frame prediction for a target block. Inter-frame prediction can be one of unidirectional and bidirectional prediction. Optionally, the inter-frame prediction indicator can represent the number of reference images used to generate prediction blocks for the target block. Optionally, the inter-frame prediction indicator can represent the number of prediction blocks used for inter-frame prediction of the target block. The reference direction can refer to the inter-frame prediction indicator. For example, the inter-frame prediction indicator can represent one of a unidirectional indicator and a bidirectional indicator. Optionally, in an inter-frame mode that uses only reference images in reference image list L0, the inter-frame prediction indicator can have a "0" as a first value; in an inter-frame mode that uses only reference images in reference image list L1, the inter-frame prediction indicator can have a "1" as a second value; and in an inter-frame mode that uses at least two of the reference images in reference image list L0 and reference images in reference image list L1, the inter-frame prediction indicator can have a "2" as a third value.

[0153] Prediction list utilization flag: The prediction list utilization flag for a specific reference image list indicates whether at least one reference image in the specific reference image list is used to generate a prediction block for the target block. For example, a prediction list utilization flag of "0" indicates that reference images from the specific reference image list are not used to generate the prediction block. A prediction list utilization flag of "1" indicates that reference images from the specific reference image list are used to generate the prediction block.

[0154] - Inter-frame prediction indicators can be derived using prediction list utilization flags. Conversely, prediction list utilization flags can be derived using inter-frame prediction indicators. For example, inter-frame prediction indicators can be derived using prediction list utilization flags from multiple reference image lists. When an inter-frame prediction indicator indicates that a specific reference list among the multiple reference image lists is used, the prediction list utilization flag for the specific reference list indicated by the inter-frame prediction indicator can be set to "1", while the prediction list utilization flags for the remaining reference image lists not indicated by the inter-frame prediction indicator can be set to "0".

[0155] Reference direction: The reference direction can indicate a list of reference images used for prediction of the target block. For example, the reference direction can indicate one or more of reference image list L0 and reference image list L1.

[0156] - The reference direction indicates only the list of reference images used for prediction of the target block, and may not indicate that the orientation of the reference images in the list is limited to either the forward or backward direction. In other words, each of the reference image lists L0 and L1 may include both forward and backward images. Here, forward can indicate the direction from the target image to an image preceding the target image. Forward inter-frame prediction can be an inter-frame prediction using an image preceding the target image as a reference image. The backward direction can indicate the direction from the target image to an image following the target image. Backward inter-frame prediction can be an inter-frame prediction using an image following the target image as a reference image.

[0157] - A unidirectional reference direction can mean using one list of reference images. A bidirectional reference direction can mean using two lists of reference images. For example, a reference direction can indicate one of the following: using only reference image list L0, using only reference image list L1, or using both reference image lists. Furthermore, the reference direction can be indicated by an inter-frame prediction indicator.

[0158] Picture Order Count (POC): The POC of an image (picture) can indicate the display order or output order of the images (pictures).

[0159] Motion information: Motion information can be information used to specify a reference block. Motion information may include information used for inter-frame prediction, such as motion vectors (MV), reference image index, reference image, inter-frame prediction indicator, and prediction list utilization flags. Furthermore, motion information may include information used in a specific inter-frame prediction mode, such as MV candidates, MV candidate indices, merge candidates, and merge indices.

[0160] - For inter-frame prediction of a target block, multiple motion information related to multiple lists of reference images can be used separately. Motion information of a specific list of reference images can be used for prediction using that specific list of reference images. Multiple (intermediate) prediction blocks can be derived separately from multiple motion information. The (final) prediction block of the target block can be generated using the statistical values ​​of the multiple (intermediate) prediction blocks.

[0161] MV: Motion vector (MV) can be a two-dimensional (2D) vector used for inter-frame prediction. MV can refer to the offset between the target block and the reference block. Alternatively, MV can indicate the difference between the position of the target block and the position of the reference block.

[0162] - For example, it can be in the form of (mv x , mv yMV is represented in the form of ) x It can indicate the horizontal component, and mv y It can indicate the vertical component.

[0163] - The zero vector can be (0,0)MV.

[0164] Block Vector (BV): BV can be a two-dimensional (2D) vector used for intra-block replication prediction. BV can refer to the offset between a target block and a reference block in the target image. In other words, BV can indicate the displacement between a target block and a reference block in the target image.

[0165] - For example, similar to MV, it can be in (bv x , bv y BV is represented in the form of ) . bv x It can indicate the horizontal component, and bv y It can indicate the vertical component.

[0166] - The zero vector can be (0,0)MV.

[0167] Motion information candidates: In a specific prediction, motion information of a target block can be selected from motion information candidates determined by a specific scheme. Motion information candidates can refer to the motion information of a reference block, and can refer to the reference block itself containing motion information. Here, the reference block can be a block determined by a specific scheme to select motion information candidates.

[0168] Candidate List: The candidate list can be a list including one or more candidates. For example, the candidate list may include a motion information candidate list, a merge candidate list, an MV candidate list, an MPM list, etc. The candidate list can be generated by both the encoding and decoding devices in the same way. In other words, the candidate list used by the encoding device and the candidate list used by the decoding device can be the same as each other, and the same candidate list can be shared between the encoding and decoding devices. The encoding device can select candidates from the candidates in the candidate list to be used for processing the target block. An indicator indicating the selected candidate can be sent from the encoding device to the decoding device via a signal. The decoding device can specify the candidate to be used for processing the target block from the candidates in the candidate list by utilizing the indicator. Optionally, the encoding and decoding devices can specify the candidate to be used for processing the target block from the candidates in the candidate list based on the same rules.

[0169] Motion information candidate list: The motion information candidate list can refer to a list constructed using one or more motion information candidates.

[0170] Motion Information Candidate Index: A motion information candidate index can be an identifier or indicator that indicates a motion information candidate in the motion information candidate list for prediction of the target block.

[0171] - In a specific inter-frame prediction mode, motion information of the target block can be derived using motion information from additional reconstructed blocks. The additional block can be a neighboring block. In this mode, the motion information of the target block itself may not be transmitted separately; instead, additional information for deriving the motion information of the target block based on the motion information of the additional reconstructed blocks may be transmitted. This additional information may include information indicating blocks within the additional reconstructed blocks (such as motion information candidate indices) whose motion information is used to derive the motion information of the target block.

[0172] For example, inter-frame prediction modes can include AMVP mode, merge mode, skip mode, etc. Motion information candidate indices can be merge indexes or MV candidate indices.

[0173] - In an embodiment, MV can be a part of motion information. In an embodiment, information related to motion information (such as motion information candidates, motion information candidate lists, and motion information candidate indexes) can be replaced with information related to MV (such as MV candidates, MV candidate lists, and MV candidate indexes), and the description of motion information can also be applied to MV.

[0174] Merging: The term "merging" can refer to merging multiple motion information from multiple blocks, or it can refer to applying motion information together with the motion information of an additional block to a target block. In other words, a merging pattern can refer to a pattern for deriving the motion information of a target block from the motion information of neighboring blocks.

[0175] Merge candidate: A merge candidate may refer to a specific (reconstructed) block used to merge the target block, and may refer to the motion information of that specific block. Optionally, a merge candidate may include the motion information of that specific block.

[0176] - Merge candidates for the target block can include spatial merge candidates, temporal merge candidates, history-based candidates, average candidates based on the average of two merge candidates, zero merge candidates, etc.

[0177] Merge candidate list: The merge candidate list can be a list constructed using one or more merge candidates.

[0178] Merge Index: A merge index can be an indicator that identifies a merge candidate in a merge candidate list for predicting the target block. The motion information of the merge candidate indicated by the merge index in the merge candidate list can be used as the motion information of the target block.

[0179] Neighboring blocks: Neighboring blocks can be blocks adjacent to the target block. Neighboring blocks can include spatially neighboring blocks and temporally neighboring blocks. A neighboring block can refer to a reconstructed neighboring block in the reference image.

[0180] Spatial neighbor block: A spatial neighbor block can be a block that is spatially adjacent to the target block.

[0181] - Target blocks and spatially neighboring blocks can be included in the target image.

[0182] - A spatially neighboring block may include a block whose boundary at least a portion is in contact with at least a portion of the boundary of the target block. Optionally, a spatially neighboring block may include a block whose distance from the target block is less than or equal to a specific value.

[0183] - Spatial neighbor blocks can include blocks that are diagonally adjacent to the vertex of the target block.

[0184] - Spatial neighbor blocks can include the top-left block adjacent to the top-left of the target block, the top block adjacent to the top of the target block, the top-right block adjacent to the top-right of the target block, the left block adjacent to the left side of the target block, the right block adjacent to the right side of the target block, the bottom-left block adjacent to the bottom-left of the target block, the bottom block adjacent to the bottom of the target block, and the bottom-right block adjacent to the bottom-right of the target block.

[0185] Temporally adjacent blocks: Temporally adjacent blocks can be blocks that are temporally adjacent to the target block.

[0186] - Temporally adjacent blocks can include co-occurring blocks (COL blocks). A COL block can be a block in a reconstructed image stored in a reference image (picture) buffer. A co-occurring image (co-occurring picture: COL picture) can refer to a picture that includes a COL block. A COL picture can be an image included in a list of reference images.

[0187] - COL blocks can be determined based on the position of target blocks in the target image. The fact that two blocks are "temporally adjacent" can imply that the positions of the two blocks meet specific conditions.

[0188] - The position of the COL block in the COL frame can be the same as the position of the target block in the target image. Optionally, the position of the COL block in the COL frame can correspond to the position of the target block in the target image. Here, the positions of corresponding blocks can mean that the areas of the blocks are the same, that the area of ​​a block is included in the area of ​​the additional block, or that a block occupies a specific position in the additional block.

[0189] For example, the position of a COL block in a COL image can be the same as the position of a target block in the target image. Alternatively, a COL block can be a block in a COL image that includes COL samples. A COL sample can be a sample point with the same coordinates as a specific sample point in the target block.

[0190] - A temporally neighboring block can be a block that is spatially adjacent to the target block in time.

[0191] Search Range: The search range can refer to the 2D extent of the search for MVs during inter-frame prediction. For example, when the best MV is derived to process the target block, the best MV is selected from the MVs within the indicated search range.

[0192] Transform coefficients: Transform coefficients can be coefficients generated by performing a transformation on the residual block. Alternatively, transform coefficients can be coefficient values ​​generated by performing dequantization on the quantization level.

[0193] Quantization level: The quantization level can be an integer number used as the input for dequantization.

[0194] Quantization: Quantization can be a process that generates quantization levels for transform coefficients. Quantization levels can be generated by applying quantization to transform coefficients. Transformation can also be considered as part of quantization.

[0195] Dequantization: Dequantization can be a process of multiplying a factor by a quantization level. (Reconstructed) transform coefficients can be generated by applying dequantization to the quantization level.

[0196] Quantization parameter (QP): QP can refer to a factor used when generating the quantization levels of the transform coefficients. Additionally, QP can refer to a factor used when generating the (reconstructed) transform coefficients for the quantization levels used in inverse quantization. Optionally, QP can be a value mapped to the quantization step size.

[0197] Incremental QP: Incremental QP can be the difference between the QP predicted through specific processing and the QP of the target block. In other words, the QP of the target block can be the sum of the predicted QP and the incremental QP.

[0198] Quantization matrix: A quantization matrix is ​​a matrix used in quantization or dequantization to improve the subjective or objective quality of an image.

[0199] Quantization matrix coefficients: Quantization matrix coefficients can be each element in the quantization matrix.

[0200] Scan: A scan can refer to a method of arranging values ​​in a block or matrix. These values ​​can be coefficients. For example, a scan can refer to the arrangement of values ​​in 2D or 1D form, and the rearrangement of values ​​in 1D or 2D form. A reverse scan can refer to the arrangement (or rearrangement) that is the opposite of the arrangement performed in a scan.

[0201] Non-zero transform coefficients: Non-zero transform coefficients can refer to transform coefficients with non-zero values ​​or quantization levels with non-zero values.

[0202] Bitstream: A bitstream can refer to a stream / sequence of bits that includes encoded information generated by encoding an image. A bitstream may include information dependent on specific syntax elements. For example, the information may include syntax elements. An encoding device can generate a bitstream that includes multiple pieces of information dependent on specific syntax elements. A decoding device can extract multiple pieces of information from the bitstream based on specific syntax elements.

[0203] Signaling: Signaling information can refer to sending information from an encoding device to a decoding device via a bitstream. For example, the information may include syntax elements. Optionally, signaling can refer to the process by which the encoding device includes information in a bitstream. Information signaled by the encoding device can be used by the decoding device. In signaling, the bitstream can be transmitted over a network and can be included in a storage / recording medium. In embodiments, a description indicating that information is signaled may include: 1) the encoding device determines and generates information to signal the information; 2) the encoding device generates encoded information by encoding the information; 3) the (encoded) information is sent from the encoding device to the decoding device via a bitstream; 4) the decoding device obtains the information by decoding the encoded information; and 5) the decoding device determines and generates information by signaling the information.

[0204] Encoding devices generate encoded information by encoding data. Encoded information can be transmitted as a signal via a bitstream. Decoding devices retrieve information by decoding the encoded information.

[0205] Signaling information for a specific target can mean that multiple messages are used for that specific target, and that the processing indicated by the multiple messages is applied to that specific target. For example, signaling information at the unit level can mean using / processing corresponding information for each specific unit.

[0206] - Information sent by signaling may include one or more sub-messages. Sending specific information by signaling can mean sending each of the one or more sub-messages included in the specific information by signaling.

[0207] Selective signaling: - The selective transmission of information via signals. Selective signaling of information can refer to the process by which an encoding device selectively includes information in a bitstream (depending on specific conditions). Selective signaling of information can also refer to the process by which a decoding device selectively retrieves information from a bitstream (depending on specific conditions).

[0208] Skipping signal transmission: - Signal transmission of information can be skipped. Skipping signal transmission of information can refer to the process by which the encoding device does not include information in the bitstream (depending on specific conditions). Skipping signal transmission of information can also refer to the process by which the decoding device does not obtain information from the bitstream (depending on specific conditions). In an embodiment, the decoding device can derive the skipped signal transmission information by utilizing other information.

[0209] Symbols: A symbol can refer to at least one piece of information about a target unit, such as the syntax elements, coding parameters, quantization level, and transform coefficients of the target unit or block. Additionally, a symbol can refer to the target of entropy coding or the result of entropy decoding.

[0210] Entropy coding: Entropy coding is a technique that allocates fewer bits to more frequently occurring symbols and more bits to less frequently occurring symbols. Since symbols are represented in this way, the size of the bitstream indicating symbols can be reduced.

[0211] Entropy coding can be implemented using any method such as Variable Length Coding (VLC) and Context-Adaptive Binary Arithmetic Coding (CABAC). For example, entropy coding can be performed using variable-length tables in Variable Length Coding (VLC). In CABAC, for example, binarization methods for symbols and probabilistic models for symbol / binary numbers can be derived to perform entropy coding, and arithmetic coding using context can be performed.

[0212] Entropy decoding: In entropy decoding, the processes performed in entropy encoding can be reversed. Symbols can be generated by performing entropy decoding on a bitstream.

[0213] Parsing: Parsing can refer to determining the value of a syntax element by performing entropy decoding on the encoded information of a bitstream. Alternatively, parsing can refer to entropy decoding itself.

[0214] Statistical values: Values ​​of multiple pieces of information related to a specific entity (multiple entities) described in the embodiments can be used as inputs for specific operations. Statistical values ​​can be values ​​derived by performing specific operations on values ​​related to such a specific entity (multiple entities). For example, statistical values ​​of multiple pieces of specific information can indicate one or more values ​​generated by the average, weighted average (or weighted mean), weighted sum, minimum, maximum, mode, median, interpolated values, sum of products, and product of sums of the values ​​of multiple pieces of specific information. Furthermore, information (such as constants, variables, and encoding parameters) having specific values ​​determined through operations (computations) in the embodiments can have specific statistical values ​​according to the embodiments.

[0215] Encoding parameters In this embodiment, the encoding parameters may be information required for encoding and decoding. The encoding parameters may include information transmitted by signals from the encoding device to the decoding device, information calculated / derived during the encoding process described in this embodiment, and information for processing the encoding described in this embodiment.

[0216] In embodiments, encoding parameters may include one or more of the following: CTU size, cell size, cell type, cell shape, cell depth, minimum cell size, maximum cell size, maximum cell depth, minimum cell depth, cell partitioning information, quadtree (QT) partitioning information, binary tree (BT) partitioning information, BT partitioning direction, BT partitioning format, ternary tree (TT) partitioning information, TT partitioning direction, TT partitioning format, multi-type tree (MTT) partitioning information, combination of MTT partitions, MTT partitioning direction, MTT partitioning format, prediction mode, intra-frame prediction mode, luma intra-frame prediction mode, chroma intra-frame prediction mode, intra-frame partitioning information. Inter-frame partitioning information, coding block partitioning information, prediction block partitioning information, transform block partitioning information, reference sample line index, reference sample filtering method, reference sample filter tap, reference sample filter coefficient, prediction block filtering method, prediction block filter tap, prediction block filter coefficient, prediction block boundary filtering method, prediction block boundary filter tap, prediction block boundary filter coefficient, inter-frame prediction mode, motion information, motion vector (MV), motion vector difference (MVD), MVD resolution, MV value, MV representation accuracy, reference image list, reference image, reference image index, inter-frame prediction direction, inter-frame prediction indicator, prediction list utilization flag, frame order count (POC), MV candidate, MV candidate index, MV candidate list Information on Advanced Motion Vector Prediction (AMVP) mode usage, merge candidates, merge index, merge candidate list, merge mode usage, motion information correction information, skip mode usage, intra-frame block copy mode usage, block vector (BV), block vector difference (BVD), BVD resolution, BV magnitude, BV representation precision, BV candidates, BV candidate index, BV candidate list, interpolation filter taps, interpolation filter coefficients, transform type, transform size, transform selection information, primary transform usage information, secondary transform usage information, primary transform selection information, secondary transform selection information, residual block presence information, coding block style, coding block flag, quantization parameter (QP), increment QP, quantization matrix. Information on deblocking filters, including: deblocking filter usage information, deblocking filter coefficients, deblocking filter taps, deblocking filter strength, deblocking filter shape / form, adaptive sample offset usage information, adaptive sample offset values, adaptive sample offset categories, and adaptive sample offset types. Information on adaptive loop filters, including: adaptive loop filter coefficients, adaptive loop filter taps, and adaptive loop filter shape / form. Binarization / debinarization methods are also included. Context models are defined, including: context model determination methods, context model update methods, normal mode usage information, bypass mode usage information, effective coefficient flags, last effective coefficient flags, coefficient group cell encoding flags, last effective coefficient position, and a flag indicating whether a coefficient value is greater than 1.Flags indicating whether the indicator coefficient value is greater than 2, flags indicating whether the indicator coefficient value is greater than 3, remaining coefficient value information, sign information, context binary number, bypass binary number, reconstructed sample points, reconstructed luminance sample points, reconstructed chrominance sample points, residual sample points, residual luminance sample points, residual chrominance sample points, transform coefficient, luminance transform coefficient, chrominance transform coefficient, transform coefficient level, luminance transform coefficient level, chrominance transform coefficient level, transform coefficient level scanning method, quantization level, quantized luminance level, quantized chrominance level, size of the MV search range on the decoding device side, and M on the decoding device side. The V search range shape, MV search iteration count on the decoding device side, frame type, stripe identification information, stripe type, stripe partition information, parallel block group identification information, parallel block group type, parallel block group partition information, parallel block identification information, parallel block type, parallel block partition information, bit depth, input sample bit depth, reconstructed sample bit depth, residual sample bit depth, transform coefficient bit depth, quantization level bit depth, mapping availability information, luminance signal information, chrominance signal information, target block color space, residual block color space, and temporal layer information.

[0217] In addition, the encoding parameters may also include 1) the value of information that may be included in the encoding parameters, 2) a combination of multiple pieces of information that may be included in the encoding parameters, 3) statistical values ​​of information that may be included in the encoding parameters, 4) information related to the encoding parameters, 5) information used to calculate / derive the encoding parameters, and 6) information calculated / derived using the encoding parameters.

[0218] In an embodiment, "X usage information" can be information indicating whether "X is used / applied / executed". Optionally, "X usage information" can be information indicating whether "X is available". For example, "specific mode usage information" can be information indicating whether a specific mode is used. Mode information can indicate the mode used for the target block among the modes described in the embodiment. In an embodiment, specific mode usage information can be replaced with mode information, and the description of specific mode usage information can also be applied to mode information. "X usage information" and "X indicator" can be used interchangeably.

[0219] In embodiments, encoding parameters and syntax elements may correspond to each other. For example, a syntax element according to an embodiment may be used as an encoding parameter, and the encoding parameter may be sent as a syntax element by a signal.

[0220] In the embodiments, "X existence information" can be regarded as "information indicating whether X exists" or "information indicating whether information indicating X exists in the bitstream".

[0221] In an embodiment, "X selection information" can be information indicating one of the candidates or methods for X. "X selection information" can be considered as an "X index".

[0222] In an embodiment, the partitioning form of a particular tree can indicate one of symmetric partitioning and asymmetric partitioning, and can indicate one of QT, BT, TT, and no partitioning. The partitioning direction of a particular tree can indicate one of horizontal and vertical directions.

[0223] In an embodiment, when the encoding parameter has one of a plurality of values, "encoding parameter" can be replaced with "information indicating whether the encoding parameter has a specific value among a plurality of values ​​that can be used for the encoding parameter".

[0224] In an embodiment, when the encoding parameter has one of multiple values, "encoding parameter" can be replaced with "indicating whether the encoding parameter indicates information of a specific target among multiple targets".

[0225] Systems for video encoding and decoding Figure 1 A system for video encoding and decoding according to an embodiment is shown.

[0226] System 100 may include at least one or a combination of encoding device 110 or decoding device 150.

[0227] Each of the encoding device 110 and the decoding device 150 may be a computer or an electronic device.

[0228] Structure of encoding devices The encoding device 110 may include a processor 120, a memory 140, and a communicator 149.

[0229] The processor 120, memory 140 and communicator 149 can be connected to each other via a bus.

[0230] Processor 120 may be a semiconductor device that executes instructions or computer-executable code, such as a central processing unit (CPU). Processor 120 may be at least one hardware processor.

[0231] In an embodiment, the processor 120 may generate and process information input to or output from the encoding device 110 or used in the encoding device 110, and may perform comparisons, determinations, etc., related to the information.

[0232] Processor 120 may include multiple components. These components may include partitioner 122, subtractor 124, transformer 125, quantizer 126, dequantizer 127, inverse transformer 128, adder 129, filter 130, and entropy encoder 139.

[0233] At least some of the aforementioned components may be program modules. Program modules may be included in the encoding device 110 in the form of operating systems, applications, and other program modules. Program modules may be instructions or computer-executable code stored in memory 140 and executed by processor 120.

[0234] The memory 140 may include various types of volatile and non-volatile storage media. For example, the memory 140 may include memory such as read-only memory (ROM) and random access memory (RAM).

[0235] The memory 140 may store instructions and computer-executable code for the operation of the encoding device 110, and may also store the information and bit streams described in the embodiments. The memory 140 may include a reference screen buffer 141.

[0236] The communicator 149 can perform functions related to communication of information in the encoding device 110. For example, the communicator 149 can send a bit stream to the decoding device 150.

[0237] In the names of components of encoding device 110, "-cell" can be used instead of "-device" or "-device". Memory 140 can be designated as a storage cell.

[0238] Operation of encoding devices Encoding device 110 can sequentially encode one or more images (pictures) of a video.

[0239] The memory 140 can store the original image. The original image can be used as the target image in the encoding device 110.

[0240] The processor 120 can generate a bitstream including encoded information by performing encoding on the target image, and can store the generated bitstream in the memory 140. The generated bitstream can be stored in a computer-readable storage medium and can be transmitted by the communicator 149 to the communicator 189 of the decoding device 150 via a wired and / or wireless transmission medium.

[0241] The partitioner 122 can determine the target block by partitioning the target image.

[0242] Predictor 123 can determine the prediction mode of the target block. Predictor 123 can generate a prediction block for the target block by performing a prediction corresponding to the prediction mode.

[0243] The prediction mode for the target block can be one of the available prediction modes. For example, available prediction modes may include intra-frame prediction, inter-frame prediction, and IBC prediction.

[0244] For example, when the prediction mode corresponds to intra-frame prediction, predictor 123 can generate a prediction block for the target block by performing intra-frame prediction on the target block.

[0245] For example, when the prediction mode corresponds to inter-frame prediction, predictor 123 can generate a prediction block for the target block by performing inter-frame prediction on the target block.

[0246] For example, when the prediction mode corresponds to IBC, predictor 123 can generate a prediction block for the target block by performing IBC on the target block.

[0247] Subtractor 124 can generate a residual block of the target block. The residual block can be the difference between the original block and the predicted block. The original block can be a region indicated by the target block in the original image. Alternatively, the residual block can refer to a block generated by applying one or more of a transformation and quantization to the difference between the original block and the predicted block.

[0248] Transformer 125 can generate transformation coefficients by performing a transformation on the residual block.

[0249] Transformer 125 can perform a transformation using one of a variety of transformation methods.

[0250] For example, various transformation methods can include Discrete Cosine Transform (DCT), Discrete Sine Transform (DST), Karhunen-Loeve Transform (KLT), and transformations based on each transform.

[0251] Transform skip mode can be a mode that uses both the reconstructed residual block and the prediction block without any transform or inverse transform performed to generate the reconstructed block. When transform skip mode is applied to the target block, the transform and inverse transform of the target block can be skipped, and only the quantization and inverse quantization (dequantization) of the target block can be performed.

[0252] Quantizer 126 can generate quantization levels by applying quantization using quantization parameters to transform coefficients. In this embodiment, the quantization level may also be referred to as a "transform coefficient".

[0253] The entropy encoder 139 generates encoded information by performing entropy encoding based on a probability distribution of information used for image decoding. The bitstream may include the encoded information.

[0254] Information used for image decoding may include quantization levels, syntax elements, etc., calculated by quantizer 126.

[0255] The probability distribution can be determined based on the quantization level and encoding parameters.

[0256] The entropy encoder 139 can use a scan to change quantization levels in 2D block form to 1D vector form in order to perform encoding on the quantization levels. During the scan, it can be determined which of the following scans—upper right diagonal scan, vertical scan, and horizontal scan—to use based on encoding parameters such as the block size and the block's intra-frame prediction mode.

[0257] When encoding a target image / block, predictor 123 uses a reference image / block to perform prediction. The encoded target image / block can be used as a reference image / block for another image / block to be processed subsequently. Therefore, processor 120 can perform reconstruction on the encoded target block and can store the reconstructed image, including the reconstructed target block generated by reconstruction, as a reference image in reference image buffer 141. Inverse quantization and inverse transform can be performed on the encoded target block to perform reconstruction.

[0258] The dequantizer 127 can generate dequantized transformation coefficients by performing dequantization on the quantization level.

[0259] The inverse transformer 128 can generate inverse quantization and inverse transform coefficients by performing an inverse transform on the inverse quantization transform coefficients. In embodiments, the inverse quantization and / or inverse transform coefficients may refer to coefficients on which at least one or a combination of inverse quantization and inverse transform has been applied. The inverse quantization and inverse transform coefficients may be reconstructed residual blocks.

[0260] Adder 129 can generate a reconstruction block by adding the prediction block and the reconstruction residual block to each other.

[0261] The reconstructed block can be processed by filter 130. Filter 130 can be configured to apply one or more of a plurality of filters to the target. Each of the plurality of filters can be a loop filter. The target can be a reconstructed sample, a reconstructed block, or a reconstructed image.

[0262] The reference frame buffer 141 can store reconstructed blocks / images provided from the filter 130. The reconstructed image can be an image that includes reconstructed blocks. Alternatively, the reconstructed image can be an image composed of reconstructed blocks.

[0263] The reference frame buffer 141 can provide the stored reconstructed image as a reference image to the predictor 123. From the viewpoint of storing the decoded (i.e., reconstructed) image (frame), the reference frame buffer 141 can also be referred to as the "decoded frame buffer (DPB)".

[0264] Structure of decoding device The decoding device 150 may include a processor 160, a memory 180, and a communicator 189.

[0265] The description of the processor 120, memory 140, and communicator 149 associated with the encoding device 110 can also be applied to the processor 160, memory 180, and communicator 189 associated with the decoding device 150. Repeated descriptions will be omitted here.

[0266] Processor 160 may include multiple components. These components may include entropy decoder 161, partitioner 162, predictor 163, dequantizer 167, inverse transformer 168, adder 169, and filter 170.

[0267] The memory 180 may include a reference screen buffer 181.

[0268] The communicator 189 can perform functions related to communication with information in the decoding device 150. For example, the communicator 189 can receive a bit stream from the encoding device 110.

[0269] In the names of components of the decoding device 150, "-unit" can be used instead of "-device" or "-device". The memory 180 can be designated as a storage unit.

[0270] Operation of decoding devices The communicator 149 of the encoding device 110 can send the bit stream generated by the encoding device 110 to the decoding device 150. Optionally, a computer-readable storage medium storing the bit stream can send the bit stream generated by the encoding device 110 to the decoding device 150.

[0271] The communicator 189 can receive a bit stream from the encoding device 110 via a wired / wireless transmission medium. The received bit stream can be stored in the memory 180.

[0272] Processor 160 may obtain bit streams from memory 180 or computer-readable storage media.

[0273] Bitstreams may include encoded information.

[0274] The entropy decoder 161 can generate information for image decoding by performing entropy decoding on the encoded information of the bitstream based on a probability distribution.

[0275] Information used for image decoding can include quantization levels, syntax elements, etc.

[0276] The entropy decoder 161 can use scanning to transform quantization levels in 1D vector form into 2D block form in order to perform decoding on the quantization levels. During scanning, it can be determined which of the following scans—upper right diagonal scan, vertical scan, and horizontal scan—will be used based on coding parameters such as the block size and the block's intra-frame prediction mode.

[0277] The entropy decoder 161 can provide syntax elements to other components of the processor 160, such as the partitioner 162.

[0278] A common description of the relationship between components of the encoding device and components of the decoding device. Decoding device 150 uses the bitstream generated by encoding device 110 to perform decoding. Encoding device 110 can use the reconstructed image derived in decoding device 150 instead of the original image not provided to decoding device 150 to encode the target block. Therefore, encoding device 110 and decoding device 150 need to be able to generate reconstructed blocks / images in the same way. In this regard, the description of partitioner 122, predictor 123, dequantizer 127, inverse transformer 128, adder 129, filter 130, and reference frame buffer 141 of encoding device 110 disclosed in the embodiments can be applied to partitioner 162, predictor 163, dequantizer 167, inverse transformer 168, adder 169, filter 170, and reference frame buffer 181 of decoding device 150, respectively. Repeated descriptions will be omitted here.

[0279] Furthermore, each of the partitioner 122, predictor 123, dequantizer 127, inverse transformer 128, adder 129, and filter 130 of the encoding device 110 can generate information about the syntax elements for processing a specified target. Furthermore, each of the partitioner 162, predictor 163, dequantizer 167, inverse transformer 168, adder 169, and filter 170 of the decoding device 150 can use the information about the syntax elements to perform processing on the target (the same processing performed by the encoding device 110).

[0280] As described above, corresponding components of the encoding device 110 and the decoding device 150 can perform the same or corresponding functions. In embodiments, the processor may refer to the processor 120 of the encoding device 110 and / or the processor 160 of the decoding device 150. For example, in functions related to prediction, the processor may refer to the predictor 123, the subtractor 124, and the adder 129, and may also refer to the predictor 163 and the adder 169. In functions related to transformation, the processor may refer to the transformer 125 and the inverse transformer 128, and may also refer to the inverse transformer 168. In functions related to quantization, the processor may refer to the quantizer 126 and the dequantizer 127, and may also refer to the dequantizer 167. In functions related to entropy encoding / decoding, the processor may refer to the entropy encoder 139 and / or the entropy decoder 161. In functions related to filtering, the processor may refer to the filter 130 and / or the filter 170. The memory may refer to the memory 140 of the encoding device 110 and / or the memory 180 of the decoding device 150. The reference frame buffer may refer to the reference frame buffer 141 of the encoding device 110 and / or the reference frame buffer 181 of the decoding device 150. The communicator may refer to the communicator 149 of the encoding device 110 and / or the communicator 189 of the decoding device 150.

[0281] Partitions of the units that make up an image Figure 2 An image partitioning structure according to an embodiment is shown.

[0282] Figure 2 An example of a cell being divided into multiple sub-cells can be illustrated.

[0283] A CU can be used as the basic unit for image encoding and decoding. Furthermore, a CU can be used as the basic unit for prediction, transform, quantization, inverse quantization, inverse transform, entropy coding, and entropy decoding.

[0284] A CU can be used as a unit for applying prediction modes. In other words, it can be determined which of the available prediction modes available for each CU will be applied to the encoding. For example, available prediction modes may include intra-frame prediction, inter-frame prediction, and intra-block copy (IBC) prediction.

[0285] The target image 200 can be sequentially partitioned into units of CTUs. For each CTU, a partitioning structure can be determined. A CTU may be partitioned into CUs depending on the partitioning structure. Optionally, a CTU may be used as a CU. The size of a CTU can be the largest CU size.

[0286] Each CU can have depth information. Depth information can refer to the depth of the CU and can also represent the size of the CU. The depth of a CTU can be 0. The depth of a CU generated by partitioning a CTU can be 1. When a parent CU is partitioned into child CUs, the depth of the child CU can be increased by 1 from the depth of the parent CU. The number of CUs generated by partitioning can be a positive integer equal to or greater than 2, including 2, 4, 8, 16, etc. Depending on the number of child CUs, at least one of the horizontal and vertical dimensions of the child CUs generated by partitioning the parent CU can be less than at least one of the horizontal and vertical dimensions of the parent CU.

[0287] The CUs can be recursively partitioned in the same way until a predefined maximum depth or a predefined minimum size is reached. The depth of the minimum coding unit (SCU) can be the predefined maximum depth, and the size of the SCU can be the predefined minimum size. The size of the SCU can be the minimum CU size.

[0288] For example, the range of CU depths can correspond to values ​​from 0 to 3. Depending on the CU depth, the CU can have dimensions ranging from 64×64 to 8×8. A CTU with a depth of 0 can be a 64×64 block. 0 can be the minimum depth. An SCU with a depth of 3 can be an 8×8 block. 3 can be the maximum depth. Depth 0 can represent a CTU as a 64×64 block. Depth 1 can represent a CU as a 32×32 block. Depth 2 can represent a CU as a 16×16 block. Depth 3 can represent an SCU as an 8×8 block.

[0289] The partition information of a CU indicates whether the CU has been partitioned. The partition information can be a 1-bit flag. All CUs except SCUs can include partition information. For example, the partition information of a CU that is not further partitioned can be "0" as the first value, while the partition information of a partitioned CU can be "1" as the second value.

[0290] Quadtree (QT) partitioning refers to dividing a parent CU into four child CUs. When a parent CU is partitioned into four child CUs, the horizontal and vertical dimensions of each child CU can be half the horizontal and vertical dimensions of the parent CU, respectively.

[0291] Binary tree (BT) partitioning refers to dividing a CU into two CUs. For example, when a parent CU is partitioned into two child CUs, the horizontal or vertical dimension of each child CU can be half the horizontal or vertical dimension of the parent CU.

[0292] Ternary tree (TT) partitioning refers to dividing a single CU (Computer Unit) into three CUs. For example, when a parent CU is partitioned into three child CUs, the three child CUs can be generated by partitioning the parent CU's horizontal or vertical dimensions in a 1:2:1 ratio. The horizontal or vertical dimensions of the child CUs can be 1 / 4, 1 / 2, and 1 / 4 of the parent CU's horizontal or vertical dimensions, respectively.

[0293] exist Figure 2 In this configuration, the QT partition is applied to the first CTU. The QT partition, BT partition, and TT partition are applied to the second CTU.

[0294] To partition a CTU, at least one of the following different partition types can be applied to the CTU: QT partition, BT partition, and TT partition. Different partition types can be applied based on specific priorities.

[0295] For example, QT partitions can be preferentially applied to CTUs. CUs that cannot be further partitioned with QT partitions can correspond to leaf nodes of QT. A CU that is a leaf node of QT can be the root node of a BT and / or TT. A CU that is a leaf node of QT can be partitioned in BT or TT form, or it can be left unpartitioned. In this case, the QT partitions may not be applied again to CUs generated by applying BT or TT partitions to CUs that are leaf nodes of QT.

[0296] QT partitioning information can be used to signal the partitioning of the CU corresponding to each node of the QT. QT partitioning information can be a flag. The QT partitioning information of a cell can indicate whether the cell is partitioned in QT format. A "0" as the first value of the QT partitioning information indicates that the CU is not partitioned in QT format. QT partitioning information with a first value can refer to Multi-Type Tree (MTT) partitioning. MTT partitioning can include BT partitioning and TT partitioning. A "1" as the second value of the QT partitioning information indicates that the CU is partitioned in QT format.

[0297] There may be no priority between BT and TT partitions. That is, CUs corresponding to leaf nodes of QT can be partitioned in either BT or TT format. Furthermore, CUs generated by BT or TT partitions can be further partitioned in either BT or TT format, or they may not be further partitioned.

[0298] The CU corresponding to a leaf node of QT can be the root node of MTT. For each CU corresponding to a node of MTT, the CU may also include partition direction information and partition format information in MTT format.

[0299] Partition direction information indicates the partitioning direction of an MTT partition. A "0" as the first value of the partition direction information indicates that the CU is partitioned horizontally. A "1" as the second value of the partition direction information indicates that the CU is partitioned vertically.

[0300] Partition format information indicates the partition format used for Multi-Type Tree (MTT) partitioning. A "0" as the first value of the partition format information indicates that the CU is partitioned in TT format. A "1" as the second value of the partition format information indicates that the CU is partitioned in BT format.

[0301] Here, each of the partition direction information and partition format information mentioned above can be a flag with a specific length (e.g., 1 bit).

[0302] The partition information of a CU may also include QT partition information, partition direction information, and partition format information.

[0303] A CU that has not been further partitioned through QT, BT, or TT partitioning can be used as a unit for specific processes such as prediction, transform, quantization, inverse quantization, inverse transform, entropy coding, and entropy decoding. In other words, a CU may not be further partitioned for a specific process. Therefore, the partitioning information used to partition a CU into PUs and / or TUs may not exist in the bitstream.

[0304] Conversely, when the size of the CU is larger than the maximum TU size, the CU can be recursively partitioned until the size of the CU becomes smaller than or equal to the maximum TU size. For example, when the size of the CU is 64×64 and the maximum TU size is 32×32, the CU can be partitioned into four 32×32 TUs to perform the transformation. Similarly, when the size of the CU is 32×64 and the maximum TU size is 32×32, the CU can be partitioned into two 32×32 TUs to perform the transformation.

[0305] In this scenario, it's not necessary to separately send a signal indicating whether the CU has been partitioned for transformation. Instead, partitioning can be determined by comparing the CU's dimensions (horizontal / vertical) with the maximum TU's dimensions (horizontal / vertical). For example, if the CU's horizontal dimension is greater than the maximum TU's horizontal dimension, the CU can be vertically bisected. Similarly, if the CU's vertical dimension is greater than the maximum TU's vertical dimension, the CU can be horizontally bisected.

[0306] For example, the minimum size of a CU can be 4×4. For example, the maximum size of a transform block can be 64×64. For example, the minimum size of a transform block can be 4×4. The minimum size of a QT can be the minimum size of the CU corresponding to a leaf node of the QT. The maximum depth of an MTT can be the maximum depth of the path from the root node to a leaf node of the MTT.

[0307] The maximum size of BT can represent the maximum size of the CU corresponding to each node of BT, and the maximum size of TT can represent the maximum size of the CU corresponding to each node of TT. The minimum size of BT and / or the minimum size of TT can be set as the minimum size of the CU.

[0308] When the depth of the CU corresponding to a node in the MTT is equal to the maximum depth of the MTT, the CU may not be partitioned in BT form and / or TT form.

[0309] Based on the various sizes and depths of the CU described above, each piece of information described in the embodiments may or may not exist in the bitstream.

[0310] Information about the maximum or minimum size described in the embodiments can be transmitted via signals at a level higher than the CU. In the embodiments, levels higher than the CU may include video level, sequence level, picture level, sub-picture level, parallel block group level, parallel block level, stripe level, etc.

[0311] The information described in the embodiments can be transmitted separately using signals for different types of stripes. Different types of stripes may include intra-frame stripes and inter-frame stripes.

[0312] Block processing depends on block attributes Whether to apply / execute a specific process described in the embodiments can be determined based on the attributes of a block associated with a specific process. Whether to apply / execute a specific process described in the embodiments can be determined based on whether the attributes of a block associated with a specific process satisfy specific conditions. For example, a block may include a target block, neighboring blocks, and reference blocks. The block may include other blocks described in the embodiments. A block may be one of the blocks and units described in the embodiments.

[0313] The blocks for a particular process described in the application examples may have a square shape or a non-square shape.

[0314] In this embodiment, the attributes of a block may include its size. When specific conditions related to the block size are met, specific processes described in this embodiment may be applied / performed.

[0315] In an embodiment, specific conditions may include a minimum block size condition and a maximum block size condition. The block applying the minimum block size condition and the block applying the maximum block size condition may be different from each other.

[0316] In the embodiments, the minimum block size and / or maximum block size for a specific process can be predefined.

[0317] In an embodiment, the processing according to the embodiment may be applied / executed when the block size is equal to or greater than the minimum block size and / or when the block size is less than or equal to the maximum block size. Optionally, in an embodiment, the processing according to the embodiment may be applied / executed when the block size is greater than the minimum block size and when the block size is less than the maximum block size.

[0318] In this embodiment, the processing according to the embodiment may be applied / executed only when the block size is equal to or greater than the minimum block size and less than or equal to the maximum block size. Alternatively, the processing according to the embodiment may be applied / executed only when the block size is greater than the minimum block size and less than or equal to the maximum block size. Alternatively, the processing according to the embodiment may be applied / executed only when the block size is equal to or greater than the minimum block size and less than the maximum block size. The processing according to the embodiment may be applied / executed only when the block size is greater than the minimum block size and less than the maximum block size.

[0319] In this embodiment, the processing according to the embodiment may be applied / executed only if the block size is a predefined block size.

[0320] In embodiments, block size can be determined based on various methods. For example, block size can represent the horizontal or vertical dimension of the block. Block size can represent both the horizontal and vertical dimensions of the block. Block size can represent the area of ​​the block. Block size can represent 1) the result of a known equation, 2) the result of an equation according to the embodiment, or 3) a statistical value using the horizontal and vertical dimensions of the block.

[0321] Furthermore, for the first size, the processing according to the first embodiment can be applied / executed, and for the second size, the processing according to the second embodiment can be applied / executed.

[0322] In embodiments, the block size can be 2×2, 4×4, 8×8, 16×16, 32×32, 64×64, or 128×128. Optionally, in embodiments, the block size can be (2 SIZE X )×(2 SIZE Y (or similar.) SIZE X It can be one of 1 or a larger integer. SIZE Y It can be one of 1 or a larger integer.

[0323] Predictive information used for prediction Prediction information can be used to generate prediction blocks for the target block.

[0324] Encoding device 110 can generate prediction information required for prediction and can generate a bit stream including the prediction information. The prediction information can be transmitted from encoding device 110 to decoding device 150 via the bit stream. Decoding device 150 can obtain the prediction information from the bit stream and can generate a prediction block by performing prediction on the target block using the prediction information.

[0325] The prediction information may include intra-frame prediction information, inter-frame prediction information, and IBC prediction information. In embodiments, the prediction information may be replaced by intra-frame prediction information, inter-frame prediction information, and / or IBC prediction information. Intra-frame prediction may include the information for intra-frame prediction described in the embodiments. Inter-frame prediction may include the information for inter-frame prediction described in the embodiments. IBC information may include the information for IBC prediction described in the embodiments.

[0326] Intra-frame prediction Figure 3 The structure of intra-frame prediction according to an embodiment is shown.

[0327] Intra-frame prediction can be performed using reference samples and coding parameters for the target block. The reference samples can be (reconstructed) samples within a (reconstructed) reference block. Optionally, intermediate prediction samples can be generated using samples such as reconstructed samples described in the embodiments, and then the intermediate prediction samples can be used to generate reference samples. Processes such as filtering described in the embodiments can be applied when generating reference samples.

[0328] The reference block can be a (spatial) neighboring block of the target block. The coding parameters can be the coding parameters used for the target block and / or the coding parameters used for the reference block. In intra-frame prediction, the reference sample can be a neighboring sample.

[0329] Predicted blocks can be generated by performing intra-frame prediction on the target block based on reference points of the target image and information related to those reference points, according to an intra-frame prediction mode. The size of the target block and the size of the predicted block can be equal.

[0330] In this embodiment, the prediction block may be a PU. Alternatively, the prediction block may correspond to the CU or TU described in the embodiment. The prediction block may have a square or rectangular shape.

[0331] Intra-frame prediction modes can be represented by at least one of mode number, mode value, mode angle, and mode direction. Figure 3The lower right portion shows the prediction directions for multiple intra prediction modes used for the target block. Among these intra prediction modes, all except DC mode and planar mode can be directional modes. A directional mode can be an intra prediction mode with a specific direction or angle. The intra prediction mode for the target block can be selected from directional modes and non-directional modes.

[0332] In the rectangle indicating the target block in the lower right part of the figure, the number "0" can indicate a planar mode as a non-directional intra-prediction mode. The number "1" can indicate a DC mode as a non-directional intra-prediction mode. In the rectangle indicating the target block in the lower right part of the figure, the arrow from the center of the rectangle to the outside indicates the prediction direction of the directional intra-prediction mode. Furthermore, the numbers indicated near the arrows can represent examples of mode values ​​assigned to the intra-prediction mode or the prediction direction of the intra-prediction mode.

[0333] Intra-prediction can be performed based on the intra-prediction mode used for the target block. One of the intra-prediction modes available for the target block can be used as the intra-prediction mode for the target block.

[0334] The number of intra-prediction modes available for the target block can be a predefined value. Alternatively, the number of intra-prediction modes available for the target block can be determined based on the attributes of the prediction block. For example, the attributes of the prediction block can include coding parameters such as shape, size, and color components.

[0335] For example, in Figure 3 In the image, the directional patterns indicated by the dashed lines (i.e., directional patterns with numbers corresponding to any one of -14 to -1 or any one of 67 to 80) can be applied only for predictions of non-square blocks. Therefore, the number of intra-frame prediction patterns available for predictions of square blocks can be 67 (planar mode, DC mode, 65 directional patterns).

[0336] For example, the number of available intra-prediction modes can vary depending on which of the luma or chroma signals corresponds to the color component of the block. The number of available intra-prediction modes for the luma component block can be greater than the number of available intra-prediction modes for the chroma component block.

[0337] Intra-frame prediction modes can include horizontally-below mode, horizontal mode, vertical mode, and vertically-right mode. Horizontally-below mode can be an intra-frame prediction mode located below the horizontal mode. Vertically-right mode can be a mode located to the right of the vertical mode. For example, in... Figure 3In the above, the mode value for the horizontal mode can be 18. The mode value for the vertical mode can be 50. Intra-prediction modes (each of which has a mode value of 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, and 66) can be vertical right-side modes. Intra-prediction modes with a mode value of 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, and 17 can be horizontal bottom-side modes.

[0338] The number of intra-prediction modes and the mode number of each intra-prediction mode described above are merely examples. The number of intra-prediction modes and the mode number of each intra-prediction mode may be defined differently depending on the embodiments, their implementation methods, and / or necessity.

[0339] When generating a prediction block for a target block in the intra-frame prediction mode of planar mode, the sample value of the prediction sample can be generated by using the weighted sum of the upper reference sample of the target sample, the left reference sample of the target sample, the upper right reference sample of the target block, and the lower left reference sample of the target block, based on the position of the prediction sample in the prediction block.

[0340] When the intra-frame prediction mode is DC mode, a prediction block can be generated based on the average of the sample values ​​of multiple reference samples. These reference samples can include the top reference sample and the left reference sample of the target block. The values ​​of the prediction samples in the prediction block can be determined based on the average of the sample values ​​of the multiple reference samples. Furthermore, filtering using the values ​​of the reference samples can be performed on specific rows and / or specific columns in the target block. A specific row can be one or more upper rows adjacent to the upper reference sample. A specific column can be one or more left columns adjacent to the left reference sample.

[0341] When the intra-frame prediction mode is directional mode, the prediction block can be generated using the top reference sample, left reference sample, top right reference sample, and / or bottom left reference sample of the target block.

[0342] The intra-prediction mode of the target block can be determined using the intra-prediction modes of its neighboring blocks. Information for determining the intra-prediction mode of the target block can be transmitted using signals.

[0343] For example, when the intra-prediction mode of the target block and the intra-prediction mode of the neighboring blocks are the same, an indicator indicating that the intra-prediction mode of the target block and the intra-prediction mode of the neighboring blocks are the same can be sent by signaling.

[0344] For example, an indicator can be sent to indicate an intra-prediction mode that is the same as the intra-prediction mode of the target block among the intra-prediction modes of multiple neighboring blocks.

[0345] For example, when the intra prediction modes of the target block and neighboring blocks are different from each other, an indicator indicating the intra prediction mode of the target block can be sent using signals. Alternatively, information for deriving the intra prediction mode of the target block based on the intra prediction modes of neighboring blocks can be sent using signals.

[0346] Reference samples used for intra-frame prediction of the target block may include lower left reference samples, left side reference samples, upper left reference samples, upper top reference samples, upper right reference samples, etc.

[0347] For example, a left reference sample can be a reconstructed reference sample adjacent to the left side of the target block. A top reference sample can be a reconstructed reference sample adjacent to the top of the target block. A top-left reference sample can be a reconstructed reference sample adjacent to the top-left diagonal of the target block. A bottom-left reference sample can be a sample located below a left reference sample on the same line as the left-side reference sample line formed by the left reference samples. A top-right reference sample can be a sample located to the right of an upper reference sample on the same line as the upper-side reference sample line formed by the upper reference samples.

[0348] Reference samples for intra-prediction of the target block can be determined based on the intra-prediction mode of the target block. One or more reference samples can be used to determine the sample values ​​for the prediction samples of the prediction block. Figure 3 In the diagram, the direction of the intra-prediction mode, indicated by the arrow, can indicate the direction from the predicted sample to each reference sample. The direction of the intra-prediction mode can represent the dependency between the reference sample and the predicted sample. For example, depending on the intra-prediction mode, the sample value of a particular reference sample can be used as the sample value of at least one sample in the prediction block. Here, the particular reference sample and at least one sample in the prediction block can be samples that are specified as a straight line in the direction of the corresponding intra-prediction mode. In other words, the sample value of the particular reference sample can be copied to the sample value of the predicted sample located in the direction opposite to the direction of the intra-prediction mode. Optionally, the sample value of the predicted sample in the prediction block can be the sample value of the reference sample located in the direction of the intra-prediction mode based on the position of the predicted sample.

[0349] Reference points used for intra-frame prediction are not limited to those points that are exactly adjacent to the target block. For example... Figure 3 As shown, for intra-frame prediction of the target block, at least one or a combination of reference sample lines 0 to 3 can be used.

[0350] Figure 3Each reference sample line can include one or more reference samples. When the reference sample line number is smaller, it indicates a line closer to the target block. Reference sample line 0 can be a line with reference samples that are exactly adjacent to the target block. When the top-left coordinate of the target block is (X, Y), the horizontal length of the target block is W, and the vertical length of the target block is H, the reference sample on reference sample line 0 can be a sample with x-coordinate X-1 or y-coordinate Y-1. Here, the y-coordinate of a reference sample with x-coordinate X-1 can be Y-1 to Y+2H. The x-coordinate of a reference sample with y-coordinate Y-1 can be X-1 to X+2W. Reference samples in reference sample line A can be samples with x-coordinate XA-1 or y-coordinate YA-1. Here, the y-coordinate of a reference sample with x-coordinate XA-1 can be YA-1 to Y+2H+A. The x-coordinate of a reference sample with y-coordinate YA-1 can be XA-1 to X+2W+A. “A” can be 1, 2 or 3.

[0351] Samples in fragments A and F can be derived by filling with the samples closest to fragments B and E, respectively, instead of being obtained from reconstructed neighboring blocks.

[0352] A reference sample line index can indicate the reference sample line among multiple reference sample lines used for intra-frame prediction of the target block. For example, a reference sample line index can have a value corresponding to one of 0 to 3. The reference sample line index can be transmitted by signal.

[0353] When inter-color component intra-frame prediction is used for a target block, a predicted block for the second color component can be generated based on the reconstructed block of the first color component of the target block. For example, the first color component can be the luma component, and the second color component can be the chroma component.

[0354] For intra-frame prediction between color components, parameters between the first and second color components can be derived based on a template. For example, the parameters can be those of a linear model.

[0355] For example, the template may include an upper reference sample and / or a left reference sample of the target block, and may include an upper reference sample and / or a left reference sample of the reconstructed block of the first color component corresponding to the reference sample.

[0356] When parameters are exported, a predicted block for the second color component can be generated for the target block by applying the reconstructed block of the first color component to the linear model. Depending on the image format or the type of inter-color component intra-frame prediction, subsampling / downsampling can be performed on the neighboring samples of the reconstructed block of the first color component and the reconstructed block of the first color component. When subsampling is performed, the corresponding samples exported through subsampling can be used to perform parameter export and inter-color component intra-frame prediction.

[0357] Intra-fragmented sub-partitioning (ISP) prediction refers to the sequential execution of intra-fragmentation over multiple sub-blocks generated by partitioning a target block. In ISP prediction, the target block can be partitioned into two or four sub-blocks in the horizontal and / or vertical directions. The sub-blocks generated from the partitions can be reconstructed sequentially. When intra-fragmentation is performed on each sub-block, a sub-prediction block can be generated for each sub-block. Furthermore, when inverse quantization and / or inverse transform is performed on each sub-block, a sub-residual block can be generated for the corresponding sub-block. The corresponding reconstructed sub-block can be generated by adding the sub-prediction block to the sub-residual block. The reconstructed sub-block can be used as reference samples for intra-fragmentation of other sub-blocks that will be processed subsequently.

[0358] When performing prediction for a target block, it can be determined whether samples included in the reconstructed neighboring blocks can be used as reference samples for the target block. When there are unusable samples among the samples in the neighboring blocks that cannot be used as reference samples for the target block, the sample value of each unusable sample can be replaced by a value generated based on the copying and / or interpolation of the sample value using at least one sample included in the reconstructed neighboring blocks. When the sample value of a sample is replaced by a value generated by copying and / or interpolation, the corresponding sample can be used as a reference sample for the target block.

[0359] When performing intra-frame prediction, the sample values ​​of prediction samples in the prediction block can be determined based on the sample values ​​of reference samples. The position of the reference sample can be specified by the position of the prediction sample and the direction of the intra-frame prediction mode. When the position specified by the position of the prediction sample and the direction of the intra-frame prediction mode is an integer position, the sample value of a reference sample indicated by the integer position can be used to determine the sample value of the prediction sample in the prediction block. When the position specified by the position of the prediction sample and the direction of the intra-frame prediction mode is not an integer position, an interpolated reference sample can be generated based on the two reference samples closest to the specified position. The sample value of the interpolated reference sample can be used to determine the sample value of the prediction sample. In other words, when the position specified by the position of the prediction sample and the direction of the intra-frame prediction mode indicates the position between two reference samples, an interpolated sample value can be generated based on the sample values ​​of the two samples.

[0360] Inter-frame prediction Figure 4 The structure of inter-frame prediction for explaining inter-frame prediction processing according to an embodiment is shown.

[0361] Figure 4 The rectangle shown can represent an image (screen). Furthermore, in Figure 4 In the image, the arrow can specify the prediction direction.

[0362] Based on the encoding type, the individual images that make up a video can be classified into I-frames (i.e., intra-frame frames), P-frames (i.e., one-way predictive frames), and B-frames (i.e., two-way predictive frames). Encoding and decoding of each frame can be performed according to its encoding and decoding type.

[0363] When the target frame is an I-frame, information from the target frame can be used to perform encoding and decoding of the target frame without referencing inter-frame prediction of other images. For example, intra-frame prediction and / or IBC prediction can be used to perform encoding and decoding of the I-frame.

[0364] Encoding and decoding of P-frames and B-frames can be performed through at least one of intra-frame prediction, IBC prediction, and inter-frame prediction using a reference image.

[0365] When the target frame is a P-frame, one-way inter-frame prediction using a list of reference images can be used to perform encoding and decoding of the target frame.

[0366] When the target frame is a B-frame, the encoding and decoding of the target frame can be performed using one-way inter-frame prediction or two-way inter-frame prediction with two reference image lists.

[0367] In the following, inter-frame prediction for a target block in inter-frame mode according to an embodiment will be described in detail.

[0368] When the prediction mode of the target block is inter-frame mode, inter-frame prediction can be performed on the target block. The target block can be a prediction block or a partition prediction block.

[0369] Inter-frame prediction can be performed using reference images and motion information. In inter-frame prediction, a reference image index can be used to select a reference image (reference frame), and motion information can be used to determine a reference block within the reference image corresponding to the target block. The determined reference block can then be used to generate a predicted block for the target block.

[0370] Motion information can be derived using encoding parameters, etc. For example, motion information can be derived using motion information from reconstructed neighboring blocks, motion information from COL blocks, and / or motion information from blocks adjacent to COL blocks.

[0371] In this embodiment, the candidate list can be used for inter-frame prediction. The candidate list may include multiple candidates. An index indicating the candidate in the candidate list used for inter-frame prediction of the target block can be transmitted by signaling. The candidate list can be derived by the encoding device 110 and the decoding device 150 in the same manner based on the same information. Here, the same information may include the reconstructed image and the reconstructed block. Furthermore, in order to execute the candidate selection using the index, the order of the candidates in the candidate list needs to be consistent.

[0372] In this embodiment, prediction for a target block can be performed by utilizing the motion information of spatial candidates or temporal candidates as the motion information of the target block. The motion information of spatial candidates can be referred to as spatial motion information. The motion information of temporal candidates can be referred to as temporal motion information.

[0373] Spatial candidates can be reconstructed spatial neighbor blocks that are spatially adjacent to the target block.

[0374] Spatial candidates can be 1) blocks that exist in the target image, 2) blocks that have been reconstructed by decoding, and 3) blocks that are adjacent to the target block.

[0375] Spatial candidates can include the left block, top block, bottom left block, top right block, and top left block of the target block.

[0376] Temporal candidates can be reconstructed time neighbors that exist in the reconstructed COL image (picture) and correspond to the target block.

[0377] In this embodiment, the motion information of spatial candidates can be the motion information of blocks including spatial candidates. The motion information of temporal candidates can be the motion information of blocks including temporal candidates.

[0378] In inter-frame prediction, COL blocks can be identified for the target block. The region of the target block in the target image and the region of the COL block in the COL frame can be the same. In other words, a COL block can be a block that occupies a specific area in the COL frame. The specific area can be the region in the COL frame that corresponds to the region of the target block.

[0379] The time candidate can indicate the position inside the COL block within the COL screen and / or the position outside the COL block.

[0380] For example, a COL block can include a first COL block and a second COL block. The top-left coordinate of the COL block is (xP, yP) and the size of the COL block is (nPSW, nPSH). The first COL block can be the block occupying coordinates (xP+nPSW, yP+nPSH). The second COL block can be the block occupying coordinates (xP+(nPSW>>1), yP+(nPSH>>1)). When the first COL block is unavailable, the second COL block can be selectively used as the COL block.

[0381] The MV of the target block can be determined based on the MV of the COL block. Scaling can be performed on the MV of the COL block. The scaled MV of the COL block can be used as the MV of the target block or the predicted MV. Optionally, the MVs of temporal candidates listed in the candidate list related to inter-frame prediction can be scaled MVs.

[0382] The ratio of the scaled MV to the COL block's MV can be the same as the ratio of the first time distance to the second time distance. The first time distance can be the distance between the reference image of the target block and the target image. The second time distance can be the distance between the reference image of the COL block and the COL image.

[0383] The scheme for deriving motion information can be determined based on the inter-frame prediction mode of the target block. For example, AMVP mode, merge mode, skip mode, merge mode with MVD, sub-block merge mode, GPM, combined inter-intra-frame prediction (CIIP) mode, or affine inter-frame prediction mode can be used as the inter-frame prediction mode. The various inter-frame prediction modes will be described in the following embodiments.

[0384] AMVP mode When the AMVP pattern is used as the prediction pattern, a list of MV candidates, including one or more MV candidates, can be created using spatial MV candidates, temporal MV candidates, history-based MV candidates, zero vectors, etc. At least one of spatial MV candidates, temporal MV candidates, zero vectors, etc., can be determined and used as an MV candidate.

[0385] Spatial candidates can include reconstructed spatial neighbor blocks. The motion vectors (MVs) of the reconstructed spatial neighbor blocks can be referred to as spatial motion vector (MV) candidates. Temporal candidates can include COL blocks and blocks adjacent to COL blocks. The MV of a COL block or the MV of a block adjacent to a COL block can be referred to as temporal motion vector (MV) candidates. History-based MV candidates can be MVs included in a list of MVs of other blocks that were encoded / decoded before the target block was encoded / decoded.

[0386] Encoding device 110 can use an MV candidate list to determine the MV to be used for encoding the target block within a search range. The maximum number of MV candidates in the MV candidate list can be predefined. N can represent the predefined maximum number. For example, N can be 2. Optionally, the maximum number of candidates can be signaled from the encoding device to the decoding device, or it can be derived by the decoding device. Encoding device 110 can determine MV candidates from the MV candidates in the MV candidate list that will be used as the predicted MV for the target block. The MV to be used for encoding the target block can be the MV that can be encoded with minimal cost. Encoding device 110 can determine whether an AMVP pattern will be used to encode the target block and can generate AMVP pattern usage information indicating whether an AMVP pattern is used.

[0387] Inter-frame prediction information may include 1) AMVP mode usage information, 2) MV candidate index, 3) MVD, 4) MVD resolution information, 5) reference orientation, and 6) reference image index, and may also include residual blocks. The inter-frame prediction information can be transmitted from the encoding device 110 to the decoding device 150 via a bitstream.

[0388] Decoding device 150 can obtain AMVP mode usage information from the bitstream. When the AMVP mode usage information indicates that the AMVP mode is used, decoding device 150 can obtain MV candidate index, MVD, MVD resolution information, reference orientation, and reference image index from the bitstream. Among the MV candidates included in the MV candidate list, the MV candidate indicated by the MV candidate index can be selected as the predicted MV of the target block.

[0389] MVD can represent the difference between the MV actually used for inter-frame prediction of the target block and the predicted MV. Encoding device 110 can derive a predicted MV that is closer to the MV actually used for inter-frame prediction of the target block in order to use an MVD with the smallest possible value. Decoding device 150 can derive the MV of the target block by adding the MVD to the predicted MV. In other words, the MV of the target block derived by decoding device 150 can be the sum of the MVD and the predicted MV candidate.

[0390] Furthermore, the encoding device 110 can generate MVD resolution information. The MVD resolution information can be used to adjust the MVD resolution. The decoding device 150 can use the MVD resolution information to adjust the MVD resolution.

[0391] Furthermore, the encoding device 110 can calculate the MVD based on an affine model. The affine control points MV of the target block can be derived based on the sum of the affine control point MV candidates and the MVD. The MV of each sub-block in the target block can be derived using the affine control points MV.

[0392] Merge mode When using the merge mode, motion information from spatial candidates, motion information from temporal candidates, etc., can be used to create a merge candidate list that includes multiple merge candidates. Motion information can include 1) motion vector (MV), 2) reference image index, 3) reference orientation, etc. Each merge candidate can be a piece of motion information.

[0393] Merge candidates may include 1) spatial merge candidates generated based on spatial candidates, 2) temporal merge candidates generated based on temporal candidates, 3) historical merge candidates, 4) average merge candidates, and 5) zero merge candidates.

[0394] Historically based merge candidates can include information from multiple motion information of other blocks that were encoded / decoded before the target block was encoded / decoded.

[0395] The average merge candidate can be a merge candidate generated based on the average of two merge candidates in the merge candidate list.

[0396] Zero-vector motion information can be zero-vector motion information. Zero-vector motion information can be motion information where MV is a zero-vector.

[0397] Merging candidates can be added to the merging candidate list according to a predefined method and a predefined order, so that the merging candidate list has a preset number of merging candidates. The encoding device 110 and the decoding device 150 can construct the same merging candidate list by using the predefined method and predefined order.

[0398] Encoding device 110 can select a merge candidate from a merge candidate list to be used for encoding the target block. Encoding device 110 can determine whether a merge mode will be used to encode the target block and can generate merge mode usage information indicating whether a merge mode is used.

[0399] Inter-frame prediction information may include 1) merging mode usage information, 2) merging index, and 3) correction information, and may include residual blocks. The inter-frame prediction information can be transmitted from the encoding device 110 to the decoding device 150 as a bit stream.

[0400] Decoding device 150 can obtain merge mode usage information from the bitstream. When the merge mode usage information indicates that a merge mode is being used, decoding device 150 can obtain information related to the merge mode from the bitstream, such as the merge index.

[0401] Encoding device 110 can select the best merge candidate from merge candidates included in the merge candidate list, and can set the value of the merge index to indicate the selected merge candidate.

[0402] The correction information can be used to correct the MV. Encoding device 110 can generate the correction information. Decoding device 150 can derive the corrected MV by correcting the MV of the merge candidate selected by the merge index based on the correction information. The corrected MV can be used as the MV of the target block.

[0403] In an embodiment, the correction information may include MVD (Modular Value Determination). The correction information may include one or more of correction usage information, correction direction information, and correction size information. The correction usage information may indicate whether correction of the MV is used. A merging mode that performs MV correction based on the correction information may be referred to as a merging mode with MVD.

[0404] In merge mode, predictions for the target block can be performed using merge candidates indicated by the merge index from among the merge candidates included in the merge candidate list.

[0405] The motion information of the target block can be derived using 1) MV, 2) reference image index, 3) reference direction, etc., of the merge candidates indicated by the merge index.

[0406] In an embodiment, the merge candidates in the merge candidate list can be specific modes for deriving inter-frame prediction information. Each merge candidate can be information indicating a specific mode for deriving inter-frame prediction information. Inter-frame prediction information for a target block can be derived based on the specific mode indicated by the merge candidate. In this regard, a specific mode can be considered a specific inter-frame prediction information deriving mode or a specific motion information deriving mode. A specific mode can include a series of processes for deriving inter-frame prediction information.

[0407] Inter-frame prediction information for the target block can be derived based on a specific mode indicated by the merge candidate selected by the merge index from the merge candidate list. For example, the specific mode may include a sub-block motion information derivation mode and an affine motion information derivation mode, and may also include other modes for deriving motion information as described in the embodiments.

[0408] Skip mode can be a mode that does not use residual blocks. In other words, when using skip mode, the reconstructed block can be the same as the predicted block. The description of merge mode in the embodiments can also be applied to skip mode. The difference between merge mode and skip mode lies in whether residual blocks are sent and used. In other words, except for not sending / using residual blocks, skip mode can be similar to merge mode, and the description of merge mode can also be applied to skip mode.

[0409] Sub-block merging mode can be a mode for deriving motion information of target sub-blocks within a target block. When applying sub-block merging mode, a list of sub-block merging candidates can be created using affine control point motion vector merging candidates and / or sub-block-based temporal merging candidates. Sub-block-based temporal merging candidates can be the motion information of the COL sub-blocks of the target sub-block.

[0410] In GPM, two motion information points of the target block can be used to generate a first prediction block and a second prediction block. For each coordinate of the target block, the weighted sum of the first prediction sample points of the first prediction block and the second prediction sample points of the second prediction block can be used to generate the final prediction block of the final prediction block.

[0411] Here, the first weight of the first predicted sample and the second weight of the second predicted sample can be determined based on the boundaries of the GPM. The boundaries can indicate the partition lines used to partition the target block. According to the boundaries, the target block can be partitioned into a first partition region and a second partition region.

[0412] When the distance between the final predicted sample point and the boundary is less than or equal to the reference value, the value of the final predicted sample point in the final prediction block can be determined by the weighted sum of the first predicted sample point in the first prediction block and the second predicted sample point in the second prediction block. When the distance between the final predicted sample point and the boundary is greater than the reference value, one of the first weight and the second weight can be 1, and the other can be 0.

[0413] The Combined Inter-Frame Intra-Frame Prediction (CIIP) mode can be a mode that uses a weighted sum of prediction samples generated by inter-frame prediction and prediction samples generated by intra-frame prediction to derive prediction samples for the target block.

[0414] In the above mode, improvements can be made to the derived motion information itself, and the improved motion information can be used as the motion information for the target block. For example, blocks within a specific region determined based on the derived motion information can be searched, and the motion information of the block with the minimum sum of absolute differences (SAD) among the found blocks can be used as the improved motion information for the target block. The specific region can be a square region within a reference image specified by the motion information. The point indicated by the motion information can be the center of the specific region.

[0415] In the aforementioned mode, optical flow can be used to perform compensation for the predicted samples derived through inter-frame prediction.

[0416] Figure 5 The order in which spatial candidates are added to the candidate list is shown according to an embodiment.

[0417] exist Figure 5 The text describes the locations of spatial candidates.

[0418] The large block at the center of the attached diagram indicates the target block. The five smaller blocks adjacent to the target block represent spatial candidates.

[0419] The target block's coordinates can be (xP, yP), and the target block's size can be (nPSW, nPSH).

[0420] Spatial candidate A0 can be the block that is adjacent to the lower left of the target block. A0 can be the block that occupies the sample point corresponding to the coordinates (xP-1, yP+nPSH).

[0421] Spatial candidate A1 can be the block to the left of the target block. A1 can be the bottommost block among the blocks to the left of the target block. Optionally, A1 can be the block to the top of A0. A1 can be the block that occupies the sample point corresponding to the coordinates (xP-1, yP+nPSH).

[0422] Spatial candidate B0 can be the block that is adjacent to the top right of the target block. B0 can be the block that occupies the sample point corresponding to the coordinates (xP+nPSW, yP-1).

[0423] Spatial candidate B1 can be the block adjacent to the top of the target block. B1 can be the rightmost block among the blocks adjacent to the top of the target block. Optionally, B1 can be the block adjacent to the left of B0. B1 can be the block that occupies the sample point corresponding to the coordinates (xP+nPSW-1,yP-1).

[0424] Spatial candidate B2 can be the block that is the top-left neighbor of the target block. B2 can be the block that occupies the sample point corresponding to the coordinates (xP-1, yP-1).

[0425] like Figure 5 As shown, when adding spatial candidates to the candidate list, the order B1, A1, B0, A0, and B2 can be used. That is, available spatial candidates can be added to the candidate list in the order of B1, A1, B0, A0, and B2. Figure 5 middle, Figure 5 The order in which spatial candidates are added to the merge candidate list in the diagram is merely an example.

[0426] The candidate list can include motion information candidate list, merged candidate list, MV candidate list, BV candidate list, MPM list, etc.

[0427] To add a spatial or temporal candidate to the candidate list, it can be determined whether the spatial or temporal candidate is available. When a candidate block exists outside the boundaries of an image, strip, parallel block, etc., the availability of the corresponding candidate block can be set to "false". The description "the availability of the corresponding candidate block can be set to false" can mean that the corresponding candidate block is designated as "unavailable".

[0428] The maximum number of candidates in the candidate list can be set. N can represent the preset maximum number of candidates. The preset maximum number of candidates can be signaled via parameter sets, headers, etc. For example, the maximum number of candidates in the candidate list for a target block in a strip can be set via the strip header. For example, N can be essentially 5.

[0429] IBC mode IBC mode can be an intra-frame block copy prediction mode, in which predicted blocks for the target block are generated by referencing previously reconstructed regions in the target image. In this sense, IBC mode can be referred to as the current image reference mode. A block vector (BV) can be used to specify the previously reconstructed region.

[0430] IBC mode usage information can be used to determine whether the target block is encoded / decoded in IBC mode. Encoding device 110 can determine whether IBC mode will be used to encode the target block and can generate IBC mode usage information indicating whether IBC mode is used. Decoding device 150 can obtain IBC mode usage information from the bitstream.

[0431] In IBC mode, predicted blocks for a target block can be generated based on BV. BV can specify a reference block. BV can refer to the displacement between the target block and the reference block. The reference block can be a block in the target image (picture). The description of MV in the embodiments can also be applied to BV.

[0432] IBC modes can include skip mode, merge mode, AMVP mode, etc. The descriptions of AMVP mode, merge mode, and skip mode in the embodiments can be similarly applied to the AMVP mode, merge mode, and skip mode of IBC modes, respectively.

[0433] In skip or merge mode, a merge candidate list can be constructed, and the merge index can specify one of the merge candidates in the merge candidate list. The BV of the specified merge candidate can be used as the BV of the target block.

[0434] In the AMVP model, BVD can be used. The description of MVD in the examples can also be applied to BVD.

[0435] In IBC mode, the reference block can be limited to a block in a previously reconstructed region of the target image. Optionally, the reference block can be included in the target CTU or in at least one of the left-hand CTUs. For example, the BV value can be restricted such that the reference block is located in a specific region. The specific region can be a region corresponding to three blocks of a specific size that are encoded / decoded before the block of a specific size including the target block is encoded / decoded. The specific size can be 64×64.

[0436] Transformation and Quantization Quantization levels can be generated by performing a transform / quantization on the residual block. A residual block can refer to the difference between the original block and the predicted block. Reconstructed residual blocks can be generated by performing inverse quantization and / or inverse transform on the quantization levels. Reconstructed residual blocks can refer to the difference between the reconstructed block and the predicted block.

[0437] When performing a transformation or inverse transformation, a separable transformation or a two-dimensional (2D) non-separable transformation can be performed on the residual block. A separable transformation can be a transformation that performs a 1D transformation on the residual block in each of the horizontal and vertical directions.

[0438] Transform kernels used for transformations may include: 1) various DCT kernels, such as DCT type 2 (DCT-II); 2) DST kernels; and 3) kernels derived through training. For 1D transforms, in addition to DCT-II, DCT type and DST type may also include DCT-V, DCT-VIII, DST-I, and DST-VII.

[0439] Transform sets can be used when determining the DCT type, DST type, or kernel derived through training for the transform. Each transform set can include multiple transform candidates. Each transform candidate can be a kernel derived through a DCT type, DST type, or training.

[0440] Encoding device 110 can perform transform and inverse transform using transform candidates included in each transform set. Decoding device 150 can perform inverse transform using transform candidates included in each transform set. Transform selection information can be transmitted by signaling, indicating which of the multiple transform candidates included in the transform set applied to the residual block will be used. Transform selection information may include vertical transform selection information and horizontal transform selection information. Vertical transform selection information can indicate which of the transforms belonging to the transform set will be used for vertical transform. Horizontal transform selection information can indicate which of the transforms belonging to the transform set will be used for horizontal transform.

[0441] The transformation may include at least one of a primary transformation and a secondary transformation. Primary transformation coefficients can be generated by performing a primary transformation on the residual block, and secondary transformation coefficients can be generated by performing a secondary transformation on the transformation coefficients. Here, the transformation coefficients may include both primary and secondary transformation coefficients.

[0442] Primary transformation can refer to multiple transformation selection (MTS) that applies different transformations to various 1D directions (i.e., vertical and horizontal directions).

[0443] Secondary transformations can be transformations used to increase the energy concentration of the transformation coefficients generated by the primary transformation. Secondary transformations can be 1) separable transformations such as the primary transformation, and 2) 2D non-separable transformations. 2D non-separable transformations can refer to low-frequency non-separable transformations (LFNST) or non-separable primary transformations (NSPT).

[0444] NSPT can be applied to specific block sizes (such as 4×4, 4×8, 8×4, 4×16, 16×4, 8×8, 8×16 and 16×8) for intra-frame coding.

[0445] The primary transformation can be performed using at least one of several predefined transformation methods. In the example, the predefined transformation methods may include DCT, DST, KLT, etc. Furthermore, depending on the transformation kernel functions that define the DCT and DST, the primary transformation can be a transformation with various transformation types. For example, depending on the multiple transformation kernels, the primary transformation may include multiple transformations such as DCT-2, DCT-4, DCT-5, DCT-7, DCT-8, DST-1, DST-2, DST-4, DST-7, and DST-8.

[0446] In an embodiment, the transform type can be determined based on coding parameters associated with the target block. For example, the transform type can be determined based on one or more of the following: 1) the prediction mode of the target block (e.g., one of intra-frame prediction and inter-frame prediction), 2) the size of the target block, 3) the shape of the target block, 4) the intra-frame prediction mode of the target block, 5) the components of the target block (e.g., one of luma component and chroma component), and 6) the partition type applied to the target block (e.g., one of QT, BT, TT, and non-separable type).

[0447] For example, in the case of a primary transformation, a transformation set can be defined even in a secondary transformation. The method for deriving and / or determining the transformation set according to the embodiment can also be applied to both secondary and primary transformations.

[0448] In this embodiment, primary and / or secondary transformations can be determined for a specific target. Transformation selection information may include transformation target information. The transformation target information may refer to a target to which primary and / or secondary transformations have been applied.

[0449] For example, primary and / or secondary transformations can be applied to one or more signal components in the luminance and chrominance components.

[0450] In this embodiment, the transform selection information may include primary transform usage information and secondary transform usage information. The primary transform usage information may indicate whether the primary transform is applied to the residual block of the target block. The secondary transform usage information may indicate whether the secondary transform is applied to the residual block of the target block.

[0451] In an embodiment, it can be determined whether to apply primary and / or secondary transformations based on the encoding parameters of the target block / neighboring block (such as the size and shape of the target block / neighboring block).

[0452] In an embodiment, the transform selection information may include primary transform selection information and secondary transform selection information. The primary transform selection information indicates the transform method to be applied to the residual block from among a variety of transform methods available in the primary transform. The primary transform selection information may be a primary transform index. The secondary transform selection information indicates the transform method to be applied to the transform coefficients from among a variety of transform methods available in the secondary transform. The secondary transform selection information may be a secondary transform index.

[0453] In embodiments, transformation methods for primary and secondary transforms can be derived separately based on specific information such as encoding parameters. For example, encoding parameters may include encoding parameters for the target block / neighboring block.

[0454] In this embodiment, information related to the transformation, such as transformation selection information and sub-information of the transformation selection information, can be transmitted via signals to a specific target. For example, the specific target may be a CU.

[0455] Information related to the transformation (such as transformation selection information) and sub-information of the transformation selection information can be exported for a specific target. For example, the specific target can be a CU.

[0456] Quantization levels can be generated by performing quantization on the results generated by performing primary and / or secondary transformations or on the residual block.

[0457] The above description of the transformation can also be applied to the inverse transformation. In this application, the inverse processing of the transformation description can be performed in the inverse transformation. The term "transformation" in the names of components related to the transformation can be changed to the term "inverse transformation." Furthermore, the input of the transformation can be regarded as the output of the inverse transformation. The output of the transformation can be regarded as the input of the inverse transformation. The decoding device 150 can acquire transformation-related information such as transformation selection information, and can perform the inverse processing of the transformation-related processing indicated by the transformation-related information by using the transformation-related information.

[0458] A target block can include multiple sub-blocks. Each sub-block can be defined based on the minimum block size or minimum block shape. A target block can be partitioned into multiple sub-blocks, each of which can include coefficients such as 4×4, 2×8, and 8×2. A target block can be a transform block. Transform coefficients or quantization levels can be represented by the shape of the block. Transform coefficients can be quantized transform coefficients.

[0459] Transform coefficients or quantization levels can be scanned using at least one scan type selected from diagonal scan, vertical scan, and horizontal scan. A diagonal scan can be either an upper right diagonal scan or a lower left diagonal scan.

[0460] For example, the coefficients of a block can be changed or arranged in 1D vector form by using a diagonal scan. A vertical scan can be an operation that scans coefficients with a 2D block shape in the column direction. A horizontal scan can be an operation that scans coefficients with a 2D block shape in the row direction.

[0461] The scan type of coefficients can be determined based on coding parameters such as intra-prediction mode, block size, and block shape. For example, it can be determined which scan—diagonal, vertical, or horizontal—will be used based on coding parameters such as intra-prediction mode, block size, and block shape. A block can be a transform unit.

[0462] Depending on the type of scan, a scan can begin at a specific starting point and can end at a specific ending point.

[0463] During a scan, the scan order corresponding to the scan type can be applied to the sub-blocks first. Next, the scan order corresponding to the scan type can be applied to the transform coefficients or quantization levels within each sub-block.

[0464] Encoding device 110 can generate a bit stream that includes entropy-coded transform coefficients or entropy-coded quantization levels by performing entropy coding on transform coefficients or quantization levels.

[0465] Decoding device 150 can obtain entropy-coded transform coefficients or entropy-coded quantization levels from the bitstream, and can generate transform coefficients or quantization levels by performing entropy decoding on the entropy-coded transform coefficients or entropy-coded quantization levels. The coefficients can be arranged in 2D blocks via inverse scanning. The inverse scanning arrangement can be a rearrangement that is the reverse of the scanning arrangement.

[0466] The inverse scan transform coefficients or the quantization level of the inverse scan can be generated by inverse scanning the coefficients. Here, the inverse scan type can include diagonal scan, vertical scan, and horizontal scan, and the inverse scan type of the inverse transform corresponding to the scan type of the transform can be selected.

[0467] Decoding device 150 can perform dequantization on the (inverse scan) coefficients. Depending on whether a secondary inverse transform is to be performed, a secondary inverse transform can be performed on the result of the dequantization. Furthermore, depending on whether a primary inverse transform is to be performed, a primary inverse transform can be performed on the result of the secondary inverse transform. Reconstructed residual blocks can be generated by selectively performing both secondary and primary inverse transforms on the coefficients.

[0468] Filtering To improve image quality, filtering can be performed on blocks. Filtering can be used to determine or update the values ​​of target samples.

[0469] The target sample point can be one of the sample points described in the embodiments. For example, the target sample point can be one or more of the sample points described in the embodiments, such as prediction sample points, reference sample points, residual sample points, reconstruction sample points, and filtered reconstruction sample points.

[0470] The target sample point can be a sample point within one or more of the following: target image, target strip, target CTB, target block, reference sample point line, and template. The target block can be one of the above-mentioned blocks. For example, the target block can be one or more of the blocks described in the embodiments, such as transform block, prediction block, reference block, residual block, and reconstruction block.

[0471] In the embodiments, the filtering process described as being applied to one object can also be applied to another object. For example, the filtering process described in a specific loop filter can also be applied to transform blocks, prediction blocks, reference blocks, residual blocks, etc.

[0472] Specific types of filtering can be used for filtering processes according to embodiments. Filter types may include filter taps (or filter tap lengths), filter shapes, filter strengths, filter coefficients (or weights), and offsets for each filter.

[0473] A filter tap can represent the number of input samples used for the filter. Input samples may include target samples. Optionally, input samples may include specific values ​​determined for the target samples. Input samples may include one or more reference samples. One or more reference samples may be determined based on the properties of the target block described in the embodiments. Properties may include encoding parameters. For example, the properties of the target sample may include the position of the target sample. One or more reference samples may be specified based on their position relative to the position of the target sample.

[0474] The filter shape can be represented by the shape formed by the input samples. A specific value determined for a target sample can be considered as the target sample. In other words, when a specific value determined for a target sample is used as an input sample for the filter, the target sample can also be considered to form the filter shape.

[0475] The samples whose values ​​are determined by filtering can include multiple samples. The filter strength can represent the range of samples whose values ​​are determined by filtering. The filter strength can be either a strong filter strength or a weak filter strength. The number of samples whose values ​​are determined by a strong filter strength can be greater than the number of samples whose values ​​are determined by a weak filter strength. Optionally, the filter strength can represent the range of values ​​changed by filtering. The range of sample values ​​changed by a strong filter strength can be wider than the range of sample values ​​changed by a weak filter strength.

[0476] Filter coefficients can be the coefficients or weights of the input samples.

[0477] An offset can be a specific value that will be added to the result of a computation (such as a weighted sum) performed using the values ​​of the input samples and coefficients.

[0478] Filtering, interpolation, and sampling all share the characteristic that the values ​​of the sample points are updated. Therefore, the description of any one of filtering, interpolation, and sampling in the embodiments can also be applied to another of filtering, interpolation, and sampling. Here, sampling can include at least one of upsampling, downsampling, and subsampling.

[0479] Filtering may include filtering performed by predictor 123, predictor 163, etc.

[0480] When encoding a target block, prediction errors may exist between the original samples in the original block and the predicted samples in the prediction block. To reduce prediction errors, filtering can be performed on at least one of the predicted samples in the prediction block and the reference samples used for prediction.

[0481] For example, in intra-frame prediction, reference samples may include one or more of the following: top-left reference sample, top-right reference sample, left-side reference sample, and bottom-left reference sample. Filtering of prediction samples can be performed by applying specific weights to the prediction sample, left-side reference sample, top-side reference sample, and / or top-left reference sample, respectively.

[0482] Filtering can be performed on at least one of the predicted sample and the reference sample based on the attributes of the target block and the attributes of the predicted sample. For example, whether to perform filtering, the type of filter, the region to which the filter is applied, the filter weights, the reference sample, the range of the reference sample, and the location of the reference sample can be determined based on the attributes of the target block and the attributes of the predicted sample.

[0483] For example, the attributes of the target block may include information related to the target block described in the embodiments, such as the target block's 1) size, 2) prediction mode, 3) intra-frame prediction mode, 4) reference sample line, 5) sample value, and 6) coding parameters.

[0484] For example, the attributes of the predicted sample may include information related to the predicted sample described in the embodiments, such as 1) the sample value of the predicted sample and 2) the position of the predicted sample in the target block, and may include encoding parameters related to the predicted sample.

[0485] Filtering may include loop filtering performed by filter 130, filter 170, etc.

[0486] Figure 6 Several loop filters are shown based on the example.

[0487] Multiple loop filters used for loop filtering may include one or more of the following: Luminance Map with Chroma Scaling (LMCS), Deblocking Filter, Sample Adaptive Offset (SAO), and Adaptive Loop Filter (ALF).

[0488] Multiple loop filters can be connected sequentially to each other. For example, multiple loop filters can be connected in the order of LMCS, deblocking filter, SAO, and ALF. Furthermore, multiple loop filters can be connected in a sequence of all available permutations of the multiple loop filters. The output of one of the multiple loop filters can be used as the input of the next filter.

[0489] like Figure 6 As shown, an input image can be fed into a first filter. The input image can be a block as described in the embodiments. For example, the input image can be a reconstructed block generated by adder 129 or adder 169. The output of one filter can be the input of the next filter. The output image can be generated by the last filter. The output image can be a filtered block as described in the embodiments. For example, the output image can be a filtered reconstructed image generated by filter 130 or filter 170.

[0490] The target block can represent the image input to the filter. The target block for filtering can represent the image output from the filter.

[0491] LMCS can include luminance signal mapping of the luminance signal of the target block and chrominance signal scaling of the chrominance signal of the target block.

[0492] Luminance signal mapping can perform codeword reallocation for luminance signals.

[0493] Luminance signal mapping can include forward mapping and backward mapping. In forward mapping, the existing dynamic range can be partitioned into multiple intervals. The mapped dynamic range can be determined by performing codeword reallocation on the input image using a linear model for the corresponding intervals. In backward mapping, a reverse mapping is performed from the mapped dynamic range back to the existing dynamic range.

[0494] Chroma scaling can correct chroma signals based on the relationship between luminance signals and their corresponding chroma signals.

[0495] Forward mapping can be performed between inter-frame prediction and reconstruction of the luma signal, and between inter-frame prediction and chroma scaling of the luma signal. Inverse mapping can be performed between reconstruction and loop filtering of the luma signal. Chroma scaling can be performed between inverse transform and reconstruction of the chroma signal.

[0496] This structure allows for the inverse quantization, inverse transform, prediction, and reconstruction of luminance and chrominance signals within the mapped dynamic range. It also enables loop filtering, inter-frame prediction, intra-frame prediction, and reconstruction of chrominance signals within the existing dynamic range.

[0497] Deblocking filters remove block distortion that occurs at the boundaries between blocks in a reconstructed image. For example, a block can be a transform block. Furthermore, a block can be a sub-block of a specific block as described in the embodiments. Here, the boundary between blocks can refer to the sample points adjacent to the boundary between blocks.

[0498] Deblocking filters can be applied to both vertical and horizontal boundaries between blocks. After filtering the vertical boundaries between blocks, filtering can be applied again to the horizontal boundaries between the filtered blocks.

[0499] Deblocking filters can be applied selectively. The decision to apply a deblocking filter to a target block can be determined based on at least one of the following: a specific number of samples included in a specific number of columns or rows in the target block, and a specific number of samples included in a specific number of columns or rows in neighboring blocks adjacent to a specific boundary.

[0500] When a deblocking filter is to be applied to a target block, the filter to be applied can be determined based on the required intensity of the deblocking filter. In other words, a filter selected from several other filters based on the intensity of the deblocking filter can be applied to the target block. These multiple filters can include one of the following: a long-tap filter, a strong filter, a weak filter, or a Gaussian filter.

[0501] The maximum length of the deblocking filter can be determined based on the properties of the target block, such as the size of the target block, the components of the target block, and the coding parameters of the target block.

[0502] SAO can perform compensation for distortion between the original and reconstructed images based on samples. To compensate, SAO can apply an appropriate offset to the sample values ​​of the samples. That is, an offset can be added to the sample values.

[0503] Offsets can be determined for a target block. For example, offsets can be determined for each component of the CTB. The determined offsets can be applied to samples within a specific component of the CTB.

[0504] SAO can include SAOs that include edge offset (EO) and SAOs that use band offset (BO). Depending on the characteristics of the samples in a particular block, such as a CTU, it can be determined whether to perform an SAO using EO or an SAO using BO.

[0505] In SAO using EO, distortion correction in samples can be performed based on the orientation of edges within the target block. EO style categories can include horizontal, vertical, 135-degree diagonal, and 45-degree diagonal styles. For the target block, information indicating the style category to be applied and multiple offsets for that style category can be signaled. The number of offsets can be up to four. For target samples within the target block, adjacent samples can be determined based on the orientation of the style category. The offset to be applied to the target sample can be determined using the styles of the adjacent samples.

[0506] In the offset using BO, the luminance values ​​in the target block can be classified into specific bands, thus correcting distortion in the samples. The bit depth of the input image can be divided into m intervals. For example, m could be 32. The specific band can be n consecutive intervals among the m intervals. For example, n could be 4. N offsets for the n intervals can be transmitted using signals. Furthermore, information indicating the first interval selected from the n intervals among the m intervals can be transmitted using signals. The offset of the interval corresponding to the target sample can be added to the sample value of the target sample in the target cell.

[0507] ALF can perform compensation for distortion between the reconstructed image and the original image.

[0508] The filter coefficients of ALF can be transmitted as a signal via a bitstream.

[0509] The filter shape of an ALF can be determined by the components of the target block. For example, a 7×7 diamond filter can be used for the luminance component. A 5×5 diamond filter can be used for the chrominance component.

[0510] In ALF, the characteristics of a specific block can be determined, and the category of that block can be determined based on those characteristics. In other words, ALF determination and category determination can be performed on a 4×4 block basis. Filter coefficients can be calculated based on the category. A specific block can be a block with a 4×4 size.

[0511] Based on the orientation and activity determined using the gradient of a specific block, one of 25 categories can be chosen as the category of a particular block. Depending on the gradient of the particular block, rotational transformations, vertical symmetric transformations, and / or diagonal symmetric transformations of the filter can be applied.

[0512] Information related to whether or not ALF is applied can be sent via signals to specific units such as CTB.

[0513] The available signal transmission indicates the index of the filter in the available filters that will be applied to a specific unit. Here, the available filters may include fixed filters and filters configured using a parameter set. For example, the parameter set may be an adaptive parameter set (APS). Fixed filters may be defined equivalently by the encoding device 110 and the decoding device 150. The filter coefficients of filters configured using a parameter set can be determined based on the encoding parameters.

[0514] Entropy encoding and entropy decoding Figure 7 The entropy encoding and entropy decoding are shown based on the example.

[0515] exist Figure 7 The upper part shows the entropy encoding process performed by the entropy encoder 139.

[0516] The entropy encoder 139 may include a context modeler, a binarization unit, and an entropy coding unit. The context modeler may include a context selection unit and a context memory.

[0517] A binarization unit generates a binary number for each syntax element by performing binarization on the syntax elements of the target block. Binarization can be a process that converts syntax elements into binary form.

[0518] Information about syntax elements and binary numbers can be provided from the binarization unit to the context selection unit.

[0519] The context modeler can perform context updates.

[0520] Context can represent the probability of occurrence of each binary number of the encoded syntax element.

[0521] The context modeler can perform context updates to apply the current probability information to the encoding of the binary numbers of the syntax elements of the target block. The updated context can be stored in the context memory. Here, the updated context corresponding to the syntax elements of the target block (the binary numbers in the syntax elements of the target block) can be derived by the context modeler.

[0522] The context selection unit can select a context corresponding to the binary number of the syntax elements of the target block. The selected context can be loaded from the context memory and can be used as an updated context for entropy encoding of the binary number of the syntax elements of the target block.

[0523] The updated context can be used for entropy encoding of the syntactic elements of the target block.

[0524] An entropy encoder can generate encoded information of the syntax elements of a target block by performing entropy encoding using the generated binary number and the updated context, and can generate a bit stream including the encoded information. The entropy encoder can use at least one of arithmetic encoding methods and bypass encoding methods.

[0525] exist Figure 7 The lower part shows the entropy decoding process performed by the entropy decoder 161.

[0526] The entropy decoder 161 may include a context modeler, an entropy decoding unit, and a debinarization unit. The context modeler may include a context selection unit and a context memory.

[0527] The context modeler can perform context updates.

[0528] The context can represent the probability of occurrence of each binary number of a decoded syntax element.

[0529] The context modeler can perform context updates to apply the currently decoded probability information to the entropy decoding of the binary numbers of the syntax elements of the target block. The updated context can be stored in the context memory. Here, the updated context corresponding to the syntax elements of the target block (the binary numbers in the syntax elements of the target block) can be derived by the context modeler.

[0530] The context selection unit can select a context corresponding to the binary number of the syntax elements of the target block. The selected context can be loaded from the context memory and can be used as an updated context for entropy decoding of the binary number of the syntax elements of the target block.

[0531] The updated context can be used for entropy decoding of the syntactic elements of the target block.

[0532] An entropy decoder can generate the binary representation of the syntax elements of a target block by entropy decoding the encoded information of the bitstream based on an updated context. The entropy decoder can use at least one of arithmetic decoding methods and bypass decoding methods.

[0533] A debinarization unit can obtain the syntax elements of a target block by performing debinarization on at least one of the generated binary numbers. Debinarization can be a process that converts at least one binary number in the binary number into the form of a syntax element.

[0534] Information about syntax elements and binary numbers can be provided to the context selection unit from the debinarization unit.

[0535] Each syntax element may be one of the encoding parameters described in the embodiment.

[0536] Methods for binarization, debinarization, entropy coding, and entropy decoding In an embodiment, in order to perform signal transmission for specific information, one or more of the binarization methods, debinarization methods, entropy coding methods, and entropy decoding methods listed below can be used.

[0537] - Signed zero-order exponent Golomb binarization / inverse binarization method (abbreviated as se(v)) - Signed k-order Exp_Golomb binarization / inverse binarization method (abbreviated as sek(v)) - A 0th-order Exp_Golomb binarization / debinarization method for unsigned positive integers (abbreviated as ue(v)). - A k-order Exp_Golomb binarization / inverse binarization method for unsigned positive integers (abbreviated as uek(v)). - Fixed-length binarization / inverse binarization method (abbreviated as f(n)) - Truncation Rice binarization / inverse binarization method or truncation univariate binarization / inverse binarization method (abbreviated as tu(v)) - Truncation of binary binarization / inverse binarization method (abbreviated as tb(v)) -Context-adaptive arithmetic coding / decoding method (abbreviated as ae(v)) - A string of bits in bytes (abbreviated as b(8)) - Signed integer binarization / debinarization method (abbreviated as i(n)) - Unsigned positive integer binarization / debinarization method (abbreviated as u(n)) (where "u(n)" can represent a fixed-length binarization / debinarization method).

[0538] - Univariate binarization / inverse binarization methods A neighboring block can refer to a block adjacent to the target block. Neighboring blocks can include spatially adjacent blocks and temporally adjacent blocks. A neighboring block can also refer to a reconstructed neighboring block within a reference image. A neighboring block does not necessarily have to be adjacent to the target block.

[0539] A spatially adjacent block can be a block that is spatially adjacent to the target block.

[0540] The target block and spatially adjacent blocks can be included within the target image.

[0541] A spatially neighboring block may include a block whose boundary is adjacent to at least a portion of the boundary of the target block. Optionally, a spatially neighboring block may include a block whose distance from the target block is less than or equal to a reference value.

[0542] Spatial neighbor blocks can include blocks that are diagonally adjacent to the vertex of the target block.

[0543] The spatial neighboring blocks can include the top-left block adjacent to the top-left of the target block, the top block adjacent to the top of the target block, the top-right block adjacent to the top-right of the target block, the left block adjacent to the left side of the target block, the right block adjacent to the right side of the target block, the bottom-left block adjacent to the bottom-left of the target block, the bottom block adjacent to the bottom of the target block, and the bottom-right block adjacent to the bottom-right of the target block.

[0544] A temporally adjacent block can be a block that is temporally adjacent to the target block.

[0545] Temporally adjacent blocks can include collocation blocks (col blocks). A collocation block can be a block in the reconstructed image within a reference image buffer. A collocation image (col frame) can refer to an image that includes a collocation block. A collocation image can be an image included in a list of reference images.

[0546] Coordinate blocks can be determined based on the position of target blocks in the target image. When two blocks are "temporally adjacent," this can mean that the positions of the two blocks meet specific conditions.

[0547] The position of a co-position block within a co-position frame can be the same as the position of a target block in the target image. Optionally, the position of a co-position block within a co-position frame can correspond to the position of a target block in the target image. Here, when the positions of the blocks correspond, this can mean that the regions of the blocks are the same, that the region of one block is included in the region of another block, or that one block occupies a specific position of another block.

[0548] For example, the position of a co-position block in a co-positional frame can be the same as the position of a target block in the target image. Optionally, a co-position block can be a block within the co-positional frame that includes co-positional samples. A co-positional sample can be a sample point having the same coordinates as a specific sample point of the target block.

[0549] A temporally neighboring block can be a block that is spatially adjacent to the target block in time.

[0550] Neighboring samples can refer to samples in neighboring blocks. Neighboring samples can include predicted samples, reconstructed samples, residual samples, and decoded samples.

[0551] The terms listed below in a single line may be used with the same meaning in the embodiments and may be used interchangeably in the embodiments.

[0552] "Motion Information", "Motion Vector", "Block Vector" "Bidirectional prediction", "Bidirectional prediction", "Inter-frame bidirectional prediction", "Bidirectional inter-frame prediction" Predefined value: A predefined value can refer to a value that is used in both the encoding and decoding devices. For example, a predefined value can be interpreted simply as a fixed value. Optionally, a predefined value can be a value shared by the encoding and decoding devices through signal transmission. Optionally, a predefined value can be a value derived in both the encoding and decoding devices through the same process, such that the encoding and decoding devices have a common value. Optionally, a predefined value can be a common value in both the encoding and decoding devices.

[0553] Values ​​derived in the encoding and decoding apparatus through the same process can include values ​​derived in the encoding and decoding apparatus through the same process for the same value and / or the same information.

[0554] Values ​​derived through the same process in the encoding and decoding apparatus can include values ​​derived by using the same conditional statements for the same value and / or the same information in the encoding and decoding apparatus.

[0555] The description of predefined values ​​can also be applied to predefined information. In the description above, "value" can be replaced with "information".

[0556] Motion information may refer to information including at least one of the following: reference frame list information, reference image, motion vector candidate, motion vector candidate index, merge candidate and merge index, block vector, block vector candidate and block vector candidate index, and motion vector, reference frame index and inter-frame prediction indicator.

[0557] In this disclosure, "when the indicator indicating whether a particular method is performed is true" can mean whether the execution of a particular method is true in the prediction pattern, motion information, encoding parameters and / or position indicated by the corresponding indicator.

[0558] For example, an indicator that indicates whether to execute a specific mode can have values ​​from 0 to 3, and the specific mode can only be executed if the indicator has a value of 1 or 3. In this case, when the indicator that indicates whether to execute a specific mode is true, it can mean that the indicator that indicates whether to execute a specific mode has a value of 1 or 3.

[0559] In this disclosure, "when the indicator indicating whether to execute a particular method is false" can mean that the indicator indicating whether to execute a particular method is not true.

[0560] In an embodiment, reordering for a specific target can refer to sorting the specific target or the elements within the specific target.

[0561] In this disclosure, "when an indicator indicating whether a particular method (or mode) is performed on a target block is true" and "when a particular method is performed in a target block" can be used with the same meaning and can be used interchangeably.

[0562] In this disclosure, "when the indicator indicating whether a particular method (or pattern) is performed on the target block is false" and "when a particular method is not performed in the target block" can be used with the same meaning and can be used interchangeably.

[0563] In this disclosure, the matching cost may include template matching cost and bidirectional matching cost.

[0564] In Adaptive Motion Vector Resolution (AMVR), the resolution of the motion vector difference can be adjusted on a block-by-block basis.

[0565] Adaptive motion vector resolution can improve encoding / decoding efficiency by adjusting the resolution of the motion vector difference.

[0566] For example, the adjusted resolution can represent one of 16 pixels (pel), 8 pixels, 4 pixels, full pixels, half pixels, and quarter pixels, but is not limited to these. pel represents a pixel unit. For example, when the adjusted resolution of the current block is 4 pixels, each component of the motion vector difference can have a number of pixels that are multiples of 4 pixels.

[0567] The resolution of the motion vector difference can be predefined.

[0568] When performing subsampling, this can mean selecting only some samples from a specific region.

[0569] When performing subsampling, this can mean selecting, when choosing only some samples from a specific region, the following samples are selected: 1) the SUBSAMPLE_START_HOR sample in the horizontal direction and the SUBSAMPLE_START_VER sample in the vertical direction, based on the specific sample; and 2) samples with a horizontal spacing of a multiple of SUBSAMPLE_STEP_HOR and a vertical spacing of a multiple of SUBSAMPLE_STEP_VER, based on the sample in 1). Optionally, when performing subsampling, this can mean selecting a subset of the samples in 1) and 2).

[0570] The specific location can be the top-left sample point of the region where subsampling is performed. However, the specific location is not limited to the top-left sample point of the region where subsampling is performed.

[0571] Each of SUBSAMPLE_START_HOR and SUBSAMPLE_START_VER can be 0 or a positive integer. Information about at least one of SUBSAMPLE_START_HOR and SUBSAMPLE_START_VER can be sent / encoded / decoded using signals, or the values ​​of SUBSAMPLE_START_HOR and / or SUBSAMPLE_START_VER can be determined as predefined values ​​without sending / encoding / decoding information using signals.

[0572] SUBSAMPLE_STEP_VER and SUBSAMPLE_STEP_HOR can be 0, 1, 2, or positive integers, respectively. Information about at least one of SUBSAMPLE_STEP_VER and SUBSAMPLE_STEP_HOR can be sent / encoded / decoded using signals, or the values ​​of SUBSAMPLE_STEP_VER and / or SUBSAMPLE_STEP_HOR can be determined to predefined values ​​without sending / encoding / decoding information using signals.

[0573] Molecular sampling methods can be distinguished by the region where subsampling is performed, the location of the subsampling region, the size of the subsampling region, SUBSAMPLE_START_HOR, SUBSAMPLE_START_VER, SUBSAMPLE_STEP_HOR, and SUBSAMPLE_STEP_VER. However, the criteria used to distinguish molecular sampling methods are not limited to the values ​​mentioned above.

[0574] Information about the subsampling method can be transmitted / encoded / decoded using signals, or a predefined subsampling method can be used without transmitting / encoding / decoding signals.

[0575] Information about the subsampling method can be used to determine the subsampling method.

[0576] For example, information about the subsampling method can be information about at least one of the following: the region where subsampling is performed, the location of the region where subsampling is performed, the size of the region where subsampling is performed, SUBSAMPLE_START_HOR, SUBSAMPLE_START_VER, SUBSAMPLE_STEP_HOR, and SUBSAMPLE_STEP_VER.

[0577] For example, the subsampling method can be determined based on at least one of the target block's motion information, encoding parameters, size, and prediction mode.

[0578] The subsampling method can be determined at least one of the following levels: sequence level, frame level, parallel block level, parallel block group level, stripe level, coding tree unit (CTU) level, coding unit (CU) level, and prediction unit (PU) level, but the determination unit is not limited to these.

[0579] Geometric Partitioning (GPM) can be a method for determining partition boundaries that divide a target block into two parts in a straight line along each direction, and performing a weighted summation on the two prediction blocks (or reference blocks) using a weight map determined based on the partition boundaries.

[0580] For example, at least one prediction block (or at least one reference block) in a geometric partitioning pattern may refer to a prediction block that is predicted unidirectionally and / or bidirectionally, or a reference block in at least one direction in unidirectional and / or bidirectional prediction.

[0581] Alternatively, for example, at least one prediction block in a geometric partitioning pattern may refer to a prediction block predicted intra-frame.

[0582] For example, in geometric partitioning mode, one prediction block can be predicted inter-frame and another prediction block can be predicted intra-frame.

[0583] Figure 8 This is an example illustrating the partition boundaries in a geometric partitioning pattern.

[0584] Figure 9 It is a diagram used to describe the partition boundaries, partition offsets, and partition angles in a geometric partitioning pattern.

[0585] Figure 10 This is an example illustrating a weighted graph for each prediction block based on specific partition boundaries.

[0586] Figure 11 This is an illustration of an example of template matching.

[0587] In template matching, the motion information of the target block can be determined and / or modified based on the calculation results of the cost function between the target template and the reference template.

[0588] The reference block may include at least one of the following: 1) a block indicated by initial motion information, 2) a block indicated by motion information derived in the template matching search process, 3) a block indicated by motion information finally improved by template matching, 4) a block whose sample points belonging to the template matching search range are one of the top left, bottom left, top right, bottom right and center, and 5) a block finally determined by template matching.

[0589] The size of the reference block can be the same as the size of the target block.

[0590] Motion information improved by template matching can be the motion information with the lowest matching cost derived from the template matching search process. However, the methods used to derive motion information are not limited to the above criteria.

[0591] Template matching cost can refer to the result calculated using a cost function that applies to the templates of the target block and the reference block used in the template matching.

[0592] Each of the reference block, reference template, and reference region may include at least one of the following: predicted samples, reconstructed samples, residual samples, and decoded samples of the reference image. Optionally, each of the reference block, reference template, and reference region may include at least one of the following: predicted samples, reconstructed samples, residual samples, and decoded samples of the target image.

[0593] The template configuration for template matching can be as follows.

[0594] First, the target template can include neighboring sample points of the target block.

[0595] The reference region for the target block can include neighboring samples of the target block.

[0596] For example, the reference region of the target block may include at least one of the sample points located in the lower left region, left side region, upper left region, upper top region, and upper right region surrounding the target block.

[0597] For example, in template matching, the target template can be the same as the reference region of the target block.

[0598] For example, the sample points of a target template based on a target block can be sample points corresponding to the sample points of a reference template based on a reference block.

[0599] For example, when configuring a target template in template matching, you can select some sample points in the reference area of ​​the target block. The target template can be configured by using the selected sample points.

[0600] For example, the sample points selected for configuring a target template based on a target block can be the same sample points selected for configuring a template based on a reference block.

[0601] For example, the reference region of a target block based on a target block can be the region corresponding to the reference region of a reference block based on a reference block.

[0602] The reference template may include neighboring samples of the reference block.

[0603] The reference region of a reference block can include neighboring sample points of the reference block.

[0604] For example, the reference region of a reference block may include at least one of the sample points located in the lower left region, left side region, upper left region, upper top region, and upper right region surrounding the reference block.

[0605] For example, in template matching, the reference template can be the same as the reference region of the reference block.

[0606] For example, the sample points of a reference template based on a reference block can be the sample points of a target template based on a target block.

[0607] For example, when configuring a reference template in template matching, you can select some sample points within the reference area of ​​the reference block. The reference template can be configured using the selected sample points.

[0608] For example, the sample points selected for configuring a reference template based on a reference block can be the same sample points selected for configuring a template based on a target block.

[0609] For example, the reference region of a reference block based on a reference block can be the region corresponding to the reference region of a target block based on a target block.

[0610] The template matching method may include at least one of intra-frame template matching mode and inter-frame template matching mode.

[0611] Intra-frame template matching mode can refer to a template matching method for each of the reference block, reference template, and reference region, including at least one of the predicted sample, reconstructed sample, residual sample, and decoded sample of the target image.

[0612] Inter-frame template matching mode can refer to a template matching method for each of the reference block, reference template, and reference region, including at least one of the predicted sample, reconstructed sample, residual sample, and decoded sample of the reference image.

[0613] The target template / reference template used for template matching may include at least one of the following: 1) at least one sample point in the TMSIZE_LEFT bar adjacent to the left side of the target block / reference block; and 2) at least one sample point in the TMSIZE_ABOVE bar adjacent to the top of the target block / reference block.

[0614] However, the positional relationship between each sample point in the template and the target / reference block and / or the method used to configure the template are not limited to the relationships or methods described above.

[0615] Each of TMSIZE_LEFT and TMSIZE_ABOVE can be 0, 1, 2, 3, 4 or a positive integer greater than or equal to 4.

[0616] TMSIZE_LEFT and TMSIZE_ABOVE can be the same. Alternatively, TMSIZE_LEFT and TMSIZE_ABOVE can be different.

[0617] Each of TMSIZE_LEFT and TMSIZE_ABOVE can be a predefined value or a value determined based on information sent / encoded / decoded by signals.

[0618] Each of TMSIZE_LEFT and TMSIZE_ABOVE can be determined based on at least one of the target block's motion information, encoding parameters, size, and prediction mode.

[0619] Subsampling can be performed when configuring a template for template matching.

[0620] When configuring a template for template matching, you can use all the sample points in the reference area, or you can use only some of the sample points in the reference area.

[0621] The template used for template matching can represent at least one of the templates of the target block and the template of the reference block.

[0622] A reference region can refer to at least one of the reference regions of the target block used for template matching and the reference regions of the reference block.

[0623] When configuring a template for template matching by using only some sample points located in the reference region, subsampling can be performed on all or part of the reference region.

[0624] When configuring a template for template matching by using only some samples located in the reference region, the reference region can be divided into at least two regions. Each divided region can be one of the following: 1) a region where subsampling is performed, 2) a region where subsampling is not performed and is used for template configuration, and 3) a region not used for template configuration. The template for template matching can be configured using samples selected by subsampling in 1) and samples within region 2).

[0625] For example, the area corresponding to 1) could be the area within the reference area that belongs to the left and / or upper left of the block.

[0626] For example, the area corresponding to 1) could be the area within the reference area that belongs to the top and / or upper left of the block.

[0627] Optionally, when configuring a template for template matching by using only some sample points located in the reference region, the reference region can be divided into at least two regions. Each divided region can be one of the following: 1) a region where subsampling is performed and 2) a region not used for template configuration. The template for template matching can be configured using sample points selected by subsampling in 1).

[0628] When calculating the cost function between a target template and a reference template in template matching, all samples within each template can be used, or only some samples within each template can be used. In other words, the cost function can be calculated for only some samples.

[0629] When calculating the cost function between templates using only some samples within the template, subsampling can be performed on all or part of the template region.

[0630] When calculating the cost function between templates using only some samples within the template, the region of the template used for template matching can be divided into at least two regions. Each divided region can be one of the following: 1) a region where subsampling is performed, 2) a region where subsampling is not performed and is used to calculate the cost function, and 3) a region not used to calculate the cost function. The cost function between templates can be calculated in template matching using samples selected by subsampling in region 1) and samples within region 2).

[0631] Optionally, when calculating the cost function between templates using only some samples within the template, the template region used for template matching can be divided into at least two regions. Each divided region can be one of the following: 1) a region where subsampling is performed and 2) a region not used to calculate the cost function. The cost function between templates can be calculated in template matching using samples selected by subsampling in 1).

[0632] When performing template matching search processing, all samples / locations within the search range can be used, or only some samples / locations within the search range can be selected. Search and / or matching cost calculations can be performed only for the selected samples / locations. Optionally, search and / or matching cost calculations can be performed only for motion information indicating the selected samples / locations.

[0633] When performing template matching search processing by using only some samples / locations within the search range, subsampling can be performed on all or part of the search range.

[0634] When template matching search processing is performed using only some samples / positions within the search range, the search range can be divided into at least two regions. Each divided region can be one of the following: 1) a region where subsampling is performed, 2) a region where subsampling is not performed but search processing is performed, and 3) a region where search processing is not performed. Template matching search processing can be performed on pixels and / or positions selected by subsampling in 1) and samples / positions within region 2). Optionally, template matching search processing can be performed on motion information indicating the samples / positions selected by subsampling in 1) and the samples / positions within region 2).

[0635] Optionally, when performing template matching search processing by using only some samples / locations within the search range, the search range can be divided into at least two regions. Each divided region can be one of the following: 1) a region where subsampling is performed and 2) a region where search processing is not performed. Template matching search processing can be performed using the samples / locations selected by subsampling in 1). Optionally, template matching search processing can be performed on motion information indicating the samples / locations selected by subsampling in 1).

[0636] Figure 12 Various examples of subsampling methods are shown.

[0637] Figure 13 Various examples of subsampling methods are shown.

[0638] Figure 12 and Figure 13 The shaded sample point (or location) in the text can refer to the sample point (or location) selected through subsampling.

[0639] For all or part of the reference area, it can be as follows: Figure 13 The subsampling is performed as shown, and the template can be configured by using only the selected sample points (or locations).

[0640] For all or part of the template region that matches the template, it can be like this: Figure 13 The example demonstrates subsampling, and the cost function can be computed only for selected sample points (or locations).

[0641] For all or part of the search range for template matching, it can be like this: Figure 12 The method performs subsampling and can perform search and / or matching cost calculations only for selected pixels and / or locations.

[0642] For all or part of the search range for template matching, it can be like this: Figure 12 The process involves subsampling and can perform search and / or matching cost calculations only for motion information indicating the selected pixels and / or locations.

[0643] The search can be performed by using the calculation of a cost function to determine the similarity between NUM_TEMPLATE_COMPARE templates.

[0644] When certain conditions are met, the search may include the process of determining at least one piece of motion information that satisfies the specific conditions within a specific search range. The motion information of the target block may be determined and / or modified based on the at least one piece of motion information determined through the search.

[0645] Motion information can refer to the motion information with the lowest matching cost among the motion information within the search range, but it is not limited to this.

[0646] Optionally, the search may include the process of identifying at least one block that satisfies specific conditions within a particular search range. Motion information of the blocks identified through the search can be used as motion information for the target block.

[0647] A block that meets certain conditions can be one of the reference blocks within the search scope.

[0648] The cost function can refer to a function used to determine the similarity between at least one sample in the target template and at least one sample in the reference template.

[0649] The similarity between the first and second values ​​can be determined by using at least one of the following methods: 1) the difference between the two values, 2) the ratio between the two values, and 3) an operation that compares the difference between the two values ​​with a specific value.

[0650] The cost function can be a function that determines the similarity between at least one sample point in the target template and a sample point in its corresponding reference template.

[0651] The cost function can be at least one of the following: Sum of Absolute Differences (SAD), Sum of Absolute Transformed Differences (SATD), Sum of Absolute Differences with Mean Removed (MR-SAD), Mean Squared Error (MSE), and Sum of Squared Errors (SSE). However, the cost function is not limited to the items listed above.

[0652] The cost function used in template matching can be predefined or determined based on information sent / encoded / decoded by signals.

[0653] For example, MR-SAD can be used as a cost function in template matching when the target block meets the enable conditions for bilateral matching and / or some of the enable conditions for bilateral matching, or when bilateral matching is performed in the target block.

[0654] For example, SAD can be used as a cost function in template matching when the target block does not meet the enable conditions for bilateral matching and / or some of the enable conditions for bilateral matching, or when bilateral matching is not performed in the target block.

[0655] For example, the type of cost function in a bilateral match can be determined based on whether specific conditions are met in the bilateral match. In this case, the type of cost function in the template match can be determined based on whether the enabling conditions of the bilateral match and the specific conditions used to determine the type of cost function in the bilateral match are met.

[0656] For example, MR-SAD can be used as the cost function in bilateral matching when the target block meets the enable conditions and specific conditions of bilateral matching; otherwise, SAD can be used as the cost function in bilateral matching.

[0657] For example, MR-SAD can be used as the cost function in template matching when the target block meets the enabled conditions for bilateral matching, and weighted inter-frame bidirectional prediction is performed or the number of samples in the target block is greater than a certain value; otherwise, SAD can be used as the cost function in template matching.

[0658] The search area can be a specific range centered on the location indicated by the initial motion information. In other words, the center of the search area can be the location indicated by the initial motion information.

[0659] Optionally, the search range can be a specific area with the initial motion information indicating the upper left position. In other words, the upper left of the search range can be the position indicated by the initial motion information.

[0660] Optionally, the search range may include a pre-reconstructed region surrounding the target block. As an example, the search range may include at least one of the sample points (or the locations of sample points) located in the lower left, left, upper left, upper, and upper right regions surrounding the target block.

[0661] At least one of the size and shape of the search range for the target block can be predefined in the encoder and decoder.

[0662] Optionally, at least one of the size and shape of the search range for the target block can be determined based on at least one of the target block's size, encoding parameters, motion information, and prediction mode.

[0663] Optionally, information indicating one of the dimensions and shape of the search range for the target block can be encoded and transmitted using a signal.

[0664] The search area can be a rectangle with a width of SR_X and a height of SR_Y. Alternatively, the search area can be a rhombus with a width of SR_X and a height of SR_Y. However, the shape and size of the search area are not limited to the embodiments described above.

[0665] Each of SR_X and SR_Y can be a positive integer. Each of SR_X and SR_Y can be a predefined value or a value determined based on information sent / encoded / decoded by signals.

[0666] Initial motion information can be determined based on at least one of the following: motion information of the target block, encoding parameters of the target block, motion vector of the target block, reference image of the target block, block vector of the target block, motion vector predictor of the target block, block vector predictor of the target block, motion information of at least one neighboring block of the target block, merging candidate of the target block, motion vector difference of the target block, and block vector difference of the target block.

[0667] Search methods can be classified based on units such as search style, search resolution, search range, initial motion information, and derived motion information.

[0668] The search method can be determined based on at least one of the following: motion information of the target block, encoding parameters of the target block, size of the target block, prediction mode of the target block, reference image of the target block, at least one sample value in the target block, target template, at least one sample value in the target template, and region of the target template.

[0669] The search style can be one of the diamond, cross, or full search styles. However, the search style is not limited to those listed above.

[0670] When (0,0) represents the position indicated by the initial motion information, a search using a diamond pattern can refer to searching for at least one of the positions (0,2×RR), (RR,RR), (2×RR,0), (RR,-RR), (0,-RR), (-RR,-RR), (-RR,0), (-RR,RR), and (0,0).

[0671] When (0,0) represents the position indicated by the initial motion information, a crosshair pattern search can refer to searching for at least one of the positions (0,RR), (RR,0), (0,-RR), (-RR,0), and (0,0).

[0672] RR can be a value determined based on the search resolution or the search resolution itself, or it can be a predefined positive number.

[0673] A search using the full search style can refer to searching all locations within a predefined search range.

[0674] For example, when FS_i has values ​​from -FS_X to FS_X and FS_j has values ​​from -FS_Y to FS_Y, a full-search style search can refer to searching the position (FS_i×RR, FS_j×RR). Here, (0,0) can be a position indicated by the initial motion information. However, the search range is not limited to the positions mentioned above. Each of FS_X and FS_Y can be a predefined positive number.

[0675] The search resolution can be one of 4 pixels, full pixels, half pixels, or quarter pixels. However, the search resolution is not limited to the pixels mentioned above.

[0676] The search resolution can be predefined, or it can be determined based on at least one of the information about the adaptive motion vector resolution, or it can be determined based on the values ​​sent / encoded / decoded by the signal.

[0677] The unit from which motion information is derived can include all block units and sub-block units.

[0678] Figures 14 to 19 The search methods in template matching based on the examples are shown respectively.

[0679] Based on the motion information of the target block, encoding parameters, prediction mode, and adaptive motion vector resolution, it is possible to determine Figures 14 to 19 The table shown contains specific columns. Searches can be performed from top to bottom of the defined columns, using the search style and search resolution corresponding to the rows marked "v".

[0680] For example, when in Figure 14 When the target block is in AMVP mode and the resolution determined by the adaptive motion vector resolution is 4 pixels, a diamond-shaped search using a 4-pixel search resolution can be performed, followed by a cross-shaped search using a 4-pixel search resolution.

[0681] ALT_IF represents the index of an adaptive interpolation filter. An interpolation filter can be applied to compute the pixel value at a sample location with a specific resolution. The adaptive interpolation filter can be one of several interpolation filters selected by its index. In other words, when applying an adaptive interpolation filter, different interpolation filters can be used based on the index to compute the pixel value at a sample location with a specific resolution.

[0682] For example, a specific resolution can be half a pixel. However, a specific resolution is not limited to half a pixel.

[0683] For example, the interpolation filter determined by the index can be one of a 6-tap interpolation filter and an 8-tap interpolation filter. However, the method used to determine the interpolation filter is not limited to the method described above.

[0684] Figure 20 This illustrates the first template configuration method under affine mode, based on the example.

[0685] Figure 21 This illustrates a second template configuration method under affine mode, based on an example.

[0686] CPMV can refer to the affine control point motion vector. The MV of each sub-block within a target block can be derived using CPMV.

[0687] When the target block is in affine mode, it can be partitioned into sub-block units of width N and height M. The motion information of each sub-block can be determined based on at least one of the target block's motion information, encoding parameters, and size. The template matching cost for the target block can be determined based on at least one of the template matching costs for the partitioned sub-blocks. For example, the template matching cost for the target block can be the sum of the template matching costs for the partitioned sub-blocks, or the average of the template matching costs for each sub-block.

[0688] N and M can be 2, 4, 8 or positive integers.

[0689] N and M can be predefined values ​​or values ​​determined based on information sent / encoded / decoded by signals.

[0690] The motion information of a target block can be determined based on the motion information of neighboring blocks. This can be represented as "the target block inherits motion information from its neighboring blocks".

[0691] For example, when the target block is in merge mode, a merge candidate can be specified from the merge candidate list based on the merge index, and the motion information of the specified merge candidate can be used as the motion information of the target block.

[0692] For example, when the target block is in AMVP mode, an MV candidate can be specified from the MV candidate list based on the MV candidate index, and the motion information of the specified MV candidate can be used as the motion information of the target block.

[0693] When the motion information inherited by the target block from neighboring blocks indicates bidirectional prediction, an example of performing template matching in the target block can be as follows.

[0694] As a first step, template matching is performed for each of the L0 and L1 directions. The template matching costs C0 and C1 for the motion information determined for the L0 and L1 directions can be calculated.

[0695] In this case, when performing template matching for each direction, the template matching can be performed in the same manner as that in the one-way prediction for the corresponding direction without considering the motion information in the other direction.

[0696] For example, in this case, when a predefined condition is satisfied in the target block, MR-SAD can be used as the cost function; otherwise, SAD can be used as the cost function.

[0697] The predefined condition can be a condition based on at least one of the following: whether a model-based prediction method is performed in the target block; an indicator indicating whether a model-based prediction method is performed in the target block; whether bilateral matching is performed in the target block; an indicator indicating whether bilateral matching is performed in the target block; the motion information of the target block; the size of the target block; the coding parameters of the target block; the motion information of the neighboring blocks of the target block; the coding parameters of the neighboring blocks of the target block; and the type of the cost function in the template matching in the neighboring blocks of the target block.

[0698] For example, when whether a model-based prediction method is performed in the target block is true or when the indicator indicating whether a model-based prediction method is performed in the target block is true, MR-SAD can be used as the cost function; otherwise, SAD can be used as the cost function.

[0699] For example, when whether bilateral matching is performed in the target block is true or the indicator indicating whether bilateral matching is performed in the target block is true, and the number of samples in the target block is greater than or equal to a specific value, MR-SAD can be used as the cost function; otherwise, SAD can be used as the cost function.

[0700] The cost function can refer to the cost function used in the search for template matching; and / or the cost function for calculating at least one of C0, C1, and C'. The cost function used in the search for template matching and the cost function for calculating at least one of C0, C1, and C' can be the same or different.

[0701] For example, MR-SAD can be used as the cost function in the search for template matching, and C0, C1, and C' can be calculated by using SAD.

[0702] As a second step, if C0 < C1, a new target template T' can be generated by using the target template and the template in the L0 direction.

[0703] For example, when the target template is T, the reference template in the L0 direction is T0, and the reference template in the L1 direction is T1, the new target template T' can be determined as T' = W T ×T + WT0 × T0.

[0704] W T and W T0 can be predefined values respectively.

[0705] W T and W T0 can be values determined based on whether weighted inter - frame bidirectional prediction is performed in the target block; and / or the weights in weighted inter - frame bidirectional prediction.

[0706] For example, W T can be 2 and W T0 can be - 1.

[0707] If C0 > C1, a new target template T' can be generated by using the target template and the template in the L1 direction.

[0708] When the value of C0 is the same as the value of C1, this situation can be considered as one of C0 < C1 or C1 > C0, and the process in the second step can be executed.

[0709] As the third step, if C0 < C1, template matching is performed by using the target template as T' for the L1 direction. The template matching cost C' for the motion information determined for the L1 direction can be calculated.

[0710] If C0 > C1, template matching can be performed by using the target template as T' for the L0 direction, and the template matching cost C' for the motion information determined for the L0 direction can be calculated.

[0711] When the value of C0 is the same as the value of C1, this situation can be considered as one of C0 < C1 or C1 > C0, and the process in the third step can be executed.

[0712] As the fourth step, if C' = W C0 × C0 + W C1 × C1, the motion information of the target block can be changed to the motion information indicating unidirectional prediction in the L0 direction or the L1 direction based on the values of C0 and C1.

[0713] If C0 < C1, the motion information of the target block can be changed to the motion information indicating unidirectional prediction in the L0 direction. Optionally, the motion information for the L1 direction can be considered unavailable in the target block.

[0714] If C0 > C1, the motion information of the target block can be changed to the motion information indicating unidirectional prediction in the L0 direction. Optionally, the motion information for the L1 direction can be considered unavailable in the target block.

[0715] When the value of C0 is the same as the value of C1, this situation can be considered as one of C0 < C1 or C1 > C0, and the process in the fourth step can be executed.

[0716] W C0 and W C1 can be predefined values respectively.

[0717] W C0 and W C1 can be values determined based on whether weighted inter-frame bidirectional prediction is performed in the target block; and / or the weights in weighted inter-frame bidirectional prediction.

[0718] For example, W C0 can be 1, and W C1 can be 1 / 8.

[0719] For example, the second step to the fourth step can be executed only when the target block meets predefined conditions.

[0720] For example, the second step to the fourth step can be executed only when the target block is bidirectionally predicted and bilateral matching is not performed in the target block or the target block does not meet the enabling conditions for bilateral matching.

[0721] In bilateral matching, the reference block in the L0 direction and the reference block in the L1 direction can be used as templates, and the motion information of the target block can be determined and / or changed based on the calculation result of the cost function between the two templates.

[0722] The reference block can include at least one of the following reference blocks: 1) the reference block indicated by the initial motion information, 2) the reference block indicated by the motion information derived in the search process of bilateral matching, and 3) the reference block indicated by the motion information finally improved through bilateral matching.

[0723] For example, when configuring the template for bilateral matching, the reference block in the L0 direction and the reference block in the L1 direction can be used as templates.

[0724] The bilateral matching cost can mean the result value calculated using the cost function of the templates for the reference block in the L0 direction and the reference block in the L1 direction used in bilateral matching.

[0725] Subsampling can be performed when configuring the template for bilateral matching.

[0726] For example, when configuring the template for bilateral matching, only some pixels and / or positions within the reference block in the L0 direction and the reference block in the L1 direction can be selected. The template can be configured by only using the selected pixels and / or the selected positions.

[0727] The template used for bilateral matching can represent at least one of the template in the L0 direction and the template in the L1 direction.

[0728] For example, when configuring a template for bilateral matching, subsampling can be used for the reference block in the L0 direction and the reference block in the L1 direction.

[0729] For example, when configuring a template for bilateral matching, subsampling can be used for a portion of the reference block in the L0 direction and a portion of the reference block in the L1 direction.

[0730] Optionally, for example, when constructing a template for bilateral matching, the reference block in the L0 direction and the reference block in the L1 direction can each be divided into at least two regions. Each divided region can be one of the following: 1) a region where subsampling is performed, 2) a region where subsampling is not performed and is used for template configuration, and 3) a region not used for template configuration. The template for bilateral matching can be configured using pixels and / or positions selected by subsampling in 1) and pixels and / or positions within region 2).

[0731] Optionally, for example, when constructing a template for bilateral matching, the reference block in the L0 direction and the reference block in the L1 direction can each be divided into at least two regions. Each divided region can be one of the following: 1) a region for which subsampling is performed and 2) a region not used for configuring the template. The template for bilateral matching can be configured using pixels and / or positions selected by subsampling in 1).

[0732] For example, when configuring a template for bilateral matching, the area used to configure the template can be a portion of the reference block in the L0 direction and a portion of the reference block in the L1 direction.

[0733] For example, when configuring a template for bidirectional matching, pixels (or positions) for configuring the template can be selected from only a portion of the reference block in the L0 direction and a portion of the reference block in the L1 direction.

[0734] The size of a portion of the reference block in the L0 direction can be smaller than the size of the reference block in the L0 direction.

[0735] For example, the height (vertical dimension) of a portion of the reference block in the L0 direction can be smaller than the height (vertical dimension) of the reference block in the L0 direction.

[0736] For example, the width (horizontal dimension) of a portion of the reference block in the L0 direction can be smaller than the width (horizontal dimension) of the reference block in the L0 direction.

[0737] The size of a portion of the reference block in the L1 direction can be smaller than the size of the reference block region in the L1 direction.

[0738] For example, the height (vertical dimension) of a portion of the reference block in the L1 direction can be smaller than the height (vertical dimension) of the reference block in the L1 direction.

[0739] For example, the width (horizontal dimension) of a portion of the reference block in the L1 direction can be smaller than the width (horizontal dimension) of the reference block in the L1 direction.

[0740] For example, when calculating the cost function between templates in bilateral matching, only some pixels and / or locations within the template region can be selected. The cost function can be calculated only for the selected pixels and / or selected locations.

[0741] For example, when calculating the cost function between templates in bilateral matching, subsampling can be performed on the template regions in the L0 direction and the template regions in the L1 direction.

[0742] For example, when calculating the cost function between templates in a bilateral matching, subsampling can be performed on a portion of the template region in the L0 direction and a portion of the template region in the L1 direction.

[0743] Optionally, for example, when calculating the cost function between templates in bilateral matching, the regions of the templates in the L0 direction and the regions of the templates in the L1 direction can each be divided into at least two regions. Each divided region can be one of the following: 1) a region where subsampling is performed, 2) a region where subsampling is not performed but used to calculate the cost function, and 3) a region not used to calculate the cost function. The cost function between templates can be calculated in bilateral matching using pixels and / or positions selected by subsampling in region 1) and pixels and / or positions within region 2).

[0744] Optionally, for example, when calculating the cost function between templates in bilateral matching, the regions of the templates in the L0 direction and the regions of the templates in the L1 direction can each be divided into at least two regions. Each divided region can be one of the following: 1) a region for which subsampling is performed and 2) a region not used for calculating the cost function. The calculation of the cost function between templates can be performed in bilateral matching using pixels and / or positions selected by subsampling in 1).

[0745] For example, when calculating the cost function between templates in a bilateral matching, the region used to calculate the cost function can be a portion of the template in the L0 direction and a portion of the template in the L1 direction.

[0746] The size of a portion of the template in the L0 direction can be smaller than the size of the template area in the L0 direction.

[0747] For example, the height (vertical dimension) of a portion of the template in the L0 direction can be less than the height (vertical dimension) of the template in the L0 direction.

[0748] For example, the width (horizontal dimension) of a portion of the template in the L0 direction can be smaller than the width (horizontal dimension) of a block of the template in the L0 direction.

[0749] The size of a portion of the template in the L1 direction can be smaller than the size of the template area in the L1 direction.

[0750] For example, the height (vertical dimension) of a portion of the template in the L1 direction can be less than the height (vertical dimension) of the template in the L1 direction.

[0751] For example, the width (horizontal dimension) of a portion of the template in the L1 direction can be smaller than the width (horizontal dimension) of the template in the L1 direction.

[0752] For example, when performing a bilateral matching search process, only some pixels and / or locations within the search range can be selected. Search and / or matching cost calculations can be performed only for the selected pixels and / or selected locations. Alternatively, search and / or matching cost calculations can be performed only for motion information indicating the selected pixels and / or locations.

[0753] For example, when performing a bilateral matching search, subsampling can be performed on all or part of the search range.

[0754] Optionally, for example, when performing bilateral matching search processing, the search range can be divided into at least two regions. Each divided region can be one of the following: 1) a region where subsampling is performed, 2) a region where subsampling is not performed but search processing is performed, and 3) a region where search processing is not performed. Bilateral matching search processing can be performed on pixels and / or positions selected by subsampling in 1) and pixels and / or positions within region 2). Optionally, bilateral matching search processing can be performed on pixels and / or positions selected by subsampling in 1) and motion information indicating pixels and / or positions within region 2).

[0755] Optionally, for example, when performing bilateral matching search processing, the search area can be divided into at least two regions. Each divided region can be one of the following: 1) a region where subsampling is performed and 2) a region where search processing is not performed. Bilateral matching search processing can be performed using pixels and / or locations selected by subsampling in 1). Optionally, bilateral matching search processing can be performed on motion information indicating pixels and / or locations selected by subsampling in 1).

[0756] Details of the Implementation Examples Figure 12 This could be a diagram illustrating various examples of subsampling methods in bilateral matching.

[0757] exist Figure 12 In this context, a shaded sample (or location) can refer to a sample (or location) selected through subsampling.

[0758] For the reference block region in the L0 direction and the reference block region in the L1 direction, such as Figure 12 As shown, subsampling can be performed, and the template can be configured by using only selected sample points (or locations).

[0759] For a portion of the reference block region in the L0 direction and a portion of the reference block region in the L1 direction, such as Figure 12 As shown, subsampling can be performed, and the template can be configured by using only selected sample points (or locations).

[0760] For all or part of the template region of bilateral matching, such as Figure 12 As shown, subsampling can be performed, and the cost function can be calculated only for selected sample points (or locations).

[0761] For both sides of the search scope, whether it is all or part, such as Figure 12 As shown, subsampling can be performed, and search and / or matching cost calculations can be performed only for selected pixels and / or locations.

[0762] For both sides of the search scope, whether it is all or part, such as Figure 12 As shown, subsampling can be performed, and search and / or matching cost calculations can be performed only for motion information indicating the selected pixels and / or locations.

[0763] Bilateral matching can only operate when predefined enabling conditions are met.

[0764] For example, bilateral matching can always be performed.

[0765] For example, bilateral matching can be performed when inter-frame prediction mode is used for the target block and at least two reference blocks are used.

[0766] For example, bilateral matching can only be performed if the first direction and the second direction are different from each other and the first POC interval and the second POC interval are the same. The first direction can be the direction from the target image to the reference image in the L0 direction. The second direction can be the direction from the target image to the reference image in the L1 direction. The first POC interval can be the difference between the POC of the target image and the POC of the reference image in the L0 direction. The second POC interval can be the difference between the POC of the target image and the POC of the reference image in the L1 direction.

[0767] For example, bilateral matching can only be performed if the first direction and the second direction are different from each other. The first direction can be the direction from the target image to the reference image in the L0 direction. The second direction can be the direction from the target image to the reference image in the L1 direction.

[0768] Here, when the first direction and the second direction are different, it can mean that the following equation 1 is satisfied.

[0769] [Equation 1](POC) t -POC0)×(POC t -POC1)<0 Here, when the first direction and the second direction are the same, it can mean that the following equation 2 is satisfied.

[0770] [Equation 2](POC) t -POC0)×(POC t -POC1)>0 POC t It can be a Proof-of-Concept (POC) of the target image.

[0771] POC0 can be the POC of the reference image in the L0 direction.

[0772] POC1 can be the POC of the reference image in the L1 direction.

[0773] Two-sided matching may include at least one search step.

[0774] For example, bilateral matching can be configured to sequentially include the steps of 1) exporting motion information for all blocks and 2) exporting motion information for the sub-blocks of a block. However, the method for exporting motion information executed in each step and the order of the steps are not limited to the above configuration.

[0775] In each search step of bilateral matching, motion information for BM_NUM directions can be improved in both the motion information for the L0 direction and the motion information for the L1 direction. BM_NUM can be 0, 1, 2, or a positive integer. The BM_NUM used in the steps of bilateral matching can be the same or different from each other.

[0776] For example, when BM_NUM is 1 in a specific search step, it can be used only for LX in that specific search step. BM Motion information in the direction is used to improve motion information.

[0777] For example, when BM_NUM is 1 and X is 1 in a particular search step BM When the value is 0, in a specific search step, a search can be performed only for the L0 direction while fixing the template in the L1 direction and the motion information in the L1 direction.

[0778] X BM It can be 0, 1, or a positive integer.

[0779] X BM It can be predefined.

[0780] For example, X BM The direction can be either the L0 direction or the L1 direction, which has a larger POC interval. The POC can be the difference between the POC of the reference image and the POC of the target image (in a specific direction).

[0781] For example, X BM The direction can be the direction with the higher template matching cost between the L0 and L1 directions. Here, the template matching cost in a specific direction can be the template matching cost of motion information in that specific direction.

[0782] For example, regarding X BM Information can be sent / encoded / decoded using signals.

[0783] For example, when the first POC difference is greater than the second POC difference, X BM It can be 0, and otherwise, X BM It can be 1. Optionally, when the first POC difference is greater than the second POC difference, X BM It can be 1, otherwise, X BM It can be 0. The first POC difference can be the difference between the POC of the target image and the POC of the reference image in the L0 direction. The second POC difference can be the difference between the POC of the target image and the POC of the reference image in the L1 direction.

[0784] For example, X can be determined based on a context model and / or a probabilistic model used for entropy encoding and entropy decoding of motion information and encoding parameters of the target block. BM .

[0785] For example, X can be determined based on at least one of the context model and / or probability model used when entropy coding and entropy decoding of inter-frame prediction indicators in the target block. BM .

[0786] For example, based on the context model and / or probability model used for entropy coding and entropy decoding of inter-frame prediction indicators, the direction more likely in the target block between unidirectional prediction in the L0 direction and unidirectional prediction in the L1 direction can be selected as LX. BM Direction. For example, based on the context model and / or probability model used for entropy coding and entropy decoding of inter-frame prediction indicators, the direction more likely in the target block between unidirectional predictions in the L0 direction and unidirectional predictions in the L1 direction can be selected as LX. BM direction.

[0787] Optionally, based on the context model and / or probability model used for entropy coding and entropy decoding of inter-frame prediction indicators, the direction more likely in the target block between unidirectional predictions in the L0 direction and unidirectional predictions in the L1 direction can be selected as L(1-X). BM ) direction. For example, based on the context model and / or probability model used for entropy coding and entropy decoding of inter-frame prediction indicators, the direction more likely in the target block between unidirectional prediction in the L0 direction and unidirectional prediction in the L1 direction can be selected as L(1-X). BM )direction.

[0788] A more probable direction may refer to a direction that uses fewer bits when performing entropy encoding using a context model and / or a probabilistic model. Alternatively, a more probable direction may refer to a direction with a higher probability indicated by the context model and / or the probabilistic model for that direction.

[0789] For example, LX can be determined based on the inter-frame bidirectional prediction weights of the target block. BM For example, LX BM It can be the direction with higher inter-frame bidirectional prediction weight in the target block, either the L0 or L1 direction. Alternatively, for example, LX can be the direction with lower inter-frame bidirectional prediction weight in the target block, either the L0 or L1 direction.

[0790] For example, X can be determined based on at least one of the motion information of neighboring blocks and encoding parameters. BM .

[0791] Figure 22 Examples of neighboring blocks of the target block are shown.

[0792] For example, it can be based on Figure 22 The motion information of the neighboring blocks corresponding to at least one of A0, A1, B0, B1 and B2 and at least one of the encoding parameters are used to determine X in the target block.

[0793] For example, X in the target block can be determined based on at least one of the inter-prediction indicator in neighboring blocks and the inter-bidirectional prediction weights. BM .

[0794] For example, one or more context models and / or probability models can be used for X BM Entropy encoding and entropy decoding.

[0795] In multiple contextual models and / or probabilistic models, X for the target block can be determined based on at least one of motion information and encoding information from neighboring blocks. BM Contextual and / or probabilistic models for entropy encoding and entropy decoding.

[0796] For example, X used in a block BM The context model and / or probability model for entropy encoding and entropy decoding can be the same. Optionally, the X used in the block... BM The context model and / or probability model for entropy encoding and entropy decoding can vary depending on at least one of the inter-frame prediction direction of neighboring blocks and the inter-frame bidirectional prediction weights.

[0797] The same X can be used in the search step of bilateral matching. BM Optionally, different X values ​​can be used in the search steps of bilateral matching. BM .

[0798] For example, when performing bilateral matching, the cost function used to calculate the cost of bilateral matching can be determined based on the weights of the weighted inter-frame bidirectional prediction of the target block and / or the weight index of the weighted inter-frame bidirectional prediction.

[0799] In the embodiments, weighted bidirectional prediction may refer to bidirectional prediction with CU-level weights (BCW).

[0800] For example, when the first and second weights of the target block are the same, the bilateral matching cost can be calculated using SAD or SATD. When the first and second weights of the target block are different, the bilateral matching cost can be calculated using MRSAD or MRSATD. Here, the first weight can be the weight for the L0 direction in weighted inter-frame bidirectional prediction. The second weight can be the weight for the L1 direction in weighted inter-frame bidirectional prediction.

[0801] BM_NUM can be 0, 1, 2 or a positive integer.

[0802] BM_NUM can be predefined.

[0803] BM_NUM in each search step of bilateral matching can be determined based on encoding parameters. Alternatively, BM_NUM can be determined based on at least one of motion information, the search step of bilateral matching, the matching cost in the previous search step, the matching cost for the initial motion information in the current search step, and BM_NUM in the previous search step.

[0804] For example, BM_NUM in the first search step of a bilateral matching can be 1 or 2.

[0805] For example, BM_NUM in the current search step can be determined based on the matching cost in the previous search step.

[0806] For example, BM_NUM in the current search step can be 0 when the difference between the matching cost for the initial motion information in the previous search step and the matching cost for the improved motion information in the previous search step is less than COSTDIFF_FORBMNUM.

[0807] COSTDIFF_FORBMNUM can be 0, 1, 2, 4, 8, 16 or a positive integer.

[0808] For example, COSTDIFF_FORBMNUM can be determined based on the size of the target block. COSTDIFF_FORBMNUM can be the product of the number of pixels in the target block and a specific value. The specific value can be 0, 1, 2, 4, 8, or a positive integer.

[0809] For example, if BM_NUM was 0 in a previous search step, BM_NUM in the current search step can be 0.

[0810] For example, if the matching cost for initial motion information in the current search step is less than COSTDIFF_FORBMNUM_INIT, the BM_NUM of the target block can be 0.

[0811] COSTDIFF_FORBMNUM can be 0, 1, 2, 4, 8, 16 or a positive integer.

[0812] For example, COSTDIFF_FORBMNUM_INIT can be determined based on the size of the target block. COSTDIFF_FORBMNUM_INIT can be the product of the number of pixels in the target block and a specific value. The specific value can be 0, 1, 2, 4, 8, or a positive integer.

[0813] For example, when motion improvement is performed in a bidirectional matching search step, motion information improvement for motion information in the L0 direction can only be performed if the matching cost of motion information in the L0 direction relative to the initial motion information is greater than COSTDIFF_FORBMNUM_INIT in the current search step.

[0814] For example, when motion improvement is performed in the search step of bilateral matching, motion information improvement for motion information in the L1 direction can only be performed if the matching cost of motion information in the L1 direction relative to the initial motion information is greater than COSTDIFF_FORBMNUM_INIT in the current search step.

[0815] COSTDIFF_FORBMNUM can be 0, 1, 2, 4, 8, 16 or a positive integer.

[0816] For example, COSTDIFF_FORBMNUM_INIT can be determined based on the size of the target block. COSTDIFF_FORBMNUM_INIT can be the product of the number of pixels in the target block and a specific value. The specific value can be 0, 1, 2, 4, 8, or a positive integer.

[0817] When BM_NUM is 0 in a specific search step of a bilateral match, this indicates that motion information improvement is not performed in that specific search step. Alternatively, when BM_NUM is 0 in a specific search step of a bilateral match, this indicates that the specific search step is not performed.

[0818] For example, when performing bilateral matching, BM_NUM in the motion information derivation step for the entire block can be 1, and BM_NUM in the motion information derivation step for sub-blocks can be 2. In this case, in the motion information derivation step for the entire block, only the step for LX can be improved. BM The motion information in the direction can be improved in the step of deriving motion information for sub-blocks, and the motion information in both the L0 and L1 directions can be improved.

[0819] Figure 23 This shows a bilateral match based on the example.

[0820] Figure 23 This illustrates the case where BM_NUM is 2 in the motion information derivation step for the entire block during bilateral matching.

[0821] MV0 can be the initial motion information for the L0 direction.

[0822] MV1 can be the initial motion information for the L1 direction.

[0823] MV diff It can represent the improved value of motion information derived through bilateral matching.

[0824] MV0' and MV1' can be motion information derived through bilateral matching.

[0825] In bilateral matching, the magnitude of the motion information improvement value for the L0 direction can be the same as the magnitude of the motion information improvement value for the L1 direction. The directions of the motion information improvement values ​​for the L0 direction and the L1 direction can be opposite to each other. In other words, equations 3 and 4 below can be established.

[0826] [Equation 3] MV0' = MV0 + MV diff [Equation 4] MV1′=MV1-MV diff Subsampling in the embodiments can be performed based on at least one of the following: whether template matching is performed; an indicator indicating whether template matching is performed; whether bilateral matching is performed; an indicator indicating whether bilateral matching is performed; motion information of the target block; encoding parameters of the target block; and search steps in template matching and / or bilateral matching.

[0827] For example, the subsampling method can be performed based on at least one of the following: whether template matching is performed; an indicator indicating whether template matching is performed; whether bilateral matching is performed; an indicator indicating whether bilateral matching is performed; motion information of the target block; encoding parameters of the target block; and search steps in template matching and / or bilateral matching.

[0828] For example, when performing template matching and / or bilateral matching, the subsampling method used for the search step can be the same.

[0829] Alternatively, for example, when performing template matching and / or bilateral matching, the subsampling method used for each search step can be different.

[0830] In an embodiment, whether to perform subsampling can be based on at least one of the following: whether to perform template matching; an indicator indicating whether to perform template matching; whether to perform bilateral matching; an indicator indicating whether to perform bilateral matching; motion information of the target block; encoding parameters of the target block; and search steps in template matching and / or bilateral matching.

[0831] For example, whether to perform subsampling can be based on at least one of the following: whether to perform template matching; an indicator indicating whether to perform template matching; whether to perform bilateral matching; an indicator indicating whether to perform bilateral matching; motion information of the target block; encoding parameters of the target block; and search steps in template matching and / or bilateral matching.

[0832] For example, whether or not subsampling is performed can be the same for the search step when performing template matching and / or bilateral matching.

[0833] Optionally, for example, when performing template matching and / or bilateral matching, whether or not subsampling is performed can be different for each search step.

[0834] For example, whether to perform subsampling in the horizontal direction and whether to perform subsampling in the vertical direction can be the same.

[0835] For example, whether to perform subsampling in the horizontal direction and whether to perform subsampling in the vertical direction can be different from each other.

[0836] The subsampling method in the search step that performs a search on the entire target block in template matching and / or bilateral matching can be the same or different from the subsampling method in the search step that performs a search on sub-blocks within the target block in template matching and / or bilateral matching.

[0837] A mapping model can be a model that represents the correlation from a first set of variables to a second set of variables.

[0838] The first set of variables can be called the set of independent variables (or a group of independent variables).

[0839] The set of independent variables can be a group of variables that are expected to have an effect on or can have an effect on other variables (dependent variables).

[0840] The second set of variables can be called the set of dependent variables (or a group of dependent variables).

[0841] The dependent variable set can be a group of variables whose values ​​are determined or expected to be determined based on other variables (independent variables).

[0842] A mapping model can be a model that represents how the dependent variable changes as the independent variable changes.

[0843] Prediction can be performed on at least one sample value of the target block based on a mapping model that calculates the correlation between independent sample sets and dependent sample sets, as well as the main sample set.

[0844] Prediction can be performed on at least one sample of the target block based on the result obtained by inputting the subject sample set into the mapping model or a set of results obtained.

[0845] At least one sample value of the target block can be changed based on a mapping model that calculates the correlation between independent sample sets and dependent sample sets, as well as the main sample set.

[0846] At least one sample value of the target block can be changed based on the result obtained by inputting the subject sample set into the mapping model or a set of results obtained.

[0847] As an example, for the inter-component prediction mode, the independent sample set can represent the reconstructed luminance component samples around the target block, the dependent sample set can represent the reconstructed chrominance component samples around the target block, and the main sample set can represent the target block of the chrominance component.

[0848] Figure 24 A to Figure 24 B is a flowchart of an encoding method for a target block according to an embodiment.

[0849] An embodiment may be part of a method for encoding a target block or a method for encoding video.

[0850] The target block can be encoded by including at least one of the following steps: a step for configuring the sample set and deriving the mapping model; a step for applying the mapping model to the target block; and a step for entropy encoding the encoded information.

[0851] Figure 25 A to Figure 25 B is a flowchart of a decoding method for a target block according to an embodiment.

[0852] An embodiment may be part of a method for decoding a target block or a method for decoding video.

[0853] The target block can be decoded by including at least one of the following steps: a step for entropy decoding of the encoded information; a step for configuring the sample set and deriving the mapping model; and a step for applying the mapping model to the target block.

[0854] For example, a model-based prediction method can be performed by including the following processing steps.

[0855] - Steps for configuring independent sample sets, dependent sample sets, and main sample sets - Steps for configuring inputs for each sample / location in an independent sample set. - Steps for configuring the target output for each sample / location in the dependent sample set corresponding to each sample / location in the independent sample set. - Steps for deriving the coefficients of the mapping model to minimize the sum (or average) of the differences between the input and the target output for all samples / locations in the independent sample set. - Steps for configuring inputs for each sample / location in the main sample set. - Used to output the input and the result value from the mapping model for each sample / location in the main sample set, and then perform at least one step (prediction, reconstruction, and decoding) in the target block using the result value. The input structure can be about how to configure the input when configuring the input for a predetermined sample point (or location).

[0856] The output structure can be about what form the output takes for a given input.

[0857] The input structure of a mapping model may include information including at least one of the following: information about each input component of the mapping model, the relationship between at least two input components of the mapping model (e.g., the positional relationship between the first input component and the second input component; e.g., the method of deriving the second input component from the first input component), and information about the number of input components of the mapping model (the size of the input).

[0858] The output structure of a mapping model may include at least one of the following: information about each output component of the mapping model, the relationship between at least two output components of the mapping model, and information about the number of output components of the mapping model (the size of the output).

[0859] Information about a particular input (or output) component may include at least one of the following: the type of the corresponding input (or output) component, the method used to derive the corresponding input (or output) component, the method used to determine the corresponding input (or output) component, and its relationship with another input (or output) component (e.g., positional relationship, deriving method).

[0860] The type of a particular input (or output) component may include at least one of the following: the value of the sample and / or a value based on the value of the sample; the position of the sample and / or a value based on the position of the sample; and an offset.

[0861] The value based on a specific sample point can be (at least one of the target block, the neighboring block of the target block, the reference block of the target block, and the neighboring block of the reference block of the target block) (at least one of the luminance component and the chrominance component) (at least one of the prediction sample point, the reconstruction sample point, the residual sample point, and the decoding sample point).

[0862] A value based on a specific sample point can represent the result of performing a specific operation on the sample point's value or a value that includes the specific sample point's value.

[0863] The value based on the location of a specific sample point can represent the result of performing a specific operation on the location of that specific sample point or the value of the location including that specific sample point.

[0864] For example, the input structure can have four input components.

[0865] For example, the input structure can have four inputs for a specific sample point, where each input can be the corresponding sample point, the sample point to the left of the corresponding sample point, the sample point to the right of the corresponding sample point, and the bias.

[0866] When the first input structure includes the second input structure, it can refer to one of the following.

[0867] - When inputs are configured for the same point (or location) using a first input structure and a second input structure, the input using the first input structure includes all input components of the input using the second input structure.

[0868] - Optionally, when inputs are configured for the same point (or location) using a first input structure and a second input structure, the input using the first input structure includes at least one input component of the input using the second input structure.

[0869] In this disclosure, bias and bias value can be used with the same meaning and can be substituted for each other.

[0870] The bias can be based on at least one of the following determined values: motion information of the target block, encoding parameters of the target block, size of the target block, range of values ​​that the luminance component of the target block can have, range of values ​​that the chrominance component of the target block can have, availability of neighboring blocks of the target block, encoding parameters of neighboring blocks of the target block, and neighboring samples of the target block.

[0871] For example, at least one of the bias values ​​may be a value obtained by dividing the range of possible values ​​of the luminance component (or chrominance component) of the target block by a specific value.

[0872] The specific value can be 2, 4, or a positive integer.

[0873] For example, at least one of the bias values ​​can be the median of the values ​​that the luminance component (or chrominance component) of the target block can have.

[0874] The median value of the luminance component (or chrominance component) can be a value determined by the bit depth or a predefined value.

[0875] For example, at least one of the bias values ​​may be a value of a sample point existing at a specific location based on the target block or a value of a sample point existing at a specific location based on the co-location block.

[0876] Figure 27 It is a diagram used to describe a specific location.

[0877] At least one of the bias values ​​can exist Figure 27 The values ​​of the sample points at at least one of the shadow positions of the target block and the shadow positions of the co-position blocks.

[0878] The size of a block can refer to at least one of the block's width, block's height, and the number of sample points in the block.

[0879] In this disclosure, a particular operation may include at least one of exponentiation, weighted average, weighted sum, arithmetic operation, and filtering.

[0880] For example, a particular operation may include at least one of a weighted average, a weighted sum, and an arithmetic operation with respect to a particular value.

[0881] A specific value can be determined based on at least one of the following: the prediction mode of the target block, the motion information of the target block, the coding parameters of the target block, the size of the target block, the range of values ​​that the luminance component of the target block can have, the range of values ​​that the chrominance component of the target block can have, the availability of neighboring blocks of the target block, the coding parameters of the neighboring blocks of the target block, and the neighboring samples of the target block.

[0882] For example, a specific value can exist in Figure 27 The value of the sample point at at least one of the shaded positions in the diagram.

[0883] For example, a specific value could be the median of the range of values ​​that the luminance component (or chrominance component) of the target block can have.

[0884] The position of a sample point can be represented by its relative position to the top-left sample point of the target block or its relative position to the top-left sample point of the image.

[0885] The position in the image can refer to the relative position to the top left sample point of the image.

[0886] For example, a specific value could be the x-component and / or y-component at the location of the top-left sample point of the target block.

[0887] The value based on the first value can refer to the result of performing a specific operation on the first value.

[0888] The operation of selecting at least one sample point can be considered as the operation of configuring the sample point set.

[0889] When determining the region of sample points and using at least one of the following to derive the mapping model, the operation of determining the region of sample points can be considered as the operation of configuring the sample point set: the value of at least one sample point in the corresponding region; the position information of at least one sample point; the value of at least one sample point; the motion information of at least one sample point; and the value based on the motion information and / or position information of at least one sample point.

[0890] Figure 26 This is a diagram illustrating the reference region in a model-based prediction method based on an example.

[0891] For example, when determined Figure 26 When a reference region is defined in a map and the value of at least one sample point within that region is used to derive a mapping model, the operation of determining the reference region can be considered as the operation of configuring a sample point set by using sample points within the reference region.

[0892] A sample set can refer to at least one of an independent sample set, a dependent sample set, and a principal sample set.

[0893] Information for determining at least one of the independent sample set, dependent sample set, or main sample set can be predefined in the encoder and decoder. Based on predefined rules in the encoder and decoder, the sample set used to derive the mapping model can be configured, and / or the sample set to which the mapping model will be applied can be configured.

[0894] An independent sample set can be a set of independent variables in a mapping model.

[0895] Alternatively, a set of independent variables for the mapping model can be determined based on an independent sample set.

[0896] The dependency sample set can be a set of dependent variables in the mapping model. Alternatively, a set of dependent variables in the mapping model can be determined based on the dependency sample set.

[0897] The main sample set can be a set of variables for which a mapping model is applied to predict at least one sample of the target block and / or to change the value of at least one sample of the target block. Optionally, the set of variables for which a mapping model is applied to predict at least one sample of the target block and / or to change the value of at least one sample of the target block can be determined based on the main sample set.

[0898] The input to the mapping model can be configured for each sample (or location) in the independent sample set and / or the main sample set.

[0899] When configuring the input of the mapping model for each sample (or location) in the independent sample set, the target output corresponding to the input can be configured for each sample (or location) in the dependent sample set that corresponds to each sample (or location) in the main sample set.

[0900] Each of the independent sample set, the dependent sample set, and the main sample set may include at least one of the following samples: samples in the reference region, samples in the target block, and samples in the block corresponding to the main block.

[0901] For example, the reference region may not include sample points in the reference region of the target block, or it may include at least one sample point in the reference region of the target block.

[0902] For example, a reference region may not include sample points in the reference region of a reference block, or it may include at least one sample point in the reference region of a reference block.

[0903] For example, the reference region can be a region adjacent to at least one of the following: the target block, the reference region, and the block corresponding to the target block.

[0904] In this disclosure, "adjacent" may not necessarily mean "contact".

[0905] Figure 26 This is a diagram illustrating the reference region in a model-based prediction method based on an example.

[0906] Each of LINENUM_L, LINENUM_LB, LINENUM_A, and LINENUM_RA can be 0, 1, 2, or a positive integer.

[0907] Each of LINENUM_L, LINENUM_LB, LINENUM_A, and LINENUM_RA can be a predefined value.

[0908] LINENUM_MORE_X can be 0, 1, 2 or a positive integer (X is A, B, L, R).

[0909] LINENUM_MORE_X can be a predefined value. Optionally, information about LINENUM_MORE_X can be sent / encoded / decoded using signals.

[0910] For example, when configuring the input of a mapping model for specific samples in an independent sample set of the mapping model, when using samples that are at most N away from the corresponding sample in a specific direction, the value of LINENUM_MORE_X for the corresponding direction can be determined as N. N can be 0, 1, or a positive integer. The specific direction can include at least one of top, bottom, left, or right.

[0911] In this disclosure, "determining specific information based on a mapping model" can mean determining specific information based on at least one of the following: MODEL_INPUT_NUM of the mapping model; MODEL_OUTPUT_NUM of the mapping model; at least one bias value in the mapping model; a method for configuring the input of the mapping model; the input structure of the mapping model; the output structure of the mapping model; a method for performing (prediction, reconstruction, decoding) of samples in a target block from the output of the mapping model; and a method for changing the (prediction, reconstruction, decoding) values ​​of samples in a target block from the output of the mapping model, but the meaning of "determining specific information based on a mapping model" is not limited thereto.

[0912] Each of the independent sample set, dependent sample set, and main sample set may include at least one of the following: a predicted sample, a reconstructed sample, a residual sample, and a decoded sample, representing at least one of the luminance component and at least one of the following: a target block, a neighboring block of the target block, a reference block of the target block, and a neighboring block of the reference block of the target block.

[0913] When a specific sample point is included in a specific sample point set, this can mean that at least one of the following is included in the specific sample point set: the sample point value corresponding to the specific sample point; the motion information corresponding to the specific sample point; the encoding parameters corresponding to the specific sample point; the prediction mode corresponding to the specific sample point; and motion information in the block corresponding to the specific sample point, the x-component of the sample point's position, the y-component of the sample point's position, and motion information at the position of the specific sample point. Optionally, when a specific sample point is included in a specific sample point set, this can mean that it is included in the specific sample point set based on at least one of the following values: the sample point value corresponding to the specific sample point; the motion information corresponding to the specific sample point; the encoding parameters corresponding to the specific sample point; the prediction mode corresponding to the specific sample point; and motion information in the block corresponding to the specific sample point, the x-component of the sample point's position, the y-component of the sample point's position, and motion information at the position of the specific sample point.

[0914] "Specific samples are included in a specific sample set" and "Specific samples are used to configure a specific sample set" can be used with the same meaning and can be used interchangeably in the embodiments.

[0915] "Value based on a specific value" can refer to the result of performing a specific operation on a specific value.

[0916] Predicted samples can refer to the luminance component prediction component and / or the chrominance component prediction component.

[0917] Reconstructed samples can refer to luminance component reconstruction and / or chrominance component reconstruction.

[0918] Residual sample points can refer to the residual components of the luminance component and / or chrominance component.

[0919] Decoding sample points can refer to the luminance component decoding component and / or the chrominance component decoding component.

[0920] The reference block may include at least one of the following: a reference block in the L0 direction of inter-frame prediction, a reference block in the L1 direction of inter-frame prediction, a reference block in intra-frame template matching, and a reference block in intra-frame block copying.

[0921] Intra-frame template matching mode can refer to a template matching method for each of the reference block, reference template, and reference region, including at least one of the predicted sample, reconstructed sample, residual sample, and decoded sample of the target image.

[0922] Information about how to configure at least one of the independent sample set, dependent sample set, and main sample set can be sent / encoded / decoded using signals.

[0923] A method for configuring a specific sample set may mean a method that includes at least one of the following: 1) the number of samples used to configure the sample set and / or the maximum number of samples used to configure the sample set; 2) the size and / or location of the region where the samples of the sample set are located; 3) selecting the shape, size, and / or location of the region where the samples of the sample set are configured; 4) the type of at least one sample used to configure the sample set; 5) whether to include an offset value; 6) a method for determining the offset value; 7) a filling method when configuring the corresponding sample set; and 8) the size and / or location of a reference region.

[0924] The method for configuring a specific sample set can be distinguished based on at least one of the following: 1) the number of samples used to configure the sample set and / or the maximum number of samples used to configure the sample set; 2) the size and / or location of the region where the samples of the sample set are located; 3) the shape, size and / or location of the region where the samples of the sample set are selected; 4) the type of at least one sample used to configure the sample set; 5) whether an offset value is included; 6) the method used to determine the offset value; 7) the filling method when configuring the corresponding sample set; and 8) the size and / or location of the reference region.

[0925] Information regarding the method for configuring the sample set may include at least one of the following: 1) the number of samples used to configure the sample set and / or the maximum number of samples used to configure the sample set; 2) the size and / or location of the region where the samples of the sample set are located; 3) the shape, size, and / or location of the region where the samples of the sample set are selected; 4) the type of at least one sample used to configure the sample set; 5) whether an offset value is included; 6) the method for determining the offset value; 7) the filling method when configuring the corresponding sample set; and 8) the size and / or location of the reference region.

[0926] Information regarding the method for configuring at least one of the independent sample set, dependent sample set, and main sample set can be determined based on at least one of the following: the prediction mode of the target block, the motion information of the target block, the encoding parameters of the target block, the size of the target block, the range of values ​​that the luminance component of the target block can have, the range of values ​​that the chrominance component of the target block can have, the availability of neighboring blocks of the target block, the encoding parameters of the neighboring blocks of the target block, and the neighboring samples of the target block.

[0927] For example, information about a method for configuring at least one of the following can be determined based on at least one of the target block's prediction mode, target block motion information, target block encoding parameters, target block size, range of values ​​that the target block's luminance component can have, range of values ​​that the target block's chrominance component can have, availability of the target block's neighboring blocks, encoding parameters of the target block's neighboring blocks, neighboring samples of the target block, and mapping model, without having to send / encode / decode that information via signaling.

[0928] For example, a method for configuring at least one of the independent sample set, dependent sample set, and main sample set can be determined based on whether neighboring blocks in a specific direction are available to the target block.

[0929] Availability can include existence.

[0930] A specific direction can be one of the following: top left, top, top right, left, right, bottom left, bottom, and bottom right.

[0931] When configuring at least one of the independent sample set, dependent sample set, and main sample set, it can be determined whether to include at least one sample in the corresponding sample set based on whether at least one neighboring block in a specific direction is available to the target block.

[0932] For example, a method for configuring at least one of the independent sample set, dependent sample set, and main sample set can be determined based on the ratio of the width to the height of the target block.

[0933] When configuring at least one of the independent sample set, dependent sample set, and main sample set, it can be determined whether to include at least one sample in the corresponding sample set in a specific direction based on the ratio of the width to the height of the target block.

[0934] A specific direction can be one of the following: top left, top, top right, left, right, bottom left, bottom, and bottom right.

[0935] When configuring at least one of the independent sample set, dependent sample set, and main sample set, the size and / or location of the reference region can be determined based on the width-to-height ratio of the target block.

[0936] For example, a method for configuring at least one of the independent sample set, dependent sample set, and main sample set can be determined based on at least one of the prediction pattern of the target block, motion information, and coding parameters.

[0937] When configuring at least one of the independent sample set, dependent sample set, and main sample set, the type of at least one sample in the sample set used for configuring the sample set can be determined based on at least one of the target block's prediction mode, motion information, and coding parameters.

[0938] The sample type may include at least one of the following: (at least one of the target block, the neighboring block of the target block, the reference block of the target block, and the neighboring block of the reference block of the target block), (at least one of the luminance component and the chrominance component), (at least one of the prediction sample, the reconstruction sample, the residual sample, and the decoding sample); and at least one bias value.

[0939] For example, when the target block is in a geometric partitioning mode, a method can be determined for configuring at least one of the independent sample set, dependent sample set, and main sample set.

[0940] For example, when the target block is a geometric partitioning pattern, at least one of the independent sample set, dependent sample set, and main sample set can be determined based on at least one of the information about the geometric partitioning pattern.

[0941] Information about geometric partitioning patterns may refer to at least one of the partition boundaries, partitioning patterns, partitioning angles, and partitioning offsets of a geometric partitioning pattern, but information about geometric partitioning patterns is not limited to these.

[0942] For example, for the boundary surface in a specific direction of the target block, it can be determined whether to use the neighboring samples in the specific direction to configure at least one of the independent sample set, dependent sample set and main sample set based on the sample value of at least one neighboring sample in BOUND_LINE_NUM adjacent rows / columns and / or at least one sample in BOUND_LINE_NUM rows / columns in the target block.

[0943] BOUND_LINE_NUM can be 1, 2, or a positive integer. BOUND_LINE_NUM can be determined based on at least one of the following: motion information of the target block, encoding parameters of the target block, size of the target block, range of possible values ​​for the luminance component of the target block, range of possible values ​​for the chrominance component of the target block, availability of neighboring blocks of the target block, encoding parameters of neighboring blocks of the target block, and neighboring samples of the target block.

[0944] Figure 28 This is an example used to describe neighboring samples within BOUND_LINE_NUM rows / columns adjacent to the target block and samples within BOUND_LINE_NUM rows / columns of the target block.

[0945] Figure 29 This is an embodiment of a configuration for determining whether to use neighboring samples in the top direction for at least one of the independent sample set, dependent sample set, and main sample set, based on the sample value of at least one neighboring sample in two adjacent rows / columns and / or at least one sample in two rows / columns within the target block, for the top boundary surface of the target block.

[0946] exist Figure 29 In the middle, when When the value of (W0×p(x,2)+W1+p(x,1)+W2×q(x,1)+W3×q(x,2)) is greater than or equal to a specific threshold, neighboring samples in the top direction of the target block may not be used to configure the independent sample set, dependent sample set, and main sample set. For example, when When the value of (W0×p(x,2)+W1+p(x,1)+W2×q(x,1)+W3×q(x,2)) is less than a certain threshold, at least one neighboring sample point in the top direction of the target block can be used to configure at least one of the independent sample point set, dependent sample point set, and topic sample point set.

[0947] Since the block size is 8×8 in this embodiment, x has a value from 0 to 7, but even when the block size is 8×8, x can have values ​​within a different range. Additionally, x can have values ​​determined in intervals greater than 1 (e.g., 0, 2, 4, 6).

[0948] For example, a mapping model can be a regression model.

[0949] For example, the mapping model can be a filter.

[0950] For example, a mapping model can be derived through regression analysis.

[0951] For example, to reduce the complexity of deriving the mapping model, at least one of LU decomposition, LDL decomposition, and Gaussian elimination can be used.

[0952] A mapping model can be a model that takes inputs including MODEL_INPUT_NUM components and outputs result values ​​with MODEL_OUTPUT_NUM components.

[0953] When a specific sample is included in the input of a mapping model, this can mean that at least one of the following is included in the specific sample set: the sample value corresponding to the specific sample; the motion information corresponding to the specific sample; the encoding parameters corresponding to the specific sample; the prediction mode corresponding to the specific sample; and the motion information in the block corresponding to the specific sample, the x-component at the sample's location, the y-component at the sample's location, and the motion information at the specific sample's location. Optionally, when a specific sample is included in a specific sample set, this can mean that the input is based on at least one of the following values: the sample value corresponding to the specific sample; the motion information corresponding to the specific sample; the encoding parameters corresponding to the specific sample; the prediction mode corresponding to the specific sample; and the motion information in the block corresponding to the specific sample, the x-component at the sample's location, the y-component at the sample's location, and the motion information at the specific sample's location.

[0954] "Specific samples are included in the input" and "Specific samples are used to configure the input" can be used with the same meaning and can be used interchangeably in the embodiments.

[0955] "Result of the mapping model" and "output of the mapping model" can be used with the same meaning and can be used interchangeably in the embodiments.

[0956] Each of MODEL_INPUT_NUM and MODEL_OUTPUT_NUM can be a positive integer.

[0957] Each of MODEL_INPUT_NUM and MODEL_OUTPUT_NUM can be a predefined value. Optionally, information about at least one of MODEL_INPUT_NUM and MODEL_OUTPUT_NUM can be sent / encoded / decoded using signals.

[0958] The mapping model can be a linear model. Alternatively, the mapping model can be a nonlinear model.

[0959] Mapping model f, input X = {x1, x2, ..., X} MODEL_INPUT_NUM} and the result value Y={y1, y2, ..., y MODEL_OUTPUT_NUM The relationship between them can be represented as shown in the following examples.

[0960] Y=f(X) y1 = a 1,1 × x1+ a 2,1 × x2 + ... + a (MODEL_INPUT_NUM),1 × x MODEL_INPUT_NUM y2 = a 1,2 × x1+ a 2,2 × x2 + ... + a (MODEL_INPUT_NUM),2 × x MODEL_INPUT_NUM ... y MODEL_OUTPUT_NUM = a 1, MODEL_OUTPUT_NUM × x1+ a 2, MODEL_OUTPUT_NUM × x2 + ... +a (MODEL_INPUT_NUM), (MODEL_OUTPUT_NUM) × x MODEL_INPUT_NUM a i,j It can represent the coefficients of the mapping model f.

[0961] i can have values ​​from 1 to MODEL_INPUT_NUM.

[0962] j can have values ​​from 1 to MODEL_OUTPUT_NUM.

[0963] When deriving the mapping model f, this can be used to derive at least one of the coefficients of the mapping model f.

[0964] For a specific sample in an independent sample set, the input comprising MODEL_INPUT_NUM components can be determined by including at least one of the following: the corresponding sample; neighboring samples adjacent to the corresponding sample; neighboring samples that are not adjacent (not in contact) to the corresponding sample; an offset value; and the result of performing a specific operation on at least one of the above elements.

[0965] For example, at least one of the neighboring samples and / or at least one of the bias values ​​used to configure inputs for a specific sample in an independent sample set may not be included in the independent sample set.

[0966] For example, the input can include bias values ​​that are independent of specific samples.

[0967] For a specific sample in the independent sample set, the target output comprising MODEL_OUTPUT_NUM components can be determined by including at least one of the following: the sample corresponding to the corresponding sample in the dependent sample set; the neighboring sample adjacent to the sample corresponding to the corresponding sample in the dependent sample set; the neighboring sample not adjacent (not in contact) to the sample corresponding to the corresponding sample in the dependent sample set; and the bias value.

[0968] A mapping model can be a model that calculates the correlation between the input and the target output for each sample in an independent sample set.

[0969] A mapping model can be derived such that, when the input and target output are determined for each sample in an independent sample set, the result value of the mapping model for the input of all samples is similar in mean to the target output.

[0970] For a specific sample in an independent sample set, when using neighboring samples as input, if the specific neighboring sample is unavailable, a sample determined based on at least one of the available samples that is closest in location to the corresponding neighboring sample can be used as input instead of the specific neighboring sample. This process can refer to performing a description of filling in the corresponding neighboring sample.

[0971] A sample in the dependent sample set corresponding to a specific sample in the independent sample set can mean at least one of the following: a sample in the dependent sample set corresponding to the position of the corresponding sample; and / or at least one of the target outputs determined for the corresponding sample.

[0972] For example, when configuring independent sample sets and / or dependent sample sets, NUM_SAMPLE_TO_REF sample points can be used.

[0973] NUM_SAMPLE_TO_REF can be a fixed constant value. Alternatively, NUM_SAMPLE_TO_REF can be an adaptively determined value based on at least one of the following: the prediction mode of the target block, the motion information of the target block, the coding parameters of the target block, the size of the target block, the range of values ​​that the luma component of the target block can have, the range of values ​​that the chroma component of the target block can have, the availability of neighboring blocks of the target block, the coding parameters of the neighboring blocks of the target block, and the neighboring samples of the target block.

[0974] For example, when the size of the target block is equal to or less than a specific value, the value of NUM_SAMPLE_TO_REF can be NUM_SAMPLE_TO_REF=(H+W); otherwise, the value of NUM_SAMPLE_TO_REF can be NUM_SAMPLE_TO_REF=(H+W) / STEP_TO_REF.

[0975] STEP_TO_REF can be 2, 4, 8 or a positive integer.

[0976] Optionally, for example, when the size of the target block is equal to or less than a specific value, the value of NUM_SAMPLE_TO_REF can be NUM_SAMPLE_TO_REF=(H+W); otherwise, the value of NUM_SAMPLE_TO_REF can be NUM_SAMPLE_TO_REF=NUM_SAMPLE_TO_REF_CONST.

[0977] NUM_SAMPLE_TO_REF_CONST can be 8, 16, 32, 64, 128, or a positive integer.

[0978] The size of a target block can refer to at least one of the following: the target block's height, width, the product of its height and width, the sum of its height and width, the larger of its height and width, the smaller of its height and width, and its diagonal length.

[0979] For a specific sample in the dependent sample set, the target output comprising MODEL_OUTPUT_NUM components can be determined by including at least one of the following: the corresponding sample; neighboring samples adjacent to the corresponding sample; neighboring samples that are not adjacent (not in contact) to the corresponding sample; the bias value; and the result of performing a specific operation on at least one of the above elements.

[0980] For example, at least one of the neighboring samples and / or at least one of the bias values ​​used to configure inputs for a specific sample in an independent sample set may not be included in the independent sample set.

[0981] For a specific sample in the dependent sample set, the input comprising MODEL_INPUT_NUM components can be determined by including at least one of the following: the sample corresponding to the corresponding sample in the independent sample set; the neighboring sample adjacent to the sample corresponding to the corresponding sample in the independent sample set; the neighboring sample not adjacent (not in contact) to the sample corresponding to the corresponding sample in the dependent sample set; the bias value; and the result of performing a specific operation on at least one of the above elements.

[0982] A mapping model can be a model that calculates the correlation between the input and the target output for each sample in a dependent sample set.

[0983] When the input and target output are determined for each sample in the dependent sample set, a mapping model can be derived such that the result value of the mapping model for the input of all samples is similar to the target output in mean.

[0984] For a specific sample in the dependent sample set, when using neighboring samples of the corresponding sample as the target output, if the specific neighboring sample is unavailable, a sample determined based on at least one of the available samples that is closest in position to the corresponding neighboring sample can be used instead of the specific neighboring sample as the target output. This process can represent the performance of padding for the corresponding neighboring sample.

[0985] A sample in the independent sample set corresponding to a specific sample in the dependent sample set can mean at least one of the following: a sample in the independent sample set corresponding to the location of the corresponding sample; and / or at least one of the inputs determined for the corresponding sample.

[0986] Mapping model f, input X = {x1, x2, ..., x} MODEL_INPUT_NUM} and target output Y target = {y target_1 ,y target_2 , ..., y target(MODEL_OUPUT_NUM) The relationship between} can be represented as shown in the following examples. ε can represent the error between the output value of the mapping model for the input and the target output.

[0987] Y target =f(X)+ε A mapping model f can be derived to minimize the sum or average of ε for all samples in the dependent sample set or for some samples in the dependent sample set.

[0988] For example, when deriving a mapping model, at least one coefficient of the mapping model can be determined as a value selected from a candidate list including predefined DEFAULT_COEF_NUM values. DEFAULT_COEF_NUM can be 1, 2, 3, or a positive integer. The value of DEFAULT_COEF_NUM can be predefined, or information about DEFAULT_COEF_NUM can be sent / encoded / decoded using signals. For example, information about which candidate value among the DEFAULT_COEF_NUM candidate values ​​was used to determine the coefficients of the mapping model can be sent / encoded / decoded using signals.

[0989] The input to the mapping model may include at least one of the following: 1) the value of at least one sample in the independent sample set and / or the value based on the value of the at least one sample; 2) the position of at least one sample in the independent sample set and / or the value based on the position of the at least one sample; 3) the value of at least one sample in the main sample set and / or the value based on the value of the at least one sample; 4) the position of at least one sample in the main sample set and / or the value based on the position of the at least one sample; and 5) a bias.

[0990] Information about model-based prediction methods within the target block can be stored. In blocks encoded / decoded after the target block, the model-based prediction method can be executed based on the information stored in the target block. Alternatively, the model-based prediction method can be executed in the target block based on information from blocks encoded / decoded before the target block.

[0991] Information regarding the model-based prediction method in the target block may include at least one of the following: whether the model-based prediction method is executed in the target block and / or an indicator indicating whether the model-based prediction method is executed in the target block; a method for configuring at least one of the independent sample set, dependent sample set, and principal sample set in the target block; at least one of the samples, locations, and values ​​included in the independent / dependent / principal sample set in the target block; a method for determining a reference region for the target block; mapping model information in the target block; whether a multi-mapping model is used in the target block and / or an indicator indicating whether a multi-mapping model is used in the target block; and / or the number and / or maximum number of mapping models in the target block.

[0992] The mapping model information may include at least one of the following: the mapping model's MODEL_INPUT_NUM; the mapping model's MODEL_OUTPUT_NUM; at least one coefficient of the mapping model; the mapping model's input structure; and the mapping model's output structure.

[0993] The type of a particular variable can relate to the components of the corresponding variable. For example, the type of a particular variable can be distinguished based on at least one of whether an operation is performed, the type of operation performed, and the type of the component.

[0994] The type of component may include at least one of the following: (at least one of the target block, the neighboring block of the target block, the reference block of the target block, and the neighboring block of the reference block of the target block), (at least one of the luminance component and the chrominance component), (at least one of the prediction sample, the reconstruction sample, the residual sample, and the decoding sample); and at least one bias value.

[0995] Based on the information about the model-based prediction method stored in the second block (encoded / decoded before the first block), the model-based prediction method in the first block can be executed. Alternatively, based on the information about the model-based prediction method stored in the second block (encoded / decoded before the first block), the information about the model-based prediction method in the first block can be determined.

[0996] In this disclosure, "inheriting information about the model-based prediction method in the second block from the first block" can mean executing the model-based prediction method in the first block based on information about the model-based prediction method stored in the second block that was encoded / decoded prior to the first block. Alternatively, in this disclosure, "inheriting information about the model-based prediction method in the second block from the first block" can mean determining the information about the model-based prediction method in the first block based on information about the model-based prediction method stored in the second block that was encoded / decoded prior to the first block.

[0997] For example, at least one coefficient of at least one mapping model included in the information about model-based prediction methods stored in the second block can be determined as one of the coefficients of the mapping model in the first block.

[0998] Inheriting information about model-based prediction methods from the second block from the first block can be considered as deriving the mapping model f from the first block.

[0999] For example, some coefficients of at least one mapping model included in the information about model-based prediction methods stored in the second block can be identified as some coefficients of the mapping model in the first block. The remaining coefficients of the mapping model in the first block can then be derived through regression analysis.

[1000] Information about the model-based prediction method for the current block can be inherited from at least one candidate block. A candidate block may include at least one of neighboring blocks adjacent to the current block and non-neighboring blocks not adjacent to the current block.

[1001] As an example, such as Figure 27As shown, adjacent blocks can include at least one of the blocks adjacent to the current block above, to the upper left, to the upper right, to the left, and to the lower left. Non-adjacent blocks can include blocks that are at a distance of N from at least one of the upper and left boundaries of the current block, such as... Figure 32 The example shown is illustrated. N can be an integer, such as 4, 8, or 16. After deriving merge candidates from adjacent blocks, merge candidates can be derived using non-adjacent blocks.

[1002] Optionally, a non-adjacent block can be a block indicated by at least one of the following: the block vector of the target block, the motion vector of the target block, the block vector of the luma component block corresponding to the target block when the target block is a chroma component block, the motion vector of the luma component block corresponding to the target block when the target block is a chroma component block, the block vector of the neighboring block, the motion vector of the neighboring block, the block vector around the target block, and the motion vector around the target block.

[1003] For example, a neighboring block can be an adjacent block.

[1004] For example, for at least two of the following blocks: a block indicated by the block vector of the target block, a block indicated by the motion vector of the target block, a block indicated by the block vector of the luma component block corresponding to the target block when the target block is a chroma component block, a block indicated by the motion vector of the luma component block corresponding to the target block when the target block is a chroma component block, a block indicated by the block vector of a neighboring block, a block indicated by the motion vector of a neighboring block, a block indicated by the block vector around the target block, and a block indicated by the motion vector around the target block, merge candidates can be derived from the N blocks among them that have the lowest template matching cost.

[1005] Merge candid...

Claims

1. An image decoding method, the method comprising: Generate a candidate list for the current block; The candidate list is reordered based on the matching cost; Obtain the prediction information for the current block from the reordered candidate list; and Execute the prediction for the current block based on the prediction information for the current block. The candidate list includes at least one of the following: spatial candidates for the current block, temporal candidates, history-based candidates, and default candidates.

2. The method according to claim 1, wherein: The spatial candidates include adjacent spatial candidates that are adjacent to the current block and non-adjacent spatial candidates that are not adjacent to the current block.

3. The method according to claim 2, wherein: The non-adjacent space candidates are determined based on the width and height of the current block.

4. The method according to claim 3, wherein: The non-adjacent spatial candidates are blocks that are N times the unit distance from a specific location in the horizontal or vertical direction.

5. The method according to claim 1, wherein: The time candidate is obtained from the block at the shifted position of the current block's time block.

6. The method according to claim 5, wherein: The time block is determined by the motion vectors of the spatially adjacent blocks of the current block.

7. The method according to claim 1, wherein: The historical candidates are obtained by using a table that stores prediction information accumulated prior to the current block.

8. The method according to claim 1, wherein: The prediction of the current block is performed by performing a weighted sum on the first prediction block obtained based on the first mapping model and the second prediction block obtained based on the second mapping model.

9. The method according to claim 8, wherein: The first mapping model is included in the prediction information of the current block obtained from the candidate list of the current block.

10. An image encoding method, the method comprising: Generate a candidate list for the current block; The candidate list is reordered based on the matching cost; Obtain the prediction information for the current block from the reordered candidate list; and Execute the prediction for the current block based on the prediction information for the current block. The candidate list includes at least one of the following: spatial candidates for the current block, temporal candidates, history-based candidates, and default candidates.

11. The method of claim 10, wherein: The spatial candidates include adjacent spatial candidates that are adjacent to the current block and non-adjacent spatial candidates that are not adjacent to the current block.

12. The method according to claim 11, wherein: The non-adjacent space candidates are determined based on the width and height of the current block.

13. The method according to claim 12, wherein: The non-adjacent spatial candidates are blocks that are N times the unit distance from a specific location in the horizontal or vertical direction.

14. The method of claim 10, wherein: The time candidate is obtained from the block at the shifted position of the current block's time block.

15. The method of claim 14, wherein: The time block is determined by the motion vectors of the spatially adjacent blocks of the current block.

16. The method of claim 10, wherein: The historical candidates are obtained by using a table that stores prediction information accumulated prior to the current block.

17. The method of claim 10, wherein: The prediction of the current block is performed by performing a weighted sum on the first prediction block obtained based on the first mapping model and the second prediction block obtained based on the second mapping model.

18. The method of claim 17, wherein: The first mapping model is included in the prediction information of the current block obtained from the candidate list of the current block.

19. A method for transmitting a bitstream generated by an image encoding method, wherein, The image encoding method includes: Generate a candidate list for the current block; The candidate list is reordered based on the matching cost; Obtain the prediction information for the current block from the reordered candidate list; To obtain the predicted block for the current block, a prediction is performed on the current block based on the prediction information of the current block; and Send the bit stream obtained by encoding the predicted block. The candidate list includes at least one of the following: spatial candidates for the current block, temporal candidates, history-based candidates, and default candidates.

Images