Method, apparatus, and recording medium for encoding / decoding an image

Abstract

The method, apparatus, and recording medium for encoding / decoding an image of the disclosure can include the steps of obtaining an initial merge candidate list of a current block; rearranging the initial merge candidate list to obtain a rearranged merge candidate list of the current block; determining one candidate of the current block among the candidates of the rearranged merge candidate list; and performing prediction of the current block based on the determined candidate.

Description

Technical Field

[0001] This disclosure can be used in methods, apparatus, and recording media for image encoding / decoding. Background Technology

[0002] With the continued development of the information and communication industry, broadcast services supporting high-definition (HD) resolution have become widespread throughout the world. Through this widespread adoption, a large number of users have become accustomed to high-resolution and high-definition images and / or videos.

[0003] To meet user demand for high definition, numerous organizations have accelerated the development of next-generation imaging devices. In addition to High Definition TV (HDTV) and Full High Definition (FHD) TV, user interest in UHD TV has also increased, with UHD TV offering more than four times the resolution of Full High Definition (FHD) TV. With this growing interest, there is now a need for image encoding / decoding technologies specifically designed for images with higher resolution and greater definition.

[0004] As an image compression technique, there are various techniques (such as inter-frame prediction, intra-frame prediction, transform, quantization, filtering, and entropy coding).

[0005] Inter-frame prediction techniques are used to predict the values ​​of pixels included in the current frame using frames preceding and / or following the current frame. Intra-frame prediction techniques are used to predict the values ​​of pixels included in the current frame using information about the pixels in the current frame. Transform and quantization techniques can be used to compress the energy of residual signals. Entropy coding techniques are used to assign short codewords to frequently occurring values ​​and long codewords to less frequent values.

[0006] By utilizing these image compression techniques, data about images can be effectively compressed, transmitted, and stored. Summary of the Invention

[0007] Technical issues This disclosure is used to improve image encoding and decoding efficiency by configuring a candidate list and reordering the candidates.

[0008] Technical solution The image encoding / decoding method, apparatus, and recording medium disclosed herein may include: obtaining an initial merging candidate list for the current block; reordering the initial merging candidate list to obtain a reordered merging candidate list for the current block; determining a candidate for the current block from the candidates in the reordered merging candidate list; and performing prediction for the current block based on the determined candidate.

[0009] In the image encoding / decoding method, apparatus and recording medium disclosed herein, the initial merging candidate list includes at least one spatial candidate, wherein the spatial candidate can be obtained based on blocks adjacent to the current block or blocks not adjacent to the current block.

[0010] In the image encoding / decoding method, apparatus and recording medium disclosed herein, the initial merging candidate list may include Automatic Relocation Block Vector Prediction (AR-BVP) candidates.

[0011] In the image encoding / decoding method, apparatus and recording medium disclosed herein, AR-BVP candidates can be derived based on the block vector at the upper left, upper right, center, lower left or lower right position of the reference block of the current block.

[0012] In the image encoding / decoding method, apparatus and recording medium disclosed herein, the initial merging candidate list may include at least one chained MV prediction (CMVP) candidate.

[0013] In the image encoding / decoding method, apparatus and recording medium disclosed herein, CMVP candidates can be derived based on the temporal MV predictor (TMVP) candidates of the initial merge candidate list.

[0014] In the image encoding / decoding method, apparatus, and recording medium disclosed herein, CMVP candidates can be derived based on the accumulation of MV and BV, wherein MV and BV are recursively tracked based on pre-derived motion vectors from an initial merge candidate list.

[0015] In the image encoding / decoding method, apparatus and recording medium disclosed herein, in response to the initial merging candidate list including multiple CMVP candidates, the multiple CMVP candidates can be reordered based on template cost.

[0016] In the image encoding / decoding method, apparatus, and recording medium disclosed herein, the template cost can be calculated based on the difference between the template of the block indicated by the CMVP candidate and the template of the current block.

[0017] In the image encoding / decoding method, apparatus, and recording medium disclosed herein, the reordered merge candidate list may include only some of the reordered multiple CMVP candidates.

[0018] Technical effect When using the methods disclosed herein for configuring candidate lists and reordering candidates, encoding and decoding efficiency can be improved. Attached Figure Description

[0019] Figure 1 This is a block diagram illustrating the configuration of an embodiment of an encoding device to which the present disclosure is applied.

[0020] Figure 2 This is a block diagram illustrating the configuration of an embodiment of the decoding device to which this disclosure is applied.

[0021] Figure 3 It is a schematic diagram illustrating the partitioning structure of an image when it is encoded and decoded.

[0022] Figure 4 This is a diagram showing the form of prediction units (PUs) that a coding unit (CU) can include.

[0023] Figure 5 This is a diagram showing the form of a transformation unit (TU) that can be included in a CU.

[0024] Figure 6 This shows the block division based on the example.

[0025] Figure 7 This is a diagram illustrating an embodiment used to explain the intra-frame prediction process.

[0026] Figure 8 This is a diagram showing the reference samples used in the intra-frame prediction process.

[0027] Figure 9 This is a diagram illustrating an embodiment used to explain the inter-frame prediction process.

[0028] Figure 10 Spatial candidates according to an embodiment are shown.

[0029] Figure 11 The order in which motion information of spatial candidates is added to the merging list is shown according to an embodiment.

[0030] Figure 12 The transformation and quantization processes are shown based on the example.

[0031] Figure 13 This shows a diagonal scan based on an example.

[0032] Figure 14 The horizontal scan is shown based on the example.

[0033] Figure 15 The vertical scan is shown based on the example.

[0034] Figure 16 This is a configuration diagram of an encoding device according to an embodiment.

[0035] Figure 17 This is a configuration diagram of a decoding device according to an embodiment.

[0036] Figure 18 The shape of the filter template is shown.

[0037] Figure 19The shape of the FIBC filter is shown.

[0038] Figure 20 An embodiment of the IBC predictive signal filtering of this disclosure is shown.

[0039] Figure 21 An example of Dbv predictive signal filtering is shown.

[0040] Figure 22 This diagram illustrates the IBC prediction signal filtering process based on the value of ibc_filtering_flag.

[0041] Figure 23 This shows an example of using a 1×1 rectangular filter when the ibc_filter_shape syntax element is the first value and using a 3×3 cross-shaped filter when the ibc_filter_shape syntax element is the second value.

[0042] Figure 24 This shows an example of using a 3×3 rectangular filter when the ibc_filter_shape syntax element is the first value and a 3×3 cross-shaped filter when the ibc_filter_shape syntax element is the second value.

[0043] Figure 25 An example of an IBC filter mode information signaling block diagram is shown.

[0044] Figure 26 An example of an IBC filter mode information signaling block diagram is shown.

[0045] Figure 27 Examples are shown of all or part of a block calculated using only Diff.

[0046] Figure 28 An example of Intra-Block Copy (IBC) is shown.

[0047] Figure 29 An example of Dbv predictive signal generation is shown.

[0048] Figure 30 An example of a 1×1 rectangular filter is shown.

[0049] Figure 31 An example of a 3×3 rectangular filter is shown.

[0050] Figure 32 An example of a 3×3 cross-shaped filter is shown.

[0051] Figure 33 An example of a 1×1 rectangular filter is shown.

[0052] Figure 34 An example of a 3×3 rectangular filter is shown.

[0053] Figure 35 An example of a 3×3 cross-shaped filter is shown.

[0054] Figure 36 This is a flowchart of an example of deriving motion vectors based on merging candidate vectors.

[0055] Figure 37 An example of merging candidate configuration order is shown.

[0056] Figure 38 An example of AR-BVP export is shown.

[0057] Figure 39 Reference block B is shown. n The five positions: top left, top right, center, bottom left, and bottom right.

[0058] Figure 40 An example of a CMVP candidate export is shown.

[0059] Figure 41 The basic operation flowchart of ARMC is shown.

[0060] Figure 42 Examples of non-adjacent spatial candidates are shown.

[0061] Figure 43 An example is shown of deriving AR-BVP by using a scaled BV obtained through two MVs.

[0062] Figure 44 An example is shown of deriving AR-BVP by using a scaled BV obtained through a MV.

[0063] Figure 45 An example of a shifted NA space candidate is shown.

[0064] Figure 46 An example of AR-BVP export using TBVP is shown.

[0065] Figure 47 This shows an example of CMVP export using TMVP.

[0066] Figure 48 An example of TM refinement for CMVP is shown. Detailed Implementation

[0067] This disclosure can be modified in various ways and can have various embodiments, specific embodiments of which will now be described in detail with reference to the accompanying drawings. However, it should be understood that this disclosure is not intended to be limited to the specific embodiments and includes all variations, equivalents, or substitutions that fall within the spirit and scope of this disclosure.

[0068] The following exemplary embodiments will be described in detail with reference to the accompanying drawings, which illustrate specific embodiments. These embodiments are described to enable those skilled in the art to readily implement them. It should be noted that the various embodiments differ from one another but are not necessarily mutually exclusive. For example, specific shapes, structures, and characteristics described herein may be implemented as other embodiments without departing from the spirit and scope of other embodiments associated with one embodiment. Furthermore, it should be understood that the position or arrangement of various components in each disclosed embodiment can be changed without departing from the spirit and scope of the embodiments. Therefore, the appended detailed description is not intended to limit the scope of this disclosure, and the scope of the exemplary embodiments is defined only by the appended claims and their equivalents (provided they are properly described).

[0069] In the accompanying drawings, similar reference numerals are used to designate the same or similar functions in various respects. The shape, size, etc., of the components in the drawings may be exaggerated for clarity of description.

[0070] Terms such as “first” and “second” may be used to describe various components, but components are not limited by these terms. These terms are used only to distinguish one component from another. For example, without departing from the scope of this specification, a first component may be referred to as a second component. Similarly, a second component may be referred to as a first component. The term “and / or” may include a combination of multiple related descriptive terms or any one of multiple related descriptive terms.

[0071] It will be understood that when a component is referred to as "connected" or "combined" to another component, the two components may be directly connected or combined with each other, or there may be an intermediate component between the two components. On the other hand, it will be understood that when a component is referred to as "directly connected or combined," there is no intermediate component between the two components.

[0072] Furthermore, the components described in the embodiments are shown independently to indicate different functional characteristics, but this does not mean that each component is formed by a single piece of hardware or software. That is, for ease of description, multiple components are arranged and included separately. For example, at least two of the multiple components may be integrated into a single component. Conversely, a component may be divided into multiple components. As long as it does not depart from the spirit of this specification, embodiments in which multiple components are integrated or embodiments in which some components are separated are included within the scope of this specification.

[0073] The terminology used in the embodiments is for describing particular embodiments only and is not intended to limit this disclosure. Singular expressions include plural expressions unless the context specifically indicates the opposite. In the embodiments, it should be understood that terms such as “comprising” or “having” are intended only to indicate the presence of features, numbers, steps, operations, components, parts, or combinations thereof, and are not intended to exclude the possibility of the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof. That is, in the embodiments, the expression describing a component as “comprising” a specific component means that additional components may be included within the practice or technical spirit of this disclosure, but do not exclude the presence of components other than the specific component stated therein.

[0074] In embodiments, the term "at least one" may mean one of one or more quantities (such as 1, 2, 3, and 4). In embodiments, the term "a plurality of" may mean one of two or more quantities (such as 2, 3, and 4).

[0075] Some components of the embodiments are not essential components for performing the necessary functions, but may be optional components used only to improve performance. Embodiments may be implemented using only the essential components necessary to achieve the essence of the embodiments. For example, a structure that includes only the essential components (excluding optional components used only to improve performance) is also included within the scope of the embodiments.

[0076] The embodiments will now be described in detail with reference to the accompanying drawings, enabling those skilled in the art to readily implement the embodiments. In the following description of the embodiments, detailed descriptions of well-known functions or configurations that are considered to obscure key points of this specification will be omitted. Furthermore, the same reference numerals are used throughout the drawings to designate the same components, and repeated descriptions of the same components will be omitted.

[0077] In the following text, "image" may refer to a single frame that constitutes a video, or it may refer to the video itself. For example, "encoding and / or decoding of an image" may mean "encoding and / or decoding of a video," and may also mean "encoding and / or decoding of any one of the multiple images that constitute a video."

[0078] In the following text, the terms “video” and “moving footage” may be used to have the same meaning and may be used interchangeably.

[0079] In the following text, the target image can be an encoded target image that is the target to be encoded and / or a decoded target image that is the target to be decoded. Furthermore, the target image can be an input image input to an encoding device or an input image input to a decoding device. Also, the target image can be the current image, i.e., the target currently to be encoded and / or decoded. For example, the terms "target image" and "current image" can be used to have the same meaning and can be used interchangeably.

[0080] In the following text, the terms “image,” “picture,” “frame,” and “screen” may be used to have the same meaning and may be used interchangeably with each other.

[0081] In the following text, a target block can be an encoding target block (i.e., the target to be encoded) and / or a decoding target block (i.e., the target to be decoded). Furthermore, a target block can be the current block, i.e., the target currently to be encoded and / or decoded. Here, the terms "target block" and "current block" can be used to have the same meaning and are interchangeable. A current block can represent an encoding target block that is the encoding target during encoding and / or a decoding target block that is the decoding target during decoding. Furthermore, a current block can be at least one of an encoding block, a prediction block, a residual block, and a transform block.

[0082] In the following text, the terms “block” and “unit” may be used to have the same meaning and may be used interchangeably. Optionally, “block” may refer to a specific unit.

[0083] In the following text, the terms “region” and “fragment” are used interchangeably.

[0084] In the following embodiments, specific information, data, flags, indexes, elements, and attributes may have their own values. The value "0" corresponding to each of the information, data, flags, indexes, elements, and attributes may indicate false, logical false, or a first predefined value. In other words, the values ​​"0," false, logical false, and the first predefined value are interchangeable. The value "1" corresponding to each of the information, data, flags, indexes, elements, and attributes may indicate true, logical true, or a second predefined value. In other words, the values ​​"1," true, logical true, and the second predefined value are interchangeable.

[0085] When variables such as i or j are used to indicate rows, columns, or indices, the value i can be an integer 0 or greater than 0, or an integer 1 or greater than 1. In other words, in the embodiments, each of the rows, columns, and indices can be counted starting from 0 or starting from 1.

[0086] In the embodiments, the term "one or more" or the term "at least one" may mean the term "multiple". The terms "one or more" or the term "at least one" may be used interchangeably with "multiple".

[0087] The terminology used in the embodiments will be described below.

[0088] Encoder: An encoder represents a device used to perform encoding. In other words, an encoder can represent an encoding device.

[0089] Decoder: A decoder refers to a device used to perform decoding. In other words, a decoder can represent a decoding device.

[0090] Unit: A unit can represent a component of image encoding and decoding. The terms "unit" and "block" can be used interchangeably and have the same meaning.

[0091] – A cell can be an M×N sample array. Each of M and N can be a positive integer. Cells can typically represent sample arrays in two-dimensional form.

[0092] In the process of image encoding and decoding, a "unit" can be a region generated by partitioning an image. In other words, a "unit" can be a specified region within an image. A single image can be partitioned into multiple units. Optionally, an image can be partitioned into sub-parts, and a unit can represent each sub-part created when encoding or decoding is performed on the partitioned sub-parts.

[0093] – During the encoding and decoding of an image, predefined processing can be performed on each unit according to its type.

[0094] Based on function, unit types can be classified as macrounits, coding units (CUs), prediction units (PUs), residual units, transform units (TUs), etc. Optionally, based on function, units can represent blocks, macroblocks, coding tree units, coding tree blocks, coding units, coding blocks, prediction units, prediction blocks, residual units, residual blocks, transform units, transform blocks, etc. For example, the target unit that serves as the object of encoding and / or decoding can be at least one of CUs, PUs, residual units, and TUs.

[0095] – The term “unit” can refer to a block of luma components, a block of chroma components corresponding to the luma components, and information about the syntax elements for each block, such that the unit is specified to be distinct from the block.

[0096] The size and shape of the unit can be implemented differently. In addition, the unit can have any of a variety of sizes and shapes. Specifically, the shape of the unit can include not only squares, but also geometric shapes that can be represented in two dimensions (2D) (such as rectangles, trapezoids, triangles and pentagons).

[0097] In addition, cell information may include one or more of the following: cell type, cell size, cell depth, cell encoding order, and cell decoding order. For example, the cell type may indicate one of CU, PU, ​​residual cell, and TU.

[0098] – A cell can be divided into sub-cells, each sub-cell having a smaller size than the related cell.

[0099] Depth: Depth represents the degree to which a cell is partitioned. Furthermore, cell depth indicates the level at which a cell exists when represented by a tree structure.

[0100] – Cell partitioning information may include depth, which indicates the depth of the cell. Depth may indicate the number of times the cell is partitioned and / or the degree to which the cell is partitioned.

[0101] In a tree structure, the root node can be considered to have the smallest depth and the leaf nodes have the largest depth. The root node can be the highest (top) node. The leaf nodes can be the lowest nodes.

[0102] A single unit can be hierarchically partitioned into multiple sub-units, with each unit possessing depth information based on a tree structure. In other words, a unit and the sub-units generated by partitioning that unit can correspond to a node and its child nodes, respectively. Each partitioned sub-unit can have a unit depth. Since depth indicates the number of times a unit is partitioned and / or the degree to which a unit is partitioned, the partitioning information of a sub-unit can include information about the size of that sub-unit.

[0103] In a tree structure, the top node can correspond to the initial node before partitioning. The top node can be called the "root node". Furthermore, the root node can have a minimum depth value. Here, the depth of the top node can be level "0".

[0104] A node with a depth of level "1" can represent a cell generated when the initial cell is partitioned once. A node with a depth of level "2" can represent a cell generated when the initial cell is partitioned twice.

[0105] – A leaf node of depth “n” can represent a cell generated when the initial cell is partitioned n times.

[0106] – Leaf nodes can be bottom nodes that cannot be further partitioned. The depth of a leaf node can be the maximum level. For example, a predefined value for the maximum level could be 3.

[0107] –QT depth can represent the depth for a four-partition drive. BT depth can represent the depth for a two-partition drive. TT depth can represent the depth for a three-partition drive.

[0108] – Sample: A sample can be the basic unit that makes up a block. It can be 0 to 2 based on the bit depth (Bd). Bd The value of -1 is used to represent the sample point.

[0109] – A sample point can be a pixel or a pixel value.

[0110] – In the following text, the terms “pixel” and “sample” may be used to have the same meaning and may be used interchangeably.

[0111] Code Tree Unit (CTU): A CTU can consist of a single luma component (Y) code tree block and two chroma component (i.e., Cb, Cr) code tree blocks associated with the luma component code tree block. Furthermore, a CTU can represent information including the aforementioned blocks and the syntax elements used for each block.

[0112] – Each coding tree unit (CTU) can be partitioned using one or more partitioning methods (such as quadtree (QT), binary tree (BT), and ternary tree (TT)) to configure sub-units, such as coding units, prediction units, and transform units. A quadtree can represent a quaternion tree. Additionally, one or more partitioning methods can be used to partition each coding tree unit using multi-type tree (MTT).

[0113] – “CTU” can be used as a term to specify a pixel block as a processing unit in image decoding and encoding processes, such as in the case of partitioning an input image.

[0114] Coding Tree Block (CTB): "CTB" can be used as a term to specify any one of the Y coding tree block, Cb coding tree block, and Cr coding tree block.

[0115] Neighboring Blocks: Neighboring blocks (or adjacent blocks) can represent blocks that are adjacent to the target block. Neighboring blocks can also represent neighboring blocks that are being reconstructed.

[0116] In the following text, the terms “nearby block” and “adjacent block” may be used to have the same meaning and may be used interchangeably.

[0117] Neighboring blocks can represent reconstructed neighboring blocks.

[0118] Spatial neighbor block: A spatial neighbor block can be a block that is spatially adjacent to the target block. Neighbor blocks can include spatial neighbor blocks.

[0119] – Target blocks and spatially adjacent blocks can be included in the target frame.

[0120] – A spatially adjacent block can represent a block whose boundary is in contact with the target block or a block located within a predetermined distance from the target block.

[0121] – A spatial neighbor block can represent a block that is adjacent to the vertex of the target block. Here, a block that is adjacent to the vertex of the target block can be a block that is vertically adjacent to a neighbor block that is horizontally adjacent to the target block, or a block that is horizontally adjacent to a neighbor block that is vertically adjacent to the target block.

[0122] Temporally neighboring blocks: Temporally neighboring blocks can be blocks that are temporally adjacent to the target block. Neighboring blocks can include temporally neighboring blocks.

[0123] –Time-proximity blocks can include col blocks.

[0124] The col block can be a block in a previously reconstructed co-location frame (col frame). The position of the col block in the col frame can correspond to the position of the target block in the target frame. Optionally, the position of the col block in the col frame can be equal to the position of the target block in the target frame. The col frame can be a frame included in the list of reference frames.

[0125] - A temporally neighboring block can be a block that is spatially adjacent to the target block in time.

[0126] Prediction mode: Prediction mode can be information indicating the mode used for intra-frame prediction or the mode used for inter-frame prediction.

[0127] Prediction Unit: A prediction unit can be a basic unit used for prediction (such as inter-frame prediction, intra-frame prediction, inter-frame compensation, intra-frame compensation, and motion compensation).

[0128] A single prediction unit can be divided into multiple partitions or sub-prediction units with smaller sizes. These partitions can also be the basic units used in performing prediction or compensation. Partitions generated by dividing the prediction unit can also be prediction units.

[0129] Prediction cell partitioning: Prediction cell partitioning can be the shape in which prediction cells are divided.

[0130] Reconstructed neighboring cells: The reconstructed neighboring cells can be cells that are adjacent to the target cell and have already been decoded and reconstructed.

[0131] - The reconstructed neighboring units can be those that are spatially adjacent to the target unit or temporally adjacent to the target unit.

[0132] – The reconstructed spatial neighbor unit can be a unit included in the target image that has already been reconstructed through encoding and / or decoding.

[0133] – The reconstructed temporal neighbor unit can be a unit included in the reference image that has already been reconstructed through encoding and / or decoding. The position of the reconstructed temporal neighbor unit in the reference image can be the same as the position of the target unit in the target image, or it can correspond to the position of the target unit in the target image. Furthermore, the reconstructed temporal neighbor unit can be a block adjacent to a corresponding block in the reference image. Here, the position of the corresponding block in the reference image can correspond to the position of the target block in the target image. The fact that the positions of the blocks correspond to each other can indicate that the positions of the blocks are the same, that one block is included in another block, or that one block occupies a specific position in another block.

[0134] Sub-screen: A screen can be divided into one or more sub-screens. A sub-screen can consist of one or more parallel block rows and one or more parallel block columns.

[0135] – A sub-screen can be an area within a screen that has a square or rectangular shape (i.e., a non-square rectangle). Furthermore, a sub-screen can include one or more CTUs.

[0136] – A sub-picture can be a rectangular area of ​​one or more strips in the picture.

[0137] A sub-picture may include one or more parallel blocks, one or more bricks, and / or one or more stripes.

[0138] Parallel blocks: Parallel blocks can be areas in the image that have a square or rectangular shape (i.e., a non-square rectangle).

[0139] – A parallel block may include one or more CTUs.

[0140] – Parallel blocks can be partitioned into one or more blocks.

[0141] Blocking: Blocks can represent one or more CTU lines in a parallel block.

[0142] – A parallel block can be partitioned into one or more blocks. Each block may contain one or more CTU rows.

[0143] – Parallel blocks that are not partitioned into two parts can also be represented as blocks.

[0144] Strip: A strip may include one or more parallel blocks of a frame. Optionally, a strip may include one or more sub-blocks of a parallel block.

[0145] A sub-picture may comprise one or more stripes that share a rectangular area that covers the entire picture. Therefore, each sub-picture boundary is always a stripe boundary, and each vertical sub-picture boundary is always a vertical parallel block boundary.

[0146] Parameter set: The parameter set corresponds to the header information in the internal structure of the bitstream.

[0147] – The parameter set may include at least one of the following: Video Parameter Set (VPS), Sequence Parameter Set (SPS), Picture Parameter Set (PPS), Adaptive Parameter Set (APS), Decoding Parameter Set (DPS).

[0148] Information sent via signals for each parameter set can be applied to the screen referencing the corresponding parameter set. For example, information in a VPS can be applied to the screen referencing the VPS. Information in an SPS can be applied to the screen referencing the SPS. Information in a PPS can be applied to the screen referencing the PPS.

[0149] Each parameter set can reference a higher-level parameter set. For example, PPS can reference SPS, and SPS can reference VPS.

[0150] Additionally, the parameter set may include parallel block groups, stripe header information, and parallel block header information. A parallel block group can be a group comprising multiple parallel blocks. Furthermore, the meaning of "parallel block group" can be the same as that of "strip".

[0151] Rate-distortion optimization: Coding devices can use rate-distortion optimization to provide high coding efficiency by utilizing a combination of the following: coding unit (CU) size, prediction mode, prediction unit (PU) size, motion information, and transform unit (TU) size.

[0152] – Rate distortion optimization schemes can calculate the rate distortion cost of each combination to select the optimal combination from these combinations. This can be achieved using equations such as " To calculate the rate distortion cost, the combination that minimizes the rate distortion cost is typically chosen as the optimal combination under the rate distortion optimization scheme.

[0153] – D can represent distortion. D can be the average of the squares of the differences between the original transform coefficients and the reconstructed transform coefficients in the transform unit (i.e., mean square error).

[0154] – R can represent the rate, which can be expressed as a bit rate using relevant context information.

[0155] – R represents the Lagrange multiplier. R can include not only coding parameter information (such as prediction mode, motion information, and coding block flags), but also bits generated from encoding the transform coefficients.

[0156] The coding device can perform processes such as inter-frame prediction and / or intra-frame prediction, transform, quantization, entropy coding, inverse quantization (dequantization), and / or inverse transform to compute accurate D and R. These processes greatly increase the complexity of the coding device.

[0157] – Bitstream: A bitstream can represent a stream of bits that encode image information.

[0158] Parsing: Parsing can be the determination of the value of a syntax element by performing entropy decoding on a bitstream. Alternatively, the term "parsing" can refer to this entropy decoding itself.

[0159] Symbols: A symbol can be at least one of the syntax elements, encoding parameters, and transform coefficients of the encoding target unit and / or the decoding target unit. Furthermore, a symbol can be the target of entropy encoding or the result of entropy decoding.

[0160] Reference frame: The reference frame can be an image referenced by the cell to perform inter-frame prediction or motion compensation. Optionally, the reference frame can be an image that includes reference cells referenced by the target cell to perform inter-frame prediction or motion compensation.

[0161] In the following text, the terms “reference screen” and “reference image” may be used to have the same meaning and may be used interchangeably.

[0162] Reference frame list: The reference frame list can be a list of one or more reference images that are used for inter-frame prediction or motion compensation.

[0163] – The types of reference screen lists can include combined list (LC), list 0 (L0), list 1 (L1), list 2 (L2), list 3 (L3), etc.

[0164] – For inter-frame prediction, one or more reference frame lists can be used.

[0165] Inter-frame prediction indicator: The inter-frame prediction indicator indicates the direction of inter-frame prediction for the target cell. Inter-frame prediction can be either unidirectional or bidirectional. Optionally, the inter-frame prediction indicator can represent the number of reference frames used to generate the prediction cell for the target cell. Optionally, the inter-frame prediction indicator can represent the number of prediction blocks used for inter-frame prediction or motion compensation of the target cell.

[0166] Prediction list utilization flags: Prediction list utilization flags indicate whether to use at least one reference screen from a specific reference screen list to generate prediction cells.

[0167] – Inter-frame prediction indicators can be derived using prediction list utilization flags. Conversely, prediction list utilization flags can be derived using inter-frame prediction indicators. For example, a prediction list utilization flag of "0" (as a first value) indicates that reference frames from the reference frame list are not used to generate prediction blocks for the target cell. A prediction list utilization flag of "1" (as a second value) indicates that the reference frame list is used to generate prediction cells for the target cell.

[0168] Reference screen index: The reference screen index can be an index that indicates a specific reference screen in the list of reference screens.

[0169] Screen Order Count (POC): The POC value of a screen indicates the order in which the corresponding screens are displayed.

[0170] Motion Vector (MV): A motion vector can be a 2D vector used for inter-frame prediction or motion compensation. A motion vector represents the offset between the target image and the reference image.

[0171] – For example, it can be in the form of (mv) x ,mv y MV is represented in the form of ) x It can indicate the horizontal component, mv y It can indicate the vertical component.

[0172] – Search Range: The search range can be a 2D region where a search for the MV is performed during inter-frame prediction. For example, the size of the search range can be M×N. M and N can both be positive integers.

[0173] Motion vector candidates: Motion vector candidates can be blocks that are used as prediction candidates when motion vectors are predicted, or motion vectors that are used as prediction candidates.

[0174] – Motion vector candidates can be included in the motion vector candidate list.

[0175] Motion vector candidate list: The motion vector candidate list can be a list using one or more motion vector candidate configurations.

[0176] Motion vector candidate index: The motion vector candidate index can be an indicator used to indicate motion vector candidates in the motion vector candidate list. Optionally, the motion vector candidate index can be an index of motion vector predictors.

[0177] Motion information: Motion information may include at least one of the following: a list of reference frames, a reference image, motion vector candidates, a motion vector candidate index, a merge candidate and a merge index, as well as information on motion vectors, reference frame indexes and inter-frame prediction indicators.

[0178] Merge candidate list: The merge candidate list can be a list that uses one or more merge candidate configurations.

[0179] Merging Candidates: Merging candidates can be spatial merging candidates, temporal merging candidates, combined merging candidates, combined dual-prediction merging candidates, history-based candidates, average merging candidates based on the average of two candidates, zero merging candidates, etc. Merging candidates may include inter-frame prediction indicators and may include motion information such as prediction type information, reference frame index for each list, motion vectors, prediction list utilization flags, and inter-frame prediction indicators.

[0180] Merge index: A merge index can be an indicator used to indicate merge candidates in a merge candidate list.

[0181] - The merge index can indicate which reconstruction cell is used to derive the merge candidate among the reconstruction cells that are spatially adjacent to the target cell and temporally adjacent to the target cell.

[0182] – The merge index can indicate at least one of the multiple motion information candidates to be merged.

[0183] Transform Unit: A transform unit can be the basic unit for residual signal encoding and / or residual signal decoding (such as transform, inverse transform, quantization, dequantization, transform coefficient encoding, and transform coefficient decoding). A single transform unit can be partitioned into multiple sub-transform units with smaller sizes. Here, the transform may include one or more primary transforms and secondary transforms, and the inverse transform may include one or more primary inverse transforms and secondary inverse transforms.

[0184] Scaling: Scaling can represent the process of multiplying a factor by a transformation coefficient.

[0185] – As a result of scaling the levels of the transform coefficients, transform coefficients can be generated. Scaling can also be referred to as “inverse quantization”.

[0186] Quantization parameter (QP): The quantization parameter can be a value used to generate a transform coefficient level for the transform coefficients during quantization. Optionally, the quantization parameter can also be a value used to generate transform coefficient values ​​by scaling the transform coefficient level during dequantization. Optionally, the quantization parameter can be a value mapped to the quantization step size.

[0187] Delta quantization parameter: The Delta quantization parameter represents the difference between the quantization parameter of the target cell and the predicted quantization parameter.

[0188] Scan: A scan can refer to a method of arranging the coefficients in a cell, block, or matrix in order. For example, a method for arranging a 2D array in the form of a one-dimensional (1D) array can be called a "scan". Alternatively, a method for arranging a 1D array in the form of a 2D array can also be called a "scan" or "inverse scan".

[0189] Transform coefficients: Transform coefficients can be coefficient values ​​generated when the encoding device performs a transform. Optionally, transform coefficients can be coefficient values ​​generated when the decoding device performs at least one of entropy decoding and dequantization.

[0190] – The level of quantization or the level of quantized transform coefficients generated by applying quantization to transform coefficients or residual signals can also be included in the meaning of the term “transform coefficients”.

[0191] Quantization level: The quantization level can be a value generated when the encoding device performs quantization on the transform coefficients or residual signal. Optionally, the quantization level can be a value that serves as the target for dequantization when the decoding device performs dequantization.

[0192] – The transformation coefficient levels of quantization, as a result of transformation and quantization, can also be included in the meaning of the quantization levels.

[0193] Non-zero transform coefficients: Non-zero transform coefficients can be transform coefficients with values ​​other than 0, or transform coefficient levels with values ​​other than 0. Optionally, non-zero transform coefficients can be transform coefficients with a value amplitude that is not zero, or transform coefficient levels with a value amplitude that is not zero.

[0194] Quantization matrix: A quantization matrix is ​​a matrix used during the quantization or dequantization process to improve the subjective or objective image quality of an image. A quantization matrix can also be referred to as a "scaling list".

[0195] Quantization matrix coefficients: Quantization matrix coefficients can be each element in the quantization matrix. Quantization matrix coefficients are also referred to as "matrix coefficients".

[0196] Default matrix: The default matrix can be a quantization matrix predefined by the encoding and decoding devices.

[0197] Non-default matrix: A non-default matrix can be a quantization matrix that is not predefined by the encoding and decoding devices. A non-default matrix can represent a quantization matrix sent by the user from the encoding device to the decoding device.

[0198] Most Probable Mode (MPM): MPM can represent an intra-prediction mode that is highly likely to be used for intra-prediction of the target block.

[0199] Encoding and decoding devices can determine one or more MPMs based on encoding parameters associated with the target block and attributes of entities associated with the target block.

[0200] Encoding and decoding devices can determine one or more MPMs based on the intra-prediction modes of a reference block. A reference block may include multiple reference blocks. These multiple reference blocks may include spatially adjacent blocks to the left of the target block and spatially adjacent blocks above the target block. In other words, one or more distinct MPMs can be determined based on which intra-prediction modes have been used for the reference blocks.

[0201] – One or more MPMs can be identified in the same way in both the encoding and decoding devices. That is, the encoding and decoding devices can share the same list of MPMs, which includes one or more MPMs.

[0202] MPM List: The MPM list can be a list that includes one or more MPMs. The number of one or more MPMs in the MPM list can be predefined.

[0203] MPM Indicator: The MPM indicator can indicate one or more MPMs in the MPM list that will be used for intra-prediction against the target block. For example, the MPM indicator can be an index used for the MPM list.

[0204] Since the MPM list is determined in the same way in both the encoding and decoding devices, it is not necessary to send the MPM list itself from the encoding device to the decoding device.

[0205] – The MPM indicator can be signaled from the encoding device to the decoding device. Because the MPM indicator is signaled, the decoding device can determine which MPM in the MPM list will be used for intra-frame prediction against the target block.

[0206] MPM Usage Indicator: The MPM usage indicator indicates whether an MPM usage mode will be used for prediction of the target block. The MPM usage mode can be determined using a list of MPMs to identify the MPMs that will be used for intra-frame prediction of the target block.

[0207] –MPM uses indicators that can be sent from the encoding device to the decoding device using signals.

[0208] Sending by signal: "Sending by signal" can indicate that information is sent from the encoding device to the decoding device. Optionally, "sending by signal" can indicate that the information is included in a bitstream or recording medium by the encoding device. Information sent by signal by the encoding device can be used by the decoding device.

[0209] An encoding device generates encoded information by encoding the information to be transmitted as a signal. The encoded information is then sent from the encoding device to a decoding device. The decoding device obtains the information by decoding the transmitted encoded information. Here, the encoding can be entropy encoding, and the decoding can be entropy decoding.

[0210] Selective signal transmission: Information can be selectively transmitted using signals. Selective signal transmission for information can mean that an encoding device (under specific conditions) selectively includes information in a bitstream or recording medium. Selective signal transmission for information can also mean that a decoding device (under specific conditions) selectively extracts information from a bitstream.

[0211] The omission of "signal transmission": The use of "signal transmission" for information can be omitted. Regarding information, the omission of "signal transmission" for information may mean that the encoding device (under certain conditions) does not include the information in the bitstream or recording medium. The omission of "signal transmission" for information may also mean that the decoding device (under certain conditions) does not extract the information from the bitstream.

[0212] Statistical values: Variables, coding parameters, constants, etc., can have computable values. Statistical values ​​can be values ​​generated by performing calculations (operations) on the values ​​of a specified target. For example, a statistical value can indicate one or more of the following: the mean, weighted average, weighted sum, minimum, maximum, mode, median, and interpolation of the values ​​of a specific variable, a specific coding parameter, a specific constant, etc.

[0213] Figure 1 This is a block diagram illustrating the configuration of an embodiment of an encoding device to which the present disclosure is applied.

[0214] Encoding device 100 can be an encoder, a video encoding device, or an image encoding device. The video may include one or more images. Encoding device 100 can sequentially encode one or more images of the video.

[0215] Reference Figure 1 The encoding device 100 includes an inter-frame prediction unit 110, an intra-frame prediction unit 120, a switcher 115, a subtractor 125, a transform unit 130, a quantization unit 140, an entropy coding unit 150, an inverse quantization unit 160, an inverse transform unit 170, an adder 175, a filter unit 180, and a reference frame buffer 190.

[0216] The encoding device 100 can perform encoding on the target image using intra-frame mode and / or inter-frame mode. In other words, the prediction mode of the target block can be one of intra-frame mode and inter-frame mode.

[0217] In the following text, the terms "intra-frame mode", "intra-frame prediction mode", "in-frame mode" and "intra-frame prediction mode" may be used to have the same meaning and may be used interchangeably.

[0218] In the following text, the terms "inter-frame mode", "inter-frame prediction mode", "inter-picture mode" and "inter-picture prediction mode" may be used to have the same meaning and may be used interchangeably.

[0219] In the following text, the term "image" may refer to only a portion of an image, or it may refer to a block. Furthermore, the processing of an "image" may refer to the sequential processing of multiple blocks.

[0220] Furthermore, the encoding device 100 can generate a bitstream including encoded information by encoding the target image, and can output and store the generated bitstream. The generated bitstream can be stored in a computer-readable storage medium and can be streamed via wired and / or wireless transmission media.

[0221] When intra-frame mode is used as prediction mode, switcher 115 can switch to intra-frame mode. When inter-frame mode is used as prediction mode, switcher 115 can switch to inter-frame mode.

[0222] The encoding device 100 can generate a prediction block for the target block. Furthermore, after the prediction block has been generated, the encoding device 100 can use the residual between the target block and the prediction block to encode the residual block for the target block.

[0223] When the prediction mode is intra-frame mode, the intra-frame prediction unit 120 can use pixels from previously encoded / decoded neighboring blocks adjacent to the target block as reference samples. The intra-frame prediction unit 120 can use the reference samples to perform spatial prediction on the target block and can generate prediction samples for the target block via spatial prediction. The prediction samples can represent samples in the prediction block.

[0224] The inter-frame prediction unit 110 may include a motion prediction unit and a motion compensation unit.

[0225] When the prediction mode is inter-frame mode, the motion prediction unit can search for the region in the reference image that best matches the target block during motion prediction, and can derive motion vectors for both the target block and the found region based on the found region. Here, the motion prediction unit can use the search range as the target region for the search.

[0226] A reference image may be stored in a reference frame buffer 190. More specifically, when the encoding and / or decoding of a reference image has been processed, the encoded and / or decoded reference image may be stored in the reference frame buffer 190.

[0227] Since it stores the decoded screen, the reference screen buffer 190 can be a decoded screen buffer (DPB).

[0228] The motion compensation unit can generate a predicted block for the target block by performing motion compensation using motion vectors. Here, the motion vector can be a two-dimensional (2D) vector used for inter-frame prediction. Furthermore, the motion vector can indicate the offset between the target image and the reference image.

[0229] When the motion vector has values ​​other than integers, the motion prediction unit and motion compensation unit can generate prediction blocks by applying interpolation filters to a portion of the reference image. To perform inter-frame prediction or motion compensation, it can be determined which of the following modes—skip mode, merge mode, advanced motion vector prediction (AMVP) mode, and current frame reference mode—corresponds to the method for predicting and compensating for the motion of the PU included in the CU based on the CU, and inter-frame prediction or motion compensation can be performed according to that mode.

[0230] Subtractor 125 generates a residual block, which is the difference between the target block and the prediction block. The residual block can also be referred to as the "residual signal".

[0231] The residual signal can be the difference between the original signal and the predicted signal. Optionally, the residual signal can be a signal generated by transforming or quantizing the difference between the original signal and the predicted signal, or a signal generated by transforming and quantizing the difference. The residual block can be the residual signal for a block unit.

[0232] The transformation unit 130 can generate transformation coefficients by transforming the residual block, and can output the generated transformation coefficients. Here, the transformation coefficients can be coefficient values ​​generated by transforming the residual block.

[0233] Transformation unit 130 may use one of a number of predefined transformation methods when performing a transformation.

[0234] The predefined transformation methods may include Discrete Cosine Transform (DCT), Discrete Sine Transform (DST), Karhunen-Loeve Transform (KLT), etc.

[0235] The transformation method for transforming the residual block can be determined based on at least one of the coding parameters for the target block and / or neighboring blocks. For example, the transformation method can be determined based on at least one of the inter-frame prediction mode for the PU, the intra-frame prediction mode for the PU, the size of the TU, and the shape of the TU. Optionally, transformation information indicating the transformation method can be transmitted from the encoding device 100 to the decoding device 200 by a signal.

[0236] When using the transform skip mode, the transform unit 130 can omit the operation of transforming the residual block.

[0237] By quantizing the transform coefficients, a quantized transform coefficient level or a quantized level can be generated. In the following examples, each of the quantized transform coefficient level and the quantized level may also be referred to as a "transform coefficient".

[0238] Quantization unit 140 can generate quantized transform coefficient levels (i.e., quantized levels or quantized coefficients) by quantizing the transform coefficients according to quantization parameters. Quantization unit 140 can output the generated quantized transform coefficient levels. In this case, quantization unit 140 can use a quantization matrix to quantize the transform coefficients.

[0239] Entropy coding unit 150 can generate a bitstream by performing probability distribution-based entropy coding based on values ​​calculated by quantization unit 140 and / or coding parameter values ​​calculated during the encoding process. Entropy coding unit 150 can output the generated bitstream.

[0240] The entropy coding unit 150 can perform entropy coding on information about the pixels of the image and information required to decode the image. For example, the information required to decode the image may include syntax elements, etc.

[0241] When applying entropy coding, fewer bits can be allocated to more frequently occurring symbols, and more bits can be allocated to less frequently occurring symbols. Because symbols are represented through this allocation, the size of the bit string used to encode the target symbol can be reduced. Therefore, entropy coding can improve the compression performance of video coding.

[0242] Furthermore, for entropy coding, the entropy coding unit 150 can use coding methods such as Exponential Golomb, Context Adaptive Variable Length Coding (CAVLC), or Context Adaptive Binary Arithmetic Coding (CABAC). For example, the entropy coding unit 150 can use a variable-length code / code (VLC) table to perform entropy coding. For example, the entropy coding unit 150 can derive a binarization method for the target symbol. Furthermore, the entropy coding unit 150 can derive a probabilistic model for the target symbol / bit. The entropy coding unit 150 can use the derived binarization method, probabilistic model, and context model to perform arithmetic coding.

[0243] The entropy coding unit 150 can transform the coefficients in 2D block form into 1D vector form through the transform coefficient scanning method, so as to encode the quantized transform coefficient levels.

[0244] Encoding parameters can be information required for encoding and / or decoding. Encoding parameters may include information encoded by encoding device 100 and sent from encoding device 100 to decoding device, and may also include information that can be derived during encoding or decoding. For example, information sent to decoding device may include syntax elements.

[0245] Encoding parameters can include not only information such as syntax elements (or flags or indexes) encoded by the encoding device and transmitted by the encoding device to the decoding device via signals, but also information derived during the encoding or decoding process. Furthermore, encoding parameters can include information required for encoding or decoding an image. For example, encoding parameters can include at least one value of the following, a combination of the following, or statistics of the following: cell / block size, cell / block shape / form, cell / block depth, cell / block partitioning information, cell / block partitioning structure, information indicating whether a cell / block is partitioned in a quadtree structure, information indicating whether a cell / block is partitioned in a binary tree structure, partitioning direction of the binary tree structure (horizontal or vertical), partitioning form of the binary tree structure (symmetric or asymmetric), information indicating whether a cell / block is partitioned in a ternary tree structure, partitioning direction of the ternary tree structure (horizontal or vertical), partitioning form of the ternary tree structure (symmetric or asymmetric), and partitioning direction of the ternary tree structure (horizontal or vertical). Information including: symmetric partitioning, whether the unit / block is partitioned in a multi-type tree structure, the combination and direction of partitions in the multi-type tree structure (horizontal or vertical, etc.), the partition form of the multi-type tree structure (symmetric or asymmetric partitioning, etc.), the partition tree in the multi-type tree form (binary tree or ternary tree), prediction type (intra-frame prediction or inter-frame prediction), intra-frame prediction mode / direction, intra-frame luma prediction mode / direction, intra-frame chroma prediction mode / direction, intra-frame partition information, inter-frame partition information, coded block partition flag, prediction block partition flag, transform block partition flag, reference sample filtering method, reference sample filter taps, reference sample filter coefficients, prediction block filtering method. Method, prediction block filter taps, prediction block filter coefficients, prediction block boundary filter method, prediction block boundary filter taps, prediction block boundary filter coefficients, inter-frame prediction mode, motion information, motion vector, motion vector difference, reference frame index, inter-frame prediction direction, inter-frame prediction indicator, prediction list utilization flag, reference frame list, reference image, POC, motion vector prediction factor, motion vector prediction index, motion vector prediction candidate, motion vector candidate list, information indicating whether the merge mode is used, merge index, merge candidate, merge candidate list, information indicating whether the skip mode is used, type of interpolation filter, interpolation filter taps, interpolation The filter parameters include: filter coefficients, motion vector magnitude, motion vector representation precision, transform type, transform size, information indicating whether the primary transform is used, information indicating whether the secondary transform is used, primary transform selection information (or primary transform index), secondary transform selection information (or secondary transform index), information indicating the presence or absence of the residual signal, code block pattern, code block flag, quantization parameters, residual quantization parameters, quantization matrix, information about the loop filter, information indicating whether the loop filter is applied, loop filter coefficients, loop filter taps, loop filter shape / form, and information indicating whether the deblocking filter is applied.Deblocking filter coefficients, deblocking filter taps, deblocking filter strength, deblocking filter shape / form, information indicating whether adaptive sample offset is applied, adaptive sample offset value, adaptive sample offset category, adaptive sample offset type, information indicating whether adaptive loop filter is applied, adaptive loop filter coefficients, adaptive loop filter taps, adaptive loop filter shape / form, binarization / debinarization method, context model, context model determination method, context model update method, information indicating whether normal mode is executed, information indicating whether bypass mode is executed, valid coefficient flag, last valid coefficient flag, coefficient group encoding flag, last valid coefficient position, information indicating whether the coefficient value is greater than 1, information indicating whether the coefficient value is greater than 2, information indicating whether the coefficient value is greater than 3, remaining coefficient value information, positive / negative sign information, reconstructed luminance sample, reconstructed chrominance sample, context binary bits. The following parameters are included: bypass binary bits, residual luminance samples, residual chrominance samples, transform coefficients, luminance transform coefficients, chrominance transform coefficients, quantization level, luminance quantization level, chrominance quantization level, transform coefficient level, transform coefficient level scanning method, size of the motion vector search area on the decoding device side, shape / form of the motion vector search area on the decoding device side, number of motion vector searches on the decoding device side, CTU size, minimum block size, maximum block size, maximum block depth, minimum block depth, image display / output order, stripe identification information, stripe type, stripe partition information, parallel block group identification information, parallel block group type, parallel block group partition information, parallel block identification information, parallel block type, parallel block partition information, screen type, bit depth, input sample bit depth, reconstructed sample bit depth, residual sample bit depth, transform coefficient bit depth, quantization level bit depth, information about the luminance signal, information about the chrominance signal, color space of the target block, and color space of the residual block. Furthermore, information related to the above encoding parameters may also be included in the encoding parameters. Information used to calculate and / or derive the above encoding parameters may also be included in the encoding parameters. Information calculated or derived using the above encoding parameters can also be included in the encoding parameters.

[0246] Primary transform selection information indicates the primary transform to be applied to the target block.

[0247] Secondary transformation selection information indicates the secondary transformations to be applied to the target block.

[0248] The residual signal can represent the difference between the original signal and the predicted signal. Optionally, the residual signal can be a signal generated by transforming the difference between the original signal and the predicted signal. Optionally, the residual signal can be a signal generated by transforming and quantizing the difference between the original signal and the predicted signal. The residual block can be the residual signal for a block.

[0249] Here, sending information via a signal can indicate that the encoding device 100 includes entropy-encoded information generated by performing entropy encoding on flags or indices in the bitstream, and can also indicate that the decoding device 200 obtains information by performing entropy decoding on the entropy-encoded information extracted from the bitstream. This information may include flags, indices, etc.

[0250] A signal can refer to information that will be transmitted using a signal. In the following text, information used for images and blocks may be referred to as a "signal". Furthermore, in the following text, the terms "information" and "signal" may be used to have the same meaning and may be used interchangeably. For example, a specific signal may be a signal representing a specific block. A raw signal may be a signal representing a target block. A prediction signal may be a signal representing a predicted block. A residual signal may be a signal representing a residual block.

[0251] The bitstream may include information based on a specific syntax. Encoding device 100 may generate a bitstream that includes information according to the specific syntax. Decoding device 200 may obtain information from the bitstream according to the specific syntax.

[0252] Since the encoding device 100 performs encoding via inter-frame prediction, the encoded target image can be used as a reference image for other images to be processed subsequently. Therefore, the encoding device 100 can reconstruct or decode the encoded target image and store the reconstructed or decoded image as a reference image in the reference frame buffer 190. For decoding, inverse quantization and inverse transform of the encoded target image can be performed.

[0253] The quantization levels can be dequantized by the dequantization unit 160 and inversely transformed by the inverse transform unit 170. The dequantization unit 160 can generate dequantized coefficients by performing an inverse transform on the quantization levels. The inverse transform unit 170 can generate coefficients that have undergone both dequantization and inverse transform by performing an inverse transform on the dequantized coefficients.

[0254] The coefficients that have undergone dequantization and inverse transform can be added to the prediction block by adder 175. Adding the coefficients that have undergone dequantization and inverse transform to the prediction block generates a reconstructed block. Here, the coefficients that have undergone dequantization and / or inverse transform can represent one or more coefficients that have undergone dequantization and inverse transform, and can also represent the reconstructed residual block. Here, the reconstructed block can represent either the recovered block or the decoded block.

[0255] The reconstructed blocks can be filtered by filter unit 180. Filter unit 180 can apply one or more filters, including deblocking filter, sample adaptive offset (SAO) filter, adaptive loop filter (ALF), and nonlocal filter (NLF), to the reconstructed samples, reconstructed blocks, or reconstructed images. Filter unit 180 may also be referred to as a "loop filter".

[0256] Deblocking filters eliminate block distortion that occurs at the boundaries between blocks in a reconstructed image. To determine whether to apply a deblocking filter, the number of columns or rows of pixels included in the block and on which the determination of whether to apply the deblocking filter to the target block is based can be determined.

[0257] When a deblocking filter is applied to a target block, the applied filter can vary depending on the required deblocking strength. In other words, among different filters, one that takes into account the strength of the deblocking filter can be applied to the target block. When a deblocking filter is applied to a target block, one or more filters, such as long-tap filters, strong filters, weak filters, and Gaussian filters, can be applied to the target block according to the required deblocking strength.

[0258] Furthermore, when performing vertical and horizontal filtering on the target block, horizontal and vertical filtering can be performed in parallel.

[0259] SAO can add an appropriate offset to the pixel value to compensate for coding errors. SAO can perform correction on the image to which deblocking is applied based on pixels, where the correction uses an offset of the difference between the original image and the image to which deblocking is applied. To perform offset correction on an image, methods can be used to divide the pixels included in the image into a specific number of regions, determine the region to be offset within the divided regions, and apply the offset to the determined region, or methods can be used to apply the offset taking into account the edge information of each pixel.

[0260] The ALF can perform filtering based on values ​​obtained by comparing the reconstructed image with the original image. After the pixels included in the image have been divided into a predetermined number of groups, the filter to be applied to each group can be determined, and filtering can be performed differently for each group. Information related to whether an adaptive loop filter is applied can be sent to each CU via a signal. This information can be sent via a signal for the luminance signal. The shape and filter coefficients of the ALF to be applied to each block can be different for each block. Alternatively, an ALF with a fixed form can be applied to the block regardless of its characteristics.

[0261] Nonlocal filters can perform filtering based on reconstructed blocks similar to the target block. Regions similar to the target block can be selected from the reconstructed image, and statistical properties of the selected similar regions can be used to perform filtering on the target block. Information regarding whether a nonlocal filter is applied can be sent to the coding unit (CU) via a signal. Furthermore, the shape and filter coefficients of the nonlocal filter applied to the block can vary depending on the block.

[0262] The reconstructed blocks or reconstructed image filtered by filter unit 180 can be stored as a reference frame in reference frame buffer 190. The reconstructed blocks filtered by filter unit 180 can be part of the reference frame. In other words, the reference frame can be a reconstructed frame composed of reconstructed blocks filtered by filter unit 180. The stored reference frame can then be used for inter-frame prediction or motion compensation.

[0263] Figure 2 This is a block diagram illustrating the configuration of an embodiment of the decoding device to which this disclosure is applied.

[0264] Decoding device 200 can be a decoder, video decoding device, or image decoding device.

[0265] Reference Figure 2 The decoding device 200 may include an entropy decoding unit 210, an inverse quantization unit 220, an inverse transform unit 230, an intra-frame prediction unit 240, an inter-frame prediction unit 250, a switcher 245, an adder 255, a filter unit 260, and a reference frame buffer 270.

[0266] Decoding device 200 can receive bit streams output from encoding device 100. Decoding device 200 can receive bit streams stored in computer-readable storage media and can also receive bit streams transmitted via wired / wireless transmission media.

[0267] The decoding device 200 can perform decoding on the bitstream in intra-frame mode and / or inter-frame mode. Furthermore, the decoding device 200 can generate a reconstructed image or a decoded image via decoding, and can output the reconstructed image or the decoded image.

[0268] For example, switcher 245 can be used to switch between intra-frame mode and inter-frame mode based on the prediction mode used for decoding. When the prediction mode used for decoding is intra-frame mode, switcher 245 can be operated to switch to intra-frame mode. When the prediction mode used for decoding is inter-frame mode, switcher 245 can be operated to switch to inter-frame mode.

[0269] The decoding device 200 can obtain the reconstructed residual block by decoding the input bitstream and can generate the prediction block. When the reconstructed residual block and the prediction block are obtained, the decoding device 200 can generate the reconstructed block, which is the target to be decoded, by adding the reconstructed residual block and the prediction block.

[0270] The entropy decoding unit 210 can generate symbols by performing entropy decoding on the bitstream based on the probability distribution of the bitstream. The generated symbols may include symbols in the form of quantized transform coefficient levels (i.e., quantized levels or quantized coefficients). Here, the entropy decoding method may be similar to the entropy coding method described above. That is, the entropy decoding method may be the inverse process of the entropy coding method described above.

[0271] The entropy decoding unit 210 can transform coefficients in one-dimensional (1D) vector form into 2D block shapes by a transform coefficient scanning method in order to decode the quantized transform coefficient levels.

[0272] For example, the coefficients of a block can be transformed into a 2D block shape by scanning the block coefficients using a top-right diagonal scan. Optionally, which of the top-right diagonal scan, vertical scan, and horizontal scan will be used can be determined based on the size of the corresponding block and / or the intra-frame prediction mode.

[0273] The quantized coefficients can be dequantized by the dequantization unit 220. The dequantization unit 220 generates dequantized coefficients by performing dequantization on the quantized coefficients. Furthermore, the dequantized coefficients can be inversely transformed by the inverse transform unit 230. The inverse transform unit 230 generates a reconstructed residual block by performing an inverse transform on the dequantized coefficients. As a result of performing dequantization and inverse transform on the quantized coefficients, a reconstructed residual block can be generated. Here, when generating the reconstructed residual block, the dequantization unit 220 can apply the quantization matrix to the quantized coefficients.

[0274] When using intra-frame mode, intra-frame prediction unit 240 can generate a prediction block by performing spatial prediction on the target block, wherein the spatial prediction uses the pixel values ​​of previously decoded neighboring blocks adjacent to the target block.

[0275] The inter-frame prediction unit 250 may include a motion compensation unit. Optionally, the inter-frame prediction unit 250 may be designated as a "motion compensation unit".

[0276] When using inter-frame mode, the motion compensation unit can generate a prediction block by performing motion compensation on the target block, wherein the motion compensation uses motion vectors and a reference image stored in the reference frame buffer 270.

[0277] The motion compensation unit can apply an interpolation filter to a portion of the reference image when the motion vector has values ​​other than integers, and can use the reference image with the interpolation filter applied to generate prediction blocks. To perform motion compensation, the motion compensation unit can determine, based on the CU, which of the following modes—skip mode, merge mode, advanced motion vector prediction (AMVP) mode, and current frame reference mode—corresponds to the motion compensation method used by the PU included in the CU, and can perform motion compensation according to the determined mode.

[0278] The reconstructed residual block and the predicted block can be added to each other by adder 255. Adder 255 generates a reconstructed block by adding the reconstructed residual block and the predicted block.

[0279] The reconstructed blocks can be filtered by filter unit 260. Filter unit 260 can apply at least one of a deblocking filter, a SAO filter, an ALF filter, and an NLF filter to the reconstructed blocks or the reconstructed image. The reconstructed image can be a picture that includes the reconstructed blocks.

[0280] The filter unit 260 can output a reconstructed image.

[0281] The reconstructed image and / or reconstructed blocks filtered by filter unit 260 can be stored as a reference image in reference image buffer 270. The reconstructed blocks filtered by filter unit 260 can be part of the reference image. In other words, the reference image can be an image composed of reconstructed blocks filtered by filter unit 260. The stored reference image can then be used for inter-frame prediction or motion compensation.

[0282] Figure 3 It is a schematic diagram illustrating the partitioning structure of an image when it is encoded and decoded.

[0283] Figure 3 An example can be illustrated by showing a single cell divided into multiple sub-cells.

[0284] To effectively partition an image, coding units (CUs) can be used in encoding and decoding. The term "unit" can be used to collectively specify 1) a block comprising image samples and 2) a syntax element. For example, "partition of a unit" can mean "partition of a block corresponding to a unit".

[0285] A CU can be used as the basic unit for image encoding / decoding. A CU can be used as the unit to which one of the intra-frame and inter-frame modes is applied during image encoding / decoding. In other words, during image encoding / decoding, it can be determined which of the intra-frame and inter-frame modes will be applied to each CU.

[0286] Furthermore, the CU can be the basic unit for predicting, transforming, quantizing, inverse transforming, dequantizing, and encoding / decoding transform coefficients.

[0287] Reference Figure 3 Image 300 can be sequentially partitioned into units corresponding to the largest coding unit (LCU), and the partitioning structure can be determined for each LCU. Here, LCU can be used to have the same meaning as coding tree unit (CTU).

[0288] Partitioning a cell can represent partitioning the block corresponding to the cell. Block partitioning information may include depth information about the depth of the cell. The depth information may indicate the number of times the cell is partitioned and / or the degree to which the cell is partitioned. A single cell may be hierarchically partitioned into multiple sub-cells, and the single cell may have depth information based on a tree structure.

[0289] Each partitioned sub-unit can have depth information. The depth information can be information indicating the size of the CU. Depth information can be stored for each CU.

[0290] Each CU can have depth information. When a CU is partitioned, the depth of the CU generated from the partition can be increased by 1 from the depth of the partitioned CU.

[0291] The partitioning structure represents the distribution of coding units (CUs) in the LCU 310 used for effectively encoding an image. This distribution can be determined by whether a single CU will be partitioned into multiple CUs. The number of CUs generated by partitioning can be a positive integer of 2 or greater, including 2, 4, 8, 16, etc.

[0292] Depending on the number of CUs generated through partitioning, the horizontal and vertical dimensions of each CU generated through partitioning can be smaller than the horizontal and vertical dimensions of the CU before partitioning. For example, the horizontal and vertical dimensions of each CU generated through partitioning can be half the horizontal and vertical dimensions of the CU before partitioning.

[0293] Each partitioned CU can be recursively partitioned into multiple CUs in the same manner. Through recursive partitioning, at least one of the horizontal and vertical dimensions of each partitioned CU can be reduced compared to at least one of the horizontal and vertical dimensions of the CU before partitioning.

[0294] The partitioning of a CU can be performed recursively until a predefined depth or predefined size is reached.

[0295] For example, the depth of a CU can range from 0 to 3. The size of a CU can range from 64×64 to 8×8, depending on its depth.

[0296] For example, the depth of LCU 310 can be 0, and the depth of the minimum coding unit (SCU) can be a predefined maximum depth. Here, as mentioned above, the LCU can be a CU with the maximum coding unit size, and the SCU can be a CU with the minimum coding unit size.

[0297] Partitioning can begin at LCU 310, and the depth of the CU can be increased by 1 whenever the horizontal and / or vertical dimensions of the CU are reduced by partitioning.

[0298] For example, for each depth, an unpartitioned CU can have a size of 2N×2N. Furthermore, when CUs are partitioned, a CU of size 2N×2N can be partitioned into four CUs, each with a size of N×N. The value of N is halved each time the depth increases by 1.

[0299] Reference Figure 3 An LCU with a depth of 0 can have 64×64 pixels or 64×64 blocks. 0 can be the minimum depth. An SCU with a depth of 3 can have 8×8 pixels or 8×8 blocks. 3 can be the maximum depth. Here, a CU with 64×64 blocks as an LCU can be represented by depth 0. A CU with 32×32 blocks can be represented by depth 1. A CU with 16×16 blocks can be represented by depth 2. A CU with 8×8 blocks as an SCU can be represented by depth 3.

[0300] Information about whether a corresponding CU is partitioned can be represented by the CU's partition information. Partition information can be 1 bit. All CUs except the SCU can include partition information. For example, the partition information value for an unpartitioned CU can be a first value. The partition information value for a partitioned CU can be a second value. When the partition information indicates whether a CU is partitioned, the first value can be "0" and the second value can be "1".

[0301] For example, when a single CU is partitioned into four CUs, the horizontal and vertical dimensions of each of the four CUs created through partitioning can be half the horizontal and vertical dimensions of the CU before partitioning. When a 32×32 CU is partitioned into four CUs, the size of each of the four partitioned CUs can be 16×16. When a single CU is partitioned into four CUs, the CU can be considered to have been partitioned using a quadtree structure. In other words, quadtree partitioning can be considered to have been applied to the CU.

[0302] For example, when a single CU is partitioned into two CUs, the horizontal or vertical dimension of each of the two resulting CUs can be half the horizontal or vertical dimension of the CU before partitioning. When a 32×32 CU is vertically partitioned into two CUs, the size of each of the two resulting CUs can be 16×32. When a 32×32 CU is horizontally partitioned into two CUs, the size of each of the two resulting CUs can be 32×16. When a single CU is partitioned into two CUs, the CU can be considered to have been partitioned using a binary tree structure. In other words, binary tree partitioning can be considered to have been applied to the CU.

[0303] For example, when a single CU is partitioned (or divided) into three CUs, the original CU before partitioning is partitioned such that its horizontal or vertical dimensions are divided in a 1:2:1 ratio, thus enabling the generation of three sub-CUs. For example, when a 16×32 CU is horizontally partitioned into three sub-CUs, the three sub-CUs generated by the partitioning can have dimensions of 16×8, 16×16, and 16×8 respectively from top to bottom. For example, when a 32×32 CU is vertically partitioned into three sub-CUs, the three sub-CUs generated by the partitioning can have dimensions of 8×32, 16×32, and 8×32 respectively from left to right. When a single CU is partitioned into three CUs, the CU can be considered to be partitioned in the form of a ternary tree. In other words, ternary tree partitioning can be considered to have been applied to the CU.

[0304] Quadtree partitioning and binary tree partitioning are both used... Figure 3 LCU 310.

[0305] In the encoding device 100, a 64×64 coding tree unit (CTU) can be partitioned into multiple smaller CUs using a recursive quadtree structure. A single CU can be partitioned into four CUs of the same size. Each CU can be recursively partitioned and can have a quadtree structure.

[0306] By using recursive partitioning of the CU, the optimal partitioning method that causes the minimum rate distortion cost can be selected.

[0307] Figure 3 The Coding Tree Unit (CTU) 320 in the example is a CTU in which quadtree partitioning, binary tree partitioning and ternary tree partitioning are all applied.

[0308] As described above, to partition a CTU, at least one of quadtree partitioning, binary tree partitioning, and ternary tree partitioning can be applied to the CTU. Partitioning can be applied based on specific priorities.

[0309] For example, quadtree partitioning can be preferentially applied to CTUs. CUs that cannot be further partitioned in quadtree form can correspond to the leaf nodes of a quadtree. CUs corresponding to the leaf nodes of a quadtree can be the root nodes of a binary tree and / or a ternary tree. That is, CUs corresponding to the leaf nodes of a quadtree can be partitioned in binary or ternary tree form, or may not be further partitioned. In this case, to prevent each CU generated by applying binary or ternary tree partitioning to the CUs corresponding to the leaf nodes of a quadtree from being quadtree partitioned again, the operations of block partitioning and / or signaling block partitioning information are effectively performed.

[0310] Four-partition information can be used to signal the partitions of a CU corresponding to each node of a quadtree. A four-partition message with a first value (e.g., "1") indicates that the corresponding CU is partitioned in quadtree form. A four-partition message with a second value (e.g., "0") indicates that the corresponding CU is not partitioned in quadtree form. The four-partition message can be a flag with a specific length (e.g., 1 bit).

[0311] There may be no priority between binary tree partitioning and ternary tree partitioning. That is, the CU corresponding to the leaf node of a quadtree can be partitioned in either binary or ternary tree form. Furthermore, CUs generated by binary or ternary tree partitioning can be further partitioned in either binary or ternary tree form, or they may not be further partitioned.

[0312] A partition executed when there is no priority between a binary tree partition and a ternary tree partition can be called a "multi-type tree partition". That is, the CU corresponding to a leaf node of a quadtree can be the root node of a multi-type tree. The partitioning of the CU corresponding to each node of the multi-type tree can be signaled using at least one of the following: information indicating whether the CU is partitioned according to the multi-type tree, partitioning direction information, and partitioning tree information. For the partitioning of the CU corresponding to each node of the multi-type tree, the information indicating whether the multi-type tree partitioning is executed, the partitioning direction information, and the partitioning tree information can be signaled sequentially.

[0313] For example, information indicating whether a CU is partitioned in a multi-type tree and having a first value (e.g., "1") can indicate that the corresponding CU is partitioned in a multi-type tree format. Information indicating whether a CU is partitioned in a multi-type tree and having a second value (e.g., "0") can indicate that the corresponding CU is not partitioned in a multi-type tree format.

[0314] When the CU corresponding to each node of the multi-type tree is partitioned in the form of a multi-type tree, the corresponding CU may further include partitioning direction information.

[0315] Partition direction information indicates the partitioning direction of a multi-type tree partition. Partition direction information with a first value (e.g., "1") indicates that the corresponding CU is partitioned in the vertical direction. Partition direction information with a second value (e.g., "0") indicates that the corresponding CU is partitioned in the horizontal direction.

[0316] When a CU corresponding to each node of a multi-type tree is partitioned in the form of a multi-type tree, the corresponding CU may further include partition tree information. The partition tree information can indicate the tree used for multi-type tree partitioning.

[0317] For example, partition tree information with a first value (e.g., "1") can indicate that the corresponding CU is partitioned in a binary tree format. Partition tree information with a second value (e.g., "0") can indicate that the corresponding CU is partitioned in a ternary tree format.

[0318] Here, each of the above-mentioned information indicating whether partitioning by multi-type tree is performed, partition tree information, and partition direction information can be a flag with a specific length (e.g., 1 bit).

[0319] Entropy encoding and / or entropy decoding can be performed on at least one of the above four partition information, information indicating whether partitioning according to multiple tree types has been executed, partition direction information, and partition tree information. To perform entropy encoding / entropy decoding of this information, information from neighboring CUs adjacent to the target CU can be used.

[0320] For example, it can be assumed that the partitioning patterns (i.e., partitioned / non-partitioned, partitioned tree, and / or partitioned direction) of the left and / or upper CUs are highly similar to the partitioning patterns of the target CU. Therefore, based on the information of neighboring CUs, contextual information for entropy encoding and / or entropy decoding of the information for the target CU can be derived. Here, the information of neighboring CUs may include at least one of the following: 1) four-partition information of neighboring CUs, 2) information indicating whether neighboring CUs are partitioned according to multiple types of trees, 3) partitioned direction information of neighboring CUs, and 4) partitioned tree information of neighboring CUs.

[0321] In another embodiment of binary tree partitioning and ternary tree partitioning, binary tree partitioning can be performed first. That is, binary tree partitioning can be applied first, and then the CUs corresponding to the leaf nodes of the binary tree can be set as the root nodes of the ternary tree. In this case, quadtree partitioning or binary tree partitioning may not be performed on the CUs corresponding to the nodes of the ternary tree.

[0322] A CU that is not further partitioned by quadtree partitioning, binary tree partitioning, and / or ternary tree partitioning can be a unit for encoding, prediction, and / or transformation. That is, a CU may not be further partitioned for use in prediction and / or transformation. Therefore, the partitioning structure used to partition CUs into prediction units (PUs) and / or transformation units (TUs), its partitioning information, etc., may not exist in the bitstream.

[0323] However, when the size of a CU (Computer Unit) used as a partitioning unit is larger than the size of the largest transform block, the CU can be recursively partitioned until the size of the CU becomes smaller than or equal to the size of the largest transform block. For example, when the size of the CU is 64×64 and the size of the largest transform block is 32×32, the CU can be partitioned into four 32×32 blocks to perform the transform. Similarly, when the size of the CU is 32×64 and the size of the largest transform block is 32×32, the CU can be partitioned into two 32×32 blocks.

[0324] In this case, it is not necessary to separately send a signal indicating whether the CU has been partitioned for transformation. Without signal transmission, partitioning of the CU can be determined by comparing its horizontal (and / or vertical) dimensions with the horizontal (and / or vertical) dimensions of the largest transform block. For example, when the horizontal dimension of the CU is greater than the horizontal dimension of the largest transform block, the CU can be vertically bisected. Furthermore, when the vertical dimension of the CU is greater than the vertical dimension of the largest transform block, the CU can be horizontally bisected.

[0325] Information regarding the maximum and / or minimum size of the CU and the maximum and / or minimum size of the transform block can be signaled or determined at a level higher than the CU level. For example, higher levels could be sequence level, frame level, parallel block level, parallel block group level, or stripe level. For example, the minimum size of the CU could be set to 4×4. For example, the maximum size of the transform block could be set to 64×64. For example, the maximum size of the transform block could be set to 4×4.

[0326] Information regarding the minimum size of the CU corresponding to the leaf node of the quadtree (i.e., the minimum size of the quadtree) and / or the maximum depth of the path from the root node to the leaf node of the multi-type tree (i.e., the maximum depth of the multi-type tree) can be signaled or determined at a level higher than the CU level. For example, higher levels could be sequence level, frame level, stripe level, parallel block group level, or parallel block level. Information regarding the minimum size of the quadtree and / or the maximum depth of the multi-type tree can be signaled or determined individually at each of the intra-strip and inter-strip levels.

[0327] Information regarding the difference between the size of the CTU and the maximum size of the transform block can be signaled or determined at a level higher than the CU level. For example, higher levels could be sequence level, frame level, stripe level, parallel block group level, or parallel block level. Information regarding the maximum size of the CU corresponding to each node of the binary tree (i.e., the maximum size of the binary tree) can be determined based on the size of the CTU and the aforementioned difference. The maximum size of the CU corresponding to each node of the ternary tree (i.e., the maximum size of the ternary tree) can have different values ​​depending on the stripe type. For example, the maximum size of the ternary tree in an intra-strip level could be 32×32. For example, the maximum size of the ternary tree in an inter-strip level could be 128×128. For example, the minimum size of the CU corresponding to each node of the binary tree (i.e., the minimum size of the binary tree) and / or the minimum size of the CU corresponding to each node of the ternary tree (i.e., the minimum size of the ternary tree) can be set to the minimum size of the CU.

[0328] In another example, the maximum size of a binary tree and / or the maximum size of a ternary tree can be signaled or determined at the stripe level. Furthermore, the minimum size of a binary tree and / or the minimum size of a ternary tree can be signaled or determined at the stripe level.

[0329] Based on the various block sizes and depths described above, the four partition information, information indicating whether partitioning by multiple tree types has been performed, partition tree information, and / or partition direction information may or may not exist in the bitstream.

[0330] For example, when the size of the CU is not greater than the minimum size of the quadtree, the CU may not include the four-partition information, and the four-partition information of the CU can be inferred as the second value.

[0331] For example, when the size (horizontal and vertical dimensions) of the CU corresponding to each node of the multi-type tree is greater than the maximum size (horizontal and vertical dimensions) of the binary tree and / or the maximum size (horizontal and vertical dimensions) of the ternary tree, the CU may not be partitioned in binary and / or ternary tree form. In this determination method, information indicating whether partitioning by multi-type tree is performed may not be sent by signal, but can be inferred as a second value.

[0332] Optionally, the CU may not be partitioned in binary and / or ternary tree form when the size (horizontal and vertical dimensions) of the CU corresponding to each node of the multi-type tree is equal to the minimum size (horizontal and vertical dimensions) of the binary tree, or when the size (horizontal and vertical dimensions) of the CU is twice the minimum size (horizontal and vertical dimensions) of the ternary tree. With this determination method, information indicating whether partitioning by multi-type tree is performed can be sent without signaling, but can be inferred as a second value. This is because when the CU is partitioned in binary and / or ternary tree form, it generates CUs smaller than the minimum size of the binary tree and / or the minimum size of the ternary tree.

[0333] Optionally, binary or ternary partitioning can be limited based on the size of the virtual pipeline data unit (i.e., the size of the pipeline buffer). For example, binary or ternary partitioning can be limited when a CU is partitioned into sub-CUs that do not fit the size of the pipeline buffer. The size of the pipeline buffer can be equal to the maximum size of the transform block (e.g., 64×64).

[0334] For example, when the size of the pipeline buffer is 64×64, the following partitions can be restricted.

[0335] - A ternary tree partition for an N×M CU (where N and / or M are 128). - Horizontal binary tree partitioning for a 128×N CU (where N<=64) - Vertical binary tree partitioning for N×128 CUs (where N<=64) Optionally, when the depth of the CU corresponding to each node of the multi-type tree is equal to the maximum depth of the multi-type tree, the CU may not be partitioned in binary and / or ternary tree form. With this determination method, information indicating whether partitioning by the multi-type tree is performed can be sent without signaling, but can be inferred as a second value.

[0336] Optionally, information indicating whether partitioning by the multi-type tree has been performed may be signaled only if at least one of vertical binary tree partitioning, horizontal binary tree partitioning, vertical ternary tree partitioning, and horizontal ternary tree partitioning is possible for each CU corresponding to each node of the multi-type tree. Otherwise, the CU may not be partitioned in binary and / or ternary tree form. With this determination method, the information indicating whether partitioning by the multi-type tree has been performed may not be signaled, but rather inferred as a second value.

[0337] Optionally, for each CU corresponding to a node in a multi-type tree, partitioning direction information may be signaled only when both vertical binary tree partitioning and horizontal binary tree partitioning are feasible, or only when both vertical ternary tree partitioning and horizontal ternary tree partitioning are feasible. Otherwise, partitioning direction information may not be signaled, but may be inferred as a value indicating the direction in which the CU can be partitioned.

[0338] Optionally, for each CU corresponding to a node of a multi-type tree, partition tree information may be signaled only when both vertical binary tree partitioning and vertical ternary tree partitioning are feasible, or only when both horizontal binary tree partitioning and horizontal ternary tree partitioning are feasible. Otherwise, partition tree information may not be signaled, but may be inferred as the value of the tree indicating the partitions that can be applied to the CU.

[0339] Figure 4 This is a diagram showing the form of prediction units that a coding unit can include.

[0340] In the CUs partitioned from the LCU, the CUs that are no longer partitioned can be divided into one or more prediction units (PUs).

[0341] A PU (Program Unit) can be the basic unit used for prediction. A PU can be encoded and decoded in any of the following modes: skip mode, inter-frame mode, and intra-frame mode. A PU can be partitioned into various shapes according to each mode. For example, refer to the above... Figure 1 The target block described and the above references Figure 2 The target blocks described can all be PUs.

[0342] A CU may not be classified as a PU. When a CU is not classified as a PU, the dimensions of the CU and the PU can be equal.

[0343] In skip mode, partitioning may not be present in the CU. Skip mode also supports a 2N×2N mode 410 without partitioning, where the PU and CU have the same size.

[0344] In inter-frame mode, eight types of partition shapes can exist in the CU. For example, in inter-frame mode, 2N×2N mode 410, 2N×N mode 415, N×2N mode 420, N×N mode 425, 2N×nU mode 430, 2N×nD mode 435, nL×2N mode 440 and nR×2N mode 445 are supported.

[0345] In intra-frame mode, 2N×2N mode 410 and N×N mode 425 are supported.

[0346] In 2N×2N mode 410, a PU with a size of 2N×2N can be encoded. A PU with a size of 2N×2N can represent a PU with the same size as the CU. For example, a PU with a size of 2N×2N can have a size of 64×64, 32×32, 16×16, or 8×8.

[0347] In N×N mode 425, PUs with dimensions of N×N can be encoded.

[0348] For example, in intra-frame prediction, when the PU size is 8×8, the PUs from four partitions can be encoded. The size of the PU from each partition can be 4×4.

[0349] When encoding a PU in intra-frame mode, the PU can be encoded using any of a number of intra-frame prediction modes. For example, HEVC technology provides 35 intra-frame prediction modes, and the PU can be encoded in any of these 35 intra-frame prediction modes.

[0350] The rate-distortion cost can be used to determine which of the 2N×2N modes 410 and N×N modes 425 will be used to encode the PU.

[0351] Encoding device 100 can perform encoding operations on a PU of size 2N×2N. Here, the encoding operation can be an operation of encoding the PU in each of a plurality of intra-prediction modes that can be used by encoding device 100. Through the encoding operation, an optimal intra-prediction mode for a PU of size 2N×2N can be derived. This optimal intra-prediction mode can be the intra-prediction mode that incurs the minimum rate-distortion cost when encoding a PU of size 2N×2N among the plurality of intra-prediction modes that can be used by encoding device 100.

[0352] Furthermore, the encoding device 100 can sequentially perform encoding operations on each PU obtained by performing N×N partitioning. Here, the encoding operation can be an operation of encoding the PU in each of a plurality of intra-prediction modes that can be used by the encoding device 100. Through the encoding operation, the optimal intra-prediction mode for a PU of size N×N can be derived. This optimal intra-prediction mode can be the intra-prediction mode that produces the minimum rate-distortion cost when encoding a PU of size N×N among the plurality of intra-prediction modes that can be used by the encoding device 100.

[0353] The encoding device 100 can determine which of the PUs, one of size 2N×2N and one of size N×N, will be encoded based on a comparison between the rate-distortion cost of a PU of size 2N×2N and the rate-distortion cost of a PU of size N×N.

[0354] A single CU can be partitioned into one or more PUs, and a PU can be partitioned into multiple PUs.

[0355] For example, when a single PU is partitioned into four PUs, the horizontal and vertical dimensions of each of the four PUs created through partitioning can be half the horizontal and vertical dimensions of the original PU. When a 32×32 PU is partitioned into four PUs, the size of each of the four partitioned PUs can be 16×16. When a single PU is partitioned into four PUs, the PU can be considered to have been partitioned in a quadtree structure.

[0356] For example, when a single PU is partitioned into two PUs, the horizontal or vertical dimension of each of the two resulting PUs can be half the horizontal or vertical dimension of the original PU. When a 32×32 PU is vertically partitioned into two PUs, the size of each of the two partitioned PUs can be 16×32. When a 32×32 PU is horizontally partitioned into two PUs, the size of each of the two partitioned PUs can be 32×16. When a single PU is partitioned into two PUs, the PU can be considered to have been partitioned in a binary tree structure.

[0357] Figure 5 This is a diagram showing the form of a transformation unit that can be included in an encoding unit.

[0358] A transform unit (TU) can be a basic unit in a CU used for processes such as transform, quantization, inverse transform, dequantization, entropy coding, and entropy decoding.

[0359] The TU can be square or rectangular. The shape of the TU can be determined based on the size and / or shape of the CU.

[0360] Within a CU partitioned from an LCU, CUs that are no longer designated as CUs can be partitioned into one or more TUs. Here, the partitioning structure of a TU can be a quadtree structure. For example, ... Figure 5 As shown, a single CU 510 can be partitioned once or more according to a quadtree structure. Through this partitioning, a single CU 510 can be composed of TUs of various sizes.

[0361] A CU can be considered to be recursively partitioned when a single CU is partitioned two or more times. Through partitioning, a single CU can be composed of transformation units (TUs) of various sizes.

[0362] Alternatively, a single CU can be divided into one or more TUs based on the number of vertical and / or horizontal lines dividing the CU.

[0363] The CU can be divided into symmetrical TUs or asymmetrical TUs. To divide into asymmetrical TUs, information about the size and / or shape of each TU can be transmitted from the encoding device 100 to the decoding device 200 via signals. Optionally, the size and / or shape of each TU can be derived from the information about the size and / or shape of the CU.

[0364] A CU may not be classified as a TU. When a CU is not classified as a TU, the dimensions of the CU and the TU may be equal.

[0365] A single CU can be partitioned into one or more TUs, and a TU can be partitioned into multiple TUs.

[0366] For example, when a single TU is partitioned into four TUs, the horizontal and vertical dimensions of each of the four TUs generated by the partitioning can be half the horizontal and vertical dimensions of the original TU. When a TU of size 32×32 is partitioned into four TUs, the size of each of the four partitioned TUs can be 16×16. When a single TU is partitioned into four TUs, the TU can be considered to have been partitioned in a quadtree structure.

[0367] For example, when a single TU is partitioned into two TUs, the horizontal or vertical dimension of each of the two resulting TUs can be half the horizontal or vertical dimension of the original TU. When a TU of size 32×32 is vertically partitioned into two TUs, the size of each of the two partitioned TUs can be 16×32. When a TU of size 32×32 is horizontally partitioned into two TUs, the size of each of the two partitioned TUs can be 32×16. When a single TU is partitioned into two TUs, the TU can be considered to have been partitioned in a binary tree structure.

[0368] Can be with Figure 5 The different methods shown illustrate how CUs are divided.

[0369] For example, a single CU can be divided into three CUs. The horizontal or vertical dimensions of the three CUs generated by the division can be 1 / 4, 1 / 2, and 1 / 4 of the horizontal or vertical dimensions of the original CU before the division, respectively.

[0370] For example, when a 32×32 CU is vertically divided into three CUs, the resulting three CUs can have sizes of 8×32, 16×32, and 8×32, respectively. In this way, when a single CU is divided into three CUs, the CU can be considered to be divided in the form of a ternary tree.

[0371] One of the exemplary partitioning forms (i.e., quadtree partitioning, binary tree partitioning, and ternary tree partitioning) can be applied to the partitioning of the CU, and multiple partitioning schemes can be combined together for the partitioning of the CU. Here, the combination of multiple partitioning schemes is referred to as "composite tree partitioning".

[0372] Figure 6 This shows the block division based on the example.

[0373] In video encoding and / or decoding processes, such as Figure 6 As shown, the target block can be divided. For example, the target block can be a CU.

[0374] For the partitioning of the target block, an indicator indicating the partitioning information can be sent from the encoding device 100 to the decoding device 200 by a signal. The partitioning information can be information indicating how the target block is partitioned.

[0375] The partitioning information can be one or more of the following: a partitioning flag (hereinafter referred to as "split_flag"), a quad-binary flag (hereinafter referred to as "QB_flag"), a quadtree flag (hereinafter referred to as "quadtree_flag"), a binary tree flag (hereinafter referred to as "binarytree_flag"), and a binary type flag (hereinafter referred to as "Btype_flag").

[0376] The "split_flag" can be a flag indicating whether a block has been split. For example, a split_flag value of 1 indicates that the corresponding block has been split, while a split_flag value of 0 indicates that the corresponding block has not been split.

[0377] The `QB_flag` can be a flag indicating whether the block is partitioned into a quadtree or a binary tree. For example, a `QB_flag` value of 0 indicates that the block is partitioned into a quadtree, while a `QB_flag` value of 1 indicates that the block is partitioned into a binary tree. Alternatively, a `QB_flag` value of 0 indicates that the block is partitioned into a binary tree, while a `QB_flag` value of 1 indicates that the block is partitioned into a quadtree.

[0378] The "quadtree_flag" can be a flag indicating whether the block is partitioned as a quadtree. For example, a quadtree_flag value of 1 indicates that the block is partitioned as a quadtree, while a quadtree_flag value of 0 indicates that the block is not partitioned as a quadtree.

[0379] The "binarytree_flag" can be a flag indicating whether the block is partitioned as a binary tree. For example, a binarytree_flag value of 1 indicates that the block is partitioned as a binary tree, while a binarytree_flag value of 0 indicates that the block is not partitioned as a binary tree.

[0380] The `Btype_flag` can be a flag indicating which of the vertical or horizontal partitions corresponds to the partitioning direction when a block is divided in a binary tree format. For example, a `Btype_flag` value of 0 indicates that the block is partitioned horizontally, and a `Btype_flag` value of 1 indicates that the block is partitioned vertically. Alternatively, a `Btype_flag` value of 0 indicates that the block is partitioned vertically, and a `Btype_flag` value of 1 indicates that the block is partitioned horizontally.

[0381] For example, it can be derived by sending at least one of quadtree_flag, binarytree_flag, and Btype_flag using signals. Figure 6 The block division information is shown in Table 1 below.

[0382] Table 1

[0383] For example, it can be derived by sending at least one of split_flag, QB_flag, and Btype_flag using signals. Figure 6 The block division information is shown in Table 2 below.

[0384] Table 2

[0385] The partitioning method may be limited to quadtrees or binary trees depending on the size and / or shape of the blocks. When this restriction is applied, the `split_flag` may be a flag indicating whether the blocks are partitioned in a quadtree or a binary tree format. The size and shape of the blocks can be derived from the block depth information, and the depth information can be transmitted from the encoding device 100 to the decoding device 200 by a signal.

[0386] When the block size falls within a certain range, it is possible to partition in quadtree form only. For example, the certain range can be defined by at least one of the maximum block size and the minimum block size that can be partitioned in quadtree form only.

[0387] Information indicating the maximum and minimum block sizes that can be partitioned only in quadtree form can be transmitted from the encoding device 100 to the decoding device 200 via a bitstream. Furthermore, this information can be transmitted via a signal for at least one of the units such as video, sequence, picture, parameter, parallel block group, and strip (or segment).

[0388] Optionally, the maximum block size and / or minimum block size can be fixed sizes predefined by the encoding device 100 and the decoding device 200. For example, when the block size is greater than or equal to 64×64 and less than or equal to 256×256, it is possible to partition the block only in the form of a quadtree. In this case, the split_flag can be a flag indicating whether to perform partitioning in the form of a quadtree.

[0389] When the size of a block is larger than the maximum size of a transform block, it is possible to partition it only in the form of a quadtree. Here, the sub-blocks generated by partitioning can be at least one of CU and TU.

[0390] In this case, split_flag can be a flag indicating whether the CU is partitioned in the form of a quadtree.

[0391] When the size of a block falls within a certain range, it is possible to partition it using only a binary tree or a ternary tree. For example, the specific range can be defined by at least one of the maximum block size and the minimum block size that allows partitioning using only a binary tree or a ternary tree.

[0392] Information indicating the maximum and / or minimum block size, which can be partitioned in a binary tree or a ternary tree manner, can be transmitted from the encoding device 100 to the decoding device 200 via a bitstream. Furthermore, this information can be transmitted via a signal for at least one of the units such as sequences, frames, and stripes (or segments).

[0393] Optionally, the maximum block size and / or minimum block size can be fixed sizes predefined by the encoding device 100 and the decoding device 200. For example, when the block size is greater than or equal to 8×8 and less than or equal to 16×16, it is possible to partition in binary tree form only. In this case, split_flag can be a flag indicating whether to perform partitioning in binary tree or ternary tree form.

[0394] The above description of partitioning in the form of a quadtree can be applied equally to binary and / or ternary tree forms.

[0395] The partitioning of a block may be limited by previous partitioning. For example, when a block is partitioned in a specific binary tree form and multiple sub-blocks are generated from said partition, each sub-block may be further partitioned only in that specific tree form. Here, the specific tree form can be at least one of a binary tree form, a ternary tree form, and a quadtree form.

[0396] When the horizontal or vertical dimensions of a partition block are dimensions that cannot be further subdivided, the aforementioned indicator does not need to be sent.

[0397] Figure 7 This is a diagram illustrating an embodiment of intra-frame prediction processing.

[0398] from Figure 7 The radially extending arrow from the center of the diagram indicates the prediction direction of the intra-prediction mode. Furthermore, the numbers appearing near the arrows indicate examples of mode values ​​assigned to the intra-prediction mode or its prediction direction.

[0399] exist Figure 7 In the diagram, number 0 can represent the planar mode as a non-directional intra-prediction mode. Number 1 can represent the DC mode as a non-directional intra-prediction mode.

[0400] Intra-frame coding and / or decoding can be performed using reference samples from neighboring blocks of the target block. A neighboring block can be a reconstructed neighboring block. Reference samples can represent neighboring samples.

[0401] For example, intra-frame encoding and / or decoding can be performed using the values ​​of reference samples included in the reconstructed neighboring blocks or the encoding parameters of the reconstructed neighboring blocks.

[0402] Encoding device 100 and / or decoding device 200 can generate a prediction block by performing intra-frame prediction on the target block based on information about samples in the target image. When intra-frame prediction is performed, encoding device 100 and / or decoding device 200 can generate a prediction block for the target block by performing intra-frame prediction based on information about samples in the target image. When intra-frame prediction is performed, encoding device 100 and / or decoding device 200 can perform directional prediction and / or non-directional prediction based on at least one reconstructed reference sample.

[0403] A prediction block can be a block generated as a result of performing intra-frame prediction. A prediction block can correspond to at least one of CU, PU, ​​and TU.

[0404] The cells of the prediction block may have a size corresponding to at least one of CU, PU, ​​and TU. The prediction block may have a square shape with a size of 2N×2N or N×N. The size N×N may include sizes such as 4×4, 8×8, 16×16, 32×32, 64×64, etc.

[0405] Optionally, the prediction block can be a square block with a size of 2×2, 4×4, 8×8, 16×16, 32×32, 64×64, etc., or a rectangular block with a size of 2×8, 4×8, 2×16, 4×16, 8×16, etc.

[0406] Intra-prediction can be performed using intra-prediction modes for the target block. The number of intra-prediction modes that a target block can have can be a predefined fixed value, or it can be a value determined differently based on the attributes of the prediction block. For example, the attributes of the prediction block can include the size of the prediction block, the type of the prediction block, etc. In addition, the attributes of the prediction block can indicate the coding parameters used for the prediction block.

[0407] For example, the number of intra-prediction modes can be fixed at N, regardless of the size of the prediction block. Alternatively, the number of intra-prediction modes can be, for example, 3, 5, 9, 17, 34, 35, 36, 65, 67, or 95.

[0408] Intra-frame prediction mode can be either non-directional or directional.

[0409] For example, intra-frame prediction modes may include... Figure 7 The numbers 0 to 66 shown correspond to two non-directional modes and 65 directional modes.

[0410] For example, using a specific intra-prediction method, the intra-prediction mode can include... Figure 7 The numbers -14 to 80 shown correspond to the two non-directional patterns and 93 directional patterns.

[0411] The two non-directional modes may include DC mode and planar mode.

[0412] Directional patterns can be prediction patterns with a specific direction or angle. Directional patterns can also be called "angle patterns".

[0413] An intra-prediction mode can be represented by at least one of a mode number, a mode value, a mode angle, and a mode direction. In other words, the terms "(mode) number of intra-prediction mode", "(mode) value of intra-prediction mode", "(mode) angle of intra-prediction mode", and "(mode) direction of intra-prediction mode" can be used to have the same meaning and can be used interchangeably with each other.

[0414] The number of intra-prediction modes can be M. The value of M can be 1 or greater. In other words, the number of intra-prediction modes can be M, where M includes the number of non-directional modes and the number of directional modes.

[0415] The number of intra-prediction modes can be fixed at M, regardless of the block size and / or color components. For example, the number of intra-prediction modes can be fixed at either 35 or 67, regardless of the block size.

[0416] Optionally, the number of intra-frame prediction modes may vary depending on the shape, size, and / or type of color components of the block.

[0417] For example, in Figure 7 In the diagram, the direction prediction pattern shown by the dashed line can only be applied to the prediction of non-square blocks.

[0418] For example, the larger the block size, the more intra-prediction modes there are. Alternatively, the larger the block size, the fewer intra-prediction modes there are. When the block size is 4×4 or 8×8, the number of intra-prediction modes can be 67. When the block size is 16×16, the number of intra-prediction modes can be 35. When the block size is 32×32, the number of intra-prediction modes can be 19. When the block size is 64×64, the number of intra-prediction modes can be 7.

[0419] For example, the number of intra-prediction modes can vary depending on whether the color component is a luma signal or a chrominance signal. Optionally, the number of intra-prediction modes corresponding to the luma component block can be greater than the number of intra-prediction modes corresponding to the chrominance component block.

[0420] For example, in the vertical mode with a mode value of 50, prediction can be performed along the vertical direction based on the pixel values ​​of the reference sample. Similarly, in the horizontal mode with a mode value of 18, prediction can be performed along the horizontal direction based on the pixel values ​​of the reference sample.

[0421] Even in directional modes other than those described above, the encoding device 100 and the decoding device 200 can still perform intra-frame prediction on the target unit using reference samples based on the angle corresponding to the directional mode.

[0422] Intra-prediction modes located to the right of the vertical mode can be called "vertical-right mode". Intra-prediction modes located below the horizontal mode can be called "horizontal-bottom mode". For example, in Figure 7 In the frame prediction mode, the mode value being one of 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, and 66 can be a vertical-right mode. The mode value being one of 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, and 17 can be a horizontal-downward mode.

[0423] Non-directional modes can include DC mode and planar mode. For example, the value for DC mode can be 1, and the value for planar mode can be 0.

[0424] Orientation modes can include angle modes. Among the various intra-frame prediction modes, all modes except DC mode and planar mode can be orientation modes.

[0425] When the intra-frame prediction mode is DC mode, a prediction block can be generated based on the average pixel values ​​of multiple reference pixels. For example, the pixel values ​​of the prediction block can be determined based on the average pixel values ​​of multiple reference pixels.

[0426] The number of intra-prediction modes and the mode values ​​of each intra-prediction mode described above are merely exemplary. The number of intra-prediction modes and the mode values ​​of each intra-prediction mode described above may be defined differently depending on the embodiment, implementation, and / or requirements.

[0427] To perform intra-frame prediction on a target block, a step can be performed to check whether samples included in the reconstructed neighboring blocks can be used as reference samples for the target block. When there are samples among the samples in the neighboring blocks that cannot be used as reference samples for the target block, a value generated by interpolation and / or duplication using at least one sample value from the samples included in the reconstructed neighboring blocks can replace the sample value of the sample that cannot be used as a reference sample. When a value generated by duplication and / or interpolation replaces the sample value of an existing sample, that sample can be used as a reference sample for the target block.

[0428] When using intra-frame prediction, filters can be applied to at least one of the reference samples and the prediction samples based on the size of the target block and at least one of the intra-frame prediction modes.

[0429] The type of filter to be applied to at least one of the reference sample and the prediction sample can vary based on at least one of the intra-prediction mode of the target block, the size of the target block, and the shape of the target block. The filter type can be classified based on one or more of the filter tap length, the value of the filter coefficients, and the filter strength. The filter tap length can represent the number of filter taps. Furthermore, the number of filter taps can represent the length of the filter.

[0430] When the intra-frame prediction mode is planar mode, the sample value of the predicted target block can be generated by weighting the top reference sample, left reference sample, upper right reference sample, and lower left reference sample of the target block according to the position of the predicted target sample in the prediction block.

[0431] When the intra-frame prediction mode is DC mode, the average of reference samples above and to the left of the target block can be used when generating the prediction block for the target block. Furthermore, filtering using the values ​​of the reference samples can be performed on specific rows or columns within the target block. The specific row can be one or more upper rows adjacent to the reference samples. The specific column can be one or more left columns adjacent to the reference samples.

[0432] When the intra-frame prediction mode is directional mode, the top reference sample, left reference sample, top right reference sample, and / or bottom left reference sample of the target block can be used to generate the prediction block.

[0433] To generate the above predicted samples, real-number-based interpolation can be performed.

[0434] The intra-prediction mode of the target block can be predicted from the intra-prediction modes of neighboring blocks adjacent to the target block, and the information used for prediction can be entropy encoded / entropy decoded.

[0435] For example, when the intra-prediction modes of the target block and neighboring blocks are the same, a predefined flag can be used to signal that the intra-prediction modes of the target block and neighboring blocks are the same.

[0436] For example, an indicator can be sent to indicate an intra-prediction mode that is the same as the intra-prediction mode of the target block among the intra-prediction modes of multiple neighboring blocks.

[0437] When the intra-prediction modes of the target block and neighboring blocks are different from each other, entropy coding and / or entropy decoding can be used to encode and / or decode information about the intra-prediction mode of the target block.

[0438] Figure 8 This is a diagram showing the reference samples used in the intra-frame prediction process.

[0439] The reconstruction reference points used for intra-frame prediction of the target block may include the lower left reference point, the left reference point, the upper left corner reference point, the upper reference point, and the upper right reference point.

[0440] For example, a left reference sample may represent a reconstructed reference pixel adjacent to the left side of the target block. A top reference sample may represent a reconstructed reference pixel adjacent to the top of the target block. A top-left reference sample may represent a reconstructed reference pixel located at the top-left corner of the target block. A bottom-left reference sample may represent a reference sample located below the left-side sample line, which is the same line as the left-side sample line formed by the left reference samples. A top-right reference sample may represent a reference sample located to the right of the upper sample line, which is the same line as the upper sample line formed by the upper reference samples.

[0441] When the size of the target block is N×N, the number of the lower left reference point, the left reference point, the upper reference point, and the upper right reference point can all be N.

[0442] A prediction block can be generated by performing intra-frame prediction on the target block. The process of generating a prediction block may include determining the values ​​of the pixels in the prediction block. The target block and the prediction block can have the same size.

[0443] The reference sample used for intra-prediction of the target block can be changed according to the intra-prediction mode of the target block. The direction of the intra-prediction mode can represent the dependency between the reference sample and the pixels of the prediction block. For example, the value of a specified reference sample can be used as the value of one or more specified pixels in the prediction block. In this case, the specified reference sample and the one or more specified pixels in the prediction block can be samples and pixels located on a straight line along the direction of the intra-prediction mode. In other words, the value of the specified reference sample can be copied as the value of a pixel located in the opposite direction to the direction of the intra-prediction mode. Optionally, the value of a pixel in the prediction block can be the value of a reference sample located in the direction of the intra-prediction mode relative to the pixel's position.

[0444] In the example, when the intra-prediction mode of the target block is vertical, the upper reference sample can be used for intra-prediction. When the intra-prediction mode is vertical, the value of a pixel in the prediction block can be the value of a reference sample vertically above that pixel. Therefore, the upper reference sample adjacent to the top of the target block can be used for intra-prediction. Furthermore, the value of a pixel in a row of the prediction block can be the same as the value of a pixel at the upper reference sample.

[0445] In the example, when the intra-prediction mode of the target block is horizontal, the left reference sample can be used for intra-prediction. When the intra-prediction mode is horizontal, the value of a pixel in the prediction block can be the value of a reference sample horizontally to the left of that pixel. Therefore, the left reference sample adjacent to the left side of the target block can be used for intra-prediction. Furthermore, the value of a pixel in a column of the prediction block can be the same as the value of the pixel in the left reference sample.

[0446] In the example, when the mode value of the intra-prediction mode for the current block is 34, at least some of the left reference samples, the top-left reference sample, and the top reference sample can be used for intra-prediction. When the mode value of the intra-prediction mode is 34, the value of a pixel in the prediction block can be the value of a reference sample located diagonally at the top-left corner of that pixel.

[0447] Furthermore, in the case of an intra-prediction mode with mode values ​​ranging from 52 to 66, at least a portion of the upper right reference samples can be used for intra-prediction.

[0448] Furthermore, in the case of intra-prediction modes with mode values ​​ranging from 2 to 17, at least a portion of the lower left reference samples can be used for intra-prediction.

[0449] Furthermore, in the case of intra-prediction modes with mode values ​​ranging from 19 to 49, the upper left reference sample can be used for intra-prediction.

[0450] The number of reference samples used to determine the pixel value of a pixel in the prediction block can be 1, 2, or more.

[0451] As described above, the pixel value of a pixel in a prediction block can be determined based on the pixel's position and the position of a reference sample indicated by the direction of the intra-prediction mode. When both the pixel's position and the position of the reference sample indicated by the direction of the intra-prediction mode are integer positions, the value of a reference sample indicated by the integer position can be used to determine the pixel value of the pixel in the prediction block.

[0452] When the pixel position and the position of the reference sample indicated by the direction of the intra-prediction mode are not integer positions, an interpolated reference sample can be generated based on the two reference samples closest to that reference sample position. The value of the interpolated reference sample can be used to determine the pixel value of the pixel in the prediction block. In other words, when the pixel position in the prediction block and the position of the reference sample indicated by the direction of the intra-prediction mode indicate the position between two reference samples, an interpolation based on the values ​​of those two samples can be generated.

[0453] The predicted block generated by prediction may differ from the original target block. In other words, there may be prediction errors, which are the differences between the target block and the predicted block, and there may also be prediction errors between pixels in the target block and pixels in the predicted block.

[0454] In the following text, the terms “difference,” “error,” and “residual” may be used to have the same meaning and may be used interchangeably with each other.

[0455] For example, in the case of intra-frame prediction, the greater the distance between the pixels of the predicted block and the reference sample, the greater the potential prediction error. This prediction error can lead to discontinuities between the generated predicted block and its neighboring blocks.

[0456] To reduce prediction error, filtering operations can be used for prediction blocks. These filtering operations can be configured to adaptively apply filters to regions within the prediction block that are considered to have large prediction errors. For example, regions considered to have large prediction errors could be the boundaries of the prediction block. Furthermore, the regions within the prediction block considered to have large prediction errors can vary depending on the intra-prediction mode, and the characteristics of the filters can also vary depending on the intra-prediction mode.

[0457] like Figure 8 As shown, for intra-frame prediction of the target block, at least one of reference lines 0 to 3 can be used.

[0458] exist Figure 8 Each reference line in the code can indicate a reference point line that includes one or more reference points. When the reference line number is smaller, it can indicate a reference point line that is closer to the target block.

[0459] Samples in fragments A and F can be obtained by padding instead of from reconstructed neighboring blocks, wherein the padding uses the samples from fragments B and E that are closest to the target block.

[0460] An index information indicating the reference sample lines to be used for intra-frame prediction of the target block can be transmitted using a signal. The index information can indicate which of a plurality of reference sample lines will be used for intra-frame prediction of the target block. For example, the index information can have a value corresponding to any one of 0 to 3.

[0461] When the upper boundary of the target block is the boundary of the CTU, only reference sample line 0 can be available. Therefore, in this case, index information does not need to be sent. When additional reference sample lines besides reference sample line 0 are used, filtering of the prediction block, which will be described later, is not required.

[0462] In the case of intra-frame prediction between colors, the predicted block of the target block of the second color component can be generated based on the corresponding reconstructed block of the first color component.

[0463] For example, the first color component can be the luminance component, and the second color component can be the chromaticity component.

[0464] To perform inter-color intra-frame prediction, the parameters of a linear model between the first and second color components can be derived based on a template.

[0465] The template may include a reference point above the target block (upper reference point) and / or a reference point to the left of the target block (left reference point), and may include the upper reference point and / or left reference point of the reconstructed block of the first color component corresponding to the reference point.

[0466] For example, the following values ​​can be used to derive the parameters of a linear model: 1) the value of the sample point of the first color component with the maximum value among the samples in the template, 2) the value of the sample point of the second color component corresponding to the sample point of the first color component, 3) the value of the sample point of the first color component with the minimum value among the samples in the template, and 4) the value of the sample point of the second color component corresponding to the sample point of the first color component.

[0467] When exporting the parameters of a linear model, the predicted block of the target block can be generated by applying the corresponding reconstructed block to the linear model.

[0468] Depending on the image format, subsampling can be performed on samples adjacent to the reconstructed block of the first color component and on the corresponding reconstructed block of the first color component. For example, when one sample of the second color component corresponds to four samples of the first color component, a corresponding sample can be calculated by subsampling the four samples of the first color component. When subsampling is performed, the parameters of the linear model and inter-color intra-frame prediction can be performed based on the subsampled corresponding sample.

[0469] In intra-frame prediction mode, information about whether to perform inter-color intra-frame prediction and / or the range of templates can be sent by signaling.

[0470] The target block can be divided into two or four sub-blocks in the horizontal and / or vertical directions.

[0471] Sub-blocks generated by partitioning can be reconstructed sequentially. That is, when intra-prediction is performed on each sub-block, a sub-prediction block for that sub-block can be generated. Furthermore, when inverse quantization and / or inverse transform is performed on each sub-block, a sub-residual block for the corresponding sub-block can be generated. Reconstructed sub-blocks can be generated by adding the sub-prediction blocks to the sub-residual blocks. The reconstructed sub-blocks can be used as reference samples for intra-prediction of sub-blocks with the next higher priority.

[0472] A sub-block can be a block containing a specific number (e.g., 16) or more samples. For example, when the target block is an 8×4 block or a 4×8 block, the target block can be divided into two sub-blocks. Furthermore, when the target block is a 4×4 block, it cannot be divided into sub-blocks. When the target block has another size, it can be divided into four sub-blocks.

[0473] Signals can be used to send information about whether to perform intra-frame prediction based on these sub-blocks and / or about the partitioning direction (horizontal or vertical).

[0474] This sub-block-based intra-prediction can be restricted so that it is performed only when reference sample line 0 is used. When performing sub-block-based intra-prediction, filtering of the prediction block, which will be described below, may not be performed.

[0475] The final prediction block can be generated by filtering the prediction block generated via intra-frame prediction.

[0476] Filtering can be performed by applying specific weights to the target sample, left reference sample, top reference sample, and / or top-left reference sample, which are the targets to be filtered.

[0477] The weights and / or reference samples (e.g., the range of reference samples, the location of reference samples, etc.) used for filtering can be determined based on at least one of the block size, intra-frame prediction mode, and the location of the target filter sample in the prediction block.

[0478] For example, filtering can be performed only in specific intra-frame prediction modes (e.g., DC mode, planar mode, vertical mode, horizontal mode, diagonal mode, and / or adjacent diagonal mode).

[0479] Adjacent diagonal patterns can be patterns with numbers obtained by adding k to the diagonal pattern's number, or patterns with numbers obtained by subtracting k from the diagonal pattern's number. In other words, the number of an adjacent diagonal pattern can be the sum of the diagonal pattern's number and k, or the difference between the diagonal pattern's number and k. For example, k can be a positive integer of 8 or less.

[0480] The intra prediction mode of the target block can be derived using the intra prediction modes of neighboring blocks that appear near the target block, and this derived intra prediction mode can be entropy encoded and / or entropy decoded.

[0481] For example, when the intra prediction mode of the target block is the same as that of the neighboring blocks, specific flag information can be used to signal information indicating that the intra prediction mode of the target block is the same as that of the neighboring blocks.

[0482] Furthermore, for example, an indicator information of neighboring blocks whose intra-prediction modes are the same as those of the target block can be transmitted using signals.

[0483] For example, when the intra prediction mode of the target block is different from that of the neighboring blocks, entropy coding and / or entropy decoding can be performed on the information about the intra prediction mode of the target block by performing entropy coding and / or entropy decoding based on the intra prediction modes of the neighboring blocks.

[0484] Figure 9 This is a diagram illustrating an embodiment used to explain the inter-frame prediction process.

[0485] Figure 9 The rectangles shown can represent images (or screens). Furthermore, in... Figure 9 In the image, arrows indicate the prediction direction. An arrow pointing from the first frame to the second frame indicates that the second frame references the first frame. In other words, each image can be encoded and / or decoded based on the prediction direction.

[0486] Images can be classified into intra-frame frames (I-frames), single-predictive or predictive-coded frames (P-frames), and double-predictive or double-predictive-coded frames (B-frames) based on their encoding type. Each frame can be encoded and / or decoded according to its encoding type.

[0487] When the target image to be encoded is an I-frame, the target image can be encoded using the data contained in the image itself without inter-frame prediction referencing other images. For example, an I-frame can be encoded solely via intra-frame prediction.

[0488] When the target image is a P-frame, it can be encoded using inter-frame prediction with reference frames existing in one direction. Here, the one direction can be a forward direction or a backward direction.

[0489] When the target image is a B-frame, the image can be encoded via inter-frame prediction using reference frames present in both directions, or via inter-frame prediction using reference frames present in one of the forward and backward directions. Here, the two directions can be the forward and backward directions.

[0490] P-frames and B-frames that are encoded and / or decoded using reference frames can be considered as images using inter-frame prediction.

[0491] The following will describe in detail the inter-frame prediction in inter-frame mode according to the embodiments.

[0492] Reference images and motion information can be used to perform inter-frame prediction or motion compensation.

[0493] In inter-frame mode, encoding device 100 may perform inter-frame prediction and / or motion compensation on the target block. Decoding device 200 may perform inter-frame prediction and / or motion compensation on the target block corresponding to the inter-frame prediction and / or motion compensation performed by encoding device 100.

[0494] Motion information of the target block can be derived separately by the encoding device 100 and the decoding device 200 during inter-frame prediction. Motion information can be derived using the motion information of reconstructed neighboring blocks, the motion information of the col block, and / or the motion information of blocks adjacent to the col block.

[0495] For example, encoding device 100 or decoding device 200 can perform prediction and / or motion compensation by using motion information of spatial candidates and / or temporal candidates as motion information for a target block. The target block may represent a PU and / or a PU partition.

[0496] Spatial candidates can be reconstructed blocks that are spatially adjacent to the target block.

[0497] The time candidate can be a reconstructed block that corresponds to the target block in a previously reconstructed co-location frame (col frame).

[0498] In inter-frame prediction, the encoding device 100 and the decoding device 200 can improve encoding efficiency and decoding efficiency by utilizing motion information from spatial candidates and / or temporal candidates. The motion information from spatial candidates can be referred to as "spatial motion information." The motion information from temporal candidates can be referred to as "temporal motion information."

[0499] Below, the motion information of spatial candidates can be the motion information of PUs including spatial candidates. The motion information of temporal candidates can be the motion information of PUs including temporal candidates. The motion information of candidate blocks can be the motion information of PUs including candidate blocks.

[0500] Inter-frame prediction can be performed using a reference frame.

[0501] The reference image can be at least one of an image preceding or following the target image. The reference image can be an image used for prediction of the target block.

[0502] In inter-frame prediction, regions within a reference frame can be specified using a reference frame index (or refIdx) used to indicate the reference frame, motion vectors that will be described subsequently, and so on. Here, the region specified in the reference frame can indicate a reference block.

[0503] Inter-frame prediction can select a reference frame, and can also select a reference block corresponding to the target block from the reference frame. In addition, inter-frame prediction can use the selected reference block to generate a prediction block for the target block.

[0504] Motion information can be derived by each of the encoding device 100 and the decoding device 200 during inter-frame prediction.

[0505] Spatial candidates can be 1) blocks that exist in the target frame, 2) blocks that have been previously reconstructed via encoding and / or decoding, and 3) blocks that are adjacent to or located at the corner of the target block. Here, a "block located at the corner of the target block" can be a block that is vertically adjacent to a horizontally adjacent neighboring block, or a block that is horizontally adjacent to a vertically adjacent neighboring block. Furthermore, "block located at the corner of the target block" can have the same meaning as "block adjacent to the corner of the target block." The meaning of "block located at the corner of the target block" can be included within the meaning of "block adjacent to the target block."

[0506] For example, a spatial candidate can be a reconstruction block located to the left of the target block, a reconstruction block located above the target block, a reconstruction block located at the lower left corner of the target block, a reconstruction block located at the upper right corner of the target block, or a reconstruction block located at the upper left corner of the target block.

[0507] Each of the encoding device 100 and the decoding device 200 can identify a block existing in the col frame at a spatial position corresponding to the target block. The position of the target block in the target frame and the position of the identified block in the col frame can correspond to each other.

[0508] Each of the encoding device 100 and the decoding device 200 can identify a col block existing at a predefined relevant location for the identified block as a time candidate. The predefined relevant location can be a location existing inside and / or outside the identified block.

[0509] For example, a col block can include a first col block and a second col block. When the coordinates of the identified block are (xP, yP) and the size of the identified block is represented by (nPSW, nPSH), the first col block can be the block located at coordinates (xP+nPSW, yP+nPSH). The second col block can be the block located at coordinates (xP+(nPSW>>1), yP+(nPSH>>1)). When the first col block is unavailable, the second col block can be used selectively.

[0510] The motion vector of the target block can be determined based on the motion vector of the col block. Each of the encoding device 100 and the decoding device 200 can scale the motion vector of the col block. The scaled motion vector of the col block can be used as the motion vector of the target block. Furthermore, the motion vectors of motion information for time candidates stored in a list can also be scaled motion vectors.

[0511] The ratio of the motion vector of the target block to the motion vector of the col block can be the same as the ratio of the first time distance to the second time distance. The first time distance can be the distance between the reference frame and the target frame of the target block. The second time distance can be the distance between the reference frame and the col frame of the col block.

[0512] The scheme used to derive motion information can be varied depending on the inter-frame prediction mode of the target block. For example, inter-frame prediction modes applied to inter-frame prediction may include Advanced Motion Vector Prediction Factor (AMVP) mode, merge mode, skip mode, merge mode with motion vector difference, sub-block merge mode, triangular partitioning mode, inter-frame / intra-frame combined prediction mode, affine inter-frame mode, and current frame reference mode. The merge mode can also be called "motion merge mode." Each mode will be described in detail below.

[0513] 1) AMVP mode When using AMVP mode, the encoding device 100 can search for similar blocks in the neighborhood of the target block. The encoding device 100 can obtain a predicted block by performing a prediction on the target block using the motion information of the found similar blocks. The encoding device 100 can encode the residual block, which is the difference between the target block and the predicted block.

[0514] 1-1) Create a list of candidate motion vectors for prediction. When the AMVP mode is used as the prediction mode, each of the encoding device 100 and the decoding device 200 can use spatial candidate motion vectors, temporal candidate motion vectors, and zero vectors to create a list of prediction motion vector candidates. The prediction motion vector candidate list may include one or more prediction motion vector candidates. At least one of the spatial candidate motion vectors, temporal candidate motion vectors, and zero vectors can be determined and used as a prediction motion vector candidate.

[0515] In the following text, the terms “predicted motion vector (candidate)” and “motion vector (candidate)” can be used to have the same meaning and can be used interchangeably.

[0516] In the following text, the terms “predicted motion vector candidate” and “AMVP candidate” can be used to have the same meaning and can be used interchangeably.

[0517] In the following text, the terms “predicted motion vector candidate list” and “AMVP candidate list” can be used to have the same meaning and can be used interchangeably.

[0518] Spatial candidates can include reconstructed spatial neighbor blocks. In other words, the motion vectors of the reconstructed neighbor blocks can be referred to as "spatial prediction motion vector candidates".

[0519] A temporal candidate can include the col block and the blocks adjacent to the col block. In other words, the motion vector of the col block or the motion vector of the blocks adjacent to the col block can be called a "temporal prediction motion vector candidate".

[0520] The zero vector can be a (0,0) motion vector.

[0521] The predicted motion vector candidate can be a motion vector predictor used to predict the motion vector. Furthermore, in the encoding device 100, each predicted motion vector candidate can be an initial search position for the motion vector.

[0522] 1-2) Searching for motion vectors using a list of predicted motion vector candidates Encoding device 100 can use a list of predicted motion vector candidates to determine, within a search range, the motion vectors that will be used to encode the target block. Furthermore, encoding device 100 can determine, from among the predicted motion vector candidates present in the list of predicted motion vector candidates, the predicted motion vector candidates that will be used as the target block.

[0523] The motion vector used to encode the target block can be a motion vector that can be encoded at the minimum cost.

[0524] In addition, the encoding device 100 can determine whether to use the AMVP mode to encode the target block.

[0525] 1-3) Transmission of inter-frame prediction information Encoding device 100 can generate a bitstream that includes inter-frame prediction information required for inter-frame prediction. Decoding device 200 can use the inter-frame prediction information of the bitstream to perform inter-frame prediction on the target block.

[0526] Inter-frame prediction information may include 1) mode information indicating whether the AMVP mode is used, 2) predicted motion vector index, 3) motion vector difference (MVD), 4) reference direction and 5) reference frame index.

[0527] In the following text, the terms “predicted motion vector index” and “AMVP index” can be used interchangeably and have the same meaning.

[0528] In addition, inter-frame prediction information may include residual signals.

[0529] When the mode information indicates that the AMVP mode is used, the decoding device 200 can obtain the predicted motion vector index, MVD, reference direction, and reference frame index from the bitstream through entropy decoding.

[0530] The predicted motion vector index indicates which of the predicted motion vector candidates included in the predicted motion vector candidate list will be used to predict the target block.

[0531] 1-4) Inter-frame prediction in AMVP mode using inter-frame prediction information The decoding device 200 can use the list of predicted motion vector candidates to derive predicted motion vector candidates, and can determine the motion information of the target block based on the derived predicted motion vector candidates.

[0532] The decoding device 200 can use the predicted motion vector index to determine a motion vector candidate for the target block from among the predicted motion vector candidates included in the predicted motion vector candidate list. The decoding device 200 can select the predicted motion vector candidate indicated by the predicted motion vector index from among the predicted motion vector candidates included in the predicted motion vector candidate list as the predicted motion vector for the target block.

[0533] Encoding device 100 can generate an entropy-coded predicted motion vector index by applying entropy coding to the predicted motion vector index, and can generate a bitstream including the entropy-coded predicted motion vector index. The entropy-coded predicted motion vector index can be transmitted from encoding device 100 to decoding device 200 via the bitstream. Decoding device 200 can extract the entropy-coded predicted motion vector index from the bitstream, and can obtain the predicted motion vector index by applying entropy decoding to the entropy-coded predicted motion vector index.

[0534] The motion vector actually used for inter-frame prediction of the target block may not match the predicted motion vector. MVD (Motion Vector Difference) can be used to indicate the difference between the actual motion vector used for inter-frame prediction of the target block and the predicted motion vector. The encoding device 100 can derive a predicted motion vector similar to the actual motion vector used for inter-frame prediction of the target block in order to use the smallest possible MVD.

[0535] Motion Vector Difference (MVD) can be the difference between the motion vector of the target block and the predicted motion vector. Encoding device 100 can compute the MVD and generate an entropy-coded MVD by applying entropy coding to the MVD. Encoding device 100 can generate a bitstream including the entropy-coded MVD.

[0536] The MVD can be sent from the encoding device 100 to the decoding device 200 via a bitstream. The decoding device 200 can extract the entropy-encoded MVD from the bitstream and obtain the MVD by applying entropy decoding to the entropy-encoded MVD.

[0537] The decoding device 200 can derive the motion vector of the target block by summing the MVD and the predicted motion vector. In other words, the motion vector of the target block derived by the decoding device 200 can be the sum of the MVD and the motion vector candidate.

[0538] Furthermore, the encoding device 100 can generate entropy-coded MVD resolution information by applying entropy coding to the calculated MVD resolution information, and can generate a bitstream including the entropy-coded MVD resolution information. The decoding device 200 can extract the entropy-coded MVD resolution information from the bitstream, and can obtain the MVD resolution information by applying entropy decoding to the entropy-coded MVD resolution information. The decoding device 200 can use the MVD resolution information to adjust the MVD resolution.

[0539] Additionally, the encoding device 100 can calculate the MVD based on an affine model. The decoding device 200 can derive the affine control motion vector of the target block from the sum of the MVD and the affine control motion vector candidates, and can use the affine control motion vector to derive the motion vector of the sub-blocks.

[0540] The reference direction can indicate a list of reference frames that will be used to predict the target block. For example, the reference direction can indicate one of reference frame list L0 and reference frame list L1.

[0541] The reference direction only indicates the list of reference frames that will be used to predict the target block, and does not necessarily mean that the direction of the reference frames is limited to the forward or backward direction. In other words, each of the reference frame lists L0 and L1 can include frames in the forward and / or backward directions.

[0542] A unidirectional reference direction can mean using a single reference screen list. A bidirectional reference direction can mean using two reference screen lists. In other words, the reference direction can indicate one of the following: using only reference screen list L0, using only reference screen list L1, or using both reference screen lists.

[0543] A reference frame index indicates a reference frame in a list of reference frames used to predict a target block. Encoding device 100 can generate an entropy-coded reference frame index by applying entropy coding to the reference frame index, and can generate a bitstream including the entropy-coded reference frame index. The entropy-coded reference frame index can be signaled from encoding device 100 to decoding device 200 via the bitstream. Decoding device 200 can extract the entropy-coded reference frame index from the bitstream, and can obtain the reference frame index by applying entropy decoding to the entropy-coded reference frame index.

[0544] When two reference frame lists are used to predict a target block, a single reference frame index and a single motion vector can be used for each of the reference frame lists. Furthermore, when two reference frame lists are used to predict a target block, two prediction blocks can be specified for the target block. For example, the (final) prediction block for the target block can be generated using the average or weighted sum of the two prediction blocks for the target block.

[0545] The motion vector of a target block can be derived by predicting the motion vector index, MVD, reference direction, and reference screen index.

[0546] The decoding device 200 can generate a predicted block for a target block based on the exported motion vectors and a reference frame index. For example, the predicted block can be a reference block indicated by the exported motion vectors in a reference frame indicated by the reference frame index.

[0547] Since the predicted motion vector index and MVD are encoded, while the motion vector of the target block itself is not encoded, the number of bits sent from the encoding device 100 to the decoding device 200 can be reduced, and the encoding efficiency can be improved.

[0548] For the target block, motion information from reconstructed neighboring blocks can be used. In certain inter-frame prediction modes, the encoding device 100 may not encode the actual motion information of the target block separately. Instead of encoding the motion information of the target block, additional information can be encoded, which enables the derivation of the target block's motion information using the motion information from reconstructed neighboring blocks. Because this additional information is encoded, the number of bits sent to the decoding device 200 can be reduced, and encoding efficiency can be improved.

[0549] For example, in inter-frame prediction modes where motion information of the target block is not directly encoded, skipping modes and / or merging modes may exist. Here, the motion information of each of the neighboring units in the encoding device 100 and decoding device 200 that can be used to indicate reconstruction will be used as the identifier and / or index of the unit's motion information for the target unit.

[0550] 2) Merge Mode Merging is a scheme used to derive motion information for a target block. The term "merging" can mean merging the motion of multiple blocks. Merging can also mean that the motion information of one block is applied to other blocks. In other words, a merging pattern can be a mode for deriving the motion information of a target block from the motion information of neighboring blocks.

[0551] When using the merging mode, the encoding device 100 can use motion information from spatial candidates and / or temporal candidates to predict motion information of the target block. Spatial candidates may include reconstructed spatially adjacent blocks that are spatially adjacent to the target block. Spatially adjacent blocks may include left-side adjacent blocks and top-side adjacent blocks. Temporal candidates may include col blocks. The terms "spatial candidate" and "spatial merging candidate" are used interchangeably and have the same meaning. The terms "temporal candidate" and "temporal merging candidate" are used interchangeably and have the same meaning.

[0552] The encoding device 100 can obtain a prediction block through prediction. The encoding device 100 can encode a residual block, which is the difference between the target block and the prediction block.

[0553] 2-1) Create a list of candidate mergers When using the merging mode, each of the encoding device 100 and the decoding device 200 can create a merging candidate list using motion information from spatial candidates and / or motion information from temporal candidates. The motion information may include 1) a motion vector, 2) a reference frame index, and 3) a reference direction. The reference direction can be unidirectional or bidirectional. The reference direction may represent an inter-frame prediction indicator.

[0554] The merge candidate list can include merge candidates. Merge candidates can be motion information. In other words, the merge candidate list can be a list that stores multiple pieces of motion information.

[0555] The merged candidate can be the motion information of multiple temporal and / or spatial candidates. In other words, the merged candidate list can include the motion information of temporal and / or spatial candidates, etc.

[0556] Furthermore, the merge candidate list may include new merge candidates generated by combining merge candidates that already exist in the merge candidate list. In other words, the merge candidate list may include new motion information generated by combining multiple motion information items that previously existed in the merge candidate list.

[0557] In addition, the merge candidate list may include history-based merge candidates. History-based merge candidates may be motion information of blocks that were encoded and / or decoded before the target block.

[0558] In addition, the list of merge candidates may include merge candidates based on the average of two merge candidates.

[0559] Merging candidates can be specific patterns for deriving inter-frame prediction information. Merging candidates can be information indicating specific patterns for deriving inter-frame prediction information. Inter-frame prediction information for a target block can be derived based on the specific patterns indicated by the merging candidates. Furthermore, the specific patterns can include the processing of deriving a series of inter-frame prediction information. Such specific patterns can be inter-frame prediction information deriving patterns or motion information deriving patterns.

[0560] Inter-frame prediction information for the target block can be derived based on the pattern indicated by the merge candidate selected in the merge candidate list via the merge index.

[0561] For example, the motion information export mode in the merge candidate list can be at least one of the following modes: 1) motion information export mode for sub-block units and 2) affine motion information export mode.

[0562] In addition, the list of merged candidates may include motion information for the zero vector. The zero vector may also be referred to as a "zero merged candidate".

[0563] In other words, the multiple motion information in the merged candidate list can be at least one of the following: 1) motion information of spatial candidates, 2) motion information of temporal candidates, 3) motion information generated by combining multiple motion information that previously existed in the merged candidate list, and 4) zero vector.

[0564] Motion information may include 1) motion vectors, 2) reference frame indexes, and 3) reference directions. The reference direction can also be referred to as an "inter-frame prediction indicator." The reference direction can be unidirectional or bidirectional. A unidirectional reference direction can indicate L0 prediction or L1 prediction.

[0565] A list of merge candidates can be created before performing predictions in merge mode.

[0566] The number of merge candidates in the merge candidate list can be predefined. Each of the encoding device 100 and the decoding device 200 can add merge candidates to the merge candidate list according to a predefined scheme and a predefined priority, so that the merge candidate list has a predefined number of merge candidates. The merge candidate list of the encoding device 100 and the merge candidate list of the decoding device 200 can be made identical to each other using a predefined scheme and a predefined priority.

[0567] Merging can be applied based on either CU or PU. When merging is performed based on CU or PU, the encoding device 100 can send a bitstream including predefined information to the decoding device 200. For example, the predefined information may include 1) information indicating whether merging is to be performed on each block partition, and 2) information about the blocks to be merged among the blocks that are spatial candidates and / or temporal candidates for the target block.

[0568] 2-2) Search for motion vectors using a merged candidate list Encoding device 100 can determine merge candidates to be used for encoding a target block. For example, encoding device 100 can perform prediction on the target block using merge candidates from the merge candidate list and can generate residual blocks for the merge candidates. Encoding device 100 can encode the target block using the merge candidate that generates the minimum cost in encoding both the prediction and the residual blocks.

[0569] In addition, the encoding device 100 can determine whether to use a merge mode to encode the target block.

[0570] 2-3) Transmission of inter-frame prediction information Encoding device 100 can generate a bitstream including inter-frame prediction information required for inter-frame prediction. Encoding device 100 can generate entropy-coded inter-frame prediction information by performing entropy coding on the inter-frame prediction information, and can send the bitstream including the entropy-coded inter-frame prediction information to decoding device 200. The entropy-coded inter-frame prediction information can be sent by encoding device 100 to decoding device 200 via a bitstream signal. Decoding device 200 can extract the entropy-coded inter-frame prediction information from the bitstream, and can obtain inter-frame prediction information by applying entropy decoding to the entropy-coded inter-frame prediction information.

[0571] The decoding device 200 can use the inter-frame prediction information of the bitstream to perform inter-frame prediction on the target block.

[0572] Inter-frame prediction information may include 1) mode information indicating whether a merging mode is used, 2) merging index, and 3) correction information.

[0573] In addition, inter-frame prediction information may include residual signals.

[0574] Decoding device 200 can obtain the merge index from the bitstream only when the mode information indicates that the merge mode is used.

[0575] Pattern information can be merge flags. The unit of pattern information can be a block. Information about a block can include pattern information, and the pattern information can indicate whether a merge pattern is applied to the block.

[0576] The merge index can indicate which merge candidate from the merge candidate list will be used to predict the target block. Optionally, the merge index can indicate which block from the spatially or temporally adjacent neighboring blocks will be merged with the target block.

[0577] Encoding device 100 can select the merge candidate with the highest encoding performance from the merge candidate list, and can set the value of the merge index to indicate the selected merge candidate.

[0578] The correction information can be information used to correct motion vectors. Encoding device 100 can generate the correction information. Decoding device 200 can correct the motion vectors of the merging candidates selected by the merging index based on the correction information.

[0579] The correction information may include at least one of information indicating whether correction will be performed, correction direction information, and correction size information. A prediction mode that corrects motion vectors based on correction information transmitted by a signal may be referred to as a "merging mode with motion vector difference".

[0580] 2-4) Inter-frame prediction using merging mode with inter-frame prediction information Decoding device 200 can perform prediction on target block using a merge candidate indicated by a merge index from among the merge candidates included in the merge candidate list.

[0581] The motion vector of the target block can be specified by the motion vector of the merge candidate indicated by the merge index, the reference screen index, and the reference direction.

[0582] 3) Skip Mode Skip mode can be a mode that applies spatial or temporal motion information to the target block without alteration. Furthermore, skip mode can be a mode that does not use the residual signal. In other words, when using skip mode, the reconstructed block can be identical to the predicted block.

[0583] The difference between merge mode and skip mode lies in whether or not residual signals are sent or used. In other words, skip mode is similar to merge mode except that residual signals are not sent or used.

[0584] When using skip mode, encoding device 100 can transmit information about blocks whose motion information will be used as motion information for target blocks via a bitstream to decoding device 200. Encoding device 100 can generate entropy-encoded information by performing entropy encoding on this information, and can transmit the entropy-encoded information as a signal to decoding device 200 via a bitstream. Decoding device 200 can extract the entropy-encoded information from the bitstream, and can obtain information by applying entropy decoding to the entropy-encoded information.

[0585] Furthermore, when using the skip mode, the encoding device 100 may not send other syntax information (such as MVD) to the decoding device 200. For example, when using the skip mode, the encoding device 100 may not send the syntax elements associated with at least one of MVD, code block flag, and transform coefficient level to the decoding device 200.

[0586] 3-1) Create a list of candidate mergers The merge candidate list can also be used in skip mode. In other words, the merge candidate list can be used in both merge mode and skip mode. In this respect, the merge candidate list can also be referred to as the "skip candidate list" or the "merge / skip candidate list".

[0587] Optionally, the skip mode may use an additional candidate list that differs from the candidate list used in the merge mode. In this case, in the following description, the merge candidate list and merge candidate may be replaced by the skip candidate list and the skip candidate, respectively.

[0588] A list of merged candidates can be created before performing predictions in skip mode.

[0589] 3-2) Search for motion vectors using a merged candidate list Encoding device 100 can determine merge candidates to be used for encoding the target block. For example, encoding device 100 can perform prediction on the target block using merge candidates from the merge candidate list. Encoding device 100 can encode the target block using the merge candidate that generates the minimum cost in the prediction.

[0590] In addition, the encoding device 100 can determine whether to use a skip mode to encode the target block.

[0591] 3-3) Transmission of inter-frame prediction information Encoding device 100 can generate a bitstream that includes inter-frame prediction information required for inter-frame prediction. Decoding device 200 can use the inter-frame prediction information of the bitstream to perform inter-frame prediction on the target block.

[0592] Inter-frame prediction information may include 1) mode information indicating whether a skip mode is used and 2) a skip index.

[0593] Skipping indexes is the same as merging indexes as described above.

[0594] When using skip mode, the target block can be encoded without using the residual signal. Inter-frame prediction information may not include the residual signal. Optionally, the bitstream may not include the residual signal.

[0595] Decoding device 200 may obtain the skip index from the bitstream only when the mode information indicates that a skip mode is used. As mentioned above, the merge index and the skip index may be the same. Decoding device 200 may obtain the skip index from the bitstream only when the mode information indicates that either a merge mode or a skip mode is used.

[0596] Skip index indicates which of the merge candidates included in the merge candidate list will be used to predict the target block.

[0597] 3-4) Inter-frame prediction in skip mode using inter-frame prediction information Decoding device 200 can perform prediction on the target block using a merge candidate indicated by a skip index from among the merge candidates included in the merge candidate list.

[0598] The motion vector of the target block can be specified by the motion vector of the merge candidate indicated by the skip index, the reference screen index, and the reference direction.

[0599] 4) Current screen reference mode The current frame reference mode can represent a prediction mode that uses the previously reconstructed area in the target frame to which the target block belongs.

[0600] Motion vectors can be used to specify previously reconstructed areas. The reference frame index of the target block can be used to determine whether the target block has been encoded in the current frame reference mode.

[0601] A flag or index indicating whether a target block is encoded in the current screen reference mode can be sent by the encoding device 100 to the decoding device 200. Optionally, whether a target block is encoded in the current screen reference mode can be inferred from the target block's reference screen index.

[0602] When a target block is encoded in the current frame reference mode, the current frame can exist in a fixed position or any position in the reference frame list for the target block.

[0603] For example, the fixed position could be the position where the reference screen index value is 0 or the last position.

[0604] When the target image exists at any position in the list of reference images, an additional reference image index indicating such an arbitrary position can be sent by the encoding device 100 to the decoding device 200 by a signal.

[0605] 5) Sub-block merging mode Sub-block merging mode can be a mode that derives motion information from sub-blocks of the CU.

[0606] When applying the sub-block merging mode, the motion information of col-sub-blocks of the target sub-block in the reference image (i.e., based on the temporal merging candidate of the sub-block) and / or affine control point motion vector merging candidates can be used to generate a list of sub-block merging candidates.

[0607] 6) Triangular partitioning mode In the triangular partitioning mode, the target block can be partitioned diagonally, and sub-target blocks generated through partitioning can be produced. For each sub-target block, motion information of the corresponding sub-target block can be exported, and the exported motion information can be used to derive the prediction samples of each sub-target block. The prediction samples of the target block can be derived by weighted summing of the prediction samples of the sub-target blocks generated through partitioning.

[0608] 7) Combined inter-frame and intra-frame prediction modes The combined inter-frame-intra-frame prediction mode can be a mode that uses a weighted sum of prediction samples generated via inter-frame prediction and prediction samples generated via intra-frame prediction to derive prediction samples for the target block.

[0609] In the above mode, the decoding device 200 can autonomously correct the exported motion information. For example, the decoding device 200 can search for motion information with the minimum sum of absolute differences (SAD) in a specific region based on a reference block indicated by the exported motion information, and can export the found motion information as corrected motion information.

[0610] In the above mode, the decoding device 200 can use optical flow to compensate for the prediction samples derived via inter-frame prediction.

[0611] In the AMVP mode, merge mode, skip mode, etc. described above, the index information of the list can be used to specify the motion information among multiple motion information in the list that will be used to predict the target block.

[0612] To improve coding efficiency, the encoding device 100 may use only the index of the element in the signal transmission list that generates the minimum cost in inter-frame prediction of the target block. The encoding device 100 may encode this index and may signal the encoded index.

[0613] Therefore, the encoding device 100 and the decoding device 200 must be able to derive the lists described above (i.e., the candidate list for predicted motion vectors and the candidate list for merging) using the same scheme and based on the same data. Here, the same data may include reconstructed frames and reconstructed blocks. Furthermore, in order to specify elements using indices, the order of elements in the lists must be fixed.

[0614] Figure 10 Spatial candidates according to an embodiment are shown.

[0615] exist Figure 10 The image shows the locations of the spatial candidates.

[0616] The large block in the center of the graph represents the target block. The five smaller blocks represent spatial candidates.

[0617] The coordinates of the target block can be (xP, yP), and the size of the target block can be represented by (nPSW, nPSH).

[0618] Spatial candidate A0 can be a block adjacent to the lower left corner of the target block. A0 can be a block that occupies the pixel located at coordinates (xP-1, yP+nPSH).

[0619] Spatial candidate A1 can be the block that is adjacent to the left side of the target block. A1 can be the bottommost block among the blocks that are adjacent to the left side of the target block. Alternatively, A1 can be the block that is adjacent to the top side of A0. A1 can be the block that occupies the pixel located at coordinates (xP-1, yP+nPSH-1).

[0620] Spatial candidate B0 can be the block adjacent to the top right corner of the target block. B0 can be a block that occupies the pixel located at coordinates (xP+nPSW, yP-1).

[0621] Spatial candidate B1 can be the block that is top-adjacent to the target block. B1 can be the rightmost block among the blocks that are top-adjacent to the target block. Alternatively, B1 can be the block that is left-adjacent to B0. B1 can be the block that occupies the pixel located at coordinates (xP+nPSW-1, yP-1).

[0622] Spatial candidate B2 can be a block adjacent to the top-left corner of the target block. B2 can be a block that occupies the pixel located at coordinates (xP-1, yP-1).

[0623] Determining the availability of spatial and temporal candidates In order to include spatial or temporal motion information in the list, it must be determined whether the spatial or temporal motion information is available.

[0624] In the following text, candidate blocks may include spatial candidates and temporal candidates.

[0625] For example, the determination can be performed by sequentially applying steps 1) through 4).

[0626] Step 1) When the PU including the candidate block is outside the boundary of the screen, the availability of the candidate block can be set to "false". The statement "availability is set to false" can have the same meaning as "set to unavailable".

[0627] Step 2) When the PU including the candidate block is outside the boundary of the stripe, the availability of the candidate block can be set to "false". When the target block and the candidate block are in different stripes, the availability of the candidate block can be set to "false".

[0628] Step 3) When the PU including the candidate block is outside the boundary of the parallel block, the availability of the candidate block can be set to "false". When the target block and the candidate block are in different parallel blocks, the availability of the candidate block can be set to "false".

[0629] Step 4) When the prediction mode of the PU including the candidate block is intra-frame prediction mode, the availability of the candidate block can be set to "false". When the PU including the candidate block does not use inter-frame prediction, the availability of the candidate block can be set to "false".

[0630] Figure 11 The order in which motion information of spatial candidates is added to the merging list is shown according to an embodiment.

[0631] like Figure 11 As shown, when multiple motion information entries from spatial candidates are added to the merge list, the order A1, B1, B0, A0, and B2 can be used. In other words, multiple motion information entries from available spatial candidates can be added to the merge list in the order A1, B1, B0, A0, and B2.

[0632] Methods for exporting merge lists in merge mode and skip mode As described above, the maximum number of merge candidates in the merge list can be set. The maximum number can be indicated by "N". The set number can be sent from the encoding device 100 to the decoding device 200. The stripe header can include N. In other words, the maximum number of merge candidates in the merge list for the target block of the stripe can be set via the stripe header. For example, the value of N can essentially be 5.

[0633] Multiple motion information (i.e., merge candidates) can be added to the merge list in the order of steps 1) to 4).

[0634] Step 1) Within the space candidates, available space candidates can be added to the merge list. This can be done by... Figure 10 The order shown in the diagram is used to add motion information of available space candidates to the merge list. Here, if the motion information of an available space candidate overlaps with other motion information already existing in the merge list, the motion information of the available space candidate may not be added to the merge list. The operation of checking whether the corresponding motion information overlaps with other motion information existing in the list can be simply referred to as "overlap check".

[0635] The maximum number of motion information entries that can be added is N.

[0636] Step 2) When the number of motion information entries in the merge list is less than N and time candidates are available, the motion information of the time candidates can be added to the merge list. However, if the available motion information of time candidates overlaps with other motion information already existing in the merge list, the available motion information of time candidates may not be added to the merge list.

[0637] Step 3) When the number of motion information entries in the merge list is less than N and the target strip type is "B", the combined motion information generated by combining bidirectional prediction (double prediction) can be added to the merge list.

[0638] The target strip can be a strip that includes the target block.

[0639] Combined motion information can be a combination of L0 motion information and L1 motion information. L0 motion information can be motion information that only refers to the L0 reference frame list. L1 motion information can be motion information that only refers to the L1 reference frame list.

[0640] The merged list may contain one or more L0 motion entries. Additionally, the merged list may contain one or more L1 motion entries.

[0641] Combined motion information may include one or more pieces of combined motion information. When generating combined motion information, the L0 motion information and L1 motion information that will be used in the step of generating combined motion information can be predefined from the one or more L0 motion information and the one or more L1 motion information. One or more pieces of combined motion information can be generated in a predefined order via bidirectional prediction using a pair of different motion information from a merge list. One of the different motion information in the pair can be L0 motion information, and the other of the different motion information in the pair can be L1 motion information.

[0642] For example, the combined motion information with the highest priority can be a combination of L0 motion information with a merge index of 0 and L1 motion information with a merge index of 1. When the motion information with a merge index of 0 is not L0 motion information, or when the motion information with a merge index of 1 is not L1 motion information, neither combined motion information is generated nor added. Next, the combined motion information with the next higher priority can be a combination of L0 motion information with a merge index of 1 and L1 motion information with a merge index of 0. Subsequent detailed combinations can conform to other combinations in the field of video encoding / decoding.

[0643] Here, when the combined motion information overlaps with other motion information already existing in the merge list, the combined motion information may not be added to the merge list.

[0644] Step 4) When the number of motion information entries in the merge list is less than N, the motion information of the zero vector can be added to the merge list.

[0645] Zero-vector motion information can be motion information where the motion vector is zero.

[0646] The number of zero-vector motion information entries can be one or more. The reference frame indices for one or more zero-vector motion information entries can be different from each other. For example, the reference frame index value for the first zero-vector motion information entry can be 0. The reference frame index value for the second zero-vector motion information entry can be 1.

[0647] The number of zero-vector motion information entries can be the same as the number of reference frames in the reference frame list.

[0648] The reference direction for zero-vector motion information can be bidirectional. Both motion vectors can be zero vectors. The number of zero-vector motion information entries can be the smaller of the number of reference frames in reference frame list L0 and the number of reference frames in reference frame list L1. Optionally, when the number of reference frames in reference frame list L0 and the number of reference frames in reference frame list L1 are different from each other, a unidirectional reference direction can be used for reference frame indexing that can be applied to only a single reference frame list.

[0649] Encoding device 100 and / or decoding device 200 may subsequently add zero-vector motion information to the merge list while changing the reference screen index.

[0650] When zero-vector motion information overlaps with other motion information already existing in the merge list, the zero-vector motion information may not be added to the merge list.

[0651] The order of steps 1) to 4) above is merely exemplary and can be changed. Furthermore, some steps in the above steps may be omitted based on predefined conditions.

[0652] Method for exporting a candidate list of predicted motion vectors in AMVP mode The maximum number of predicted motion vector candidates in the candidate list can be predefined. N can be used to indicate the predefined maximum number. For example, the predefined maximum number could be 2.

[0653] Multiple pieces of motion information (i.e., predicted motion vector candidates) can be added to the predicted motion vector candidate list in the order of steps 1) to 3).

[0654] Step 1) Available spatial candidates can be added to the list of predicted motion vector candidates. Spatial candidates may include a first spatial candidate and a second spatial candidate.

[0655] The first spatial candidate can be one of A0, A1, scaled A0, and scaled A1. The second spatial candidate can be one of B0, B1, B2, scaled B0, scaled B1, and scaled B2.

[0656] Multiple motion information entries from available spatial candidates can be added to the predicted motion vector candidate list in the order of first spatial candidate and second spatial candidate. In this case, if the motion information of an available spatial candidate overlaps with other motion information already existing in the predicted motion vector candidate list, the motion information of the available spatial candidate may not be added to the predicted motion vector candidate list. In other words, when the value of N is 2, if the motion information of the second spatial candidate is the same as the motion information of the first spatial candidate, then the motion information of the second spatial candidate may not be added to the predicted motion vector candidate list.

[0657] The maximum number of motion information entries that can be added is N.

[0658] Step 2) When the number of motion information entries in the predicted motion vector candidate list is less than N and a time candidate is available, the motion information of the time candidate can be added to the predicted motion vector candidate list. In this case, if the motion information of the available time candidate overlaps with other motion information already existing in the predicted motion vector candidate list, the motion information of the available time candidate may not be added to the predicted motion vector candidate list.

[0659] Step 3) When the number of motion information entries in the candidate list of predicted motion vectors is less than N, zero-vector motion information can be added to the candidate list of predicted motion vectors.

[0660] Zero-vector motion information may include one or more zero-vector motion information pieces. The reference frame indices of the one or more zero-vector motion information pieces may be different from each other.

[0661] Encoding device 100 and / or decoding device 200 can sequentially add multiple zero-vector motion information to the candidate list of predicted motion vectors while changing the reference frame index.

[0662] When zero-vector motion information overlaps with other motion information already existing in the candidate list of predicted motion vectors, the zero-vector motion information may not be added to the candidate list of predicted motion vectors.

[0663] The description of zero-vector motion information presented above, combined with the merged list, can also be applied to zero-vector motion information. Repeated descriptions will be omitted.

[0664] The order of steps 1) to 3) described above is merely exemplary and can be changed. Furthermore, some steps may be omitted based on predefined conditions.

[0665] Figure 12 The transformation and quantization processes are shown based on the example.

[0666] like Figure 12 As shown, quantization levels can be generated by performing transformation and / or quantization processing on the residual signal.

[0667] The residual signal can be generated as the difference between the original block and the predicted block. Here, the predicted block can be a block generated via intra-frame prediction or inter-frame prediction.

[0668] The residual signal can be transformed into a signal in the frequency domain through a transformation process that is part of the quantization process.

[0669] Transform kernels used for transformations can include various DCT kernels, such as Discrete Cosine Transform (DCT) Type 2 (DCT-II) and Discrete Sine Transform (DST) kernels.

[0670] These transform kernels can perform separable or two-dimensional (2D) non-separable transforms on the residual signal. A separable transform can be a transform that indicates performing a one-dimensional (1D) transform on the residual signal in each of the horizontal and vertical directions.

[0671] In addition to DCT-II, the DCT and DST types adaptively used for 1D transformations may also include DCT-V, DCT-VIII, DST-I, and DST-VII, as shown in each of Tables 3 and 4 below.

[0672] Table 3

[0673] Table 4

[0674] As shown in Tables 3 and 4, transform sets can be used when deriving the DCT or DST type to be used for the transform. Each transform set can include multiple transform candidates. Each transform candidate can be a DCT type or a DST type.

[0675] Table 5 below shows examples of the transform sets that will be applied in the horizontal direction and the transform sets that will be applied in the vertical direction according to the intra-frame prediction mode.

[0676] Table 5

[0677] Table 5 shows the numbers of the vertical transform set and horizontal transform set that will be applied to the horizontal direction of the residual signal according to the intra-frame prediction mode of the target block.

[0678] As illustrated in Table 5, the transform sets to be applied in the horizontal and vertical directions can be predefined based on the intra-prediction mode of the target block. Encoding device 100 can use transforms included in the transform set corresponding to the intra-prediction mode of the target block to perform transforms and inverse transforms on the residual signal. Furthermore, decoding device 200 can use transforms included in the transform set corresponding to the intra-prediction mode of the target block to perform an inverse transform on the residual signal.

[0679] In the transform and inverse transform, as illustrated in Tables 3, 4, and 5, the set of transforms to be applied to the residual signal can be determined and may not be transmitted by signal. Transform indication information can be transmitted from the encoding device 100 to the decoding device 200 by signal. The transform indication information may be information indicating which of the multiple transform candidates included in the set of transforms to be applied to the residual signal is used.

[0680] For example, when the target block size is 64×64 or smaller, all three transform sets can be configured according to the intra-frame prediction mode. The optimal transform method can be selected from a total of nine multi-transform methods generated by combinations of three horizontal and three vertical transforms. With such an optimal transform method, the residual signal can be encoded and / or decoded, thus improving coding efficiency.

[0681] Here, information indicating which of the multiple transformations belonging to each transform set has been used for at least one of the vertical and horizontal transformations can be entropy encoded and / or entropy decoded. Here, truncated univariate binarization can be used to encode and / or decode such information.

[0682] As mentioned above, various transformation methods can be applied to residual signals generated via intra-frame prediction or inter-frame prediction.

[0683] The transformation may include at least one of a primary transformation and a secondary transformation. Transform coefficients can be generated by performing a primary transformation on the residual signal, and secondary transform coefficients can be generated by performing a secondary transformation on the transform coefficients.

[0684] The primary transformation can be referred to as the "initial transformation". Furthermore, the primary transformation can also be called the "Adaptive Multiple Transformation (AMT) scheme". As mentioned above, AMT can represent applying different transformations to various 1D directions (i.e., the vertical and horizontal directions).

[0685] Secondary transformations can be transformations used to increase the energy concentration of the transformation coefficients generated by primary transformations. Similar to primary transformations, secondary transformations can be separable or non-separable. Such non-separable transformations can be non-separable secondary transformations (NSSTs).

[0686] The primary transformation can be performed using at least one of a predefined set of transformation methods. For example, the predefined set of transformation methods may include the Discrete Cosine Transform (DCT), the Discrete Sine Transform (DST), the Karhunen-Loeve Transform (KLT), and so on.

[0687] Furthermore, depending on the kernel function defined for the Discrete Cosine Transform (DCT) or Discrete Sine Transform (DST), the primary transform can be of various types.

[0688] For example, the transform type can be determined based on at least one of the following: 1) the prediction mode of the target block (e.g., one of intra-frame prediction and inter-frame prediction), 2) the size of the target block, 3) the shape of the target block, 4) the intra-frame prediction mode of the target block, 5) the components of the target block (e.g., one of luma component and chroma component), and 6) the partition type applied to the target block (e.g., one of quadtree, binary tree and ternary tree).

[0689] For example, based on the transform kernels presented in Table 6 below, the primary transform can include transforms such as DCT-2, DCT-5, DCT-7, DST-7, DST-1, DST-8, and DCT-8. Table 6 below illustrates various transform types and transform kernel functions used for Multiple Transform Selection (MTS).

[0690] MTS can refer to the selection of a combination of one or more DCT and / or DST kernels to transform the residual signal in the horizontal and / or vertical directions.

[0691] Table 6

[0692] In Table 6, i and j can be integer values ​​that are equal to or greater than 0 and less than or equal to N-1.

[0693] Secondary transformations can be performed on the transformation coefficients generated by performing primary transformations.

[0694] For example, in a primary transformation, a transformation set can also be defined in a secondary transformation. The methods used to derive and / or determine the above transformation set can be applied not only to primary transformations but also to secondary transformations.

[0695] Primary and secondary transformations can be determined for specific targets.

[0696] For example, primary and secondary transforms can be applied to signal components corresponding to one or more of the luma and chroma components. Whether to apply primary and / or secondary transforms can be determined based on at least one of the coding parameters for the target block and / or neighboring blocks. For example, the application of primary and / or secondary transforms can be determined based on the size and / or shape of the target block.

[0697] In the encoding device 100 and the decoding device 200, transformation information indicating the transformation method to be used for the target can be derived by utilizing specified information.

[0698] For example, the transformation information may include transformation indices that will be used for primary and / or secondary transformations. Optionally, the transformation information may indicate that primary and / or secondary transformations are not used.

[0699] For example, when the target of the primary and secondary transforms is a target block, the transform method to be applied to the primary and / or secondary transforms, as indicated by the transform information, can be determined based on at least one of the encoding parameters for the target block and / or blocks adjacent to the target block.

[0700] Optionally, the encoding device 100 may send transformation information indicating the transformation method for a specific target to the decoding device 200 via a signal.

[0701] For example, for a single CU, the decoding device 200 can derive transformation information such as whether a primary transformation is used, the index indicating the primary transformation, whether a secondary transformation is used, and the index indicating the secondary transformation. Alternatively, for a single CU, transformation information indicating the following can be transmitted via signals: whether a primary transformation is used, the index indicating the primary transformation, whether a secondary transformation is used, and the index indicating the secondary transformation.

[0702] Quantized transform coefficients (i.e., quantization levels) can be generated by quantizing the results produced by performing primary and / or secondary transforms, or by quantizing the residual signal.

[0703] Figure 13 This shows a diagonal scan based on an example.

[0704] Figure 14 The horizontal scan is shown based on the example.

[0705] Figure 15 The vertical scan is shown based on the example.

[0706] The quantized transform coefficients can be scanned via at least one of (top right) diagonal scan, vertical scan, and horizontal scan, based on at least one of intra-frame prediction mode, block size, and block shape. The block can be a transform unit (TU).

[0707] Each scan can be started at a specific start point and terminated at a specific end point.

[0708] For example, by using Figure 13 A diagonal scan is used to scan the coefficients of the block to transform the quantized transform coefficients into a 1D vector form. Optionally, this can be used depending on the block size and / or intra-frame prediction mode. Figure 14 Horizontal scan or Figure 15 It uses vertical scanning instead of diagonal scanning.

[0709] A vertical scan can be an operation that scans 2D block-type coefficients in the column direction. A horizontal scan can be an operation that scans 2D block-type coefficients in the row direction.

[0710] In other words, the choice between diagonal, vertical, and horizontal scans can be determined based on the block size and / or inter-frame prediction mode.

[0711] like Figure 13 , Figure 14 and Figure 15 As shown, the quantized transform coefficients can be scanned along the diagonal, horizontal, or vertical direction.

[0712] The quantized transformation coefficients can be represented by block shapes. Each block can include multiple sub-blocks. Each sub-block can be defined based on either the minimum block size or the minimum block shape.

[0713] During scanning, the scanning order, based on the type or direction of the scan, can be applied first to the sub-blocks. Furthermore, the scanning order, based on the direction of the scan, can be applied to the quantized transform coefficients within each sub-block.

[0714] For example, such as Figure 13 , Figure 14 and Figure 15 As shown, when the target block size is 8×8, quantized transform coefficients can be generated by primary transform, secondary transform, and quantization of the residual signal of the target block. Therefore, one of the three types of scan sequences can be applied to four 4×4 sub-blocks, and the quantized transform coefficients can be scanned for each 4×4 sub-block according to the scan sequence.

[0715] Encoding device 100 can generate entropy-coded quantized transform coefficients by performing entropy coding on scanned quantized transform coefficients, and can generate a bit stream including the entropy-coded quantized transform coefficients.

[0716] The decoding device 200 can extract entropy-encoded quantized transform coefficients from the bitstream, and can generate quantized transform coefficients by performing entropy decoding on the entropy-encoded quantized transform coefficients. The quantized transform coefficients can be arranged in a 2D block format via inverse scanning. Here, as a method of inverse scanning, at least one of upper right diagonal scanning, vertical scanning, and horizontal scanning can be performed.

[0717] In the decoding device 200, inverse quantization can be performed on the quantized transform coefficients. A secondary inverse transform can be performed on the result generated by inverse quantization, depending on whether a secondary inverse transform is performed. Furthermore, a first inverse transform can be performed on the result generated by the secondary inverse transform, depending on whether a first inverse transform will be performed. The reconstructed residual signal can be generated by performing a first inverse transform on the result generated by the secondary inverse transform.

[0718] For luminance components reconstructed via intra-frame prediction or inter-frame prediction, an inverse mapping with dynamic range can be performed before loop filtering.

[0719] The dynamic range can be divided into 16 equal segments, and the mapping functions for the corresponding segments can be signaled. These mapping functions can be signaled at the stripe level or the parallel block group level.

[0720] An inverse mapping function can be derived from the mapping function to perform the inverse mapping.

[0721] Loop filtering, reference frame storage, and motion compensation can be performed in the inverse mapping region.

[0722] Predicted blocks generated via inter-frame prediction can be transformed to a mapped region using a mapping function, and the transformed predicted blocks can be used to generate reconstructed blocks. However, since intra-frame prediction is performed in the mapped region, predicted blocks generated via intra-frame prediction can be used to generate reconstructed blocks without requiring mapping and / or inverse mapping.

[0723] For example, when the target block is a residual block of the chrominance component, the residual block can be transformed into the inverse mapping region by scaling the chrominance component of the mapping region.

[0724] Scaling availability can be signaled at the stripe level or the parallel block group level.

[0725] For example, scaling can be applied only when the mapping is available for the luminance component and the partitions for the luminance and chrominance components follow the same tree structure.

[0726] Scaling can be performed based on the average value of the samples in the luminance prediction block corresponding to the chrominance prediction block. Here, when the target block uses inter-frame prediction, the luminance prediction block can represent the mapped luminance prediction block.

[0727] The scaling values ​​can be derived by using an index-referenced lookup table of the segment to which the average of the sample values ​​of the brightness prediction block belongs.

[0728] The residual block can be transformed into the inverse mapping region by scaling the residual block using the final derived values. Subsequently, for blocks of the chroma components, reconstruction, intra-frame prediction, inter-frame prediction, loop filtering, and storage of reference frames can be performed in the inverse mapping region.

[0729] For example, information indicating whether the mapping and / or inverse mapping of the luminance and chrominance components is available can be sent via a sequence parameter set using signals.

[0730] A predicted block for the target block can be generated based on a block vector. The block vector indicates the displacement between the target block and a reference block. The reference block can be a block in the target image.

[0731] In this way, the prediction mode that generates prediction blocks by referencing the target image can be called the "intra-block copy (IBC) mode".

[0732] The IBC mode can be applied to CUs with specific dimensions. For example, the IBC mode can be applied to an M×N CU. Here, M and N can be less than or equal to 64.

[0733] IBC modes can include skip mode, merge mode, AMVP mode, etc. In skip mode or merge mode, a merge candidate list can be configured, and the merge index is signaled, allowing a single merge candidate to be specified from among the existing merge candidates in the merge candidate list. The block vector of the specified merge candidate can be used as the block vector of the target block.

[0734] In AMVP mode, the differential block vector can be signaled. Additionally, the prediction block vector can be derived from the target block's left and top neighboring blocks. Furthermore, the index of which neighboring block will be used can be signaled.

[0735] In IBC mode, the predicted block can be included in the target CTU or the left CTU, and can be limited to blocks within the previously reconstructed region. For example, the value of the block vector can be restricted such that the predicted block of the target block is located in a specific region. The specific region can be defined by three 64×64 blocks that are encoded and / or decoded before the 64×64 block including the target block. Restricting the value of the block vector in this way reduces memory consumption and device complexity caused by the implementation of IBC mode.

[0736] Figure 16 This is a configuration diagram of an encoding device according to an embodiment.

[0737] Encoding device 1600 can correspond to encoding device 100 described above.

[0738] Encoding device 1600 may include a processing unit 1610, a memory 1630, a user interface (UI) input device 1650, a UI output device 1660, and a storage 1640 that communicate with each other via a bus 1690. Encoding device 1600 may also include a communication unit 1620 connected to a network 1699.

[0739] The processing unit 1610 may be a central processing unit (CPU) or a semiconductor device for executing processing instructions stored in memory 1630 or storage 1640. The processing unit 1610 may be at least one hardware processor.

[0740] The processing unit 1610 can generate and process signals, data, or information input to, output from, or used in the encoding device 1600, and can perform checks, comparisons, determinations, etc., related to the signals, data, or information. In other words, in this embodiment, the processing unit 1610 can perform the generation and processing of data or information, as well as the checks, comparisons, and determinations related to the data or information.

[0741] The processing unit 1610 may include an inter-frame prediction unit 110, an intra-frame prediction unit 120, a switcher 115, a subtractor 125, a transform unit 130, a quantization unit 140, an entropy coding unit 150, an inverse quantization unit 160, an inverse transform unit 170, an adder 175, a filter unit 180, and a reference frame buffer 190.

[0742] At least some of the following components—inter-frame prediction unit 110, intra-frame prediction unit 120, switcher 115, subtractor 125, transform unit 130, quantization unit 140, entropy coding unit 150, inverse quantization unit 160, inverse transform unit 170, adder 175, filter unit 180, and reference frame buffer 190—may be program modules and capable of communicating with external devices or systems. These program modules may be included in the encoding device 1600 in the form of an operating system, application module, or other program modules.

[0743] The program modules can be physically stored in various types of known storage devices. Furthermore, at least some of the program modules can also be stored in a remote storage device capable of communicating with the encoding device 1600.

[0744] Program modules may include, but are not limited to, routines, subroutines, programs, objects, components, and data structures for performing functions or operations according to an embodiment or for implementing abstract data types according to an embodiment.

[0745] The program module can be implemented using instructions or code that are executed by at least one processor of the encoding device 1600.

[0746] The processing unit 1610 can execute instructions or codes in the inter-frame prediction unit 110, intra-frame prediction unit 120, switcher 115, subtractor 125, transform unit 130, quantization unit 140, entropy coding unit 150, dequantization unit 160, inverse transform unit 170, adder 175, filter unit 180 and reference frame buffer 190.

[0747] The storage unit may represent memory 1630 and / or storage 1640. Each of memory 1630 and storage 1640 may be any of a variety of volatile or non-volatile storage media. For example, memory 1630 may include at least one of read-only memory (ROM) 1631 and random access memory (RAM) 1632.

[0748] The storage unit can store data or information used for the operation of the encoding device 1600. In an embodiment, the data or information of the encoding device 1600 can be stored in the storage unit.

[0749] For example, storage units can store images, blocks, lists, motion information, inter-frame prediction information, bitstreams, etc.

[0750] The encoding device 1600 can be implemented in a computer system that includes a computer-readable storage medium.

[0751] The storage medium may store at least one module required for the operation of the encoding device 1600. The memory 1630 may store at least one module and may be configured such that the at least one module is operated by the processing unit 1610.

[0752] The communication unit 1620 can be used to perform functions related to communication of data or information with the encoding device 1600.

[0753] For example, communication unit 1620 can send a bit stream to decoding device 1700, which will be described later.

[0754] Figure 17 This is a configuration diagram of a decoding device according to an embodiment.

[0755] Decoding device 1700 can be used in conjunction with decoding device 200 as described above.

[0756] The decoding device 1700 may include a processing unit 1710, a memory 1730, a user interface (UI) input device 1750, a UI output device 1760, and a storage 1740 that communicate with each other via a bus 1790. The decoding device 1700 may also include a communication unit 1720 connected to a network 1799.

[0757] Processing unit 1710 may be a central processing unit (CPU) or semiconductor device for executing processing instructions stored in memory 1730 or storage 1740. Processing unit 1710 may be at least one hardware processor.

[0758] The processing unit 1710 can generate and process signals, data, or information input to, output from, or used in the decoding device 1700, and can perform checks, comparisons, determinations, etc., related to the signals, data, or information. In other words, in this embodiment, the processing unit 1710 can perform the generation and processing of data or information, as well as the checks, comparisons, and determinations related to the data or information.

[0759] The processing unit 1710 may include an entropy decoding unit 210, an inverse quantization unit 220, an inverse transform unit 230, an intra-frame prediction unit 240, an inter-frame prediction unit 250, a switcher 245, an adder 255, a filter unit 260, and a reference frame buffer 270.

[0760] At least some of the entropy decoding unit 210, inverse quantization unit 220, inverse transform unit 230, intra-frame prediction unit 240, inter-frame prediction unit 250, adder 255, switcher 245, filter unit 260, and reference frame buffer 270 of decoding device 200 may be program modules and are capable of communicating with external devices or systems. These program modules may be included in decoding device 1700 in the form of an operating system, application module, or other program modules.

[0761] The program modules can be physically stored in various types of known storage devices. Furthermore, at least some of the program modules can also be stored in a remote storage device capable of communicating with the decoding device 1700.

[0762] Program modules may include, but are not limited to, routines, subroutines, programs, objects, components, and data structures for performing functions or operations according to an embodiment or for implementing abstract data types according to an embodiment.

[0763] The program module can be implemented using instructions or code executed by at least one processor of the decoding device 1700.

[0764] The processing unit 1710 can run instructions or codes in the entropy decoding unit 210, the dequantization unit 220, the inverse transform unit 230, the intra-frame prediction unit 240, the inter-frame prediction unit 250, the switcher 245, the adder 255, the filter unit 260, and the reference frame buffer 270.

[0765] The storage unit may represent memory 1730 and / or storage 1740. Each of memory 1730 and storage 1740 may be any of a variety of volatile or non-volatile storage media. For example, memory 1730 may include at least one of ROM 1731 and RAM 1732.

[0766] The storage unit can store data or information used for the operation of the decoding device 1700. In an embodiment, the data or information of the decoding device 1700 can be stored in the storage unit.

[0767] For example, storage units can store images, blocks, lists, motion information, inter-frame prediction information, bitstreams, etc.

[0768] The decoding device 1700 can be implemented in a computer system that includes a computer-readable storage medium.

[0769] The storage medium may store at least one module required for the operation of the decoding device 1700. The memory 1730 may store at least one module and may be configured such that the at least one module is operated by the processing unit 1710.

[0770] The communication unit 1720 can be used to perform functions related to communication of data or information with the decoding device 1700.

[0771] For example, communication unit 1720 can receive bit streams from encoding device 1600.

[0772] In the following text, "processing unit" may refer to processing unit 1610 of encoding device 1600 and / or processing unit 1710 of decoding device 1700. For example, regarding prediction-related functions, the processing unit may represent switch 115 and / or switch 245. Regarding inter-frame prediction-related functions, the processing unit may represent inter-frame prediction unit 110, subtractor 125, and adder 175, and may also represent inter-frame prediction unit 250 and adder 255. Regarding intra-frame prediction-related functions, the processing unit may represent intra-frame prediction unit 120, subtractor 125, and adder 175, and may also represent intra-frame prediction unit 240 and adder 255. Regarding transform-related functions, the processing unit may represent transform unit 130 and inverse transform unit 170, and may also represent inverse transform unit 230. Regarding quantization-related functions, the processing unit may represent quantization unit 140 and inverse quantization unit 160, and may also indicate inverse quantization unit 220. Regarding functions related to entropy encoding and / or entropy decoding, the processing unit may represent entropy encoding unit 150 and / or entropy decoding unit 210. Regarding functions related to filtering, the processing unit may represent filter unit 180 and / or filter unit 260. Regarding functions related to reference frames, the processing unit may instruct reference frame buffer 190 and / or reference frame buffer 270.

[0773] The embodiments can be performed in the same and / or corresponding manner in the encoding device 1600 and the decoding device 1700. Furthermore, a combination of at least one of the embodiments can be used when encoding and / or decoding images.

[0774] In the encoding device 1600 and the decoding device 1700, the order in which the embodiments are applied may differ from each other. Alternatively, the order in which the embodiments are applied may be (at least partially) the same in the encoding device 1600 and the decoding device 1700.

[0775] The embodiment can be executed for each of the luminance and chrominance signals. The embodiment can be executed in the same manner for both the luminance and chrominance signals.

[0776] The blocks in the application examples may be square or non-square.

[0777] The embodiments can be applied based on the size of at least one of the target block, coding block, prediction block, transform block, block, current block, coding unit, prediction unit, transform unit, unit, and current unit. Here, the size can be defined as the minimum and / or maximum size of the application embodiment, or it can be defined as a fixed size of the application embodiment. Furthermore, in the embodiments, a first embodiment can be applied to a first size, and a second embodiment can be applied to a second size. In other words, embodiments can be applied based on combinations of sizes. Moreover, embodiments can only be applied when the size is greater than or equal to the minimum size and less than or equal to the maximum size. In other words, embodiments can only be applied when the block size is included within a specific range.

[0778] Whether to apply and / or execute at least one of the embodiments can be determined based on block size conditions. In other words, when the block size condition is met, at least one of the embodiments can be applied and / or executed. Conditions may include minimum block size and maximum block size. A block can be one of the blocks described above in the embodiments and the units described above in the embodiments. The block to which the minimum block size is applied and the block to which the maximum block size is applied may be different from each other.

[0779] For example, the above embodiments can be applied and / or executed when the block size is greater than or equal to the minimum size and / or when the block size is less than or equal to the maximum size.

[0780] For example, the above embodiments can only be applied when the block size is a predefined block size. The predefined block size can be 2×2, 4×4, 8×8, 16×16, 32×32, 64×64, or 128×128. The predefined block size can be (2×SIZE) X )x(2×SIZE Y SIZE X It can be any integer greater than or equal to 1. SIZE Y It can be any integer greater than or equal to 1.

[0781] For example, the above embodiments can be applied only when the block size is greater than or equal to the minimum block size. The minimum block size can be 2×2, 4×4, 8×8, 16×16, 32×32, 64×64, or 128×128. Optionally, the minimum block size can be (2×SIZE) MIN_X )x(2×SIZE MIN_Y SIZE MIN_X It can be any integer greater than or equal to 1. SIZE MIN_Y It can be any integer greater than or equal to 1.

[0782] For example, the above embodiments can be applied only when the block size is less than or equal to the maximum block size. The above embodiments can also be applied only when the block size is less than the maximum block size. The maximum block size can be 2×2, 4×4, 8×8, 16×16, 32×32, 64×64, or 128×128. Optionally, the maximum block size can be (2×SIZE) MAX_X )×(2×SIZE MAX_Y SIZE MAX_X It can be any integer greater than or equal to 1. SIZE MAX_Y It can be any integer greater than or equal to 1.

[0783] For example, the above embodiments can be applied only when the block size is greater than or equal to the minimum block size and less than or equal to the maximum block size.

[0784] In the above embodiments, the block size can represent the width or height of the block. The block size can represent both the width and height of the block. Furthermore, the block size can mean the area of ​​the block. Each of the area, minimum block size, and maximum block size can be an integer greater than or equal to 1. Additionally, the block size can represent the result (or value) of an equation in the embodiment or the result (or value) of a known equation using the width and height of the block.

[0785] In addition, in the implementation, the first embodiment can be applied to a first size, and the second embodiment can be applied to a second size.

[0786] Implementation examples can be applied based on time layers. Separate identifiers can be signaled to identify the time layer to which an implementation example can be applied, and the implementation example can be applied to the time layer specified by the corresponding identifier. Here, the identifier can be defined as the lowest and / or highest layer to which an implementation example can be applied, or it can be defined as a specific layer indicating the application of an implementation example. Furthermore, a fixed time layer for applying the implementation example can be defined.

[0787] For example, the embodiments can be applied only when the time layer of the target image is the lowest layer. For example, the embodiments can be applied only when the time layer identifier of the target image is greater than or equal to 1. For example, the embodiments can be applied only when the time layer of the target image is the highest layer.

[0788] The stripe type or parallel block group type of the application embodiment can be defined, and the application embodiment can be applied according to the corresponding stripe type or parallel block group type.

[0789] The terms used in this disclosure may be defined as follows.

[0790] A block vector can be a vector that specifies the location of a reference block in Intra-Block Copy (IBC) mode.

[0791] Paired averages can represent two-way predictions of paired averages or combinations.

[0792] Skip mode can be a mode that uses the block vector of neighboring blocks and does not send residual image signals.

[0793] The merging mode can be a mode that uses the block vectors of neighboring blocks and sends the residual image signal.

[0794] Merge modes can include skip mode, merge mode, or both.

[0795] The merge candidate list can be the merge candidate list used by the skip mode or the merge mode.

[0796] The predicted block vector can be the temporal or spatial motion vector of the target block's neighboring blocks.

[0797] The residual block vector can be the difference vector between the predicted block vector and the target block vector.

[0798] Block vector information can be composed of predicted block vectors and residual block vectors.

[0799] The reference image index can indicate the image (or stripe, parallel block) referenced by the target block. When the reference image index represents an image that includes the target block, it can indicate that the coding mode of the target block is block vector prediction mode, and when the reference image index represents an image that does not include the target block, it can indicate that the coding mode of the target block is inter-frame prediction mode.

[0800] In this disclosure, the target block may represent the current block.

[0801] A motion vector can be represented as a motion vector or MV.

[0802] A list of reference images can be represented as a list of reference frames.

[0803] The reference image list L0 can be represented as reference screen list 0 or RefPicL0.

[0804] The reference image list L1 can be represented as reference screen list 1 or RefPicL1.

[0805] The motion vector using the reference image list L0 can be represented as MVL0.

[0806] The motion vector using the reference image list L1 can be represented as MVL1.

[0807] The MV predictor for non-adjacent spatial candidates can be represented as NA-MVP.

[0808] The BV predictor for non-adjacent spatial candidates can be represented as NA-BVP.

[0809] Scaled non-adjacent spatial candidates can be represented as shifted non-adjacent spatial candidates or sNASPs.

[0810] The MV predictors of the scaled non-neighboring spatial candidates can be represented as sNA-MVP.

[0811] The scaled non-neighboring spatial candidate BV predictor can be represented as sNA-BVP.

[0812] IBC Local Illumination Compensation (IBC-LIC) is a technique for applying linear filtering to IBC prediction signals. The equation for linear filtering consists of P'(x,y)=aP(x,y)+b, and the target block (P) t ) and IBC reference block (P ref The adjacent reconstructed pixels of ) can be specified as filter templates (T) t ,T ref This is used to derive filter coefficients from the corresponding filter template. Here, P(x,y) is the IBC prediction signal before applying filtering to the (x,y) coordinate pixels, P'(x,y) is the prediction signal after applying filtering to the (x,y) coordinate pixels, and a and b can refer to the filter coefficients.

[0813] Figure 18 The shape of the filter template is shown.

[0814] As in Figure 18 In the middle, the filter template size (size) T () can refer to the size of the template to be used to derive the filter coefficients. For example, it could be size. T =4.

[0815] IBC-LIC Extension is an extended mode of IBC-LIC technology that adds multi-directional and multi-model modes to IBC-LIC.

[0816] IBC-LIC multidirectional can be used to select L-shaped templates (T) through blocks. L-shape Top template (T) Top ) and left template (T) Left One of the template shapes is used to derive one of the IBC-LIC extended modes of the filter coefficients. For example... Figure 18 As shown in (A), T L-shape This can represent the top and left reconstructed pixels adjacent to the block. For example, in... Figure 18 In (B), T Top This can represent the top reconstructed pixel adjacent to the block. For example, in... Figure 18 In (C), T Left It can represent the reconstructed pixel to the left of the block.

[0817] The IBC-LIC multi-model can derive thresholds based on target block filter template values ​​and supports one of the IBC-LIC extended modes that uses two filter coefficient sets based on the corresponding threshold.

[0818] Filtered IBC (FIBC) is a technique for applying 7-tap filtering to IBC predicted signals.

[0819] Figure 19 The shape of the FIBC filter is shown.

[0820] The equations used for FIBC filtering include P'(x,y) = a0·C + a1·N + a2·S + a3·W + a4·E + a5·P + a5·β, and the target block (P t ) and IBC reference block (P ref The adjacent reconstructed pixels of ) can be specified as filter templates (T) t ,T ref The filter coefficients are derived from the corresponding filter template. Here, C represents the predicted signal P(x,y) of the (x,y) coordinate pixel before filtering is applied, and N, S, W, and E, including C, can respectively correspond to Figure 19 The pixel at the indicated position. N can be the predicted signal P(x,y-1) of the pixel at coordinates (x,y-1) before filtering is applied, S can be the predicted signal P(x,y+1) of the pixel at coordinates (x,y+1) before filtering is applied, W can be the predicted signal P(x-1,y) of the pixel at coordinates (x-1,y) before filtering is applied, and E can be the predicted signal P(x+1,y) of the pixel at coordinates (x+1,y) before filtering is applied. P can represent P(x,y). 2 Additionally, a i These can represent the coefficients of the i-th filter, and β can represent the bias value.

[0821] like Figure 19 As shown, a 7-tap filter can perform filtering using a total of 7 values: the pixel to which the filter is applied (C), the four pixels adjacent to the corresponding pixel at the top (N), bottom (S), left (W), and right (E), the non-linear term (P), and the bias value.

[0822] Figure 20 An embodiment of the IBC predictive signal filtering of this disclosure is shown.

[0823] Figure 21 An example of Dbv predictive signal filtering is shown.

[0824] [D1] In the IBC prediction signal filtering information acquisition unit, the template of the target block (T) can be used. t ), ibc_filtering_flag, ibc_filter_shape, ibc_filter_tpl_shape, ibc_filter_set_num, pred_mode_ibc_flag, general_merge_flag, merge_idx, Dbv mode, block vector BV (BV x BV y The method determines whether to perform IBC filtering by at least one of the following: ibc_lic_efibc_idx, ibc_filter_tpl_shape1, ibc_filter_tpl_shape2, the average value of the IBC prediction signal, the average value of the filter template of the target block, the histogram distribution of the IBC prediction signal, or the histogram distribution of the filter template of the target block; the IBC filter shape, the IBC filter template shape, the number of IBC filter coefficient sets, the IBC filter model, the IBC filter mode, or the Dbv prediction signal filtering information.

[0825] [D2] When generating the IBC prediction signal, the block vector (BV) can be used as a reference. x BV y ), target block location (x) t_lf ,y t_lf ), IBC reference block Blk ref The location of the IBC reference block (x ref_lf ,y ref_lf The target block width xCbW, target block height xCbH, or Dbv mode are used to generate the IBC prediction signal P for the target block. t Or the chromaticity prediction signal P used for the target block c_t At least one of them.

[0826] [D3-1] When specifying the IBC filter model, it can be based on the IBC prediction signal filtering information, IBC prediction pixels, and P... t The filter model is specified by at least one of the following: neighboring pixels of the IBC predicted pixel, bias, filter coefficients, filter shape, ibc_filter_shape, or IBC filter mode.

[0827] [D3-2] When obtaining the IBC predictive signal filter coefficients, the values ​​can be determined based on ibc_filter_shape, ibc_filter_set_num, the target block template, and T. t Template for IBC reference block, T ref Alternatively, at least one of the IBC filter models can be used to obtain the filter model and filter coefficient set (FilterSet).

[0828] [D3-3] When performing IBC prediction signal filtering, the filter coefficient set FitlerSet, ibc_filter_set_num, and IBC prediction signal P can be obtained. t Or at least one of the IBC prediction signals P' in the filter model performs filtering. t .

[0829] [D4-1] When specifying the Dbv filter model, it can be based on the Dbv mode of the corresponding luma block, the Dbv predicted pixel, and the P... c_t The Dbv filter model is specified by at least one of the following: Dbv prediction signal filtering information, bias, neighboring pixels of the Dbv prediction pixel, filter shape, ibc_filter_shape, filter coefficients, or filter mode.

[0830] [D4-2] When obtaining the filter coefficients of the Dbv predictive signal, it can be based on the ibc_filter_shape of the corresponding luminance block, the ibc_filter_set_num of the corresponding luminance block, the template of the target block, and T. c_t Template for Dbv reference blocks, T c_ref Alternatively, at least one of the Dbv filter models can be used to obtain the filter model and filter coefficient set (FilterSet).

[0831] [D4-3] When performing Dbv predictive signal filtering, the filter coefficient set, FitlerSet, ibc_filter_set_num, Dbv predictive signal, and P can be obtained according to the corresponding luminance block. c_t Or at least one of the IBC prediction signals P' in the Dbv filter model performs filtering. c_t .

[0832] In this disclosure, a filter template refers to a template used to derive filter coefficients for filtering the IBC prediction signal, and the corresponding filter template can represent adjacent reconstructed pixels of the target block and the IBC reference block. For example, in Figure 18 In the template, size T It can be set to a predefined value.

[0833] In this disclosure, the filter shape may represent a pixel used for filtering. The filter shape may be a square of a predetermined size or a cross of a predetermined size. As an example, the filter shape may be a 1×1 square, a 3×3 cross, or a 3×3 square.

[0834] In this disclosure, template shape may refer to the shape of a filter template. (See reference) Figure 18 The left template of the block (T) is shown. Left ), top template of the block (T) Top ) and the L-shaped template (T) using both the left and top templates of the block. L-shape ). It can be done by using T Left T Top or T L-shape At least one of them is used to perform filtering.

[0835] In this disclosure, a filter coefficient set may represent the collection of filter coefficients used in a filter model. The number of filter coefficient sets may be at least one. One or more filter coefficient sets may be used when filtering the signal predicted by IBC for a target block.

[0836] In this disclosure, the filter model refers to an equation for performing IBC prediction signal filtering, and the equation can be configured by using at least one of the IBC prediction pixel to be filtered, the neighboring pixels of the IBC prediction pixel, the bias, or the filter coefficients.

[0837] As an example, the filter model could be P'(x,y)=aP(x,y)+b.

[0838] As an example, the filter model could be P'(x,y) = a0·C + a1·N + a2·S + a3·W + a4·E + a5·P + a5·β.

[0839] As an example, the filter model could be P'(x,y) = a0·C + a1·N + a2·S + a3·W + a4·E + a0·C 2 + a5·β.

[0840] [D1] In the IBC prediction signal filtering information acquisition unit, information about whether to perform IBC prediction signal filtering can be obtained.

[0841] The `ibc_filtering_flag` syntax element indicates whether filtering should be performed. When `ibc_filtering_flag` is the first value, filtering can be performed on the IBC prediction block, and when `ibc_filtering_flag` is the second value, filtering can be omitted.

[0842] Figure 22 A block diagram of IBC prediction signal filtering processing based on the value of ibc_filtering_flag is shown.

[0843] As an example, such as Figure 22 (A) When ibc_filtering_flag is the first value, filtering can be performed on the IBC prediction block, and as... Figure 22 In (B), when ibc_filtering_flag is the second value, filtering may not be performed. As an example, the first value can be 1 and the second value can be 0. Alternatively, as an example, the first value can be 0 and the second value can be 1.

[0844] Figure 23 An example is shown where a 1×1 rectangular filter is used when the ibc_filter_shape syntax element is the first value and a 3×3 cross-shaped filter is used when the ibc_filter_shape syntax element is the second value.

[0845] Figure 24 An example is shown where a 3×3 rectangular filter is used when the ibc_filter_shape syntax element is the first value and a 3×3 cross-shaped filter is used when the ibc_filter_shape syntax element is the second value.

[0846] Information about the filter shape used for IBC predictive signal filtering can be obtained. The ibc_filter_shape syntax element indicates the filter shape. The filter shape can be one of a square filter of a predetermined size or a cross-shaped filter of a predetermined size.

[0847] As an example, when ibc_filter_shape is not sent with a signal, it can be determined to be a square filter shape. As another example, when ibc_filter_shape is not sent with a signal, it can be determined to be a cross-shaped filter shape.

[0848] As an example, the `ibc_filter_shape` syntax element can indicate the filter shape. When the `ibc_filter_shape` syntax element is a first value, IBC predictive signal filtering can be applied using a 1×1 square filter, and when the `ibc_filter_shape` syntax element is a second value, IBC predictive signal filtering can be applied using a 3×3 cross-shaped filter. In other words, when the `ibc_filter_shape` syntax element is a first value, when filtering is performed on pixel p0, as shown... Figure 23As shown in (A), filtering can be performed using the p0 pixel value. In other words, when the ibc_filter_shape syntax element is a second value, as... Figure 23 As shown in (B), when filtering is performed on pixel p0, filtering can be performed by using pixels p0 through p4 together. For example, the first value can be 1 and the second value can be 0. Alternatively, the first value can be 0 and the second value can be 1.

[0849] Information about the shape of the filter template used for IBC predictive signal filtering can be obtained.

[0850] The `ibc_filter_tpl_shape` syntax element indicates the shape of the filter template. For example... Figure 18 As shown, the filter template shape can be T. L-shape T Top or T Left one of the.

[0851] As an example, when ibc_filter_tpl_shape is the first value, an L-shaped template T can be used. L-shape As the filter template shape. When ibc_filter_tpl_shape is the second value, the top template T of the block is... Top It can be used as the filter template shape. When ibc_filter_tpl_shape is a third value, the left-hand template T of the block... Left It can be used as a filter template shape.

[0852] For example, the first value can be 0, the second value can be 1, and the third value can be 2.

[0853] When ibc_filter_tpl_shape is not signaled, a predefined template shape can be used. For example, the predefined template shape could be T. L-shape Optionally, the predefined template shape can be T. Top Optionally, the predefined template shape can be T. Left .

[0854] Information about the number of filter coefficient sets used for IBC predictive signal filtering can be obtained.

[0855] The ibc_filter_set_num syntax element indicates the number of filter coefficient sets.

[0856] It can be indicated that a filter coefficient set is used when ibc_filter_set_num is the first value, and i (i=2, 3, ...) filter coefficient sets are used when ibc_filter_set_num is the i-th value.

[0857] For example, the first value can be 0. The i-th value can be i-1.

[0858] This allows you to specify that one set of filter coefficients should be used when ibc_filter_set_num is the first value, and multiple sets of filter coefficients should be used when ibc_filter_set_num is the second value. For example, the first value can be 0, and the second value can be 1.

[0859] When ibc_filter_set_num is not sent with a signal, a predefined set of filter coefficients can be used.

[0860] For example, a set of filter coefficients can be used for IBC predictive signal filtering. In other words, it can be derived as ibc_filter_set_num=1.

[0861] Optionally, the two filter coefficient sets can be used for IBC predictive signal filtering. In other words, it can be derived as ibc_filter_set_num=2.

[0862] Optionally, the three filter coefficient sets can be used for IBC predictive signal filtering. In other words, it can be derived as ibc_filter_set_num=3.

[0863] For P t '(x,y)=F(P t (x,y),FilterSet),P t (x,y) can represent the IBC prediction signal pixel value at (x,y) coordinates, and P t '(x,y) can represent the relationship between P and P. t (x,y) are the pixel values ​​to which the filter is applied.

[0864] F(P t (x,y),FilterSet) can represent that FilterSet is used to filter pixel P. t (x,y) Apply filtering.

[0865] As an example, it can be indicated that one set of filter coefficients is used when ibc_filter_set_num is a first value, and two sets of filter coefficients are used when ibc_filter_set_num is a second value.

[0866] When using a set of filter coefficients, P can be used with a set of filter coefficients. t Filter (x,y).

[0867] When ibc_filter_set_num is the first value, Pt '(x,y)=F(P t (x,y), FilterSet).

[0868] When two filter coefficient sets are used, the filter coefficient set may be selected and used according to the average value Thres of the pixels belonging to the filter template. In other words, when P t (x,y) is less than or equal to Thres, the first filter coefficient set FilterSet1 may be used, and when P t (x,y) is greater than Thres, the second filter coefficient set FilterSet2 may be used.

[0869] When ibc_filter_set_num is a second value, In the case of P t (x,y)≤Thres, P t '(x,y)=F(P t (x,y), FilterSet1), In the case of P t (x,y)>Thres, P t '(x,y)=F(P t (x,y), FilterSet2).

[0870] As an embodiment, it may be indicated that one filter coefficient set is used when ibc_filter_set_num is a first value, and three filter coefficient sets are used when ibc_filter_set_num is a second value.

[0871] When one filter coefficient set is used, P t (x,y) may be filtered with the one filter coefficient set.

[0872] When ibc_filter_set_num is the first value, P t '(x,y)=F(P t (x,y), FilterSet).

[0873] When three filter coefficient sets are used, the filter coefficient set may be selected and used according to two thresholds Thres0, Thres1 (Thres0<Thres1). In other words, when P t (x,y) is less than or equal to Thres0, the first filter coefficient set FilterSet0 may be used, and when P t (x,y) is greater than Thres0 and less than or equal to Thres1, the second filter coefficient set FilterSet1 may be used. When P tWhen the (x,y) value is greater than Thres1, the third filter coefficient set FilterSet2 may be used.

[0874] Thres0 may denote a first threshold, and Thres1 may denote a second threshold (Thres0<Thres1). When a total of N pixels belong to the filter template, the pixels belonging to the filter template may be sorted in ascending order based on pixel value magnitude, then the value of the N / 3-th pixel may be set as Thres0, and the value of the 2N / 3-th pixel may be set as Thres1.

[0875] When ibc_filter_set_num is a second value, P t In the case of (x,y)≤Thres0, P t '(x,y)=F(P t (x,y),FilterSet1), In the case of Thres0<P t (x,y)≤Thres1, P t '(x,y)=F(P t (x,y),FilterSet2), In the case where P t (x,y)>Thres1, P t '(x,y)=F(P t (x,y),FilterSet3).

[0876] As an embodiment, it may be specified that one filter coefficient set is used when ibc_filter_set_num is a first value, two filter coefficient sets are used when ibc_filter_set_num is a second value, and three filter coefficient sets are used when ibc_filter_set_num is a third value.

[0877] When one filter coefficient set is used, P may be filtered with the single filter coefficient set t (x,y).

[0878] When ibc_filter_set_num is a first value, P t '(x,y)=F(P t (x,y),FilterSet).

[0879] When two filter coefficient sets are used, the filter coefficient set may be selected and used according to the average value Thres of the pixels belonging to the filter template. In other words, when P tWhen the (x,y) value is less than or equal to Thres, the first filter coefficient set FilterSet1 may be used, and when P t the (x,y) value is greater than Thres, the second filter coefficient set FilterSet2 may be used.

[0880] When ibc_filter_set_num is a second value, In the case where P t (x,y)≤Thres, P t '(x,y)=F(P t (x,y),FilterSet1), In the case where P t (x,y)>Thres, P t '(x,y)=F(P t (x,y),FilterSet2).

[0881] When three filter coefficient sets are used, the filter coefficient set may be selected and used according to two thresholds Thres0 and Thres1 (Thres0<Thres1). In other words, when P t the (x,y) value is less than or equal to Thres0, the first filter coefficient set FilterSet1 may be used, and when P t the (x,y) value is greater than Thres0 and less than or equal to Thres1, the second filter coefficient set FilterSet2 may be used. When P t the (x,y) value is greater than Thres1, the third filter coefficient set FilterSet3 may be used.

[0882] Thres0 may represent the first threshold, and Thres1 may represent the second threshold (Thres0<Thres1). When a total of N pixels belong to the filter template, the pixels belonging to the filter template may be sorted in ascending order according to the pixel value size, then the value of the pixel at the N / 3 position may be set as Thres0, and the value of the pixel at the 2N / 3 position may be set as Thres1.

[0883] When ibc_filter_set_num is a third value, In the case where P t (x,y)≤Thres0, P t '(x,y)=F(P t (x,y),FilterSet1), In the case where Thres0<P t (x,y)≤Thres1, P t'(x,y)=F(P t (x,y),FilterSet2), In P t When (x,y)>Thres1, P t '(x,y)=F(P t (x,y),FilterSet3).

[0884] The IBC filter mode can be defined by at least one of the following: whether filtering is performed, filter shape, filter template shape, and the number of filter coefficient sets.

[0885] As an example, the IBC filter mode can be defined by at least one of the syntax elements ibc_filter_flag, ibc_filter_shape, ibc_filter_tpl_shape, or ibc_filter_set_num sent by signals.

[0886] As an example, the IBC filter mode can be defined by a syntax element sent by a signal to indicate the IBC filter mode.

[0887] As an example, the IBC filter mode can be defined by at least one of the syntax elements ibc_filter_flag, ibc_filter_shape, ibc_filter_tpl_shape1, ibc_filter_tpl_shape2, or ibc_filter_set_num sent by signals.

[0888] The IBC filter modes can be defined in nine modes as follows.

[0889] 1. Non-IBC-LIC mode can be a mode in which no filtering is applied.

[0890] 2. The L-shaped IBC-LIC mode can utilize T L-shape A square filter and a filter set are used to apply filtering patterns.

[0891] 3. The multi-model IBC-LIC mode can utilize T L-shape The filtering modes include square filters and multiple filter sets. Here, "multiple" can refer to two filters.

[0892] 4. Only the top IBC-LIC mode is available for utilizing T. top A square filter and a filter set are used to apply filtering patterns.

[0893] 5. Only the left-side IBC-LIC mode can utilize T LeftA square filter and a filter set are used to apply filtering patterns.

[0894] 6. The L-shaped FIBC mode can utilize T L-shape A cross-shaped filter and a filter set are used to apply filtering patterns.

[0895] 7. The multi-model FIBC mode can utilize T L-shape The filtering pattern can be applied using cross-shaped filters and multiple filter sets. Here, "multiple" can refer to two filters.

[0896] 8. Only the top FIBC mode can utilize T top A cross-shaped filter and a filter set are used to apply filtering patterns.

[0897] 9. Only the left-side FIBC mode can utilize T. Left A cross-shaped filter and a filter set are used to apply filtering patterns.

[0898] As an example, the IBC filter mode can be defined by at least one of ibc_filter_flag, ibc_filter_shape, ibc_filter_tpl_shape, or ibc_filter_set_num sent by a signal.

[0899] When both ibc_filter_shape and ibc_filter_flag are first values, non-IBC-LIC can be specified. As another example, when ibc_filter_flag is a second value, filtering by at least one of IBC-LIC and FIBC can be performed.

[0900] When ibc_filter_shape has a first value (1×1 square filter), ibc_filter_flag has a second value, and ibc_filter_tpl_shape has a first value (T) L-shape When ibc_filter_set_num has a first value (1 filter coefficient set), an L-shaped IBC-LIC can be specified.

[0901] When ibc_filter_shape has a first value (1×1 square filter), ibc_filter_flag has a second value, and ibc_filter_tpl_shape has a first value (T) L-shape When ibc_filter_set_num has a second value (multiple filter coefficient sets), a multi-model IBC-LIC can be specified. Here, "multiple" can be two.

[0902] When ibc_filter_shape has a first value (1×1 square filter), ibc_filter_flag has a second value, and ibc_filter_tpl_shape has a second value (T) top Furthermore, when ibc_filter_set_num has a first value (1 filter coefficient set), only the top IBC-LIC can be specified.

[0903] When ibc_filter_shape has a first value (1×1 square filter), ibc_filter_flag has a second value, and ibc_filter_tpl_shape has a third value (T) Left Furthermore, when ibc_filter_set_num has a first value (1 filter coefficient set), only left-side IBC-LIC can be specified.

[0904] When ibc_filter_shape has a second value (3×3 cross filter), ibc_filter_tpl_shape has a first value (T). L-shape When ibc_filter_set_num has a first value (1 filter coefficient set), an L-shaped FIBC can be specified.

[0905] When ibc_filter_shape has a second value (3×3 cross filter), ibc_filter_tpl_shape has a first value (T). L-shape Furthermore, when ibc_filter_set_num has a second value (multiple filter coefficient sets), a multi-model FIBC can be specified. Here, "multiple" can be two.

[0906] When ibc_filter_shape has a second value (3×3 cross filter), ibc_filter_tpl_shape has a second value (T). top Furthermore, when ibc_filter_set_num has a first value (1 filter coefficient set), only the top FIBC can be specified.

[0907] When ibc_filter_shape has a second value (3×3 cross filter), ibc_filter_tpl_shape has a third value (T). Left When ibc_filter_set_num has a first value (1 filter coefficient set), only left-hand FIBC can be specified.

[0908] As an example, the IBC filter mode can be defined by a syntax element sent by a signal to indicate the IBC filter mode.

[0909] When this syntax element has the first value, a non-IBC-LIC can be specified. When this syntax element has the second value, an L-shaped IBC-LIC can be specified. When this syntax element has the third value, a multi-model IBC-LIC (L-shaped) can be specified. When this syntax element has the fourth value, a top-only IBC-LIC can be specified.

[0910] When this syntax element has a fifth value, a left-side IBC-LIC can be specified. When this syntax element has a sixth value, an L-shaped FIBC can be specified. When this syntax element has a seventh value, a multi-model FIBC (L-shaped) can be specified. When this syntax element has an eighth value, a top-only FIBC can be specified. When this syntax element has a ninth value, a left-side FIBC can be specified.

[0911] ibc_lic_efibc_index can be binarized as shown in the table below.

[0912] (As shown in Table 7), the first value can be binarized to 00, the second value can be binarized to 0100, the third value can be binarized to 0101, the fourth value can be binarized to 0110, the fifth value can be binarized to 0111, the sixth value can be binarized to 100, the seventh value can be binarized to 101, the eighth value can be binarized to 110, and the ninth value can be binarized to 111.

[0913] Table 7

[0914] (As shown in Table 8), the first value can be binarized to 00, the second value can be binarized to 010, the third value can be binarized to 0110, the fourth value can be binarized to 01110, the fifth value can be binarized to 01111, the sixth value can be binarized to 10, the seventh value can be binarized to 110, the eighth value can be binarized to 1110, and the ninth value can be binarized to 1111.

[0915] Table 8

[0916] (As shown in Table 9), the first value can be binarized to 00, the second value can be binarized to 010, the third value can be binarized to 011, the fourth value can be binarized to 110, the fifth value can be binarized to 101, the sixth value can be binarized to 1100, the seventh value can be binarized to 1101, the eighth value can be binarized to 1110, and the ninth value can be binarized to 1111.

[0917] Table 9

[0918] As an example, as shown in Table 10, ibc_filter_flag, ibc_lic_flag, and ibc_lic_filter_index can be sent using signals to indicate at least one of L-shaped IBC-LIC, multi-model IBC-LIC, top-only IBC-LIC, left-only IBC-LIC, L-shaped IBC-LIC, multi-model FIBC, top-only FIBC, or left-only FIBC.

[0919] When both ibc_filter_flag and ibc_lic_flag are first, non-IBC-LIC can be specified. As another example, when ibc_filter_flag is second, filtering by at least one of IBC-LIC and FIBC can be performed.

[0920] When ibc_filter_flag has a first value (1×1 square filter), ibc_lic_flag has a second value (perform filtering), and ibc_lic_filter_index has a first value, an L-shaped IBC-LIC can be specified in the IBC filter mode.

[0921] When ibc_filter_flag has a first value (1×1 square filter), ibc_lic_flag has a second value (perform filtering), and ibc_lic_filter_index has a second value, a multi-model IBC-LIC can be specified in the IBC filtering mode.

[0922] When ibc_filter_flag has a first value (1×1 square filter), ibc_lic_flag has a second value (perform filtering), and ibc_lic_filter_index has a third value, only top IBC-LIC can be specified in the IBC filtering mode.

[0923] When ibc_filter_flag has a first value (1×1 square filter), ibc_lic_flag has a second value (perform filtering), and ibc_lic_filter_index has a fourth value, only left-side IBC-LIC can be specified in the IBC filter mode.

[0924] An L-shaped FIBC can be specified when ibc_filter_flag has a second value (3×3 cross-shaped filter), ibc_lic_flag has a first value (perform filtering), and ibc_lic_filter_index has a first value.

[0925] Multi-model FIBC can be specified when ibc_filter_flag has a second value (3×3 cross-shaped filter), ibc_lic_flag has a first value (perform filtering), and ibc_lic_filter_index has a second value.

[0926] When ibc_filter_flag has a second value (3×3 cross-shaped filter), ibc_lic_flag has a second value (perform filtering), and ibc_lic_filter_index has a third value, only the top FIBC can be specified.

[0927] When ibc_filter_flag has a second value (3×3 cross-shaped filter), ibc_lic_flag has a second value (perform filtering), and ibc_lic_filter_index has a fourth value, only left-side FIBC can be specified.

[0928] Table 10

[0929] Figure 25 An example of an IBC filter mode information signaling block diagram is shown.

[0930] As an example, such as in Figure 25 In this context, the IBC filter mode can be defined by at least one of the syntax elements ibc_filter_flag, ibc_filter_shape, ibc_filter_tpl_shape1, ibc_filter_tpl_shape2, or ibc_filter_set_num, which are sent by signals.

[0931] When ibc_filter_tpl_shape1 is the first value, it indicates that the filter template shape is T. L-shape And when it is the second value, it indicates that the filter template shape is T. Top or T Left one of the.

[0932] When ibc_filter_tpl_shape2 is the first value, it indicates that the filter template shape is T. Top And when it is the second value, it indicates that the filter template shape is T.Left .

[0933] When both ibc_filter_shape and ibc_filter_flag are first values, non-IBC-LIC can be specified. As another example, when ibc_filter_flag is first, non-IBC-LIC is used.

[0934] When ibc_filter_shape has a first value, ibc_filter_flag has a second value, ibc_filter_tpl_shape1 has a first value, and ibc_filter_set_num has a first value, an L-shaped IBC-LIC can be specified.

[0935] When ibc_filter_shape has a first value, ibc_filter_flag has a second value, ibc_filter_tpl_shape1 has a first value, and ibc_filter_set_num has a second value, a multi-model IBC-LIC can be specified.

[0936] When ibc_filter_shape has a first value, ibc_filter_flag has a second value, ibc_filter_tpl_shape1 has a second value and ibc_filter_tpl_shape2 has a first value, only top IBC-LIC can be specified.

[0937] When ibc_filter_shape has a first value, ibc_filter_flag has a second value, ibc_filter_tpl_shape1 has a second value, and ibc_filter_tpl_shape2 has a second value, it is possible to specify only left-side IBC-LIC.

[0938] An L-shaped FIBC can be specified when ibc_filter_shape has a second value, ibc_filter_tpl_shape1 has a first value, and ibc_filter_set_num has a first value.

[0939] A multi-model FIBC can be specified when ibc_filter_shape has a second value, ibc_filter_tpl_shape1 has a first value, and ibc_filter_set_num has a second value.

[0940] When ibc_filter_shape has a second value, ibc_filter_tpl_shape1 has a second value and ibc_filter_tpl_shape2 has a first value, only the top FIBC can be specified.

[0941] When ibc_filter_shape has a second value, ibc_filter_tpl_shape1 has a second value, and ibc_filter_tpl_shape2 has a second value, it is possible to specify only the left FIBC.

[0942] Figure 26 An example of an IBC filter mode information signaling block diagram is shown.

[0943] As an example, such as in Figure 26 In this context, the IBC filter mode can be defined by at least one of the syntax elements ibc_filter_flag, ibc_filter_shape, ibc_filter_tpl_shape1, ibc_filter_tpl_shape2, or ibc_filter_set_num, which are sent by signals.

[0944] When ibc_filter_tpl_shape1 is the first value, it can indicate that the filter template shape is T. L-shape It uses a set of filter coefficients, and when it is the second value, it can indicate the use of multiple models or that the filter template shape is T. Top or T Left one of the.

[0945] When ibc_filter_tpl_shape2 is the first value, it can indicate that the filter template shape is T. Top And when it is the second value, it can indicate that the filter template shape is T. Left .

[0946] When both ibc_filter_shape and ibc_filter_flag are first, non-IBC-LIC is used. As another example, when ibc_filter_flag is first, non-IBC-LIC can be used.

[0947] An L-shaped IBC-LIC can be used when ibc_filter_shape has a first value, ibc_filter_flag has a second value, and ibc_filter_tpl_shape1 has a first value.

[0948] When ibc_filter_shape has a first value, ibc_filter_flag has a second value, ibc_filter_tpl_shape1 has a first value, and ibc_filter_set_num has a second value, a multi-model IBC-LIC can be specified.

[0949] When ibc_filter_shape has a first value, ibc_filter_flag has a second value, ibc_filter_tpl_shape1 has a second value and ibc_filter_tpl_shape2 has a first value, only top IBC-LIC can be specified.

[0950] When ibc_filter_shape has a first value, ibc_filter_flag has a second value, ibc_filter_tpl_shape1 has a second value, and ibc_filter_tpl_shape2 has a second value, it is possible to specify only left-side IBC-LIC.

[0951] An L-shaped FIBC can be specified when ibc_filter_shape has a second value, ibc_filter_tpl_shape1 has a first value, and ibc_filter_set_num has a first value.

[0952] A multi-model FIBC can be specified when ibc_filter_shape has a second value, ibc_filter_tpl_shape1 has a first value, and ibc_filter_set_num has a second value.

[0953] When ibc_filter_shape has a second value, ibc_filter_tpl_shape1 has a second value and ibc_filter_tpl_shape2 has a first value, only the top FIBC can be specified.

[0954] When ibc_filter_shape has a second value, ibc_filter_tpl_shape1 has a second value, and ibc_filter_tpl_shape2 has a second value, it is possible to specify only the left FIBC.

[0955] The IBC prediction signal filtering information acquisition unit can signal that it is in IBC merging mode by using at least one of pred_mode_ibc_flag, general_merge_flag, or merge_idx.

[0956] As an example, for the IBC merging mode, at least one of the following can be inherited: whether filtering is performed, filter shape, filter template shape, filter coefficients, number of filter coefficient sets, or IBC filter mode.

[0957] As an example, for the IBC merging mode, at least one of ibc_filter_flag, ibc_filter_shape, ibc_filter_tpl_shape, ibc_filter_set_num, or IBC filter mode can be inherited.

[0958] As an example, in the IBC merging mode, when the candidate block to be merged uses multiple models, the target block can also use multiple models.

[0959] As an example, in the IBC merging mode, when the candidate block to be merged uses an L-shaped template, the target block can also use an L-shaped template.

[0960] As an example, in the IBC merge mode, when the merge candidate block uses the top-only template, the target block can also use the top-only template.

[0961] As an example, in the IBC merge mode, when the candidate block to be merged uses a left-only template, the target block can also use a left-only template.

[0962] The filtering of the target block can be determined based on the IBC merging candidates.

[0963] As an example, when the IBC merging candidates are not filtered in the non-IBC-LIC filter mode, the target block may not perform filtering.

[0964] As an example, when filtering the IBC merging candidates in L-shaped IBC-LIC filter mode, the target block can also perform filtering in L-shaped IBC-LIC mode.

[0965] As an example, when filtering the IBC merging candidates in the multi-model IBC-LIC filtering mode, the target block can also be filtered in the multi-model IBC-LIC mode.

[0966] As an example, when filtering the IBC merging candidates in the top-only IBC-LIC filter mode, the target block can also perform filtering in the top-only IBC-LIC mode.

[0967] As an example, when filtering the IBC merging candidate in the left-only IBC-LIC filter mode, the target block can also be filtered in the left-only IBC-LIC mode.

[0968] As an example, when filtering the IBC merging candidate in L-shaped FIBC filter mode, the target block can also be filtered in L-shaped FIBC mode.

[0969] As an example, when filtering the IBC merging candidates in the multi-model FIBC filter mode, the target block can also perform filtering in the multi-model FIBC mode.

[0970] As an example, when filtering IBC merging candidates in the top-only FIBC filter mode, the target block can also perform filtering in the top-only FIBC mode.

[0971] As an example, when filtering the IBC merging candidate in the left-only FIBC filter mode, the target block can also perform filtering in the left-only FIBC mode.

[0972] In this disclosure, DBv can be indicated as DBV.

[0973] Dbv can be a chroma prediction mode, which, when predicting chroma blocks, scales the block vector used in the luma block and uses it to generate the chroma prediction signal when the luma block corresponding to the chroma target block is encoded with IBC or intraTMP.

[0974] The IBC predictive signal filtering information acquisition unit can inherit the luminance filter information in the chromaticity Dbv mode.

[0975] As an example, when encoding chroma blocks via Dbv, the block vector of the luma block can be inherited, and at least one of the following can be inherited together: whether filtering is performed, filter shape, filter template shape, filter coefficients, number of filter coefficient sets, or IBC filter mode.

[0976] As an example, when encoding chroma blocks via Dbv, the block vector of the luma block can be inherited, and at least one of ibc_filter_flag, ibc_filter_shape, ibc_filter_tpl_shape, ibc_filter_set_num, or IBC filter mode can be inherited together.

[0977] As an example, when encoding chroma blocks using Dbv, if the corresponding luma block uses a multi-model, the target chroma block can also use a multi-model.

[0978] As an example, when encoding chroma blocks using Dbv, if the corresponding luminance block uses an L-shaped template, the target chroma block can also use an L-shaped template.

[0979] As an example, when encoding chroma blocks via Dbv, if the corresponding luma block uses a top-only template, the target chroma block can also use a top-only template.

[0980] As an example, when encoding chroma blocks using Dbv, if the corresponding luma block uses a left-only template, the target chroma block can also use a left-only template.

[0981] As an example, when encoding chroma blocks in Dbv mode, the block vector of the luma block can be inherited, and at least one of the following can be inherited: whether filtering is performed, filter shape, filter template shape, filter coefficients, number of filter coefficient sets, IBC filter mode, ibc_filter_set_num, ibc_filtering_flag, ibc_filter_shape, ibc_filter_tpl_shape, intraTMP_LIC_flag, intraTMP_LIC_index, or IntraTMP_FLM_flag. intraTMP_LIC_flag can be information indicating that the luma block is in intra-frame TMP mode and an L-shaped IntraTMP-LIC filter is applied to the luma block. intraTMP_LIC_index can be information indicating that the luma block is in IntraTMP mode and at least one of multi-model, L-shaped template, top-only template, or left-only template is used when LIC is applied. IntraTMP_FLM_flag can be information indicating that the luma block is in intra-frame TMP mode and an IntraTMP-FLM filter is applied to the luma block.

[0982] The “DBV Predictive Signal Filtering Information” described in this disclosure may include at least one of the following: whether filtering is performed, filter shape, filter template shape, filter coefficients, number of filter coefficient sets, IBC filter mode, ibc_filter_set_num, ibc_filtering_flag, ibc_filter_shape, ibc_filter_tpl_shape, intraTMP_LIC_flag, intraTMP_LIC_index, or IntraTMP_FLM_flag.

[0983] The filtering of the chroma block can be determined based on the filter information of the corresponding luminance block.

[0984] As an example, when the corresponding luminance block is not filtered in the non-IBC-LIC filter mode, the chrominance target block may also not be filtered.

[0985] As an example, when the corresponding luminance block is filtered in the L-shaped IBC-LIC filter mode, the chrominance target block can also be filtered in the L-shaped IBC-LIC mode.

[0986] As an example, when the corresponding luminance block is filtered in the multi-model IBC-LIC filter mode, the chrominance target block can also be filtered in the multi-model IBC-LIC filter mode.

[0987] As an example, when the corresponding luminance block is filtered in the top-only IBC-LIC filter mode, the chrominance target block can also be filtered in the top-only IBC-LIC mode.

[0988] As an example, when the corresponding luminance block is filtered in the left-only IBC-LIC filter mode, the chrominance target block can also be filtered in the left-only IBC-LIC mode.

[0989] As an example, when the corresponding luminance block is filtered in the L-shaped FIBC filter mode, the chrominance target block can also be filtered in the L-shaped FIBC mode.

[0990] As an example, when the corresponding luminance block is filtered in the multi-model FIBC filter mode, the chrominance target block can also be filtered in the multi-model FIBC mode.

[0991] As an example, when the corresponding luminance block is filtered in the top-only FIBC filter mode, the chrominance target block can also be filtered in the top-only FIBC mode.

[0992] As an example, when the corresponding luminance block is filtered in the left-only FIBC filter mode, the chrominance target block can also be filtered in the left-only FIBC mode.

[0993] As an example, when filtering the corresponding luma block in L-shaped IntraTMP-LIC filter mode, the chroma target block can also be filtered in L-shaped IntraTMP-LIC filter mode. L-shaped IntraTMP-LIC filter mode can utilize IntraTMP for luma blocks to generate prediction signals and use T... L-shape A square filter and a filter set apply filtering modes to the generated predicted signal. At least one of intraTMP_LIC_flag or intraTMP_LIC_index can indicate whether the luminance block is in L-shaped IntraTMP-LIC filter mode.

[0994] As an example, when filtering the corresponding luma block in the multi-model IntraTMP-LIC filter mode, the chroma target block can also be filtered in the multi-model IntraTMP-LIC filter mode. The multi-model IntraTMP-LIC filter mode can utilize IntraTMP for the luma block to generate a prediction signal and use T... L-shape The mode in which a square filter and two filter sets perform filtering on the generated predicted signal. At least one of intraTMP_LIC_flag or intraTMP_LIC_index can indicate whether the luma block is in the multi-model IntraTMP-LIC filter mode.

[0995] As an example, when filtering the corresponding luma block in the top-only IntraTMP-LIC filter mode, the chroma target block can also be filtered in the top-only IntraTMP-LIC filter mode. The top-only IntraTMP-LIC filter mode can utilize the IntraTMP used for the luma block to generate a prediction signal and use T... Top A square filter and a filter set are used to perform filtering on the generated predicted signal. At least one of intraTMP_LIC_flag or intraTMP_LIC_index can indicate whether the luminance block is in the top-only IntraTMP-LIC filter mode.

[0996] As an example, when filtering the corresponding luma block in the left-only IntraTMP-LIC filter mode, the chroma target block can also be filtered in the left-only IntraTMP-LIC filter mode. The left-only IntraTMP-LIC filter mode can utilize the IntraTMP used for the luma block to generate a prediction signal and use T... Left A square filter and a filter set are used to perform filtering on the generated predicted signal. At least one of intraTMP_LIC_flag or intraTMP_LIC_index can indicate whether the luma block is in the left-only IntraTMP-LIC filter mode.

[0997] As an example, when filtering a corresponding luma block in IntraTMP-FLM filter mode, the chroma target block can also be filtered in IntraTMP-FLM filter mode. IntraTMP-FLM filter mode can be a mode that uses IntraTMP for the luma block to generate a prediction signal and performs FLM filtering on the generated prediction signal. `intraTMP_FLM_flag` indicates whether the luma block is in IntraTMP-FLM filter mode.

[0998] As an example, when filtering the corresponding luminance block in IntraTMP-FLM filter mode, the chrominance target block can be filtered in FIBC filter mode.

[0999] The IBC prediction signal filtering information acquisition unit can derive the IBC filter template shape from the neighboring reconstructed pixel (filter template pixel) values ​​of the target block and the IBC prediction signal.

[1000] All or part of the IBC prediction signal can be used to derive the template shape.

[1001] The ibc_filter_tpl_shape syntax element derives the IBC filter template shape from the neighboring reconstructed pixel (filter template pixel) values ​​of the target block and the IBC reference block.

[1002] [D2] First, the IBC prediction signal generation unit is executed to obtain the prediction signal P of the IBC mode. t Then, the IBC filter template shape can be derived from the neighboring reconstructed pixel (filter template pixel) values ​​of the target block and the IBC prediction signal.

[1003] The IBC prediction signal filtering information acquisition unit can derive the shape of the IBC filter template from the difference between the average value of the filter template of the target block and the average value of the IBC prediction signal.

[1004] As an example, when the average value of the IBC prediction signal is given and the top template T of the target block is... t,top The difference between the averages (Diff) Top m ) and the average value of the IBC prediction signal and the left template T of the target block t,left The difference between the averages (Diff) Left m When Diff is satisfied Top m ≤a1×Diff Left m When +β1, the filter template shape can be T. Top When the Diff condition is met... Left m ≤a2×Diff Top m When +β2, the filter template shape can be T. Left When none of the conditions are met, the filter template shape can be T. L-shape Here, a1, a2 and β1, β2 can be predetermined values. For example, a1 and a2 can be 1.2, and β1 and β2 can be 0.

[1005] As in Figure 27 In (A), the average value of the above IBC prediction signals can be any of the IBC prediction signals (Area). A The average value of ).

[1006] As another example, such as Figure 27 As shown in (B), when the above calculation of Diff... Top m At that time, the average value of the IBC prediction signal can belong to the top region (Area) of the IBC prediction signal, which is the same size as the template size. Top The average of the pixel coordinates of (). When the above is used to calculate Diff Left m At that time, the average value of the IBC prediction signal can belong to the left region (Area) of the IBC prediction signal, which is the same size as the template size. Left The average value of the pixel coordinates.

[1007] The IBC prediction signal filtering information acquisition unit can derive the shape of the IBC filter template from the difference between the average value of the filter template of the target block and the average value of the IBC prediction signal.

[1008] For example, when the average value of the IBC prediction signal is given and the L-shaped template T of the target block is... t,L-shape The difference between the averages (Diff) L-shape m The average value of the IBC prediction signal and the top template T of the target block t,Top The difference between the averages (Diff) Top m ) and the average value of the IBC prediction signal and the left template T of the target block t,Left The difference between the averages (Diff) Left m When ), 1) when the minimum average difference is Diff L-shape m When the filter template shape can be T L-shape Optionally, 2) when the minimum mean difference is Diff Top m When the filter template shape is T, it can be T. Top Optionally, 3) when the minimum mean difference is Diff Left m When the filter template shape is T, it can be T. Left .

[1009] As in Figure 27 In (A), the average value of the IBC prediction signal can be any of the IBC prediction signals (Area). A The average value of ).

[1010] As another example, such as Figure 27 As shown in (B), when the above calculation of Diff... L-shape m At that time, the average value of the IBC prediction signal can belong to an L-shaped region (Area) of the IBC prediction signal that is the same size as the template size. L-shape The average of the pixel coordinates of (). When the above is used to calculate Diff Top m At that time, the average value of the IBC prediction signal can belong to the top region (Area) of the IBC prediction signal, which is the same size as the template size. Top The average of the pixel coordinates of (). When the above is used to calculate Diff Left m At that time, the average value of the IBC prediction signal can belong to the left region (Area) of the IBC prediction signal, which is the same size as the template size. Left The average value of the pixel coordinates.

[1011] The IBC prediction signal filtering information acquisition unit can derive the shape of the IBC filter template based on the difference between the histogram distribution of the filter template of the target block and the histogram distribution of the IBC prediction signal.

[1012] When the histogram distribution of the IBC prediction signal is given and the top template T of the target block is... t,top The difference between histogram distributions (Diff) Top h ) and the histogram distribution of the IBC prediction signal and the left template T of the target block t,left The difference between histogram distributions (Diff) Left h When Diff is satisfied Top h ≤a1×Diff Left h When +β1, the filter template shape can be T. Top When the Diff condition is met... Left h ≤a2×Diff Top h When +β2, the filter template shape can be T. Left When none of the conditions are met, the filter template shape can be T. L-shape Here, α and β can be predetermined values. For example, α can be 1.2 and β can be 0. As another example, α can be 1 and β can be 20.

[1013] Figure 27 Examples of blocks, either all or a portion, calculated using only Diff are shown.

[1014] As in Figure 27 In (A), the average value of the above IBC prediction signals can be any of the IBC prediction signals (Area). A The average value of ).

[1015] As another example, such as in Figure 27 In (B), when the above calculation of Diff... top h At that time, the histogram distribution of the IBC prediction signal can belong to the top region (Area) of the IBC prediction signal, which is the same size as the template size. Top The histogram distribution values ​​of the pixel coordinates of ). When the above calculates Diff Left h At that time, the histogram distribution of the IBC prediction signal can belong to the left region (Area) of the IBC prediction signal, which is the same size as the template size. Left The histogram distribution of pixel coordinates.

[1016] The IBC predictive signal filtering information acquisition unit can obtain information by using block vector BV(BV). x BV y To determine whether a template is available, use at least one of the following: the coordinates of the current block (x, y) or the current block's coordinates.

[1017] When a template is unavailable, IBC predictive signal filtering may not be performed. Optionally, when a template is unavailable, ibc_filtering_flag may be exported as a second value.

[1018] As an example, for the filter template shape T L-shape , When at coordinates (x+BV) x -size T y+BV y -size T If no reconstructed pixel value exists at a given location, IBC predictive signal filtering may not be performed.

[1019] Optionally, when at coordinates (x+BV) x -size T y+BV y -size T When no reconstructed pixel value exists at a given location, ibc_filtering_flag can be exported as the second value. In other words, IBC predictive signal filtering can be omitted.

[1020] As an example, for the filter template shape T Top , When at coordinates (x + BV) x -size T ,y+BV yIf no reconstructed pixel value exists at a given location, IBC predictive signal filtering may not be performed.

[1021] Optionally, when at coordinates (x+BV) x -size T ,y+BV y When no reconstructed pixel value exists at a given location, ibc_filtering_flag can be exported as the second value. In other words, IBC predictive signal filtering can be omitted.

[1022] As an example, for the filter template shape T Left , When at coordinates (x+BV) x y+BV y -size T If no reconstructed pixel value exists at a given location, IBC predictive signal filtering may not be performed.

[1023] Optionally, when at coordinates (x+BV) x y+BV y -size T When no reconstructed pixel value exists at a given location, ibc_filtering_flag can be exported as the second value. In other words, IBC predictive signal filtering can be omitted.

[1024] Figure 28 An example of intra-block copy (IBC) is shown.

[1025] The IBC prediction signal generation unit can generate the prediction signal P through the IBC prediction signal search process. t .

[1026] When IBC mode is used for target block prediction, it can be achieved by using BV, pred_mode_ibc_flag, general_merge_flag, merge_idx, RRIBC flag, RRIBC index, and the target block position (x). t_lf y t_lf The prediction signal P of the IBC mode is obtained by selecting at least one of the following: the width (xCbW) of the target block, the height (xCbH) of the target block, the Dbv mode, or the components of the target block. t .

[1027] exist Figure 28 In the IBC reference block Blk ref It can be the prediction signal P of the IBC mode t .

[1028] It can be P t (x,y)=Rec(x+BV x ,y+BVy Here, (BV) x BV y ) is a block vector, and Rec(a,b) can be the reconstructed pixel value at coordinates (a,b).

[1029] Figure 29 An example of Dbv predictive signal generation is shown.

[1030] When the chroma is in Dbv mode, the IBC prediction signal generation unit can generate the chroma Dbv prediction signal P by using the BV inherited from the corresponding luminance prediction signal. c_t .

[1031] When the target chroma block is in Dbv mode, at least one of the following inherited from the corresponding luma signal—BV, pred_mode_ibc_flag, general_merge_flag, merge_idx, RRIBC flag, RRIBC index, intraTMP_LIC_flag, intraTMP_LIC_index, or IntraTMP_FLM_flag—and the current chroma prediction target block's position (x) can be used. t_lf y t_lf The Dbv prediction signal P of the Dbv mode of chroma is obtained by using at least one of the target block width (xCbW) and target block height (xCbH). c_t .

[1032] exist Figure 29 In the middle, Dbv reference block Blk c_ref It can be the predicted signal of the Dbv mode.

[1033] It can be P c_t (x,y)=Rec(x+BVx,y+BV y Here, (BV) x BV y ) is a block vector, and Rec(a,b) can be the reconstructed pixel value at coordinates (a,b).

[1034] The IBC prediction signal filtering unit can use the IBC prediction signal filtering information obtained by the [D1] IBC prediction signal filtering information acquisition unit to filter the IBC prediction signal P generated by the [D2] IBC prediction signal generation unit. t Perform filtering to obtain the filtered prediction signal P' t .

[1035] In addition, the IBC prediction signal filtering unit can specify the filter model by using IBC prediction signal filtering information.

[1036] In addition, the IBC prediction signal filtering unit can derive the filter coefficients by using the filter model and IBC prediction signal filtering information.

[1037] In addition, the IBC prediction signal filtering unit can use filter models and filter coefficients to filter the IBC prediction signal P. t Perform filtering to obtain the filtered prediction signal P' t .

[1038] The IBC filter model specification unit can specify the filter model by using IBC predictive signal filtering information.

[1039] A filter model is an equation used to perform filtering of IBC prediction signals, and the equation can be specified by using at least one of the IBC prediction pixels to be filtered, the neighboring pixels of the IBC prediction pixels, the bias, or the filter coefficients.

[1040] Alternatively, the filter model can be specified by using at least one of the filter shape or IBC filter mode.

[1041] Alternatively, the filter model can be specified by using at least one of ibc_filter_shape or IBC filter mode.

[1042] As an example, when the filter shape is an M×N square, the filter model can be specified by using a neighboring square of M×N pixels centered on the pixel to be filtered at (x, y) coordinates, as shown in equation [D3-1]-1.

[1043] Equation [D3-1]-1:

[1044] Here, it could be m=(M-1)>>2, n=(N-1)>>2. α,β c b It can be a filter coefficient, and b can be a bias.

[1045] As an example, when the IBC filter mode is one of the L-shaped IBC-LIC, multi-model IBC-LIC, top-only IBC-LIC, and left-only IBC-LIC modes, the filter model can be specified by using a neighboring square of M×N pixels centered on the pixel to be filtered at (x,y) coordinates, as in equation [D3-1]-1.

[1046] Figure 30 An example of a 1×1 rectangular filter is shown.

[1047] As an example, when the filter shape is a 1×1 square, such as Figure 30 As shown, the filter model can be specified as in equation [D3-1]-2 by using a 1×1 pixel neighboring square centered on the pixel to be filtered.

[1048] Equation [D3-1]-2:

[1049] Here, c 0,0 c b It can be a filter coefficient, and b can be a bias.

[1050] In this disclosure, the square filter with M=1 and N=1 can be referred to as the first filter model.

[1051] As an example, when the IBC filter mode is one of L-shaped IBC-LIC, multi-model IBC-LIC, top-only IBC-LIC, and left-only IBC-LIC modes, such as Figure 30 As shown, the filter model can be specified by using a 1×1 pixel neighboring square centered on the pixel to be filtered, as in equation [D3-1]-2.

[1052] Figure 31 An example of a 3×3 rectangular filter is shown.

[1053] As an example, when the filter shape is a 3×3 square, such as Figure 31 As shown, the filter model can be specified as in equation [D3-1]-3 by using a 3×3 pixel neighboring square centered on the pixel to be filtered.

[1054] Equation [D3-1]-3:

[1055]

[1056] Here, c α,β c b It can be a filter coefficient, and b can be a bias.

[1057] The equation can be calculated as follows: the equation calculates m and n from equation [D3-1]-1.

[1058] m=(M-1)>>2=(3-1)>>2=1, n=(N-1)>>2=(3-1)>>2=1 In this disclosure, the square filter with M=3 and N=3 can be referred to as the second filter model.

[1059] As an example, when the IBC filter mode is one of L-shaped IBC-LIC, multi-model IBC-LIC, top-only IBC-LIC, and left-only IBC-LIC modes, such as Figure 30 As shown, the filter model can be specified by using a 3×3 pixel neighboring square centered on the pixel to be filtered, as in equation [D3-1]-3.

[1060] As an example, when the filter shape is an M×N cross, the filter model can be specified by using neighboring M×N pixels centered on the pixel to be filtered at (x, y) coordinates, as shown in Equation [D3-1]-4.

[1061] Equation [D3-1]-4:

[1062] Here, it could be m=(M-1)>>2, n=(N-1)>>2. α,β c b It can be a filter coefficient, and b can be a bias.

[1063] As an example, when the IBC filter mode is one of the L-shaped FIBC, multi-model FIBC, top-only FIBC, and left-only FIBC modes, the filter model can be specified by using a neighboring cross-shaped M×N pixel centered on the pixel to be filtered at (x, y) coordinates, as shown in Equation [D3-1]-4.

[1064] Figure 32 An example of a 3×3 cross-shaped filter is shown.

[1065] As an example, when the filter shape is a 3×3 cross shape, such as Figure 32 As shown, the filter model can be specified as in equation [D3-1]-5 by using a 3×3 pixel cross-shaped neighborhood centered on the pixel to be filtered.

[1066] Equation [D3-1]-5:

[1067]

[1068] Here, c α,β c b It can be a filter coefficient, and b can be a bias.

[1069] The equation can be calculated as follows: the equation calculates m and n from equation [D3-1]-4.

[1070] m=(M-1)>>2=(3-1)>>2=1, n=(N-1)>>2=(3-1)>>2=1 In this disclosure, the cross-shaped filter with M=3 and N=3 can be referred to as the third filter model.

[1071] As an example, when the IBC filter mode is one of the L-shaped FIBC, multi-model FIBC, top-only FIBC, and left-only FIBC modes, the filter model can be specified by using a neighboring cross-shaped 3×3 pixel centered on the pixel to be filtered at (x, y) coordinates, as shown in Equation [D3-1]-5.

[1072] As an example, when the filter shape is an M×N cross, it can be achieved by using a neighboring M×N cross-shaped pixel centered at the pixel to be filtered at coordinates (x,y) and including a nonlinear term (P). t (x,y) 2 Use ) to specify the filter model, as in equation [D3-1]-6.

[1073] Equation [D3-1]-6:

[1074] Here, it could be m=(M-1)>>2, n=(N-1)>>2. α,β c P c b It can be a filter coefficient, and b can be a bias.

[1075] As an example, when the IBC filter mode is one of L-shaped FIBC, multi-model FIBC, top-only FIBC, and left-only FIBC modes, it can be achieved by using a neighboring cross-shaped M×N pixel array centered on the pixel to be filtered at (x, y) coordinates and including a nonlinear term (P). t (x, y) 2 ) to specify the filter model, as shown in equation [D3-1]-6.

[1076] As an example, when the filter shape is a 3×3 cross, it can be achieved by using neighboring 3×3 pixels centered on the pixel to be filtered and including a non-linear term (P). t (x,y) 2 To specify the filter model as in equation [D3-1]-7, such as... Figure 32 As shown in the image.

[1077] Equation [D3-1]-7:

[1078]

[1079] Here, c α,β c P c b It can be a filter coefficient, and b can be a bias.

[1080] The equation can be calculated as follows: the equation calculates m and n from equation [D3-1]-6.

[1081] m=(M-1)>>2=(3-1)>>2=1, n=(N-1)>>2=(3-1)>>2=1 In this disclosure, the cross-shaped filter with M=3 and N=3 can be referred to as the fourth filter model.

[1082] When the IBC filter mode is one of L-shaped FIBC, multi-model FIBC, top-only FIBC, and left-only FIBC modes, it uses a 3×3 pixel cross-shaped filter centered on the pixel to be filtered at (x, y) coordinates and includes a non-linear term (P). t (x, y) 2 The filter model can be specified as in equation [D3-1]-7, such as... Figure 32 As shown.

[1083] The IBC filter coefficient acquisition unit of the IBC predictive signal filtering unit can derive the filter coefficient set FilterSet(c) of the IBC filter model. α,β c P c b (at least one of them), to ensure that by applying the IBC filter model to the IBC reference block template T ref The result value obtained by the pixel value and the corresponding coordinates of the target block template T t The pixel values ​​have the best correlation (similarity).

[1084] The filter coefficients can be derived using the methods used to derive the filter coefficients for IBC-LIC and FIBC.

[1085] The IBC filter coefficient acquisition unit can export a filter coefficient set FilterSet (including c) α,β c P and c b (at least one of the following), or multiple filter coefficient sets (FilterSet) n (including c) α,β n c P n and c b n (at least one of them).

[1086] As an example, when using a set of filter coefficients, the filter coefficients c can be derived. α,β c P and c b At least one of them, such that for all IBC reference block templates T ref The pixel values ​​are obtained by applying the IBC filter model to the IBC reference block template T. ref The result value obtained by the pixel value and the corresponding coordinates of the target block template T t The pixel values ​​have the best correlation (similarity).

[1087] As an example, when using two filter coefficient sets (FilterSet1, FilterSet2), the filter coefficients c can be derived for pixels with small values ​​based on the threshold (Thres). α,β 1 c P 1 and c b 1 The set of at least one of them and the filter coefficients c for pixels with large values α,β 2 c P 2 and c b 2 A set of at least one of them.

[1088] When using two sets of filter coefficients, the threshold can be the average value of the pixels belonging to the filter template.

[1089] As an example, when using three filter coefficient sets (FilterSet1, FilterSet2, FilterSet3), it is also possible to derive a filter for satisfying P based on two thresholds Thres0 and Thres1 (Thres0 < Thres1). t The inclusion filter coefficients c of pixels whose (x, y) ≤ Thres0 α,β 1 c P 1 and c b 1 A set of at least one of the following, for satisfying Thres0 < P t The inclusion filter coefficients c of pixels whose (x, y) ≤ Thres1 α,β 2 c P 2 and c b 2 A set of at least one of them, and for satisfying P t The filter coefficients c of pixels whose (x, y) > Thres1α,β 3 , c P 3 and c b 3 A set of at least one of them.

[1090] Thres0 may represent a first threshold, and Thres1 may represent a second threshold (Thres0<Thres1). When a total of N pixels belong to the filt...

Claims

1. An image decoding method, the method comprising: Obtain the initial list of merge candidates for the current block; The initial merge candidate list is reordered to obtain a reordered merge candidate list for the current block; Determine a candidate for the current block from the candidates in the reordered list of merge candidates; as well as The prediction of the current block is performed based on the determined candidates.

2. The method according to claim 1, wherein: The initial merge candidate list includes at least one spatial candidate, and The spatial candidates are obtained based on blocks that are adjacent to the current block or blocks that are not adjacent to the current block.

3. The method according to claim 1, wherein: The initial merge candidate list includes Automatic Relocation Block Vector Prediction (AR-BVP) candidates.

4. The method according to claim 3, wherein: The AR-BVP candidate is derived based on the block vector at the top left, top right, center, bottom left, or bottom right position of the reference block of the current block.

5. The method according to claim 1, wherein: The initial merge candidate list includes at least one chained MV prediction CMVP candidate.

6. The method according to claim 5, wherein: The CMVP candidate is derived from the time MV predictor TMVP candidate in the initial merge candidate list.

7. The method according to claim 5, wherein: The CMVP candidate is derived based on the accumulation of MV and BV, wherein the MV and BV are recursively tracked based on pre-derived motion vectors from the initial merged candidate list.

8. The method according to claim 5, wherein: In response to the initial merge candidate list including multiple CMVP candidates, the multiple CMVP candidates are reordered based on template cost.

9. The method according to claim 8, wherein: The template cost is calculated based on the difference between the template of the block indicated by the CMVP candidate and the template of the current block.

10. The method according to claim 9, wherein: The reordered merged candidate list includes only a portion of the reordered CMVP candidates.

11. An image encoding method, the method comprising: Obtain the initial list of merge candidates for the current block; The initial merge candidate list is reordered to obtain a reordered merge candidate list for the current block; Determine a candidate for the current block from the candidates in the reordered list of merge candidates; as well as The prediction of the current block is performed based on the determined candidates.

12. The method according to claim 11, wherein: The initial merge candidate list includes at least one spatial candidate, and The spatial candidates are obtained based on blocks that are adjacent to the current block or blocks that are not adjacent to the current block.

13. The method according to claim 11, wherein: The initial merge candidate list includes Automatic Relocation Block Vector Prediction (AR-BVP) candidates.

14. The method of claim 13, wherein: The AR-BVP candidate is derived based on the block vector at the top left, top right, center, bottom left, or bottom right position of the reference block of the current block.

15. The method according to claim 11, wherein: The initial merge candidate list includes at least one chained MV prediction CMVP candidate.

16. The method of claim 15, wherein: The CMVP candidate is derived from the time MV predictor TMVP candidate in the initial merge candidate list.

17. The method of claim 15, wherein: The CMVP candidate is derived based on the accumulation of MV and BV, wherein the MV and BV are recursively tracked based on pre-derived motion vectors from the initial merged candidate list.

18. The method of claim 15, wherein: In response to the initial merge candidate list including multiple CMVP candidates, the multiple CMVP candidates are reordered based on template cost.

19. The method of claim 18, wherein: The template cost is calculated based on the difference between the template of the block indicated by the CMVP candidate and the template of the current block.

20. A method for transmitting a bit stream, the method comprising: Obtain the initial list of merge candidates for the current block; The initial merge candidate list is reordered to obtain a reordered merge candidate list for the current block; Determine a candidate for the current block from the candidates in the reordered list of merge candidates; The prediction for the current block is performed based on the identified candidates; as well as Send the bit stream generated based on the prediction of the current block.

Images