Methods for encoding / decoding images and storage media containing bitstreams

Abstract

A method for encoding / decoding an image and a storage medium containing a bitstream are disclosed. The method for decoding the image includes: determining whether a merging mode is applied to the current block based on a merging flag decoded from the bitstream; when a merging mode is applied to the current block, generating a merging candidate list for the current block, which includes at least one of spatial merging candidates derived based on motion information of spatially neighboring blocks and temporal merging candidates derived based on motion information of temporally neighboring blocks; updating the merging candidate list for the current block; selecting a merging candidate from the updated merging candidate list; deriving a first initial motion vector and a second initial motion vector for the current block from the merging candidate; obtaining a first corrected motion vector and a second corrected motion vector by performing motion information correction; generating a prediction block for the current block using the first corrected motion vector and the second corrected motion vector; and obtaining a reconstructed block based on the prediction block.

Description

[0001] This application is a divisional application of Chinese patent application filed on June 5, 2018, with international application number PCT / KR2018 / 006401, entitled "Video Encoding / Decoding Method and Apparatus and Recording Medium for Storing Bit Streams", and application number 201880038074.0 after entering the Chinese national phase. Technical Field

[0002] This invention relates to a method and apparatus for encoding / decoding images. Specifically, this invention relates to a method and apparatus for performing motion compensation by correcting motion information, and a recording medium for storing the bitstream generated by the image encoding method / apparatus of this invention. Background Technology

[0003] Recently, the demand for high-resolution and high-quality images, such as high-definition (HD) and ultra-high-definition (UHD) images, has increased across various application areas. However, higher resolution and quality image data involve a significant increase in data volume compared to traditional image data. Consequently, transmission and storage costs increase when transmitting image data using media such as traditional wired and wireless broadband networks, or when storing image data using traditional storage media. To address these issues arising from the increasing resolution and quality of image data, efficient image encoding / decoding technologies are needed for higher resolution and higher quality images.

[0004] Image compression techniques encompass a variety of methods, including: inter-frame prediction techniques, which predict pixel values ​​included in the current frame from previous or subsequent frames; intra-frame prediction techniques, which predict pixel values ​​included in the current frame using pixel information from the current frame; transform and quantization techniques, used to compress the energy of residual signals; entropy coding techniques, which assign short codes to values ​​with high frequency of occurrence and long codes to values ​​with low frequency of occurrence; and so on. Image data can be efficiently compressed using such image compression techniques and can then be transmitted or stored.

[0005] Because traditional image encoding / decoding methods and devices perform motion compensation based on motion information of spatial / temporal neighboring blocks adjacent to the current block (i.e., the encoding / decoding target block), they have limitations in improving encoding efficiency. Summary of the Invention

[0006] Technical issues

[0007] This invention provides a method and apparatus for performing motion compensation by modifying motion information.

[0008] Technical solution

[0009] According to a method for decoding an image according to the present invention, the method may include: deriving motion correction candidates from motion information of spatially neighboring blocks, motion information of temporally neighboring blocks, predefined motion information, and motion information that appears most frequently in a reference frame; performing motion information correction on the derived motion correction candidates; and generating a predicted block for the current block by using the motion correction candidates that have undergone motion information correction.

[0010] In the method for decoding an image according to the present invention, motion information candidates are deduced sequentially from motion information of spatially neighboring blocks, motion information of temporally neighboring blocks, defined motion information, and motion information that appears most frequently in a reference frame, in a predetermined order.

[0011] In the method for decoding an image according to the present invention, a predetermined order represents the order of motion information of spatially neighboring blocks, motion information of temporally neighboring blocks, and defined motion information.

[0012] In the method for decoding an image according to the present invention, the step of performing motion information correction includes: applying bilateral template matching to motion vectors present in the derived motion correction candidates.

[0013] In the method for decoding an image according to the present invention, bilateral template matching includes: generating a bilateral template by using motion vectors present in the derived motion correction candidates as initial motion vectors; and correcting the initial motion vector by comparing samples within the bilateral template with reconstructed samples in a reference image indicated by the initial motion vector.

[0014] In the method for decoding an image according to the present invention, the initial motion vector is a bidirectional predicted motion vector as a non-zero vector in the derived motion correction candidate.

[0015] In the method for decoding an image according to the present invention, when there is no bidirectional predicted motion vector as a non-zero vector in the derived motion correction candidates, the initial motion vector is set to a zero vector.

[0016] In the method for decoding an image according to the present invention, a temporally adjacent block is included in a reference frame selected by a reference frame index of a spatially adjacent block.

[0017] In the method for decoding an image according to the present invention, bilateral template matching is performed based on integer pixels and subpixels.

[0018] In the method for decoding an image according to the present invention, the step of correcting the initial motion vector includes: searching for a motion vector indicating a region in a reference image, wherein the region has the minimum distortion value relative to a corresponding region in a bilateral template; and setting the found motion vector as the correction value of the initial motion vector.

[0019] In the method for decoding an image according to the present invention, the step of searching for an initial motion vector includes: searching a limited search range within a reference frame.

[0020] In the method for decoding an image according to the present invention, the limited search range is set to a predetermined range delimited by integer pixels.

[0021] In the method for decoding an image according to the present invention, the step of searching for an initial motion vector includes:

[0022] Motion vectors are searched within a predetermined range based on sub-pixels.

[0023] In the method for decoding an image according to the present invention, the initial motion vector is corrected recursively.

[0024] In the method for decoding an image according to the present invention, when the current block does not correspond to the one-way prediction merging candidate, the local illumination compensation mode, and the affine motion compensation mode, motion information correction is performed on the derived motion correction candidate.

[0025] The method for decoding an image according to the present invention further includes: decoding a motion correction mode using information; and determining a motion correction mode based on the decoded motion correction mode using information, wherein motion correction candidates are deduced under the motion correction mode.

[0026] In the method for decoding an image according to the present invention, the step of decoding the motion correction mode using information includes: determining whether to decode the motion correction mode using information based on a skip flag or a merge flag.

[0027] In the method for decoding an image according to the present invention, the step of deriving motion information candidates when there are multiple spatial neighbor blocks includes: deriving motion information from spatial neighbor blocks having bidirectional predicted motion vectors, and subsequently deriving motion information from spatial neighbor blocks having unidirectional predicted motion vectors.

[0028] According to a method for encoding an image according to the present invention, the method may include: deriving motion correction candidates from motion information of spatially neighboring blocks, motion information of temporally neighboring blocks, predefined motion information, and motion information that appears most frequently in a reference frame; performing motion information correction on the derived motion correction candidates; and generating a predicted block for the current block by using the motion correction candidates that have undergone motion information correction.

[0029] A non-transitory storage medium comprising a bitstream generated by an image coding method, the method comprising: deriving motion correction candidates from motion information of spatially neighboring blocks, motion information of temporally neighboring blocks, predefined motion information, and motion information most frequently occurring in a reference frame; performing motion information correction on the derived motion correction candidates; and generating a predicted block for the current block by using the motion correction candidates that have undergone motion information correction.

[0030] According to an embodiment of the present invention, a method for decoding an image includes: determining whether a merging mode is applied to a current block based on a merging flag decoded from a bitstream; when the merging mode is applied to the current block, generating a merging candidate list for the current block, the merging candidate list including at least one of spatial merging candidates derived based on motion information of spatially neighboring blocks and temporal merging candidates derived based on motion information of temporally neighboring blocks; updating the merging candidate list for the current block, the updated merging candidate list including new merging candidates generated by using two merging candidates in the merging candidate list; selecting a merging candidate from the updated merging candidate list; deriving a first initial motion vector and a second initial motion vector for the current block from the merging candidate; obtaining a first corrected motion vector and a second corrected motion vector by performing motion information correction; generating a predicted block for the current block using the first corrected motion vector and the second corrected motion vector; and obtaining a reconstructed block based on the predicted block.

[0031] According to an embodiment of the present invention, a method for encoding an image includes: generating a merge candidate list for the current block when a merging mode is applied to the current block, the merge candidate list including at least one of spatial merge candidates derived based on motion information of spatially neighboring blocks and temporal merge candidates derived based on motion information of temporally neighboring blocks; updating the merge candidate list for the current block, the updated merge candidate list including new merge candidates generated by using two merge candidates in the merge candidate list; selecting a merge candidate from the updated merge candidate list; deriving a first initial motion vector and a second initial motion vector for the current block from the merge candidates; obtaining a first corrected motion vector and a second corrected motion vector by performing motion information correction; generating a prediction block for the current block using the first corrected motion vector and the second corrected motion vector; encoding a merging flag indicating whether a merging mode is applied to the current block; and obtaining a residual block based on the prediction block.

[0032] According to an embodiment of the present invention, a non-transitory storage medium comprising a bitstream generated by an image coding method includes: generating a merge candidate list for the current block when a merging mode is applied to the current block, the merge candidate list including at least one of spatial merge candidates derived based on motion information of spatially neighboring blocks and temporal merge candidates derived based on motion information of temporally neighboring blocks; updating the merge candidate list for the current block, the updated merge candidate list including new merge candidates generated by using two merge candidates in the merge candidate list; selecting a merge candidate from the updated merge candidate list; deriving a first initial motion vector and a second initial motion vector for the current block from the merge candidate; obtaining a first corrected motion vector and a second corrected motion vector by performing motion information correction; generating a prediction block for the current block using the first corrected motion vector and the second corrected motion vector; encoding a merging flag indicating whether a merging mode is applied to the current block; and obtaining a residual block based on the prediction block.

[0033] Beneficial effects

[0034] According to the present invention, an image encoding / decoding method and apparatus with improved compression efficiency can be provided.

[0035] According to the present invention, the efficiency of image encoding and decoding can be improved.

[0036] According to the present invention, the computational complexity of image encoders and image decoders can be reduced. Attached Figure Description

[0037] Figure 1 This is a block diagram illustrating the configuration of an encoding device according to an embodiment of the present invention;

[0038] Figure 2This is a block diagram illustrating the configuration of a decoding device according to an embodiment of the present invention;

[0039] Figure 3 It is a schematic diagram illustrating the partitioning structure of an image when it is encoded and decoded;

[0040] Figure 4 This is a diagram illustrating inter-frame prediction processing;

[0041] Figure 5 This is a flowchart illustrating an image encoding method according to an embodiment of the present invention;

[0042] Figure 6 This is a flowchart illustrating an image decoding method according to an embodiment of the present invention;

[0043] Figure 7 This is a flowchart illustrating an image encoding method according to another embodiment of the present invention;

[0044] Figure 8 This is a flowchart illustrating an image decoding method according to another embodiment of the present invention;

[0045] Figure 9 This is a diagram illustrating an exemplary method for deriving candidates for the spatial motion vector of the current block;

[0046] Figure 10 This is a diagram illustrating an exemplary method for deriving candidates for the time motion vector of the current block;

[0047] Figure 11 This is a diagram illustrating an example of adding spatial merge candidates to the merge candidate list;

[0048] Figure 12 This is a diagram illustrating an example of adding time merging candidates to the merge candidate list;

[0049] Figure 13 This is a diagram illustrating the method for performing bilateral template matching;

[0050] Figure 14 and Figure 15 This is a diagram illustrating the search range of motion vectors used to perform search corrections using bilateral template matching; and

[0051] Figure 16 This is a flowchart illustrating an image decoding method according to yet another embodiment of the present invention. Detailed Implementation

[0052] Various modifications can be made to this invention, and various embodiments of the invention exist, wherein examples of these various embodiments will now be provided with reference to the accompanying drawings, and examples of these various embodiments will be described in detail. However, the invention is not limited thereto, although exemplary embodiments may be interpreted as including all modifications, equivalents, or substitutions within the technical concept and scope of the invention. Similar reference numerals refer to functions that are identical or similar in respect of each other. In the drawings, the shapes and sizes of elements may be exaggerated for clarity. In the following detailed description of the invention, reference is made to the accompanying drawings, which illustrate specific embodiments in which the invention may be practiced. These embodiments are described in sufficient detail to enable those skilled in the art to practice this disclosure. It should be understood that the various embodiments of this disclosure, though different, are not necessarily mutually exclusive. For example, specific features, structures, and characteristics described herein in connection with one embodiment may be implemented in other embodiments without departing from the spirit and scope of this disclosure. Furthermore, it should be understood that the position or arrangement of various elements within each disclosed embodiment may be modified without departing from the spirit and scope of this disclosure. Therefore, the following detailed description should not be construed in a limiting sense, and the scope of this disclosure is defined only by the appended claims (and, where appropriate, the full scope of the equivalents claimed in the claims).

[0053] The terms "first," "second," etc., used in this specification may be used to describe various components, but these components are not to be construed as limited by these terms. These terms are only used to distinguish one component from another. For example, without departing from the scope of the invention, a "first" component may be referred to as a "second" component, and a "second" component may similarly be referred to as a "first" component. The term "and / or" includes a combination of multiple items or any one of multiple items.

[0054] It will be understood that, in this specification, when an element is referred to only as "connected to" or "joined to" another element rather than "directly connected to" or "directly joined to" another element, the element may be "directly connected to" or "directly joined to" the other element, or connected to or joined to the other element if there are other elements between the element and the other element. Conversely, it should be understood that when an element is referred to as "directly joined" or "directly connected" to another element, there are no intermediate elements.

[0055] Furthermore, the components shown in the embodiments of the present invention are illustrated independently to present distinct functionalities. Therefore, this does not imply that each component is composed as a separate hardware or software unit. In other words, for convenience, each component includes every one of the enumerated components. Thus, at least two components in each component may be combined to form a single component, or a single component may be divided into multiple components for performing each function. Embodiments where each component is combined and embodiments where a component is divided are also included within the scope of the invention without departing from its spirit.

[0056] The terminology used in this specification is for describing particular embodiments only and is not intended to limit the invention. Expressions used in the singular include plural expressions unless they have a distinct meaning in the context. In this specification, it will be understood that terms such as “comprising,” “having,” etc., are intended to indicate the presence of the features, quantities, steps, actions, elements, components, or combinations thereof disclosed in the specification, and are not intended to exclude the possibility that one or more other features, quantities, steps, actions, elements, components, or combinations thereof may be present or added. In other words, when a particular element is referred to as “comprising,” elements other than the corresponding element are not excluded; rather, additional elements may be included in embodiments of the invention or within the scope of the invention.

[0057] Furthermore, some components may not be essential for performing the necessary functions of the invention, but rather optional components that merely enhance its performance. The invention can be implemented by including only the essential components necessary for carrying out the invention, excluding components used to enhance performance. Structures that include only the essential components and exclude optional components used solely for enhancing performance are also included within the scope of the invention.

[0058] In the following, embodiments of the present invention will be described in detail with reference to the accompanying drawings. In describing exemplary embodiments of the invention, well-known functions or structures will not be described in detail, as they would unnecessarily obscure the understanding of the invention. The same constituent elements in the drawings are denoted by the same reference numerals, and repeated descriptions of the same elements will be omitted.

[0059] In the following text, an image may refer to a frame that constitutes a video, or it may refer to the video itself. For example, "encoding or decoding an image or both" may refer to "encoding or decoding a moving image or both," and may also refer to "encoding or decoding an image within an image of a moving image or both."

[0060] In the following text, the terms "moving images" and "video" may be used to mean the same thing and may be used interchangeably.

[0061] In the following text, the target image can be an encoded target image that serves as an encoding target and / or a decoded target image that serves as a decoding target. Furthermore, the target image can be an input image input to an encoding device and an input image input to a decoding device. Here, the target image may have the same meaning as the current image.

[0062] In the following text, the terms “image,” “picture,” “frame,” and “screen” may be used as having the same meaning and interchangeably with each other.

[0063] In the following text, a target block can be an encoding target block that serves as an encoding target and / or a decoding target block that serves as a decoding target. Furthermore, a target block can be the current block that serves as the current encoding and / or decoding target. For example, the terms "target block" and "current block" can be used to mean the same thing and are interchangeable.

[0064] In the following text, the terms “block” and “unit” may be used to mean the same thing and are interchangeable. Alternatively, “block” may refer to a specific unit.

[0065] In the following text, the terms “region” and “fragment” are used interchangeably.

[0066] In the following text, a specific signal can be a signal representing a specific block. For example, the original signal can be a signal representing the target block. The prediction signal can be a signal representing the prediction block. The residual signal can be a signal representing the residual block.

[0067] In this embodiment, each of the specific information, data, flags, indexes, elements, and attributes may have a value. A value equal to "0" for information, data, flags, indexes, elements, and attributes may represent logical false or a first predefined value. In other words, the values ​​"0", false, logical false, and the first predefined value can be interchanged with each other. A value equal to "1" for information, data, flags, indexes, elements, and attributes may represent logical true or a second predefined value. In other words, the values ​​"1", true, logical true, and the second predefined value can be interchanged with each other.

[0068] When variables i or j are used to represent columns, rows, or indices, the value of i can be an integer equal to or greater than 0, or an integer equal to or greater than 1. That is, columns, rows, indices, etc., can be counted starting from 0, or they can be counted starting from 1.

[0069] Terminology Description

[0070] Encoder: Represents the device that performs encoding. In other words, it refers to the encoding device.

[0071] Decoder: Represents the device that performs decoding. In other words, it refers to the decoding device.

[0072] A block is an M×N sample array. Here, M and N can represent positive integers, and a block can represent a two-dimensional sample array. A block can refer to a unit. The current block can represent a coding target block that becomes the target during encoding, or a decoding target block that becomes the target during decoding. Furthermore, the current block can be at least one of a coding block, a prediction block, a residual block, and a transform block.

[0073] Samples are the basic units that make up a block. Based on the bit depth (Bd), samples can be represented as numbers from 0 to 2. Bd The value is -1. In this invention, a sample point can be used to represent a pixel. That is, a sample point, a pel, and a pixel can have the same meaning.

[0074] Unit: Refers to an encoding and decoding unit. When encoding and decoding an image, a unit can be a region generated by partitioning a single image. Furthermore, a unit can represent a sub-partitioning unit when a single image is partitioned into sub-partitioning units during encoding or decoding. That is, an image can be partitioned into multiple units. When encoding and decoding an image, predetermined processing can be performed for each unit. A single unit can be partitioned into sub-units smaller than the unit's size. Depending on the function, a unit can represent a block, macroblock, coding tree unit, coding tree block, coding unit, coding block, prediction unit, prediction block, residual unit, residual block, transform unit, transform block, etc. Furthermore, to distinguish a unit from a block, a unit can include a luma component block, a chroma component block associated with the luma component block, and syntax elements for each chroma component block. Units can have various sizes and shapes; specifically, the shape of a unit can be a two-dimensional geometric figure, such as a square, rectangle, trapezoid, triangle, pentagon, etc. In addition, the cell information may include at least one of the following: cell type indicating coding cell, prediction cell, transform cell, etc., cell size, cell depth, and the order of encoding and decoding of the cell.

[0075] A coding tree unit is a single coding tree block configured with the luminance component Y and two coding tree blocks associated with the chrominance components Cb and Cr. Furthermore, a coding tree unit can represent a block and the syntax elements of each block. Each coding tree unit can be partitioned using at least one of quadtree partitioning, binary tree partitioning, and ternary tree partitioning methods to configure lower-level units such as coding units, prediction units, transform units, etc. A coding tree unit can be used as a term to specify a sample block that becomes a processing unit when encoding / decoding an image as an input image. Here, a quadtree can represent a quaternion tree.

[0076] Encoding block: Can be used as a term to specify any one of the Y encoding block, Cb encoding block, and Cr encoding block.

[0077] Neighboring blocks: These can represent blocks adjacent to the current block. A block adjacent to the current block can be a block that touches the boundary of the current block, or a block located within a predetermined distance from the current block. A neighboring block can also represent a block adjacent to a vertex of the current block. Here, a block adjacent to a vertex of the current block can be a block that is vertically adjacent to a block horizontally adjacent to the current block, or a block that is horizontally adjacent to a block vertically adjacent to the current block.

[0078] Reconstructed neighboring blocks: These can represent neighboring blocks that are adjacent to the current block and have already been spatially / temporally encoded or decoded. Here, reconstructed neighboring blocks can represent reconstructed neighboring units. Reconstructed spatial neighboring blocks can be blocks within the current frame that have already been reconstructed through encoding or decoding, or both. Reconstructed temporally neighboring blocks are blocks within a reference image that are located at the position corresponding to the current block in the current frame, or neighboring blocks of said block.

[0079] Cell depth: Represents the degree of cell partitioning. In a tree structure, the highest node (root node) corresponds to the first cell that is not partitioned. Furthermore, the highest node can have the minimum depth value. In this case, the depth of the highest node can be level 0. A node with a depth of level 1 represents a cell generated by partitioning the first cell once. A node with a depth of level 2 represents a cell generated by partitioning the first cell twice. A node with a depth of level n represents a cell generated by partitioning the first cell n times. Leaf nodes can be the lowest-level nodes and cannot be further partitioned. The depth of a leaf node can be the highest-level. For example, the predefined value for the highest-level can be 3. The root node can have the lowest depth, and the leaf nodes can have the deepest depth. Additionally, when cells are represented as a tree structure, the level at which the cell exists can represent the cell depth.

[0080] Bitstream: A bitstream that can represent encoded image information.

[0081] Parameter set: Corresponds to the header information in the configuration within the bitstream. At least one of the video parameter set, sequence parameter set, frame parameter set, and adaptive parameter set may be included in the parameter set. In addition, the parameter set may include slice header and tile header information.

[0082] Explanation: This could mean determining the value of a syntax element by performing entropy decoding, or it could mean entropy decoding itself.

[0083] Symbols: can represent at least one of the syntax elements, encoding parameters, and transform coefficient values ​​of the encoding / decoding target unit. Additionally, symbols can represent entropy encoding targets or entropy decoding results.

[0084] Prediction mode: This can be information indicating the mode of intra-frame predictive coding / decoding or the mode of inter-frame predictive coding / decoding.

[0085] Prediction Unit: A basic unit that can be represented when performing prediction (such as inter-frame prediction, intra-frame prediction, inter-frame compensation, intra-frame compensation, and motion compensation). A single prediction unit can be partitioned into multiple partitions with smaller sizes, or it can be partitioned into multiple lower-level prediction units. Multiple partitions can be the basic units when performing prediction or compensation. Partitions generated by dividing prediction units can also be prediction units.

[0086] Prediction cell partitioning: can represent the shape obtained by partitioning prediction cells.

[0087] Reference frame list: This can refer to a list of one or more reference frames used for inter-frame prediction or motion compensation. Several types of available reference frame lists exist, including LC (list combination), L0 (list 0), L1 (list 1), L2 (list 2), and L3 (list 3).

[0088] Inter-frame prediction indicator: This can indicate the direction of inter-frame prediction (one-way prediction, two-way prediction, etc.) for the current block. Optionally, the inter-frame prediction indicator can indicate the number of reference frames used to generate the prediction blocks for the current block. Optionally, the inter-frame prediction indicator can indicate the number of prediction blocks used when performing inter-frame prediction or motion compensation on the current block.

[0089] Prediction list utilization flag: Indicates whether at least one reference frame from a specific reference frame list is used to generate the prediction block. The prediction list utilization flag can be used to derive the inter-frame prediction indicator, and conversely, the inter-frame prediction indicator can be used to derive the prediction list utilization flag. For example, when the prediction list utilization flag has a first value of zero (0), it indicates that reference frames from the reference frame list are not used to generate the prediction block. On the other hand, when the prediction list utilization flag has a second value of one (1), it indicates that the reference frame list is used to generate the prediction block.

[0090] Reference screen index: can refer to the index of a specific reference screen in the reference screen list.

[0091] Reference frame: This can refer to a reference frame used by a specific block for inter-frame prediction or motion compensation of that block. Alternatively, a reference frame can be a frame that includes a reference block used by the current block for inter-frame prediction or motion compensation. In the following text, the terms "reference frame" and "reference image" have the same meaning and are used interchangeably.

[0092] Motion vector: This can be a two-dimensional vector used for inter-frame prediction or motion compensation. A motion vector can represent the offset between the encoded / decoded target block and the reference block. For example, (mvX, mvY) can represent a motion vector. Here, mvX can represent the horizontal component, and mvY can represent the vertical component.

[0093] Search range: This can be a two-dimensional region searched during inter-frame prediction to retrieve motion vectors. For example, the size of the search range can be M×N. Here, M and N are both integers.

[0094] Motion vector candidate: This can refer to a prediction candidate block or the motion vector of a prediction candidate block when predicting motion vectors. Furthermore, motion vector candidates can be included in a motion vector candidate list.

[0095] Candidate list of motion vectors: can represent a list consisting of one or more candidates of motion vectors.

[0096] Motion vector candidate index: This can represent an indicator that points to a motion vector candidate in the motion vector candidate list. Alternatively, the motion vector candidate index can be an index of the motion vector predictor.

[0097] Motion information: may represent information including at least one of the following items, said items including motion vector, reference frame index, inter-frame prediction indicator, prediction list utilization flag, reference frame list information, reference frame, motion vector candidate, motion vector candidate index, merge candidate and merge index.

[0098] Merge candidate list: can represent a list consisting of one or more merge candidates.

[0099] Merge Candidates: These can represent spatial merge candidates, temporal merge candidates, combined merge candidates, combined bidirectional prediction merge candidates, or zero merge candidates. Merge candidates may include motion information such as inter-frame prediction indicators, reference frame indices for each list, motion vectors, prediction list utilization flags, and inter-frame prediction indicators.

[0100] Merge Index: Can represent an indicator that points to a merge candidate in the merge candidate list. Optionally, the merge index can indicate a block from which a merge candidate has been derived among the reconstructed blocks that are spatially / temporally adjacent to the current block. Optionally, the merge index can indicate at least one piece of motion information for a merge candidate.

[0101] Transform unit: This can represent the basic unit used when performing encoding / decoding (such as transform, inverse transform, quantization, dequantization, transform coefficient encoding / decoding) on ​​the residual signal. A single transform unit can be partitioned into multiple lower-level transform units with smaller sizes. Here, the transform / inverse transform may include at least one of a first transform / first inverse transform and a second transform / second inverse transform.

[0102] Scaling: This refers to the process of multiplying the quantization level by a factor. Transform coefficients can be generated by scaling the quantization level. Scaling can also be called inverse quantization.

[0103] Quantization parameters: These represent values ​​used when generating quantization levels using transform coefficients during quantization. Quantization parameters can also represent values ​​used when generating transform coefficients by scaling the quantization levels during dequantization. Quantization parameters can be values ​​mapped to the quantization step size.

[0104] Incremental quantization parameter: can represent the difference between the predicted quantization parameter and the quantization parameter of the encoding / decoding target unit.

[0105] Scan: This can refer to a method of sorting coefficients within a cell, block, or matrix. For example, changing a two-dimensional matrix of coefficients into a one-dimensional matrix can be called a scan, and changing a one-dimensional matrix of coefficients into a two-dimensional matrix can be called a scan or inverse scan.

[0106] Transform coefficients: These represent the coefficient values ​​generated after a transform is performed in the encoder. Transform coefficients can also represent the coefficient values ​​generated after at least one of entropy decoding and dequantization is performed in the decoder. The quantization level obtained by quantizing the transform coefficients or residual signal, or the quantized transform coefficient level, can also fall within the meaning of transform coefficients.

[0107] Quantization level: This can represent the value generated in the encoder by quantizing the transform coefficients or residual signal. Alternatively, the quantization level can represent the value of the dequantization target to be dequantized in the decoder. Similarly, the transform coefficient level, as a result of transform and quantization, can also fall within the meaning of quantization level.

[0108] Non-zero transform coefficients: can represent transform coefficients with values ​​other than zero, or transform coefficient levels or quantization levels with values ​​other than zero.

[0109] Quantization matrix: A matrix used in quantization or dequantization processes to improve subjective or objective image quality. The quantization matrix can also be referred to as a scaling list.

[0110] Quantization matrix coefficients: These represent each element within the quantization matrix. Quantization matrix coefficients can also be called matrix coefficients.

[0111] Default matrix: can represent a predefined quantization matrix in the encoder or decoder.

[0112] Non-default matrix: can represent a quantization matrix that is not predefined in the encoder or decoder but is sent by the user via signal.

[0113] Statistical value: For at least one of the following variables, coding parameters, constant values, etc., that have a computable specific value, a statistical value can be one or more of the following: mean, weighted average, weighted sum, minimum, maximum, most frequent value, median, interpolation value.

[0114] Figure 1 This is a block diagram illustrating the configuration of an encoding device according to an embodiment of the present invention.

[0115] Encoding device 100 may be an encoder, a video encoding device, or an image encoding device. The video may include at least one image. Encoding device 100 may encode at least one image sequentially.

[0116] Reference Figure 1 The encoding device 100 may include a motion prediction unit 111, a motion compensation unit 112, an intra-frame prediction unit 120, a switcher 115, a subtractor 125, a transform unit 130, a quantization unit 140, an entropy coding unit 150, an inverse quantization unit 160, an inverse transform unit 170, an adder 175, a filter unit 180, and a reference frame buffer 190.

[0117] Encoding device 100 can encode the input image using intra-frame mode, inter-frame mode, or both. Furthermore, encoding device 100 can generate a bitstream including encoding information by encoding the input image and output the generated bitstream. The generated bitstream can be stored in a computer-readable recording medium or streamed via a wired / wireless transmission medium. When intra-frame mode is used as the prediction mode, switcher 115 can switch to intra-frame mode. Alternatively, when inter-frame mode is used as the prediction mode, switcher 115 can switch to inter-frame mode. Here, intra-frame mode can refer to intra-frame prediction mode, and inter-frame mode can refer to inter-frame prediction mode. Encoding device 100 can generate prediction blocks for input blocks of the input image. Furthermore, encoding device 100 can encode residual blocks using the residual between the input block and the prediction block after generating the prediction blocks. The input image can be referred to as the current image as the current encoding target. The input block can be referred to as the current block as the current encoding target, or as the encoding target block.

[0118] When the prediction mode is intra-frame mode, the intra-frame prediction unit 120 can use samples from blocks that have been encoded / decoded and are adjacent to the current block as reference samples. The intra-frame prediction unit 120 can perform spatial prediction on the current block using the reference samples, or generate prediction samples for the input block by performing spatial prediction. Here, intra-frame prediction can refer to prediction within a frame.

[0119] When the prediction mode is inter-frame mode, the motion prediction unit 111 can retrieve the region that best matches the input block from the reference image during motion prediction and derive the motion vector using the retrieved region. In this case, the search region can be used as the region. The reference image can be stored in the reference frame buffer 190. Here, the reference image can be stored in the reference frame buffer 190 when encoding / decoding the reference image is performed.

[0120] The motion compensation unit 112 can generate a prediction block by performing motion compensation on the current block using motion vectors. Here, inter-frame prediction can refer to prediction or motion compensation between frames.

[0121] When the value of the motion vector is not an integer, the motion prediction unit 111 and the motion compensation unit 112 can generate prediction blocks by applying an interpolation filter to a portion of the reference frame. To perform inter-frame prediction or motion compensation on the coding unit, it can be determined which mode—skip mode, merge mode, advanced motion vector prediction (AMVP) mode, or current frame reference mode—will be used for motion prediction and motion compensation on the prediction unit included in the corresponding coding unit. Then, depending on the determined mode, inter-frame prediction or motion compensation can be performed differently.

[0122] Subtractor 125 can generate a residual block by using the residual between the input block and the prediction block. The residual block can be referred to as a residual signal. The residual signal can represent the difference between the initial signal and the prediction signal. Furthermore, the residual signal can be a signal generated by transforming or quantizing, or transforming and quantizing, the difference between the initial signal and the prediction signal. The residual block can be the residual signal of a block cell.

[0123] Transform unit 130 can generate transform coefficients by performing a transform on the residual block and output the generated transform coefficients. Here, the transform coefficients can be coefficient values ​​generated by performing a transform on the residual block. When a transform skip mode is applied, transform unit 130 can skip the transform on the residual block.

[0124] The level of quantization can be generated by applying quantization to the transform coefficients or to the residual signal. In the following examples, the level of quantization may also be referred to as the transform coefficients.

[0125] The quantization unit 140 can generate a quantization level by quantizing the transform coefficients or residual signal according to parameters, and output the generated quantization level. Here, the quantization unit 140 can quantize the transform coefficients using a quantization matrix.

[0126] Entropy coding unit 150 can generate a bitstream by performing entropy coding on the values ​​calculated by quantization unit 140 according to a probability distribution or on the coding parameter values ​​calculated during encoding, and output the generated bitstream. Entropy coding unit 150 can perform entropy coding on sample information of the image and information used for decoding the image. For example, the information used for decoding the image may include syntax elements.

[0127] When entropy coding is applied, symbols are represented such that fewer bits are allocated to symbols with high generation probability and more bits are allocated to symbols with low generation probability, thus reducing the size of the bitstream used to encode the symbols. The entropy coding unit 150 can use coding methods for entropy coding such as Exponential Golomb, Context Adaptive Variable Length Coding (CAVLC), and Context Adaptive Binary Arithmetic Coding (CABAC). For example, the entropy coding unit 150 can perform entropy coding by using a variable-length code (VLC) table. Furthermore, the entropy coding unit 150 can derive a binarization method for the target symbol and a probability model for the target symbol / bits, and perform arithmetic coding by using the derived binarization method and context model.

[0128] In order to encode the transform coefficient levels (quantization levels), the entropy coding unit 150 can change the two-dimensional block form coefficients into one-dimensional vector form by using the transform coefficient scanning method.

[0129] Encoding parameters may include information such as syntax elements (flags, indexes, etc.) encoded in the encoder and signaled to the decoder, as well as information derived during encoding or decoding. Encoding parameters can represent the information required when encoding or decoding an image. For example, at least one value or combination of the following may be included in the encoding parameters: cell / block size, cell / block depth, cell / block partitioning information, cell / block shape, cell / block partitioning structure, whether quadtree partitioning is performed, whether binary tree partitioning is performed, binary tree partitioning direction (horizontal or vertical), binary tree partitioning type (symmetric or asymmetric), whether the current encoding unit is partitioned via ternary tree partitioning, the direction of ternary tree partitioning (horizontal or vertical), the type of ternary tree partitioning (symmetric or asymmetric), whether the current encoding unit is partitioned via multi-type tree partitioning, and the direction of multi-type tree partitioning. The text discusses various aspects of prediction, including: orientation (horizontal or vertical), type of multi-type tree partitions (symmetric or asymmetric), tree structure (binary or ternary), prediction mode (intra-frame prediction or inter-frame prediction), intra-frame prediction mode / direction for luma and chroma, intra-frame partition information, inter-frame partition information, coded block partition flags, prediction block partition flags, transform block partition flags, reference sample filtering methods, reference sample filter taps, reference sample filter coefficients, prediction block filtering methods, prediction block filter taps, prediction block filter coefficients, prediction block boundary filtering methods, prediction block boundary filter taps, prediction block boundary filter coefficients, and intra-frame prediction mode. Inter-frame prediction mode, motion information, motion vector, motion vector difference, reference frame index, inter-frame prediction angle, inter-frame prediction indicator, prediction list utilization flag, reference frame list, reference frame, motion vector predictor index, motion vector predictor candidate, motion vector candidate list, whether to use merge mode, merge index, merge candidate, merge candidate list, whether to use skip mode, interpolation filter type, interpolation filter taps, interpolation filter coefficients, motion vector magnitude, motion vector representation accuracy, transform type, transform size, information on whether the primary (first) transform is used, information on whether the secondary transform is used, primary transform index, secondary transform index Information on the presence of residual signals, code block style, code block flag (CBF), quantization parameters, quantization parameter residuals, quantization matrix, whether an intra-loop filter is applied, intra-loop filter coefficients, intra-loop filter taps, intra-loop filter shape / form, whether a deblocking filter is applied, deblocking filter coefficients, deblocking filter taps, deblocking filter strength, deblocking filter shape / form, whether adaptive sample offset is applied, adaptive sample offset value, adaptive sample offset category, adaptive sample offset type, whether an adaptive loop filter is applied, adaptive loop filter coefficients, adaptive loop filter taps, adaptive loop filter shape / form.Binarization / inverse binarization method, context model determination method, context model update method, whether to execute normal mode, whether to execute bypass mode, context binary bits, bypass binary bits, valid coefficient flag, last valid coefficient flag, encoding flag for the unit of the coefficient group, position of the last valid coefficient, flag indicating whether the coefficient value is greater than 1, flag indicating whether the coefficient value is greater than 2, flag indicating whether the coefficient value is greater than 3, information about the remaining coefficient values, symbol information, reconstructed luminance samples, reconstructed chrominance samples, residual luminance samples, residual chrominance samples, luminance transformation coefficient, chrominance transformation coefficient, quantized luminance level, quantized chrominance level, transformation coefficient level scanning method The following parameters are considered: the size of the motion vector search region on the decoder side, the shape of the motion vector search region on the decoder side, the number of motion vector searches on the decoder side, information about the CTU size, information about the minimum block size, information about the maximum block size, information about the maximum block depth, information about the minimum block depth, image display / output order, stripe identification information, stripe type, stripe partition information, parallel block identification information, parallel block type, parallel block partition information, picture type, bit depth of input samples, bit depth of reconstructed samples, bit depth of residual samples, bit depth of transform coefficients, bit depth of quantization levels, and information about the luminance signal or the chrominance signal.

[0130] Here, sending a flag or index with a signal can indicate that the encoder entropy-encodes the corresponding flag or index and includes it in the bitstream, and can also indicate that the decoder entropy-decodes the corresponding flag or index from the bitstream.

[0131] When the encoding device 100 performs encoding via inter-frame prediction, the encoded current image can be used as a reference image for another image to be processed subsequently. Therefore, the encoding device 100 can reconstruct or decode the encoded current image, or store the reconstructed or decoded image as a reference image in the reference frame buffer 190.

[0132] The quantization level can be dequantized in dequantization unit 160 or inverse transformed in inverse transform unit 170. The coefficients that have undergone dequantization or inverse transform, or both, can be added to the prediction block by adder 175. A reconstructed block can be generated by adding the coefficients that have undergone dequantization or inverse transform, or both, to the prediction block. Here, the coefficients that have undergone dequantization or inverse transform, or both, can represent coefficients for which at least one of dequantization and inverse transform has been performed, and can represent the reconstructed residual block.

[0133] The reconstructed block can be passed through filter unit 180. Filter unit 180 can apply at least one of deblocking filter, sample adaptive offset (SAO), and adaptive loop filter (ALF) to the reconstructed sample, reconstructed block, or reconstructed image. Filter unit 180 may be referred to as an in-loop filter.

[0134] Deblocking filters remove block distortion generated at the boundaries between blocks. To determine whether to apply a deblocking filter, the number of samples included in several rows or columns within the block can be used. When a deblocking filter is applied to a block, another filter can be applied based on the desired deblocking intensity.

[0135] To compensate for coding errors, a suitable offset value can be added to the sample value using a sample-adaptive offset. The sample-adaptive offset corrects the offset between the deblocked image and the original image on a sample-by-sample basis. This can be achieved by considering edge information about each sample point when applying the offset, or by dividing the image's samples into a predetermined number of regions, determining the regions where the offset will be applied, and then applying the offset to those regions.

[0136] An adaptive loop filter (ALF) can perform filtering based on a comparison between the filtered reconstructed image and the original image. Samples included in the image can be partitioned into predetermined groups, the filter to be applied to each group can be determined, and differential filtering can be performed on each group. Information regarding whether to apply the ALF can be transmitted via a signal through the coding unit (CU), and the form and coefficients of the ALF to be applied to each block can vary.

[0137] The reconstructed blocks or reconstructed image that have passed through filter unit 180 can be stored in reference frame buffer 190. The reconstructed blocks processed by filter unit 180 can be a portion of the reference image. That is, the reference image is a reconstructed image composed of the reconstructed blocks processed by filter unit 180. The stored reference image can be used later for inter-frame prediction or motion compensation.

[0138] Figure 2 This is a block diagram illustrating the configuration of a decoding device according to an embodiment of the present invention.

[0139] Decoding device 200 can be a decoder, video decoding device, or image decoding device.

[0140] Reference Figure 2 The decoding device 200 may include an entropy decoding unit 210, an inverse quantization unit 220, an inverse transform unit 230, an intra-frame prediction unit 240, a motion compensation unit 250, an adder 225, a filter unit 260, and a reference frame buffer 270.

[0141] Decoding device 200 can receive bitstreams output from encoding device 100. Decoding device 200 can receive bitstreams stored on a computer-readable recording medium, or bitstreams streamed via wired / wireless transmission media. Decoding device 200 can decode the bitstreams using intra-frame mode or inter-frame mode. Furthermore, decoding device 200 can generate and output reconstructed or decoded images generated through decoding.

[0142] When the prediction mode used during decoding is intra-frame mode, the switcher can be switched to intra-frame mode. Alternatively, when the prediction mode used during decoding is inter-frame mode, the switcher can be switched to inter-frame mode.

[0143] Decoding device 200 obtains a reconstructed residual block and generates a prediction block by decoding the input bitstream. Once the reconstructed residual block and prediction block are obtained, decoding device 200 generates a reconstructed block that becomes the decoding target by adding the reconstructed residual block and the prediction block. The decoding target block can be referred to as the current block.

[0144] The entropy decoding unit 210 can generate symbols by performing entropy decoding on the bitstream according to a probability distribution. The generated symbols may include symbols in quantized hierarchical form. Here, the entropy decoding method can be the inverse process of the entropy encoding method described above.

[0145] In order to decode the transform coefficient levels (quantization levels), the entropy decoding unit 210 can change the unidirectional vector form coefficients into two-dimensional block form by using the transform coefficient scanning method.

[0146] The quantization level can be dequantized in the dequantization unit 220, or inversely transformed in the inverse transform unit 230. The quantization level can be the result of dequantization, inverse transformation, or both, and can be generated as a reconstructed residual block. Here, the dequantization unit 220 can apply the quantization matrix to the quantization level.

[0147] When using intra-frame mode, intra-frame prediction unit 240 can generate a prediction block by performing spatial prediction on the current block, wherein the spatial prediction uses sample values ​​of blocks that are adjacent to the target block and have already been decoded.

[0148] When using inter-frame mode, motion compensation unit 250 can generate a prediction block by performing motion compensation on the current block, wherein the motion compensation uses motion vectors and a reference image stored in reference frame buffer 270.

[0149] Adder 225 generates a reconstructed block by adding the reconstructed residual block to the prediction block. Filter unit 260 can apply at least one of a deblocking filter, a sample adaptive offset, and an adaptive loop filter to the reconstructed block or reconstructed image. Filter unit 260 can output a reconstructed image. The reconstructed block or reconstructed image can be stored in a reference frame buffer 270 and used when performing inter-frame prediction. The reconstructed block processed by filter unit 260 can be a portion of the reference image. That is, the reference image is a reconstructed image composed of the reconstructed blocks processed by filter unit 260. The stored reference image can be used later during inter-frame prediction or motion compensation.

[0150] Figure 3 It is a schematic diagram illustrating the partitioning structure of an image when it is encoded and decoded. Figure 3 An example of partitioning a single cell into multiple lower-level cells is illustrated.

[0151] To effectively partition an image, coding units (CUs) are used during encoding and decoding. A coding unit can serve as the basic unit when encoding / decoding an image. Furthermore, a coding unit can be used to distinguish between intra-frame prediction modes and inter-frame prediction modes during image encoding / decoding. A coding unit can be the basic unit for prediction, transform, quantization, inverse transform, inverse quantization, or encoding / decoding of transform coefficients.

[0152] Reference Figure 3 Image 300 is partitioned sequentially according to the Largest Coding Unit (LCU), and the LCU unit is determined as the partitioning structure. Here, LCU can be used with the same meaning as Coding Tree Unit (CTU). Unit partitioning can represent partitioning of the block associated with that unit. The block partitioning information may include information about the unit depth. The depth information may represent the number or degree to which the unit is partitioned, or both the number and degree to which the unit is partitioned. A single unit can be partitioned into multiple lower-level units hierarchically associated with the depth information based on a tree structure. In other words, the unit and the lower-level units generated by partitioning that unit may correspond to a node and the child nodes of that node, respectively. Each of the partitioned lower-level units may have depth information. The depth information may be information representing the size of the CU and may be stored in each CU. The unit depth represents the number and / or degree associated with partitioning the unit. Therefore, the partitioning information of the lower-level units may include information about the size of the lower-level units.

[0153] The partitioning structure represents the distribution of coding units (CUs) within the LCU 310. This distribution can be determined by whether a single CU is partitioned into multiple CUs (positive integers equal to or greater than 2, including 2, 4, 8, 16, etc.). The horizontal and vertical dimensions of the CUs generated by partitioning can be half the horizontal and vertical dimensions of the CU before partitioning, respectively, or they can have dimensions smaller than the horizontal and vertical dimensions before partitioning, depending on the number of partitions. CUs can be recursively partitioned into multiple CUs. Through recursive partitioning, at least one of the height and width of the CU after partitioning can be reduced compared to at least one of the height and width of the CU before partitioning. CU partitioning can be performed recursively until a predefined depth or a predefined size is reached. For example, the depth of the LCU can be 0, and the depth of the minimum coding unit (SCU) can be a predefined maximum depth. Here, as mentioned above, the LCU can be the coding unit with the maximum coding unit size, and the SCU can be the coding unit with the minimum coding unit size. Partitioning begins at the LCU 310, and the CU depth increases by 1 as the horizontal or vertical dimension of the CU, or both the horizontal and vertical dimensions, decrease through partitioning. For example, for each depth, the size of an unpartitioned CU can be 2N×2N. Furthermore, in the case of partitioned CUs, a CU of size 2N×2N can be partitioned into four CUs of size N×N. When the depth increases by 1, the size of N can be halved.

[0154] Furthermore, partition information of a CU can be used to indicate whether a CU is partitioned. Partition information can be 1 bit. All CUs except SCUs can include partition information. For example, when the partition information value is 1, the CU may not be partitioned; when the partition information value is 2, the CU may be partitioned.

[0155] Reference Figure 3 An LCU with depth 0 can be a 64×64 block. 0 can be the minimum depth. An SCU with depth 3 can be an 8×8 block. 3 can be the maximum depth. CUs with 32×32 blocks and 16×16 blocks can be represented as depth 1 and depth 2, respectively.

[0156] For example, when a single coding unit is partitioned into four coding units, the horizontal and vertical dimensions of the four partitioned coding units can be half the size of the original horizontal and vertical dimensions of the CU before partitioning. In one embodiment, when a 32×32 coding unit is partitioned into four coding units, the size of each of the four partitioned coding units can be 16×16. When a single coding unit is partitioned into four coding units, it can be said that the coding unit can be partitioned into a quadtree form.

[0157] For example, when a coding unit is partitioned into two sub-coding units, the horizontal or vertical dimension (width or height) of each of the two sub-coding units can be half the horizontal or vertical dimension of the original coding unit. For example, when a coding unit of size 32×32 is vertically partitioned into two sub-coding units, each of the two sub-coding units can have a size of 16×32. For example, when a coding unit of size 8×32 is horizontally partitioned into two sub-coding units, each of the two sub-coding units can have a size of 8×16. When a coding unit is partitioned into two sub-coding units, it can be said that the coding unit is binary partitioned, or partitioned according to a binary tree partitioning structure.

[0158] For example, when a coding unit is divided into three sub-coding units, the horizontal or vertical dimensions of the coding unit can be divided in a 1:2:1 ratio, resulting in three sub-coding units with a horizontal or vertical dimension ratio of 1:2:1. For instance, when a 16×32 coding unit is horizontally divided into three sub-coding units, the three sub-coding units, in order from the top to the bottom, can have dimensions of 16×8, 16×16, and 16×8 respectively. Similarly, when a 32×32 coding unit is vertically divided into three sub-coding units, the three sub-coding units, in order from the left to the right, can have dimensions of 8×32, 16×32, and 8×32 respectively. When a coding unit is divided into three sub-coding units, it can be said that the coding unit is tri-partitioned or partitioned according to a ternary tree partitioning structure.

[0159] exist Figure 3 In the example, the coding tree unit (CTU) 320 is a CTU that applies quadtree partitioning, binary tree partitioning, and ternary tree partitioning structures.

[0160] As described above, to partition the CTU, at least one of a quadtree partitioning structure, a binary tree partitioning structure, and a ternary tree partitioning structure can be applied. Various tree partitioning structures can be applied sequentially to the CTU according to a predetermined priority order. For example, a quadtree partitioning structure can be preferentially applied to the CTU. Encoding units that cannot be further partitioned using a quadtree partitioning structure can correspond to leaf nodes of a quadtree. Encoding units corresponding to leaf nodes of a quadtree can be used as root nodes of binary and / or ternary tree partitioning structures. That is, encoding units corresponding to leaf nodes of a quadtree can be further partitioned according to a binary or ternary tree partitioning structure, or they can be left unpartitioned. Therefore, by preventing the encoded blocks generated by binary or ternary tree partitioning of encoding units corresponding to leaf nodes of a quadtree from being further quadtree partitioned, block partitioning and / or signaling of partition information can be effectively performed.

[0161] The fact that a coding unit corresponding to a node in a quadtree is partitioned can be signaled using four-partition information. Four-partition information with a first value (e.g., "1") indicates that the current coding unit is partitioned according to the quadtree partitioning structure. Four-partition information with a second value (e.g., "0") indicates that the current coding unit is not partitioned according to the quadtree partitioning structure. The four-partition information can be a flag with a predetermined length (e.g., one bit).

[0162] There may be no priority between binary tree partitions and ternary tree partitions. That is, the coding unit corresponding to the leaf node of the quadtree can be further partitioned by either binary tree partitions or ternary tree partitions. In addition, the coding unit generated by binary tree partitions or ternary tree partitions may be further partitioned by binary tree partitions or ternary tree partitions, or may not be further partitioned.

[0163] A tree structure in which there is no priority in binary and ternary tree partitioning is called a multi-type tree structure. The coding unit corresponding to the leaf node of a quadtree can be used as the root node of a multi-type tree. At least one of multi-type tree partitioning indication information, partitioning direction information, and partitioning tree information can be used to signal whether to partition the coding unit corresponding to a node in the multi-type tree. To partition the coding unit corresponding to a node in the multi-type tree, the multi-type tree partitioning indication information, partitioning direction, and partitioning tree information can be signaled sequentially.

[0164] A multi-type tree partitioning indication with a first value (e.g., "1") indicates that the current coding unit will be subjected to multi-type tree partitioning. A multi-type tree partitioning indication with a second value (e.g., "0") indicates that the current coding unit will not be subjected to multi-type tree partitioning.

[0165] When a coding unit corresponding to a node of a multi-type tree is further partitioned according to the multi-type tree partitioning structure, the coding unit may include partitioning direction information. The partitioning direction information indicates in which direction the current coding unit will be partitioned according to the multi-type tree partitioning. Partitioning direction information with a first value (e.g., "1") indicates that the current coding unit will be vertically partitioned. Partitioning direction information with a second value (e.g., "0") indicates that the current coding unit will be horizontally partitioned.

[0166] When the coding unit corresponding to a node of a multi-type tree is further partitioned according to the multi-type tree partitioning structure, the current coding unit may include partitioning tree information. The partitioning tree information may indicate the tree partitioning structure that will be used to partition the nodes of the multi-type tree. Partitioning tree information with a first value (e.g., "1") may indicate that the current coding unit will be partitioned according to a binary tree partitioning structure. Partitioning tree information with a second value (e.g., "0") may indicate that the current coding unit will be partitioned according to a ternary tree partitioning structure.

[0167] Partition indication information, partition tree information, and partition direction information can all be flags with a predetermined length (e.g., one bit).

[0168] At least one of the following—quadtree partitioning indication information, multi-type tree partitioning indication information, partitioning direction information, and partitioning tree information—can be entropy encoded / decoded. To entropy encode / decode those types of information, information about neighboring coding units adjacent to the current coding unit can be used. For example, there is a high probability that the partitioning type (partitioned or unpartitioned, partitioning tree, and / or partitioning direction) of the left-hand neighboring coding unit and / or the upper-hand neighboring coding unit of the current coding unit is similar to the partitioning type of the current coding unit. Therefore, contextual information for entropy encoding / decoding of information about the current coding unit can be derived from the information about neighboring coding units. Information about neighboring coding units may include at least one of the following: quadtree partitioning information, multi-type tree partitioning indication information, partitioning direction information, and partitioning tree information.

[0169] As another example, in binary tree partitioning and ternary tree partitioning, binary tree partitioning can be performed first. That is, the current coding unit can be preferentially partitioned into a binary tree, and subsequently, the coding unit corresponding to the leaf node of the binary tree can be set as the root node for ternary tree partitioning. In this case, for the coding unit corresponding to the node of the ternary tree, neither quadtree partitioning nor binary tree partitioning can be performed.

[0170] Encoding units that cannot be partitioned according to quadtree, binary tree, and / or ternary tree partitioning structures become basic units for encoding, prediction, and / or transformation. In other words, the encoding units cannot be further partitioned for prediction and / or transformation. Therefore, partitioning structure information and partitioning information for dividing encoding units into prediction and / or transformation units may not exist in the bitstream.

[0171] However, when the size of the coding unit (i.e., the basic unit used for partitioning) is larger than the size of the maximum transform block, the coding unit can be partitioned recursively until the size of the coding unit is reduced to be equal to or smaller than the size of the maximum transform block. For example, when the size of the coding unit is 64×64 and the size of the maximum transform block is 32×32, the coding unit can be partitioned into four 32×32 blocks for transformation. For example, when the size of the coding unit is 32×64 and the size of the maximum transform block is 32×32, the coding unit can be partitioned into two 32×32 blocks for transformation. In this case, the partitions of the coding unit for transformation are not sent separately by signal, and the partitions of the coding unit for transformation can be determined by comparing the horizontal or vertical dimensions of the coding unit with the horizontal or vertical dimensions of the maximum transform block. For example, when the horizontal dimension (width) of the coding unit is greater than the horizontal dimension (width) of the maximum transform block, the coding unit can be vertically divided into two. For example, when the vertical dimension (length) of the coding unit is greater than the vertical dimension (length) of the maximum transform block, the coding unit can be horizontally divided into two.

[0172] Information regarding the maximum and / or minimum size of the coding unit and the maximum and / or minimum size of the transform block can be transmitted or determined at a higher level than the coding unit. This higher level can be, for example, the sequence level, the picture level, or the stripe level. For example, the minimum size of the coding unit can be determined to be 4×4. For example, the maximum size of the transform block can be determined to be 64×64. For example, the minimum size of the transform block can be determined to be 4×4.

[0173] Information regarding the minimum size of the coding unit corresponding to the leaf node of the quadtree (minimum size of the quadtree) and / or the maximum depth of the multi-type tree from the root node to the leaf node (maximum depth of the multi-type tree) can be signaled or determined at a higher level of the coding unit. For example, the higher level could be the sequence level, the frame level, or the strip level. Information regarding the minimum size of the quadtree and / or the maximum depth of the multi-type tree can be signaled or determined for each of the intra-frame and inter-frame frame stripes.

[0174] The difference information between the size of the CTU and the maximum size of the transform block can be signaled or determined at a higher level of the coding unit. For example, the higher level could be the sequence level, the frame level, or the stripe level. The maximum size of the coding unit corresponding to each node of the binary tree (hereinafter referred to as the maximum size of the binary tree) can be determined based on the size and difference information of the coding tree units. The maximum size of the coding unit corresponding to each node of the ternary tree (hereinafter referred to as the maximum size of the ternary tree) can vary depending on the type of stripe. For example, for an intra-frame stripe, the maximum size of the ternary tree could be 32×32. For example, for an inter-frame stripe, the maximum size of the ternary tree could be 128×128. For example, the minimum size of the coding unit corresponding to each node of the binary tree (hereinafter referred to as the minimum size of the binary tree) and / or the minimum size of the coding unit corresponding to each node of the ternary tree (hereinafter referred to as the minimum size of the ternary tree) can be set as the minimum size of the coding block.

[0175] As another example, the maximum size of a binary tree and / or the maximum size of a ternary tree can be signaled or determined at the stripe level. Alternatively, the minimum size of a binary tree and / or the minimum size of a ternary tree can be signaled or determined at the stripe level.

[0176] Based on the size and depth information of the various blocks mentioned above, the four-partition information, multi-type tree partition indication information, partition tree information and / or partition direction information may or may not be included in the bitstream.

[0177] For example, when the size of the coding unit is no greater than the minimum size of the quadtree, the coding unit does not contain four-partition information. Therefore, the four-partition information can be derived from the second value.

[0178] For example, when the size (horizontal and vertical dimensions) of the coding unit corresponding to a node of a multi-type tree is greater than the maximum size (horizontal and vertical dimensions) of a binary tree and / or the maximum size (horizontal and vertical dimensions) of a ternary tree, the coding unit may not be divided into two or three partitions. Therefore, multi-type tree partitioning indication information can be derived from a second value instead of being sent by signal.

[0179] Alternatively, when the size (horizontal and vertical dimensions) of the coding unit corresponding to a node of a multi-type tree is the same as the maximum size (horizontal and vertical dimensions) of a binary tree, and / or twice the maximum size (horizontal and vertical dimensions) of a ternary tree, the coding unit may not be further divided into two or three partitions. Therefore, multi-type tree partitioning indication information can be derived from a second value instead of being sent via signal transmission. This is because when the coding unit is partitioned according to the binary tree partitioning structure and / or the ternary tree partitioning structure, coding units smaller than the minimum size of the binary tree and / or the minimum size of the ternary tree are generated.

[0180] Alternatively, when the depth of the coding unit corresponding to a node in the multi-type tree is equal to the maximum depth of the multi-type tree, further binary and / or ternary partitioning of the coding unit is not required. Therefore, multi-type tree partitioning indication information can be derived from a second value instead of being sent via signal transmission.

[0181] Alternatively, multi-type tree partitioning indication information may be signaled only if at least one of the vertical binary tree partitioning, horizontal binary tree partitioning, vertical ternary tree partitioning, and horizontal ternary tree partitioning is feasible for the coding unit corresponding to the node of the multi-type tree. Otherwise, it may not be possible to partition the coding unit into two and / or three partitions. Therefore, multi-type tree partitioning indication information may not be signaled, but rather derived from a second value.

[0182] Alternatively, partition direction information may be signaled only if both vertical binary tree partitioning and horizontal binary tree partitioning, or both vertical ternary tree partitioning and horizontal ternary tree partitioning, are feasible for the coding units corresponding to nodes of multiple tree types. Otherwise, partition direction information may not be signaled, but may be derived from values ​​indicating possible partition directions.

[0183] Optionally, partition tree information may be signaled only if both vertical binary tree partitioning and vertical ternary tree partitioning, or both horizontal binary tree partitioning and horizontal ternary tree partitioning, are feasible for the encoded tree corresponding to the nodes of the multi-type tree. Otherwise, partition tree information may not be signaled, but it may be derived from values ​​indicating possible partition tree structures.

[0184] Figure 4 This is a diagram illustrating an embodiment of inter-screen prediction processing.

[0185] exist Figure 4 In this context, rectangles can represent the image. Figure 4In the image, the arrow indicates the prediction direction. Based on the encoding type of the image, images can be classified into intra-frame images (I-frames), predictive images (P-frames), and bidirectional predictive images (B-frames).

[0186] I-frames can be encoded via intra-frame prediction when inter-frame prediction is not required. P-frames can be encoded via inter-frame prediction using a reference frame present in one direction (i.e., forward or backward) relative to the current block. B-frames can be encoded via inter-frame prediction using reference frames present in two directions (i.e., forward and backward) relative to the current block. When using inter-frame prediction, the encoder can perform inter-frame prediction or motion compensation, and the decoder can perform the corresponding motion compensation.

[0187] The following section will describe in detail an embodiment of inter-screen prediction.

[0188] Reference frames and motion information can be used to perform inter-frame prediction or motion compensation.

[0189] Motion information of the current block can be derived during inter-frame prediction by each of the encoding device 100 and the decoding device 200. The motion information of the current block can be derived using motion information of reconstructed neighboring blocks, motion information of co-located blocks (also called col blocks or co-position blocks), and / or motion information of blocks adjacent to the co-position block. A co-position block can represent a block that is spatially located at the same position as the current block within a previously reconstructed co-located frame (also called a col frame or co-position frame). A co-position frame can be one of one or more reference frames included in a list of reference frames.

[0190] The method for deriving motion information for the current block can be based on changes in the prediction mode of the current block. For example, prediction modes used for inter-frame prediction can include AMVP mode, merge mode, skip mode, and current frame reference mode. The merge mode can be called the motion merge mode.

[0191] For example, when AMVP is used as a prediction mode, at least one of the motion vectors of reconstructed neighboring blocks, co-located blocks, blocks adjacent to co-located blocks, and (0,0) motion vectors can be identified as motion vector candidates for the current block, and a motion vector candidate list is generated using these motion vector candidates. Motion vector candidates for the current block can be derived using the generated motion vector candidate list. Motion information for the current block can be determined based on the derived motion vector candidates. The motion vectors of co-located blocks or blocks adjacent to co-located blocks can be referred to as temporal motion vector candidates, and the motion vectors of reconstructed neighboring blocks can be referred to as spatial motion vector candidates.

[0192] Encoding device 100 can calculate the motion vector difference (MVD) between the motion vector of the current block and motion vector candidates, and can perform entropy encoding on the motion vector difference (MVD). Furthermore, encoding device 100 can perform entropy encoding on the motion vector candidate index and generate a bitstream. The motion vector candidate index can indicate the best motion vector candidate among the motion vector candidates included in the motion vector candidate list. Decoding device 200 can perform entropy decoding on the motion vector candidate index included in the bitstream, and can select motion vector candidates for the target block to be decoded from the motion vector candidates included in the motion vector candidate list by using the entropy-decoded motion vector candidate index. Furthermore, decoding device 200 can add the entropy-decoded MVD to the motion vector candidate extracted by entropy decoding, thereby deriving the motion vector of the target block to be decoded.

[0193] The bitstream may include a reference frame index indicating a reference frame. The reference frame index may be entropy encoded by the encoding device 100 and subsequently transmitted as a bitstream to the decoding device 200. The decoding device 200 may generate a predicted block for the decoded target block based on the derived motion vectors and the reference frame index information.

[0194] Another example of a method for deriving motion information for the current block could be a merge pattern. A merge pattern can represent a method for merging the motion of multiple blocks. A merge pattern can represent a pattern for deriving motion information for the current block from the motion information of neighboring blocks. When a merge pattern is applied, the motion information of reconstructed neighboring blocks and / or co-located blocks can be used to generate a list of merge candidates. Motion information may include at least one of motion vectors, reference frame indices, and inter-frame prediction indicators. Prediction indicators may indicate unidirectional prediction (L0 prediction or L1 prediction) or bidirectional prediction (L0 prediction and L1 prediction).

[0195] The merge candidate list can be a list of stored motion information. The motion information included in the merge candidate list can be at least one of zero merge candidates and new motion information, wherein the new motion information is a combination of motion information of a neighboring block adjacent to the current block (spatial merge candidate), motion information of a co-located block of the current block included in the reference frame (temporal merge candidate), and motion information existing in the merge candidate list.

[0196] Encoding device 100 can generate a bitstream by performing entropy encoding on at least one of a merge flag and a merge index, and can transmit the bitstream as a signal to decoding device 200. The merge flag may be information indicating whether a merge mode is performed for each block, and the merge index may be information indicating which neighboring block among the current block's neighboring blocks is the target block for merging. For example, neighboring blocks of the current block may include a left-hand neighboring block to the left of the current block, an upper-hand neighboring block positioned above the current block, and a time-adjacent neighboring block.

[0197] Skip mode can be a mode in which motion information of neighboring blocks is applied to the current block as is. When skip mode is applied, encoding device 100 can perform entropy encoding on information about which block's motion information will be used as the current block's motion information to generate a bitstream, and can send the bitstream to decoding device 200 as a signal. Encoding device 100 may not send syntax elements regarding at least one of motion vector difference information, coded block flags, and transform coefficient levels to decoding device 200 as a signal.

[0198] The current frame reference mode can represent the prediction mode used for the prediction of the previously reconstructed region within the current frame to which the current block belongs. Here, a vector can be used to specify the previously reconstructed region. Information indicating whether the current block will be encoded in the current frame reference mode can be encoded using the current block's reference frame index. A flag or index indicating whether the current block is a block encoded in the current frame reference mode can be signaled, and the flag or index can be derived based on the current block's reference frame index. When encoding the current block in the current frame reference mode, the current frame can be added to the reference frame list for the current block so that the current frame is located at a fixed position or an arbitrary position in the reference frame list. A fixed position can be, for example, the position indicated by reference frame index 0, or the last position in the list. When the current frame is added to the reference frame list so that the current frame is located at an arbitrary position, a reference frame index indicating an arbitrary position can be signaled.

[0199] Based on the above description, the image encoding method and image decoding method according to embodiments of the present invention will be described in detail below.

[0200] Figure 5 This is a flowchart illustrating an image encoding method according to an embodiment of the present invention. Figure 6 This is a flowchart illustrating an image decoding method according to an embodiment of the present invention.

[0201] Reference Figure 5 The encoding device can deduce motion vector candidates (step S501) and generate a motion vector candidate list based on the derived motion vector candidates (step S502). After generating the motion vector candidate list, motion vectors can be determined based on the generated motion vector candidate list (step S503), and motion compensation can be performed based on the determined motion vectors (step S504). Subsequently, the encoding device can encode the information associated with motion compensation (step S505).

[0202] Reference Figure 6The decoding device can perform entropy decoding on the motion compensation-related information received from the encoding device (step S601) and deduce motion vector candidates (step S602). The decoding device can generate a motion vector candidate list based on the deduced motion vector candidates (step S603) and use the generated motion vector candidate list to determine the motion vector (step S604). Subsequently, the decoding device can perform motion compensation by using the determined motion vector (step S605).

[0203] Figure 7 This is a flowchart illustrating an image encoding method according to another embodiment of the present invention. Figure 8 This is a flowchart illustrating an image decoding method according to another embodiment of the present invention.

[0204] Reference Figure 7 The encoding device can deduce merging candidates (step S701) and generate a merging candidate list based on the deduced merging candidates. After generating the merging candidate list, the encoding device can use the generated merging candidate list to determine motion information (step S702) and can use the determined motion information to perform motion compensation on the current block (step S703). Subsequently, the encoding device can perform entropy encoding on the information associated with motion compensation (step S704).

[0205] Reference Figure 8 The decoding device can perform entropy decoding (S801) on the motion compensation-related information received from the encoding device, deduce merging candidates (S802), and generate a merging candidate list based on the deduced merging candidates. After generating the merging candidate list, the decoding device can determine the motion information of the current block by using the generated merging candidate list (S803). Subsequently, the decoding device can use the motion information to perform motion compensation (S804).

[0206] Figure 5 and Figure 6 The application is shown in Figure 4 The example of AMVP shown in the image, and Figure 7 and Figure 8 The application is shown in Figure 4 An example of a merge pattern is shown in the image.

[0207] In the following text, it will be described Figure 5 and Figure 6 Each step in the process, and then described. Figure 7 and Figure 8 Each step in the process will be described. However, the motion compensation steps corresponding to S504, S605, S703, and S804, and the entropy encoding / decoding steps corresponding to S505, S601, S704, and S801 will be described together.

[0208] The following will describe it in detail. Figure 5 and Figure 6 Each step in the process.

[0209] First, the steps for deriving the motion vector candidates will be described in detail (S501, S602).

[0210] The motion vector candidates for the current block may include at least one of spatial motion vector candidates and temporal motion vector candidates, or both of spatial motion vector candidates and temporal motion vector candidates.

[0211] The temporal motion vector of the current block can be derived from the reconstructed blocks adjacent to it. For example, the motion vectors of the reconstructed blocks adjacent to the current block can be used as candidates for the spatial motion vector of the current block.

[0212] Figure 9 This is a diagram illustrating an example of deriving the spatial motion vector candidates for the current block.

[0213] Reference Figure 9 The spatial motion vector candidate of the current block can be derived from the neighboring blocks adjacent to the current block X. The neighboring blocks adjacent to the current block X include at least one of the following: block B1 adjacent to the top edge of the current block, block A1 adjacent to the left edge of the current block, block B0 adjacent to the top right corner of the current block, block B2 adjacent to the top left corner of the current block, and block A0 adjacent to the bottom left corner of the current block. The neighboring blocks adjacent to the current block can have a square shape or a non-square shape. When one of the neighboring blocks adjacent to the current block has a motion vector, the motion vector of that neighboring block can be determined as a spatial motion vector candidate for the current block. Whether a neighboring block has a motion vector or whether its motion vector can be used as a spatial motion vector candidate for the current block can be determined based on the determination of whether a neighboring block exists or whether it has been encoded through inter-frame prediction processing. The determination of whether a specific neighboring block has a motion vector or whether its motion vector can be used as a spatial motion vector candidate for the current block can be performed in a predetermined order. For example, as... Figure 9 As shown, the availability determination of motion vectors can be performed in the order of blocks A0, A1, B0, B1, and B2.

[0214] When the reference frame of the current block and the reference frames of neighboring blocks with motion vectors are different from each other, the motion vectors of the neighboring blocks are scaled, and the scaled motion vectors can then be used as spatial motion vector candidates for the current block. Motion vector scaling can be performed based on at least one of the distance between the current frame and the reference frame of the current block, and the distance between the current frame and the reference frames of neighboring blocks. Here, the spatial motion vector candidates for the current block can be derived by scaling the motion vectors of neighboring blocks according to the ratio of the distance between the current frame and the reference frame of the current block to the distance between the current frame and the reference frames of neighboring blocks.

[0215] However, when the reference frame index of the current block differs from the reference frame index of a neighboring block with a motion vector, the scaled motion vector of the neighboring block can be determined as a candidate spatial motion vector for the current block. Even in this case, scaling can be performed based on at least one of the distance between the current frame and the reference frame of the current block, and the distance between the current frame and the reference frames of neighboring blocks.

[0216] Regarding scaling, motion vectors of neighboring blocks can be scaled based on reference frames indicated by reference frame indices with predefined values, and the scaled motion vectors can be determined as spatial motion vector candidates for the current block. The predefined values ​​can be zero or positive integers. For example, spatial motion vector candidates for the current block can be derived by scaling the motion vectors of neighboring blocks based on the ratio of the distance between the current frame and the reference frame of the current block to the distance between the current frame and the reference frames of neighboring blocks, where the reference frame of the current block is indicated by a reference frame index with a predefined value, and the reference frames of neighboring blocks are indicated by reference frame indices with predefined values.

[0217] Alternatively, spatial motion vector candidates for the current block can be derived based on at least one of the encoding parameters of the current block.

[0218] The temporal motion vector candidate for the current block can be derived from the reconstructed block included in the co-frame. The co-frame is a frame that has been encoded / decoded prior to the current frame and may differ from the current frame in temporal order.

[0219] Figure 10 This is a diagram illustrating an example of deriving the candidate time motion vector for the current block.

[0220] Reference Figure 10The temporal motion vector candidate of the current block can be derived from blocks located outside the block that is spatially located at the same position as the current block X within the co-positional frame (also called the same-position frame) including the current frame, or from blocks located inside the block that is spatially located at the same position as the current block X. Here, the temporal motion vector candidate can represent the motion vector of the co-positional block of the current block. For example, the temporal motion vector candidate of the current block X can be derived from block H, which is adjacent to the lower left corner of block C that is spatially located at the same position as the current block X, or from block C3, which includes the center position of block C. Blocks H, C3, etc., used to derive the temporal motion vector candidate of the current block are called co-positional blocks.

[0221] Alternatively, at least one of the following can be derived based on at least one of the encoding parameters: time motion vector candidate, co-position frame, co-position block, prediction list utilization flag, and reference frame index.

[0222] When the distance between the current frame (including the current block) and its reference frame differs from the distance between the co-frame (including the co-block) and its reference frame, a candidate temporal motion vector for the current block can be obtained by scaling the motion vector of the co-frame. Here, scaling can be performed based on at least one of the distance between the current frame and its reference frame, and the distance between the co-frame and its reference frame. For example, the candidate temporal motion vector for the current block can be derived by scaling the motion vector of the co-frame according to the ratio of the distance between the current frame and its reference frame to the distance between the co-frame and its reference frame.

[0223] Next, the steps for generating a list of motion vector candidates based on the derived motion vector candidates will be described (S502, S603).

[0224] The steps of generating a motion vector candidate list may include processes for adding motion vector candidates to or deleting motion vector candidates from the motion vector candidate list, as well as processes for adding combined motion vector candidates to the motion vector candidate list.

[0225] First, the process of adding or deleting derived motion vector candidates from the motion vector candidate list will be described. Encoding and decoding devices can add derived motion vector candidates to the motion vector candidate list in the order they were derived.

[0226] Suppose that the motion vector candidate list mvpListLX can represent the motion vector candidate lists corresponding to the reference frame lists L0, L1, L2, and L3. That is, the motion vector candidate list corresponding to the reference frame list L0 can be represented by mvpListL0.

[0227] In addition to adding spatial and temporal motion vector candidates to the motion vector candidate list, motion vectors with predetermined values ​​can also be added. For example, when the number of motion vector candidates in the list is less than the maximum number of motion vector candidates that can be included in the list, motion vectors with a value of zero can be added to the list.

[0228] Next, the process of adding combined motion vector candidates to the motion vector candidate list will be described.

[0229] When the number of motion vector candidates in the motion vector candidate list is less than the maximum number of motion vector candidates that can be included in the motion vector candidate list, one or more motion vector candidates in the motion vector candidate list are combined to generate one or more combined motion vector candidates, and the generated combined motion vector candidates can be added to the motion vector candidate list. For example, at least one or more of the spatial motion vector candidates, temporal motion vector candidates, and zero motion vector candidates included in the motion vector candidate list can be used to generate combined motion vector candidates, and the generated combined motion vector candidates can be added to the motion vector candidate list.

[0230] Alternatively, combined motion vector candidates can be generated based on at least one encoding parameter, and the combined motion vector candidates generated based on at least one encoding parameter can be added to the motion vector candidate list.

[0231] Next, the steps for selecting the predicted motion vector of the current block from the motion vector candidate list will be described below (S503, S604).

[0232] Among the motion vector candidates included in the motion vector candidate list, the motion vector candidate indicated by the motion vector candidate index can be determined as the predicted motion vector for the current block.

[0233] The encoding device calculates the difference between the motion vector of the current block and the predicted motion vector of the current block, thus generating the motion vector difference. The decoding device generates the motion vector of the current block by adding the predicted motion vector to the motion vector difference.

[0234] Here, motion information correction can be applied to any of the following: motion vector candidates included in the motion vector candidate list, predicted motion vectors, or motion vectors calculated by adding the predicted motion vector to the difference between the two motion vectors. A detailed description of motion information correction will be provided later.

[0235] We will describe them together later. Figure 5 and Figure 6The steps for performing motion compensation (S504, S605) and the steps for entropy encoding / decoding of information associated with motion compensation (S505, S601) shown in the figure are as follows: Figure 7 and Figure 8 The steps for performing motion compensation (S703, S804) and entropy encoding / decoding (S704, S801) are shown in the figure.

[0236] The following text will describe it in detail. Figure 7 and Figure 8 Each step is shown in the diagram.

[0237] First, the steps for deriving the candidate mergers will be described (S701, 802).

[0238] The merge candidates for the current block may include at least one of spatial merge candidates, temporal merge candidates, and additional merge candidates. Here, the expression "derive spatial merge candidates" refers to the process of deriving spatial merge candidates and adding the derived merge candidates to the merge candidate list.

[0239] Reference Figure 9 Spatial merge candidates for the current block can be derived from the neighboring blocks adjacent to the current block X. The neighboring blocks adjacent to the current block X may include at least one of the following: block B1 adjacent to the top edge of the current block, block A1 adjacent to the left edge of the current block, block B0 adjacent to the top right corner of the current block, block B2 adjacent to the top left corner of the current block, and block A0 adjacent to the bottom left corner of the current block.

[0240] To derive spatial merge candidates for the current block, it is determined whether each neighboring block adjacent to the current block can be used to derive a spatial merge candidate for the current block. This determination is performed on neighboring blocks in a predetermined priority order. For example, it can be done by... Figure 9 The example demonstrates the availability of spatial merge candidates for blocks A1, B1, B0, A0, and B2 in that order. Spatial merge candidates determined based on availability are added sequentially to the merge candidate list for the current block.

[0241] Figure 11 This is a diagram illustrating an example of the process of adding spatial merge candidates to the merge candidate list.

[0242] Reference Figure 11 Four spatial merge candidates are derived from four neighboring blocks A1, B0, A0 and B2, and the derived spatial merge candidates can be added sequentially to the merge candidate list.

[0243] Alternatively, spatial merging candidates can be derived based on at least one of the encoding parameters.

[0244] Here, the motion information of the spatial merging candidate may include three or more motion information entries, which, in addition to L0 and L1 motion information, also include L2 and L3 motion information. There may be at least one list of reference frames, for example, including L0, L1, L2, and L3.

[0245] Next, we will describe the method for deriving the time merge candidates for the current block.

[0246] Temporal merge candidates for the current block can be derived from the reconstructed blocks included in the co-frames of the current frame. A co-frame can be a frame that was encoded / decoded before the current frame and may differ from the current frame in temporal order.

[0247] The expression "derive time merging candidate" refers to the process of deriving time merging candidates and adding the derived time merging candidates to the merging candidate list.

[0248] Reference Figure 10 The time-merging candidate for the current block can be derived from blocks located outside the block that are spatially aligned with the current block X in the same frame (also called co-positional frames) or from blocks located inside the block that are spatially aligned with the current block X in the same frame (also called co-positional frames). The term "time-merging candidate" can represent the motion information of co-positional blocks. For example, the time-merging candidate for the current block X can be derived from block H, which is adjacent to the lower left corner of block C that is spatially aligned with the current block X, or from block C3, which includes the center position of block C. Blocks H, C3, etc., used to derive the time-merging candidate for the current block are called co-positional blocks (also called co-positional blocks).

[0249] When the timing merge candidate for the current block can be derived from block H, which is located outside block C, block H is set as the sibling block of the current block. In this case, the timing merge candidate for the current block can be derived based on the motion information of block H. Conversely, when the timing merge candidate for the current block cannot be derived from block H, block C3, which is located inside block C, can be set as the sibling block of the current block. In this case, the timing merge candidate for the current block can be derived based on the motion information of block C3. When no timing merge candidate for the current block can be derived from either block H or block C3 (e.g., both block H and block C3 are intra-coded blocks), the timing merge candidate for the current block cannot be derived at all, or it can be derived from blocks other than block H and block C3.

[0250] Alternatively, for example, multiple time merging candidates for the current block can be derived from multiple blocks included in the same frame. That is, multiple time candidates for the current block can be derived from block H, block C3, etc.

[0251] Figure 12 This is a diagram illustrating an example of the process of adding time merging candidates to the merge candidate list.

[0252] Reference Figure 12 When a time merge candidate is derived from the co-occurring block at position H1, the derived time merge candidate can be added to the merge candidate list.

[0253] When the distance between the current frame (including the current block) and its reference frame differs from the distance between the co-frame (including the co-block) and its reference frame, the motion vector of the current block's time-merging candidate can be obtained by scaling the motion vector of the co-frame. Here, the motion vector scaling can be performed based on at least one of the distance between the current frame and its reference frame, and the distance between the co-frame and its reference frame. For example, the motion vector of the current block's time-merging candidate can be derived by scaling the co-frame's motion vector according to the ratio of the distance between the current frame and its reference frame to the distance between the co-frame and its reference frame.

[0254] In addition, at least one of the following can be derived based on the encoding parameters of the current block, neighboring blocks, or co-occurring blocks: time merging candidate, co-occurring picture, co-occurring block, prediction list utilization flag, and reference picture index.

[0255] The merge candidate list can be generated by generating at least one of the spatial merge candidates and the temporal merge candidates and adding the derived merge candidates sequentially to the merge candidate list in the order of derivation.

[0256] Next, we will describe the method for deriving additional merge candidates for the current block.

[0257] The term "additional merge candidate" can refer to at least one of a modified spatial merge candidate, a modified temporal merge candidate, a combined merge candidate, and a predetermined merge candidate with a predetermined motion information value. Here, the expression "derive additional merge candidate" can refer to the process of deriving additional merge candidates and adding the derived additional merge candidates to the merge candidate list.

[0258] A modified spatial merging candidate can be defined as a merging candidate obtained by modifying at least one of the motion information of the derived spatial merging candidates.

[0259] A modified time merging candidate can be defined as a modified merging candidate obtained by modifying at least one of the motion information of the derived time merging candidates.

[0260] A combined merge candidate can be defined as a merge candidate obtained by combining motion information from at least one of the following: spatial merge candidates, temporal merge candidates, modified spatial merge candidates, modified temporal merge candidates, combined merge candidates, and predetermined merge candidates with predetermined motion information values, all of which are included in the merge candidate list.

[0261] Alternatively, a combined merge candidate may represent a merge candidate derived by combining motion information of at least one of the following merge candidates: a spatial merge candidate and a temporal merge candidate not included in the merge candidate list but derived from a block from which at least one of spatial merge candidates and temporal merge candidates can be derived; a modified spatial merge candidate and a modified temporal merge candidate derived from a block from which spatial merge candidates and temporal merge candidates can be derived; a combined merge candidate; and a predetermined merge candidate having a predetermined motion information value.

[0262] Alternatively, motion information obtained by performing entropy decoding on the bitstream in the decoder can be used to derive the combined merging candidates. In this case, the motion information used to derive the combined merging candidates can be entropy encoded into a bitstream in the encoder.

[0263] Combined merging candidates can be represented as combined bidirectional prediction merging candidates. Combined bidirectional prediction merging candidates are merging candidates that use bidirectional prediction, and they can be merging candidates with both L0 motion information and L1 motion information.

[0264] Merging candidates with predetermined motion information values ​​can be zero merging candidates with motion vectors of (0,0). Merging candidates with predetermined motion information values ​​can be set in such a way that the predetermined motion information values ​​have the same value in both the encoding and decoding devices.

[0265] At least one of the following can be derived or generated based on at least one of the encoding parameters of the current block, neighboring blocks, or co-occurring blocks: a modified spatial merge candidate, a modified temporal merge candidate, a combined merge candidate, and a merge candidate with a predetermined motion information value. Furthermore, at least one of the following can be added to the merge candidate list based on at least one of the encoding parameters of the current block, neighboring blocks, or co-occurring blocks: a modified spatial merge candidate, a modified temporal merge candidate, a combined merge candidate, and a merge candidate with a predetermined motion information value.

[0266] The size of the merge candidate list can be determined based on the encoding parameters of the current block, neighboring blocks, or co-occurring blocks, and the size of the merge candidate list can vary according to the encoding parameters.

[0267] Next, the steps for determining the motion information of the current block using the generated list of merge candidates will be described (S702, S803).

[0268] The encoder can select merge candidates from the merge candidate list that will be used for motion compensation of the current block through motion estimation, and encode the merge candidate index merge_idx that indicates the determined merge candidate into a bit stream.

[0269] To generate a prediction block for the current block, the encoder selects a merge candidate from the merge candidate list using a merge candidate index and determines the motion information for the current block. The encoder then performs motion compensation based on the determined motion information to generate the prediction block for the current block.

[0270] The decoder decodes the merge candidate indices in the received bitstream and determines the merge candidates included in the merge candidate list and indicated by the merge candidate indices. The determined merge candidates can be used as motion information for the current block. This motion information is then used for motion compensation of the current block. Here, the term "motion compensation" can have the same meaning as inter-frame prediction.

[0271] Additionally, motion information corrections can be applied to any of the motion information determined based on the merge candidate index or merge candidates included in the merge candidate list. A detailed description of motion information corrections will be provided later.

[0272] Next, the steps for performing motion compensation using motion vectors or motion information will be described (S504, S605, S703, S804).

[0273] The encoding and decoding devices can calculate the motion vector of the current block using the predicted motion vector and the motion vector difference. After calculating the motion vector, the encoding and decoding devices can use the calculated motion vector to perform inter-frame prediction or motion compensation (S504, S605).

[0274] The encoding and decoding devices can use determined motion information to perform inter-frame prediction or motion compensation (S703, S804). Here, the current block may have determined motion information of merging candidates.

[0275] The current block can have one (minimum) to N (maximum) motion vectors depending on the prediction direction of the current block. One to N motion vectors can be used to generate one (minimum) to N (maximum) prediction blocks, and the final prediction block can be selected from the generated prediction blocks.

[0276] For example, when the current block has a motion vector, the predicted block generated using the motion vector (or motion information) can be determined as the final predicted block for the current block.

[0277] Additionally, when the current block has multiple motion vectors (or multiple motion information), multiple prediction blocks are generated using these multiple motion vectors (or multiple motion information), and the final prediction block for the current block is determined based on the weighted sum of these multiple prediction blocks. Multiple reference frames, each including multiple prediction blocks indicated by multiple motion vectors (or multiple motion information), can be listed in different reference frame lists or in a single reference frame list.

[0278] For example, multiple prediction blocks for the current block can be generated based on at least one of spatial motion vector candidates, temporal motion vector candidates, motion vectors with predetermined values, and combined motion vector candidates, and the final prediction block for the current block can then be determined based on a weighted sum of the multiple prediction blocks.

[0279] Alternatively, for example, multiple prediction blocks for the current block can be generated based on motion vector candidates indicated by a preset motion vector candidate index, and the final prediction block for the current block can then be determined based on a weighted sum of the multiple prediction blocks. Furthermore, multiple prediction blocks can be generated based on motion vector candidates indicated by indices within a predetermined range of motion vector candidate indices, and the final prediction block for the current block can then be determined based on a weighted sum of the multiple prediction blocks.

[0280] The weighting factor for each prediction block can be an equal 1 / N (where N is the number of prediction blocks generated). For example, when two prediction blocks are generated, the weighting factor for each prediction block is 1 / 2. Similarly, when three prediction blocks are generated, the weighting factor for each prediction block is 1 / 3. When four prediction blocks are generated, the weighting factor for each prediction block can be 1 / 4. Alternatively, the final prediction block for the current block can be determined by applying different weighting factors to each prediction block.

[0281] The weighting factors for multiple prediction blocks are not fixed but variable. The weighting factors for multiple prediction blocks can be unequal and different. For example, when generating two prediction blocks, the weighting factors for the two blocks can be equal, such as (1 / 2, 1 / 2), or unequal, such as (1 / 3, 2 / 3), (1 / 4, 3 / 4), (2 / 5, 3 / 5), or (3 / 8, 5 / 8). The weighting factors can be positive or negative real values. That is, the values ​​of the weighting factors can include negative real values, such as (-1 / 2, 3 / 2), (-1 / 3, 4 / 3), or (-1 / 4, 5 / 4).

[0282] To apply a variable weighting factor, one or more weighting factor messages for the current block can be signaled via a bitstream. Weighting factor information can be signaled block-by-block prediction or frame-by-frame prediction. Optionally, multiple prediction blocks can share a single weighting factor.

[0283] The encoding and decoding devices can determine whether to use the predicted motion vector (or predicted motion information) based on the prediction block list utilization flag. For example, for each reference frame list, when the prediction block list utilization flag has a first value of 1 (1), the encoding and decoding devices can use the predicted motion vector for the current block to perform inter-frame prediction or motion compensation for the current block. However, when the prediction block list utilization flag has a second value of 0 (0), the encoding and decoding devices may not use the predicted motion vector for the current block to perform inter-frame prediction or motion compensation for the current block. The first and second values ​​of the prediction block list utilization flag can be set to 0 and 1 respectively, respectively. Expressions 3 to 5 are examples of methods for generating the final predicted block of the current block when the inter-frame prediction indicator of the current block is PRED_BI, PRED_TRI, or PRED_QUAD and when the prediction direction for each reference frame list is unidirectional.

[0284]

Expression 1

[0285]

[0286]

Expression 2

[0287]

[0288]

Expression 3

[0289]

[0290] In expressions 1 through 3, each of P_BI, P_TRI, and P_QUAD represents the final predicted block for the current block, and LX (X = 0, 1, 2, 3) represents the reference frame list. WF_LX represents the weighting factor for the predicted block generated using the LX reference frame list. OFFSET_LX represents the offset value for the predicted block generated using the LX reference frame list. P_LX represents the predicted block generated using the motion vectors (or motion information) of the LX reference frame list for the current block. RF represents the rounding factor, and RF can be set to 0, a positive integer, or a negative integer. The LX reference screen list may include at least one of the following: long-term reference screens, reference screens that have not undergone deblocking filtering, reference screens that have not undergone sample adaptive offset, reference screens that have not undergone adaptive loop filtering, reference screens that have only undergone deblocking filtering and adaptive offset, reference screens that have only undergone deblocking filtering and adaptive loop filtering, reference screens that have undergone sample adaptive offset and adaptive loop filtering, and reference screens that have undergone all of the following: deblocking filtering, sample adaptive offset, and adaptive loop filtering. In this case, the LX reference screen list may be at least one of the L2 reference screen list and the L3 reference screen list.

[0291] Even when there are multiple prediction directions for a predetermined list of reference screens, the final prediction block for the current block can be obtained based on the weighted sum of multiple prediction blocks. In this case, the weighting factors of the multiple prediction blocks derived using a single list of reference screens can be equal or different from each other.

[0292] At least one of the weighting factor WF_LX or offset OFFSET_LX of multiple prediction blocks can be an encoding parameter that will be entropy encoded / entropy decoded. Alternatively, for example, the weighting factor and offset can be derived from previously encoded / decoded neighboring blocks adjacent to the current block. Here, neighboring blocks adjacent to the current block may include at least one of blocks used to derive spatial motion vector candidates for the current block and blocks used to derive temporal motion vector candidates for the current block.

[0293] Further, alternatively, for example, the weighting factor and offset can be determined based on the display order of the current frame (Frame Order Count (POC)) and the POC of each reference frame. In this case, the value of the weighting factor or offset can decrease as the distance between the current frame and the reference frame increases. That is, when the current frame and the reference frame are closer to each other, a larger value can be set for the weighting factor or offset. For example, when the difference between the POC of the current frame and the POC of the L0 reference frame is 2, the value of the weighting factor applied to the prediction block generated using the L0 reference frame can be set to 1 / 3. Simultaneously, when the difference between the POC of the current frame and the POC of the L0 reference frame is 1, the value of the weighting factor applied to the prediction block generated using the L0 reference frame can be set to 2 / 3. As described above, the weighting factor or offset can be inversely proportional to the difference between the display order (POC) of the current frame and the display order (POC) of the reference frames. Alternatively, the weighting factor or offset can be directly proportional to the difference between the display order (POC) of the current frame and the display order (POC) of the reference frames.

[0294] Alternatively, for example, at least one of the weighting factors and the offset can be entropy encoded / entropy decoded based on at least one encoding parameter. Furthermore, a weighted sum of the prediction blocks can be computed based on at least one encoding parameter.

[0295] The weighted sum of multiple prediction blocks can be applied only to a portion of the prediction block. This portion can be a boundary region adjacent to the boundary of each prediction block. As mentioned above, in order to apply the weighted sum only to a portion of the region, the weighted sum can be calculated block by block within each prediction block.

[0296] In a block with a block size indicated by region information, inter-frame prediction or motion compensation can be performed on sub-blocks smaller than that block by using the same prediction block or the same final prediction block.

[0297] In a block with a block depth indicated by region information, inter-frame prediction or motion compensation can be performed on sub-blocks with a block depth greater than that of the block by using the same prediction block or the same final prediction block.

[0298] Furthermore, when calculating the weighted sum of a prediction block using motion vector prediction, at least one of the motion vector candidates included in the motion vector candidate list can be used to calculate the weighted sum, and the calculation result can be used as the final prediction block for the current block.

[0299] For example, prediction blocks can be generated using only spatial motion vector candidates, a weighted sum of prediction blocks can be computed, and the computed weighted sum can be used as the final prediction block for the current block.

[0300] For example, spatial motion vector candidates and temporal motion vector candidates can be used to generate prediction blocks, a weighted sum of prediction blocks can be calculated, and the calculated weighted sum can be used as the final prediction block for the current block.

[0301] For example, a prediction block can be generated using only combined motion vector candidates, a weighted sum of the prediction blocks can be calculated, and the calculated weighted sum can be used as the final prediction block for the current block.

[0302] For example, prediction blocks can be generated using only motion vector candidates indicated by a specific index, a weighted sum of prediction blocks can be calculated, and the calculated weighted sum can be used as the final prediction block for the current block.

[0303] For example, a prediction block can be generated using only motion vector candidates indicated by indices within a predetermined index range, a weighted sum of the prediction blocks can be calculated, and the calculated weighted sum can be used as the final prediction block for the current block.

[0304] When using a merge mode to compute a weighted sum of predicted blocks, at least one merge candidate from the merge candidate list included in the merge candidate list can be used to compute the weighted sum, and the computed result can be used as the final predicted block for the current block.

[0305] For example, prediction blocks can be generated using only spatial merging candidates, a weighted sum of prediction blocks can be calculated, and the calculated weighted sum can be used as the final prediction block for the current block.

[0306] For example, spatial merge candidates and temporal merge candidates can be used to generate prediction blocks, a weighted sum of prediction blocks can be calculated, and the calculated weighted sum can be used as the final prediction block for the current block.

[0307] For example, a prediction block can be generated by simply combining and merging candidates, a weighted sum of prediction blocks can be generated, and the calculated weighted sum can be used as the final prediction block for the current block.

[0308] For example, a prediction block can be generated using only merge candidates indicated by a specific index, a weighted sum of the prediction blocks can be generated, and the calculated weighted sum can be used as the final prediction block for the current block.

[0309] For example, a prediction block can be generated using only merge candidates indicated by indexes within a predetermined index range, a weighted sum of the prediction blocks can be calculated, and the calculated weighted sum can be used as the final prediction block for the current block.

[0310] In the encoder and decoder, motion compensation can be performed using the motion vector or motion information of the current block. At this point, at least one prediction block can be used to determine the final prediction block as a result of the motion compensation. Here, the current block can represent at least one of the current encoded block and the current prediction block.

[0311] The final predicted block of the current block can be generated by performing overlapping block motion compensation on the boundary region of the current block.

[0312] The boundary region of the current block can be a region placed within the current block and adjacent to the boundaries between the current block and its neighboring blocks. The boundary region of the current block can include at least one of the following: top boundary region, left boundary region, bottom boundary region, right boundary region, top-right corner region, bottom-right corner region, top-left corner region, and bottom-left corner region. The boundary region of the current block can also be a region corresponding to a portion of the predicted block of the current block.

[0313] Overlapping block motion compensation can be represented as a motion compensation process performed by calculating a weighted sum of the predicted blocks corresponding to the boundary regions of the current block and the predicted blocks generated using motion information from the coded / decoded blocks adjacent to the current block.

[0314] The weighted sum can be calculated block by block by dividing the current block into multiple sub-blocks. In other words, motion compensation for the current block can be performed block by block using motion information from the coded / decoded sub-blocks adjacent to the current block. A sub-block can represent a lower-level block than the current block.

[0315] Furthermore, when calculating the weighted sum, a first prediction block and a second prediction block can be used. The first prediction block is generated using motion information of the current block for each sub-block of the current block, and the second prediction block is generated using motion information of spatially adjacent neighboring sub-blocks. In this case, the expression "using motion information" means "deriving motion information." The first prediction block can represent a prediction block generated by using motion information of the encoded / decoded target sub-block within the current block. The second prediction block can be a prediction block generated by using motion information of spatially adjacent neighboring sub-blocks of the encoded / decoded target sub-block within the current block.

[0316] The final predicted block for the current block can be generated by using a weighted sum of the first and second predicted blocks. In other words, overlapping block motion compensation uses the motion information of the current block and the motion information of the other block to find the final predicted block for the current block.

[0317] Furthermore, when using at least one of the Advanced Motion Vector Prediction (AMVP) mode, merging mode, affine motion compensation mode, decoder-side motion vector derivation mode, adaptive motion vector resolution mode, local illumination compensation mode, and bidirectional optical flow mode, the current block can be divided into multiple sub-blocks, and overlapping block motion compensation can be performed on a sub-block-by-sub-block basis.

[0318] When the merging mode is used for motion compensation, overlapping block motion compensation can be performed on at least one of the Advanced Temporal Motion Vector Predictor (ATMVP) candidates and the Spatiotemporal Motion Vector Predictor (STMVP) candidates.

[0319] Next, the process of performing entropy encoding / entropy decoding on the information associated with motion compensation will be described (S505, S601, S704, S801).

[0320] An encoding device can entropy encode information associated with motion compensation into a bitstream, and a decoding device can decode the motion compensation-related information included in the bitstream. The motion compensation-related information targeted by entropy encoding or decoding may include at least one of the following: inter-frame prediction indicator inter_pred_idc, reference frame indices ref_idx_l0, ref_idx_l1, ref_idx_l2, and ref_idx_l3, motion vector candidate indices mvp_l0, mvp_l1_idx, mvp_l2_idx, and mvp_l3_idx, motion vector difference, skip mode use / disuse information cu_skip_flag, merge mode use / disuse information merge_flag, merge index information merge_index, weighting factors wf_l0, wf_l1, wf_l2, and wf_l3, and offset values ​​offset_l0, offset_l1, offset_l2, and offset_l3.

[0321] When encoding / decoding the current block via inter-frame prediction, the inter-frame prediction indicator can indicate the prediction direction, the number of prediction directions, or both. For example, the inter-frame prediction indicator can indicate unidirectional or multidirectional prediction (such as bidirectional, tridirectional, and quaddirectional prediction). The inter-frame prediction indicator can also indicate the number of reference frames used to generate the prediction blocks for the current block. Optionally, a reference frame can be used for prediction in multiple directions. In this case, M reference frames are used to perform prediction in N directions (where N > M). The inter-frame prediction indicator can also indicate the number of prediction blocks used for inter-frame prediction or motion compensation of the current block.

[0322] The reference screen indicator can indicate one direction PRED_LX, two directions PRED_BI, three directions PRED_TRI, four directions PRED_QUAD, or more directions, depending on the number of predicted directions for the current block.

[0323] The prediction list for a specific reference screen list uses flags to indicate whether the reference screen list is used to generate prediction blocks.

[0324] For example, when the prediction list utilization flag for a specific reference screen list has a first value of 1, this indicates that the reference screen list is used to generate the prediction block. When the prediction list utilization flag has a second value of 0, this indicates that the reference screen list is not used to generate the prediction block. Here, the first and second values ​​of the prediction list utilization flag can be set to 0 and 1 respectively, in reverse order.

[0325] In other words, when the prediction list of a specific reference frame list has a first value, the prediction block of the current block can be generated using the motion information corresponding to the reference frame list.

[0326] A reference frame index indicates a specific reference frame that exists in the reference frame list and is referenced by the current block. For each reference frame list, one or more reference frame indices can be entropy encoded / decoded. One or more reference frame indices can be used for motion compensation of the current block.

[0327] The motion vector candidate index indicates the motion vector candidate for the current block among the motion vector candidates included in each reference frame list or a motion vector candidate list prepared for each reference frame index. For each motion vector candidate list, at least one or more motion vector candidate indices can be entropy encoded / entropy decoded. At least one or more motion vector candidate indices can be used for motion compensation of the current block.

[0328] Motion vector difference represents the difference between the current motion vector and the predicted motion vector. For each motion vector candidate list generated for each reference frame list or each reference frame index for the current block, one or more motion vector differences can be entropy encoded / entropy decoded. One or more motion vector differences can be used for motion compensation of the current block.

[0329] Regarding the skip mode use / disuse information cu_skip_flag, skip mode can be used when it has a first value of 1 (1). Conversely, skip mode can be disused when it has a second value of 0 (0). Skip mode can be used to perform motion compensation for the current block based on the skip mode use / disuse information.

[0330] Regarding the merge mode use / disuse information merge_flag, the merge mode can be used when it has a first value of 1 (1). Conversely, the merge mode can be disused when it has a second value of 0 (0). The merge mode can be used to perform motion compensation on the current block based on the merge mode use / disuse information.

[0331] The merge index information, merge_index, can indicate information about merge candidates within the merge candidate list.

[0332] Alternatively, merge index information can represent information about the merge index.

[0333] In addition, the merge index information can indicate the reconstructed blocks used to derive merge candidates among multiple reconstructed blocks that are spatially / temporally adjacent to the current block.

[0334] The merge index information can indicate one or more motion information items possessed by a merge candidate. For example, when the merge index information has a first value of zero (0), the merge index information can indicate the first merge candidate listed as the first item in the merge candidate list; when the merge index information has a second value of one (1), the merge index information can indicate the second merge candidate listed as the second item in the merge candidate list; when the merge index information has a third value of two (2), the merge index information can indicate the third merge candidate listed as the third item in the merge candidate list. Similarly, when the merge index information has values ​​from a fourth value to an Nth value, the merge index information can indicate the merge candidate listed in the merge candidate list at the position according to the order of the values. Here, N can be 0 or a positive integer.

[0335] Motion compensation can be performed on the current block based on the merge mode index information using the merge mode.

[0336] When two or more prediction blocks are generated during motion compensation of the current block, the final prediction block of the current block can be determined based on the weighted sum of the prediction blocks. When calculating the weighted sum, a weighting factor, an offset, or both can be applied to each prediction block. The weighting factors used to calculate the weighted sum (e.g., weighting factor and offset) can be entropy-encoded / entropy-decoded by at least some of the following or corresponding to at least one of the following: a list of reference frames, reference frames, motion vector candidate indices, motion vector differences, motion vectors, skip mode use / disuse information, merge mode use / disuse information, and merge index information. Furthermore, the weighting factors for each prediction block can be entropy-encoded / entropy-decoded based on inter-frame prediction indicators. The weighting factors can include at least one of a weighting factor and an offset.

[0337] Information associated with motion compensation can be entropy-encoded / entropy-decoded block by block, or it can be entropy-encoded / entropy-decoded at a higher level of unit. For example, information associated with motion compensation can be entropy-encoded / entropy-decoded block by block (e.g., per CTU, per CU, or per PU). Alternatively, information associated with motion compensation can be entropy-encoded / entropy-decoded at a higher level of unit (such as video parameter sets, sequence parameter sets, picture parameter sets, adaptive parameter sets, or strip headers).

[0338] Information associated with motion compensation can be entropy encoded / decoded based on the motion compensation information difference, wherein the motion compensation information difference indicates the difference between the information associated with motion compensation and the predicted value of the information associated with motion compensation.

[0339] Information associated with motion compensation of the coded / decoded blocks adjacent to the current block can be used as information associated with motion compensation of the current block, without entropy encoding / entropy decoding of the information associated with motion compensation of the current block.

[0340] At least one piece of information associated with motion compensation can be derived based on at least one encoding parameter.

[0341] A bitstream can be decoded based on at least one encoding parameter to generate at least one piece of information associated with motion compensation. Conversely, at least one piece of information associated with motion compensation can be entropy encoded into a bitstream based on at least one encoding parameter.

[0342] At least one piece of information associated with motion compensation may include at least one of the following: motion vector, motion vector candidate, motion vector candidate index, motion vector difference, motion vector prediction value, skip mode use / disuse information skip_flag, merge mode use / disuse information merge_flag, merge index information merge_index, motion vector resolution information, overlapping block motion compensation information, local illumination compensation information, affine motion compensation information, decoder-side motion vector derivation information, and bidirectional optical flow information. Here, decoder-side motion vector derivation may represent pattern matching motion vector derivation.

[0343] Motion vector resolution information can be information indicating which specific resolution is used for at least one of the motion vector and the motion vector difference. Here, resolution can represent precision. A specific resolution can be set to at least one of 16-pixel (16-pel) units, 8-pixel (8-pel) units, 4-pixel (4-pel) units, integer-pixel (integer-pel) units, 1 / 2-pixel (1 / 2-pel) units, 1 / 4-pixel (1 / 4-pel) units, 1 / 8-pixel (1 / 8-pel) units, 1 / 16-pixel (1 / 16-pel) units, 1 / 32-pixel (1 / 32-pel) units, and 1 / 64-pixel (1 / 64-pel) units.

[0344] Overlapping block motion compensation information can be information indicating whether, during the motion compensation of the current block, the motion vectors of spatially adjacent neighboring blocks are additionally used to calculate the weighted sum of the predicted blocks for the current block.

[0345] Local illumination compensation information can be information indicating whether one of a weighting factor and an offset is applied when generating the prediction block for the current block. Here, at least one of the weighting factor and the offset can be a value calculated based on a reference block.

[0346] Affine motion compensation information can be information indicating whether to use an affine motion model for motion compensation of the current block. Here, an affine motion model can be a model that uses multiple parameters to divide a block into multiple sub-blocks and uses representative motion vectors to calculate the motion vectors of these sub-blocks.

[0347] Decoder-side motion vector derivation information can indicate whether the motion vectors required for motion compensation are derived by the decoder and subsequently used in the decoder. Based on the decoder-side motion vector derivation information, entropy encoding / decoding of information associated with the motion vectors may not be performed. When the decoder-side motion vector derivation information indicates that the motion vectors are derived by the decoder and subsequently used in the decoder, entropy encoding / decoding of information associated with the merging mode may be performed. In other words, decoder-side motion vector derivation information can indicate whether a merging mode is used in the decoder.

[0348] Bidirectional optical flow information can indicate whether motion vectors are modified pixel-by-pixel or sub-block-by-subblock and subsequently used for motion compensation. Based on bidirectional optical flow information, entropy encoding / decoding of motion vectors can be performed without modifying them pixel-by-pixel or sub-block-by-subblock. Modifying motion vectors means converting block-based motion vector values ​​to pixel-based or sub-block-based motion vector values.

[0349] Motion compensation can be performed on the current block based on at least one piece of information associated with motion compensation, and entropy encoding / decoding can be performed on at least one piece of information associated with motion compensation.

[0350] When entropy encoding / decoding information associated with motion compensation, binarization methods can be used, such as truncated Rice binarization, K-order exponential Golomb binarization, finite K-order exponential Golomb binarization, fixed-length binarization, univariate binarization, and truncated univariate binarization.

[0351] When entropy encoding / decoding information associated with motion compensation, the context model can be determined based on at least one of the following: information associated with motion information or region information of neighboring blocks adjacent to the current block; previously encoded / decoded motion compensation-related information or previously encoded / decoded region information; information about the depth of the current block; and information about the size of the current block.

[0352] Alternatively, when entropy encoding / decoding is performed on information associated with motion compensation, entropy encoding / decoding can be performed by using at least one of the following: information associated with motion compensation of neighboring blocks, previously encoded / decoded information associated with motion compensation, information about the depth of the current block, and information about the size of the current block as a prediction of the information associated with motion compensation of the current block.

[0353] In the following text, reference will be made to Figures 13 to 15 The motion information correction method is described in detail. Here, motion information correction can refer to the process of correcting at least one piece of motion information. That is, the information to be corrected can be at least one piece of motion information including: motion vector, reference frame index, reference frame, inter-frame prediction indicator, prediction list utilization flag, weighting factor, offset, etc. Motion information correction means correcting the value of at least one of the following: motion vector, reference frame index, reference frame, inter-frame prediction indicator, prediction list utilization flag, weighting factor, and offset.

[0354] The information to be modified for motion information can be information such as at least one of various encoding parameters. In this case, motion information modification means modifying the value of information such as at least one of the encoding parameters.

[0355] Motion information correction generates corrected motion information. This corrected motion information can be used for motion compensation of the current block to be encoded / decoded (hereinafter also referred to as the encoding / decoding target block). Figure 5 S504 in Figure 6 S605 in Figure 7 S703 in Figure 8 (S804 in the text).

[0356] Regarding the timing of motion information correction, it can be performed before motion compensation. That is, before performing motion compensation on the encoded / decoded target block, motion information correction is performed first to generate corrected motion information, and then the generated corrected motion information is used to perform motion compensation.

[0357] For example, in the AMVP mode, the step of determining the motion vector can be ( Figure 5 S503 and Figure 6 Motion information correction is performed in S604. In this case, motion information correction can be applied to any of the following information types: motion vector candidates included in the motion vector candidate list, predicted motion vectors, and motion vectors calculated by adding the predicted motion vector to the difference between the motion vectors.

[0358] For example, for the merging mode, in the step of determining motion information ( Figure 7 S702 and Figure 8 Motion information correction is performed in S803. In this case, motion information correction can be applied to motion information included in the merge candidate list or extracted from the merge candidate list based on the merge candidate index.

[0359] On the other hand, motion information correction can be performed even for skip mode. In this case, motion information correction can be applied to skip candidates included in the skip candidate list or to motion information extracted from the skip candidate list based on the skip index.

[0360] Regarding the timing used to perform motion information correction, it can be found during motion compensation ( Figure 5 S504 in Figure 6 S605 in Figure 7 S703 in Figure 8Motion information correction is performed during S804. That is, when performing motion compensation on the encoded / decoded target block, motion information correction is performed to generate corrected motion information, and the corrected motion information is used to perform motion compensation.

[0361] For example, the encoder / decoder can be based on the motion vector determination step or the motion information determination step ( Figure 5 S503 in Figure 6 604 in Figure 7 702 and Figure 8 The encoder / decoder uses the motion information determined in S803 to generate a prediction block, and can generate corrected motion information by performing motion information correction using the generated prediction block. Here, the encoder / decoder can use the corrected motion information to generate the final prediction block.

[0362] Motion information can be corrected on both the encoder and decoder sides according to the same rules. Since the motion information is corrected on both the encoder and decoder sides according to the same rules, entropy encoding / decoding of the "information on the use of motion information correction" (hereinafter referred to as motion information correction utilization information) is not required.

[0363] In the following text, reference will be made to Figures 13 to 15 Bilateral template matching is described as an exemplary motion information correction method, and as an image decoding method according to one embodiment.

[0364] Bilateral template matching is a method for correcting motion vectors across multiple motion data points. It allows correction of at least one of two motion vectors used for bidirectional prediction. Here, the bilateral template refers to a prediction block obtained by calculating a weighted sum of multiple prediction blocks generated using the two motion vectors in the bidirectional prediction process. The term "bidirectional prediction" refers to predictions in both forward and backward directions.

[0365] Specifically, when performing bilateral template matching on the bidirectional prediction block, at least one of the first motion vector corresponding to reference screen list 0 and the second motion vector corresponding to reference screen list 1 can be modified. In this case, the modified motion vector may have a motion vector value different from the initial motion vector.

[0366] Figure 13 It is a graph used to describe bilateral template matching.

[0367] Reference Figure 13 The bilateral template matching includes: (1) generating a bilateral template using the initial motion vector; and (2) comparing the samples of the bilateral template with the reconstructed samples of the reference image to correct the motion vector.

[0368] (1) Generation of bilateral templates

[0369] The encoder / decoder can generate a bilateral template using prediction blocks generated with a first motion vector corresponding to reference frame list 0 and prediction blocks generated with a second motion vector corresponding to reference frame list 1. Here, the bilateral template can be generated by calculating a weighted sum of the prediction blocks, and the weighting factor used to calculate the weighted sum can be 0.5:0.5.

[0370] The motion vector used in generating the bilateral template is called the initial motion vector. The initial motion vector can be represented by a motion vector calculated using at least one motion information derivation method, such as merge mode, AMVP mode, and skip mode.

[0371] (2) Correction of motion vector

[0372] The encoder / decoder can correct the motion vectors corresponding to each list of reference frames by comparing the values ​​of the samples in the bilateral template with the values ​​of the reconstructed samples in the reference frames.

[0373] The motion vector indicating the internal reference position (i.e., the position within the reference screen) of reference screen list 0 can be determined as the corrected first motion vector, wherein the motion vector has a value with minimum distortion relative to the corresponding position in the bilateral template. The motion vector indicating the internal reference position of reference screen list 1 can be determined as the corrected second motion vector, wherein the motion vector has a value with minimum distortion relative to the corresponding position in the bilateral template.

[0374] Specifically, when the distortion value (difference) between the value of the region in the reference frame indicated by the initial first motion vector and the value of the corresponding region in the bilateral template is greater than the distortion value between the value of the region in the reference frame indicated by the corrected first motion vector and the value of the corresponding region in the bilateral template, the first motion vector can be corrected. Similarly, when the distortion value between the value of the region in the reference frame indicated by the initial second motion vector and the value of the corresponding region in the bilateral template is greater than the distortion value between the value of the region in the reference frame indicated by the corrected second motion vector and the value of the corresponding region in the bilateral template, the second motion vector can be corrected.

[0375] When comparing the values ​​of samples (target samples) in the bilateral template with the values ​​of reconstructed samples (reference samples) in the reference image, the encoder / decoder calculates the inter-sample distortion using distortion calculation methods such as: Sum of Absolute Differences (SAD), Sum of Absolute Transform Differences (SATD), Sum of Squared Errors (SSE), Mean Squared Errors (MSE), or Mean-Demeaned SAD (MR-SAD). In this way, the encoder / decoder determines the motion vector indicating the position of the reference sample that presents the minimum distortion value relative to the corresponding target sample position as the corrected motion vector. In this case, the distortion value can be calculated for at least one of the luminance and chrominance components.

[0376] When the SAD values ​​of the internal reference regions indicated by the initial first motion vector and the internal reference regions indicated by the initial second motion vector are greater than predetermined values, bilateral template matching can be performed to correct the initial first motion vector and the initial second motion vector. Conversely, when the SAD values ​​of the internal reference regions indicated by the initial first motion vector and the internal reference regions indicated by the initial second motion vector are less than predetermined values, bilateral template matching may not be performed.

[0377] In the case of performing bidirectional prediction, when the value of the initial first motion vector corresponding to the first prediction direction (e.g., the L0 prediction direction) is equal to the value of the corrected first motion vector obtained by motion information correction, motion information correction may not be performed on the initial second motion vector corresponding to the second direction (e.g., the L1 prediction direction).

[0378] On the other hand, bilateral template matching can be performed recursively.

[0379] Specifically, after generating a second bilateral template using the modified first motion vector and the modified second motion vector, another bilateral template matching can be performed using the second bilateral template to perform secondary corrections on the previously modified first motion vector and the previously modified second motion vector. In this case, the secondary correction based on the bilateral template matching can be performed iteratively up to M times. Here, M is a positive integer (e.g., 2), and it can be a fixed value preset in the encoder / decoder or a variable value encoded by the encoder and transmitted as a signal. Alternatively, M can be determined based on the size of the target block to be encoded / decoded.

[0380] For example, when the height or width of the target block being encoded / decoded is less than 8, M can be set to 2.

[0381] For example, when the height or width of the target block being encoded / decoded is 8, M can be set to 2.

[0382] For example, when the width of the encoded / decoded target block is 8 and the height is 16 (or vice versa), M can be set to 4.

[0383] In addition to the above, M can be set to 8. In this case, four bilateral template matches can be performed for the first reference screen (e.g., L0 reference screen) and four bilateral template matches can be performed for the second reference screen (e.g., L1 reference screen).

[0384] On the other hand, bilateral template matching can also be performed recursively based on integer pixels, in the case of recursively performing bilateral template matching based on subpixels.

[0385] The corrected motion vectors generated by bilateral template matching can replace the initial motion vectors for motion compensation of the target block during encoding / decoding. Bilateral template matching can be performed on both the encoder and decoder sides according to the same rules.

[0386] In bilateral template matching, only restricted areas within the reference frame can be searched to find the corrected motion vectors.

[0387] Figure 14 and Figure 15 This is a diagram showing the area (hereinafter referred to as the search range) searched during bilateral template matching to find the corrected motion vector.

[0388] Reference Figure 14 The search range can be determined based on integer pixels, for example, a range of -M to +N pixels in both the horizontal and vertical directions. Here, M and N are positive integers.

[0389] Alternatively, the search range can be determined based on subpixels, for example, a range of -O to +P subpixels in both the horizontal and vertical directions. Here, O and P can be fractions. For example, the unit of subpixel can represent 1 / 2 pixel, 1 / 4 pixel, 1 / 8 pixel, 1 / 16 pixel, 1 / 32 pixel, etc. Furthermore, O and P representing subpixels can have positive integer values.

[0390] When the encoder / decoder searches for corrected motion vectors based on subpixels, the search range may include the region of neighboring subpixels that are spatially adjacent to a specific integer pixel indicated by the integer pixel motion vector. Figure 14 This shows an example where the unit of subpixel is 1 / 2 pixel.

[0391] Furthermore, the subpixel-based search range can be limited to reduce memory access bandwidth. For example, the subpixel-based search range can be limited to only subpixels that are arranged within the integer-pixel-based search region.

[0392] Figure 15 This is a diagram illustrating a method for limiting the search range of subpixels to a search range based on integer pixels.

[0393] Reference Figure 15The subpixels represented by the shaded circle are not included in the integer-pixel-based search range, where the motion vector for which the search is performed is based on integer pixels. In this case, the subpixel-based search range can be limited such that subpixels outside the integer-pixel-based search range are not searched when searching for the corrected motion vector. That is, the subpixel-based search range can be restricted such that only subpixels represented by the unshaded circle are searched to find the corrected motion vector. In this case, it is not necessary to obtain the integer-pixel information used to generate the shaded subpixels, thereby reducing memory access bandwidth.

[0394] On the other hand, the search range based on integer pixels may include at least one of the center point, the top point, the bottom point, the left point, and the right point.

[0395] Here, based on the distortion values ​​of the top, bottom, left, and right points, at least one point can be further searched. This at least one point can be at least one of the top-left, bottom-left, top-right, and bottom-right points. In this case, the sub-pixel-based search range can be set to at least one of the center point, top point, bottom point, left point, right point, top-left, bottom-left, top-right, and bottom-right points. For example, the sub-pixel-based search range can be set to the center point, top point, bottom point, left point, and right point.

[0396] The shape of the search area around the pixel indicated by the initial motion vector can be a two-dimensional graph (such as a square, rectangle, rhombus (diagonal square), or cross). The search area can have a fixed shape predefined in the encoder / decoder or a variable shape specified by information encoded by the encoder and transmitted as a signal.

[0397] Furthermore, the range of motion vector values ​​can be limited for at least one of the initial motion vector and the modified motion vector, ensuring that the search range does not extend beyond a specific distance or greater than the boundaries of the image. This limitation of motion vector values ​​can be achieved through cropping.

[0398] In this case, the range of motion vector values ​​can be defined as falling within a predetermined range preset in the encoder and decoder. Here, the predetermined range may represent a critical value that limits the motion vector values, and the critical value may be determined by at least one of a minimum value and a maximum value. Simultaneously, the range of motion vector values ​​can be a variable range encoded by the encoder and transmitted as a signal.

[0399] On the other hand, at least one of the search range based on integer pixels and the search range based on subpixels can be limited to a specific distance or greater than the boundary of the image.

[0400] On the other hand, motion information correction based on subpixels is only allowed after motion information correction has been performed based on integer pixels.

[0401] When using bilateral template matching, motion vectors on the decoder side can be improved without sending additional syntax elements. Alternatively, bilateral template matching can be used in bidirectional prediction merging mode or decoder motion vector derivation mode.

[0402] According to one embodiment, bilateral template matching can be performed when the current block does not correspond to at least one of the unidirectional prediction merging candidate, local illumination compensation mode, affine motion compensation mode, and sub-CU merging mode.

[0403] The following sections will describe motion correction patterns related to motion information correction. Here, a motion correction pattern can refer to a prediction pattern used to derive motion information for inter-frame prediction. For example, a motion correction pattern could be a merging pattern performed based on the motion information correction using bilateral template matching described above.

[0404] Using motion correction mode, areas with zero or minimal motion in the image can be effectively encoded.

[0405] The motion correction mode is a motion information derivation scheme based on the above motion information correction, and the above motion information correction method can be applied to the motion correction mode.

[0406] On the other hand, the availability or applicability of a motion correction mode can be determined based on information transmitted via signals through a bitstream. This information can be transmitted unit by unit, where a unit can be video, sequence, frame, strip, parallel block, coding tree unit (CTU), coding unit (CU), prediction unit (PU), transform unit (TU), or block. Whether a motion correction mode is available indicates whether the motion correction mode is available for the corresponding unit. Whether a motion correction mode is applicable indicates whether the motion correction mode is applicable to the corresponding unit.

[0407] The motion correction mode may include the following steps: (1) deriving motion information, and (2) generating corrected motion information by performing motion information correction on the derived motion information. Here, the motion information correction method may include the bilateral template matching described above. The corrected motion information obtained by performing the motion correction mode may be used for motion compensation of the current block to be encoded / decoded (also known as the encoding / decoding target block).

[0408] Meanwhile, the motion information that can be corrected through the motion correction mode may include at least one of the following: motion vector, reference frame, inter-frame indicator, reference frame index, prediction list utilization flag, weighting factor, and offset.

[0409] (1) Derivation of motion information

[0410] In motion correction mode, motion information can be derived from at least one of the following information types: spatially adjacent neighboring blocks, temporally adjacent neighboring blocks, predefined motion information, and motion information that appears most frequently in the reference frame. Here, the derived motion information can be recorded as motion correction candidates in a list of motion correction modes (motion correction mode list). Motion information derived from the motion information of spatially adjacent neighboring blocks is called spatial motion correction candidate, and motion information derived from the motion information of temporally adjacent neighboring blocks is called temporal motion correction candidate.

[0411] The method for deriving a motion vector as a piece of motion information will be described in detail below.

[0412] In motion correction mode, motion vectors can be derived from the motion vectors of spatially adjacent neighboring blocks, the motion vectors of temporally adjacent neighboring blocks, predefined motion vectors, and the motion vectors that appear most frequently in the reference frame.

[0413] A. Spatially adjacent neighboring blocks

[0414] In motion correction mode, a motion vector can be derived from the motion vectors of at least one spatially adjacent neighboring block. Here, spatially adjacent neighboring blocks may include... Figure 9 The neighboring blocks shown in the image.

[0415] In motion correction mode, motion vectors can be derived from multiple motion vectors of neighboring blocks that satisfy at least one of the following conditions:

[0416] - A block whose coded block flag is 0 (i.e., a block without residual signals).

[0417] - The block corresponding to the merge skip mode

[0418] - Blocks whose motion vector difference is zero (e.g., MVD = (0,0)) in spatially adjacent neighboring blocks.

[0419] - The block corresponding to the merge pattern,

[0420] - Blocks with inter-frame prediction indicators of PRED_LX (where X is an integer including zero) and

[0421] - The inter-frame prediction indicator is any one of PRED_BI, PRED_TRI, and PRED_QUAD for the block.

[0422] In motion correction mode, motion vectors can be derived from multiple motion vectors that skip / merge at least one of the candidates in space.

[0423] For example, when the reference frame of a spatial skip / merge candidate is different from the reference frame of the encoded / decoded target block (i.e., when the frame order count (POC) of the reference frame of the spatial skip / merge candidate is different from the frame order count (POC) of the reference frame of the encoded / decoded target block), motion vectors can be derived based on the POC value of the frame by scaling motion vectors derived from multiple motion vectors of spatially adjacent neighboring blocks.

[0424] B. Neighboring blocks that are temporally adjacent

[0425] In motion correction mode, motion vectors can be derived from multiple motion vectors of at least one of temporally adjacent neighboring blocks. Here, temporally adjacent neighboring blocks may include co-position blocks within a reference frame and co-position blocks within a co-position frame.

[0426] Here, the corresponding block in the reference image can be at least one of the following blocks:

[0427] -Refer to the corresponding block in the upper left corner (0,0) of the target block in the reference frame (hereinafter referred to as the corresponding reference block).

[0428] -Refer to the corresponding block in the image ( Figure 10 (C3 in the middle)

[0429] -Refer to the corresponding block in the image ( Figure 10 C1 in the middle), and

[0430] -Refer to the corresponding block in the image ( Figure 10 (H in the text).

[0431] Here, the corresponding block in the same frame can be at least one of the following blocks:

[0432] - The corresponding block in the same frame at the top left position (0,0) of the encoded / decoded target block.

[0433] -Coordinate blocks in the same frame ( Figure 10 (C3 in the middle)

[0434] -Coordinate blocks in the same frame ( Figure 10 C1 in the middle), and

[0435] -Coordinate blocks in the same frame ( Figure 10 (H in the text).

[0436] In motion correction mode, motion vectors can be derived from multiple motion vectors of neighboring blocks that satisfy at least one of the following conditions among temporally adjacent neighboring blocks:

[0437] - Blocks whose coded block flag is 0 in blocks placed within and outside the same block (i.e., blocks without residual signals).

[0438] - Blocks corresponding to the merge skip pattern placed within and outside of sibling blocks.

[0439] - A block whose motion vector difference is zero (e.g., MVD = (0,0)) is placed between blocks inside and outside the same block.

[0440] - Blocks corresponding to the merge pattern placed within and outside of sibling blocks.

[0441] - Blocks with an inter-frame prediction indicator of PRED_LX (where X is an integer including zero), and

[0442] - A block whose inter-frame prediction indicator is any one of PRED_BI, PRED_TRI, and PRED_QUAD and is placed in blocks within or outside the same block.

[0443] In motion correction mode, motion vectors can be derived from multiple motion vectors of at least one of the time skip / merge candidates.

[0444] For example, when the reference frame or co-position frame of the time skip / merge candidate is different from the reference frame of the encoding / decoding target block (i.e., when the frame order counts (POCs) of the time skip / merge candidate and the reference frame of the encoding / decoding target block are different from each other), motion vectors can be derived based on the POC value of the frame by scaling motion vectors derived from multiple motion vectors of temporally adjacent neighboring blocks.

[0445] Additionally, reference screen indices indicating reference screens including temporally adjacent neighboring blocks and co-located screens can be derived from the reference screen indices of spatially adjacent neighboring blocks.

[0446] C. Predefined motion vectors

[0447] In motion correction mode, a zero motion vector with a value of (0,0) can be derived as a motion vector.

[0448] D. The motion vector that appears most frequently in the reference frame.

[0449] In motion correction mode, the motion vectors that appear most frequently in the reference frame can be derived.

[0450] In this scenario, the motion vectors are arranged in order of frequency of appearance in the reference frame, and the maximum number of L motion vectors can be derived based on this frequency order. Here, L is a positive integer. In this case, L can be a fixed value preset in the encoder / decoder, or it can be a variable value encoded by the encoder and transmitted as a signal.

[0451] On the other hand, in motion correction mode, motion vectors can be derived in the order of items A, B, C, and D. In this case, the derivation of motion vectors can be repeated until a total of M motion vectors are derived. Furthermore, motion vectors can be derived sequentially in the order described above. When a newly derived motion vector already exists among the previously derived motion vectors, the derivation of motion vectors stops, and thus a maximum of M motion vectors can be derived and used in the motion correction mode. Here, M is a positive integer, and it can be a fixed value preset in the encoder / decoder or a variable value encoded by the encoder and transmitted as a signal. Here, when there are multiple spatially adjacent neighboring blocks, bidirectional predicted motion vectors can be derived first and inserted into the motion correction mode list, and subsequently, unidirectional predicted motion vectors can be derived and inserted into the motion correction mode list.

[0452] The motion vector derivation under the above motion correction mode can be performed according to the processing described below.

[0453] For example, motion vectors can be derived in the order of spatially adjacent neighboring blocks, temporally adjacent neighboring blocks, and zero motion vector (0,0).

[0454] As another example, motion vectors can be derived in the order of motion vectors of temporally adjacent neighboring blocks, motion vectors of spatially adjacent neighboring blocks, and zero motion vector (0,0).

[0455] Here, the motion vectors of spatially adjacent neighboring blocks can be derived in the order of deriving spatial skip / merge candidates, and the motion vectors of temporally adjacent neighboring blocks can be derived in the order of deriving temporal skip / merge candidates.

[0456] In motion correction mode, the derived motion vector can only be used if the value of the motion vector derived by the above method is not (0,0).

[0457] For example, when using bidirectional prediction in motion correction mode, the derived motion vector can only be used if neither the first nor the second derived motion vector has a value of (0,0).

[0458] When using N-direction prediction (N is an integer of 2 or greater) in motion correction mode, the derived motion vectors can only be used if the values ​​of the N derived motion vectors are not (0,0).

[0459] In N-direction prediction (where N is an integer of 2 or greater), only some motion vectors with values ​​other than (0,0) can be used for motion vectors in motion correction mode from the derived motion vectors.

[0460] Additionally, when using bidirectional prediction in motion correction mode, if the derived first motion vector and the derived second motion vector are not equal to each other, one or both of the derived first motion vector and the derived second motion vector can be used in motion correction mode.

[0461] On the other hand, when the inter-frame prediction indicator of spatially adjacent neighboring blocks or temporally adjacent neighboring blocks indicates at least one of bidirectional, tridirectional, and quadridirectional prediction, the derived motion vector can be used in motion correction mode. That is, when the inter-frame prediction indicator of spatially adjacent neighboring blocks or temporally adjacent neighboring blocks does not indicate unidirectional prediction, the derived motion vector can be used in motion correction mode.

[0462] When the inter-frame prediction indicator for spatially adjacent neighboring blocks or temporally adjacent neighboring blocks indicates unidirectional prediction, the inter-frame prediction indicator is changed to indicate bidirectional prediction, and the sign of the motion vector for the unidirectional prediction is reversed. Therefore, a motion vector for a unidirectional prediction with a direction opposite to the previous (original) unidirectional prediction is derived and used as the motion vector in the motion correction mode.

[0463] For example, when the inter-frame prediction indicator for spatially or temporally adjacent neighboring blocks indicates L0 unidirectional prediction and the motion vector for the spatially or temporally adjacent neighboring blocks is (-4, 6), the inter-frame prediction indicator is changed to indicate bidirectional prediction, and the motion vector (4, -6) is derived for L1 directional prediction. Therefore, the two motion vectors (-4, 6) and (4, -6) can be used as motion vectors in the motion correction mode. In this case, a reference frame located at a first distance from the current frame can be determined as the reference frame for L1 unidirectional prediction, wherein the first distance is the same as or proportional to a second distance, which is the distance between the current frame and the reference frame for L0 unidirectional prediction.

[0464] In motion correction mode using N-direction prediction (N being an integer of 2 or greater), when the number of derived motion vectors is not N, one or more motion vectors can be further derived from one or more previously derived motion vectors to obtain a total of N motion vectors. Specifically, multiple motion vectors can be generated by scaling the previously derived motion vectors generated based on the frame sequence count of the current frame and / or reference frame.

[0465] For example, when using bidirectional prediction in motion correction mode and only the previously derived first motion vector exists, the first motion vector is scaled based on reference frames in reference frame list 1 to generate a second motion vector. Therefore, the previously derived first motion vector and the newly generated second motion vector can be used as motion vectors in motion correction mode.

[0466] Alternatively, when using bidirectional prediction in motion correction mode and only the previously derived second motion vector exists, the second motion vector is scaled based on reference frames in reference frame list 0 to generate a first motion vector. Therefore, the previously derived second motion vector and the newly generated first motion vector can be used as motion vectors in motion correction mode.

[0467] The following text will describe in detail the method of deriving a reference image that also serves as a piece of motion information.

[0468] In motion correction mode, at least one of the following methods can be used to derive the reference frame.

[0469] For example, a reference frame with reference frame index 0 can be derived from the reference frames included in reference frame list 0 and the reference frames included in reference frame list 1, and used as a reference frame for motion correction mode. Among the reference frames included in reference frame list 0 and reference frame list 1, there may be cases where the reference frame with reference frame index 0 in reference frame list 0 and reference frame list 1 are the same. In this case, among the reference frames included in reference frame list 1, a reference frame different from the reference frame included in reference frame list 0 with reference frame index 0 can be derived as a reference frame for use in motion correction mode. Again, among the reference frames included in reference frame list 0 and reference frame list 1, there may be cases where the reference frame with reference frame index 0 in reference frame list 0 and reference frame list 1 are the same. In this case, among the reference frames included in reference frame list 0, a reference frame different from the reference frame included in reference frame list 1 with reference frame index 0 can be derived as a reference frame for use in motion correction mode.

[0470] To derive the reference frame to be used in motion correction mode as described above, reference frame index 0 can be used. However, the invention is not limited thereto, and a reference frame index with a non-zero value can be used to derive the reference frame to be used in motion correction mode.

[0471] The following describes a method for deriving the reference frame to be used in motion correction mode by using a non-zero reference frame index.

[0472] The median of the reference frame indices for at least one of the locations A1, B1, B0, A0, and B2 spatially adjacent to the encoding / decoding target block can be used as the reference frame index to be used. Here, instead of the median, various statistical values ​​such as the minimum, maximum, average, weighted average, and mode can be used. The reference frames indicated by the determined reference frame indices can be derived as the reference frames to be used in motion correction mode. In this case, the reference frame indices can be determined for each list of reference frames using the method described above.

[0473] When a block exists only at location A1 and that block has an inter-picture prediction mode, the reference picture for that block can be determined solely by the reference picture index at location A1.

[0474] When a block exists only at location B1 and that block has an inter-picture prediction mode, the reference picture for that block can be determined solely by the reference picture index at location B1.

[0475] When a block exists only at location B0 and that block has an inter-picture prediction mode, the reference picture for that block can be determined solely by the reference picture index at location B0.

[0476] When a block exists only at location A0 and that block has an inter-picture prediction mode, the reference picture for that block can be determined solely by the reference picture index at location A0.

[0477] When a block exists only at location B2 and that block has an inter-picture prediction mode, the reference picture for that block can be determined solely by the reference picture index at location B2.

[0478] A corrected reference frame can be generated by applying the motion information correction method to a reference frame derived using the above method, and the corrected reference frame can be used as a reference frame for the motion correction mode.

[0479] In the above embodiment, it is assumed that two reference screen lists will be used. However, the number of reference screen lists is not limited to this. There may be N reference screen lists. In this case, N can be an integer of 2 or greater. When there are N reference screen lists, the derivation of reference screens can be performed in a similar manner to the method described above (i.e., based on the determination of the consistency of reference screens in the N reference screen lists). That is, N distinct reference screens can be derived.

[0480] In another embodiment, among the reference frames included in the reference frame list i (i = 0, ..., or N, where N is an integer of 0 or greater), the reference frame that has the smallest difference in frame order count (POC) with the encoded / decoded target frame can be derived as the reference frame for the motion correction mode.

[0481] Alternatively, among the reference frames included in the reference frame list i (i = 0, ..., or N, where N is an integer of 1 or greater), the reference frame with the smallest difference in frame order count (POC) with the encoded / decoded target frame and / or with the smallest time layer identifier value can be deduced as the reference frame for motion correction mode.

[0482] Alternatively, reference frames of spatially adjacent neighboring blocks can be derived as reference frames for the motion correction mode.

[0483] Alternatively, reference frames of temporally adjacent neighboring blocks can be derived as reference frames for the motion correction mode.

[0484] Further, alternatively, one or more reference frames selected from the skip / merge candidates can be derived as reference frames for the motion correction mode.

[0485] Further, alternatively, motion information correction processing is performed on all or some of the reference frames included in the reference frame list, and then the reference frame that presents the minimum distortion value after the correction processing is derived as the reference frame of the motion correction mode.

[0486] The following section will describe in detail the method for deriving an inter-frame prediction indicator as a piece of motion information.

[0487] In motion correction mode, at least one of the following methods can be used to derive the inter-frame prediction indicator.

[0488] For example, in motion correction mode, the inter-frame prediction indicator can be fixed to one of unidirectional prediction, bidirectional prediction, tridirectional prediction, quadridirectional prediction, and N-directional prediction.

[0489] As another example, the inter-frame prediction indicator for spatially adjacent neighboring blocks can be derived as the inter-frame prediction indicator to be used in motion correction mode.

[0490] As another example, the inter-frame prediction indicator for temporally adjacent neighboring blocks can be derived as the inter-frame prediction indicator to be used in motion correction mode.

[0491] As another example, one or more inter-frame prediction indicators from the skip / merge candidates can be derived as inter-frame prediction indicators that will be used in motion correction mode.

[0492] As another example, an inter-frame prediction indicator can be derived based on the number of available reference frames derived using the method described above. This inter-frame prediction indicator will be used in motion compensation mode. Specifically, when only one available reference frame exists, the inter-frame prediction indicator can be derived using unidirectional prediction. Specifically, when two available reference frames exist, the inter-frame prediction indicator can be derived using bidirectional prediction.

[0493] A modified inter-frame prediction indicator is generated by applying the motion information correction method to the previously derived inter-frame prediction indicator, and the modified inter-frame prediction indicator can be used as the inter-frame prediction indicator in the motion correction mode of the current block.

[0494] (2) Correction of motion information

[0495] The encoder / decoder generates a corrected reference frame by performing motion information correction on a reference frame derived using the method described above, and uses the corrected motion information as motion information in the motion correction mode. Specifically, motion compensation can be performed by generating a prediction block using at least one of the corrected motion vector, the corrected reference frame, and the corrected inter-frame prediction indicator. Similarly, the encoder / decoder generates a corrected reference frame index, a corrected prediction list utilization flag, a corrected weighting factor, and a corrected offset by applying the motion information correction method to at least one of the previously derived reference frame index, the previously derived prediction list utilization flag, the previously derived weighting factor, and the previously derived offset, and creates a prediction block using at least one of the corrected reference frame index, the corrected prediction list utilization flag, the corrected weighting factor, and the corrected offset. Here, the motion information correction method may include the bilateral template matching described above. In this case, a weighted sum of the prediction blocks is calculated based on the frame order count between the encoded / decoded target frame and each reference frame to obtain the final prediction block, and thus motion compensation can be performed.

[0496] On the other hand, when motion information correction processing is applied to previously derived motion vectors, the previously derived motion vectors can be used to determine the initial motion search position. The corrected motion vector can be determined based on the initial motion search position.

[0497] The encoder / decoder can perform motion information correction on up to M derived motion information pieces. In this case, M can be a fixed value preset in the encoder / decoder, or it can be a variable value encoded by the encoder and transmitted by a signal. For example, at least one motion information piece among the first, second, ..., and Nth motion information pieces can be corrected. In this case, M is an integer equal to or greater than 4.

[0498] Furthermore, without applying motion information correction to all or part of the previously derived motion information, the previously derived motion information can be used as the motion information for the motion correction mode. Specifically, motion compensation can be performed by generating prediction blocks using at least one of the derived motion vectors, derived reference frames, and derived inter-frame prediction indicators.

[0499] Simultaneously, motion information correction (e.g., bilateral template matching) can be selectively used in motion correction mode.

[0500] According to one embodiment, motion information correction is performed when the current block is unrelated to at least one of the unidirectional prediction merging candidate, local illumination compensation mode, affine motion compensation mode, and sub-CU merging mode.

[0501] Motion information correction can be performed when only one of the first difference and the second difference is a negative integer value, wherein the first difference is the POC difference between the first reference frame and the encoded / decoded target frame corresponding to the first prediction direction (e.g., the L0 prediction direction). ref0 - POC curr The second difference is the POC difference between the second reference frame and the encoded / decoded target frame corresponding to the second prediction direction (e.g., the L1 prediction direction). ref1 - POC curr ).

[0502] Furthermore, motion information correction can be performed when the first prediction direction and the second prediction direction are different from each other.

[0503] The encoder / decoder can perform bilateral template matching on at least one derived motion information obtained through the above method, generate a prediction block using the motion information with the least distortion, and perform motion compensation. In this case, the at least one motion information may include at least one of L0 motion information and L1 motion information.

[0504] The encoder / decoder can perform bilateral template matching on one or more candidates that exist in the skip candidate list or merge candidate list, generate a prediction block using the candidate with the least distortion, and perform motion compensation.

[0505] The encoder / decoder can perform bilateral template matching on at least one derived motion information obtained through the above method, generate M prediction blocks using M motion information pieces, each with minimum distortion, calculate the weighted sum of the M prediction blocks, and use the weighted sum as the final prediction block for encoding / decoding the target block. Here, M can be a positive integer and can be equal to or greater than 2.

[0506] The encoder / decoder can perform bilateral template matching on one or more candidates included in the skip candidate list or merge candidate list, generate M prediction blocks using the M candidates that all have the minimum distortion, calculate the weighted sum of the M prediction blocks, and use the weighted sum as the final prediction block for encoding / decoding the target block. Here, M can be a positive integer and can be equal to or greater than 2.

[0507] The encoder / decoder can perform entropy encoding / decoding on the skip index or merge index of a unidirectional prediction candidate that exists in a skip candidate list or merge candidate list, and can perform motion compensation by generating a prediction block using at least one candidate with the minimum distortion from M or more bidirectional prediction candidates. In this case, an entropy encoding / decoding is performed on a flag or an index to indicate M or more bidirectional prediction candidates. That is, a skip index or merge index is assigned to each unidirectional prediction candidate, while a single skip index or a single merge index is shared among the M or more bidirectional prediction candidates. For M or more bidirectional prediction candidates, since at least one candidate representing the minimum distortion can be determined using bilateral template matching, it is not necessary to assign a skip index or a merge index to each bidirectional prediction candidate.

[0508] Whether to use a motion correction mode can be determined based on the motion correction mode utilization information. Here, at least one of the flag information and index information can be used to entropy encode / decode the motion correction mode utilization information.

[0509] The motion correction mode utilization information can be encoded / decoded based on the value of the skip flag. The encoding / decoding timing of the motion correction mode utilization information can be determined based on the encoding / decoding timing of the skip flag. For example, when the skip flag is 1 (i.e., when skip mode is used), the motion correction mode utilization information can be entropy encoded / decoded. In this case, the motion correction mode utilization information can be entropy encoded / decoded after the skip flag is entropy encoded / decoded.

[0510] Conversely, the skip flag can be encoded / decoded based on the value of the motion correction mode utilization information. For example, when the motion correction mode utilization information is 1 (i.e., when the motion correction mode is used), the skip flag can be entropy-decoded / entropy-decoded. In this case, the motion correction mode utilization information can be entropy-coded / entropy-decoded before the skip flag is entropy-coded / entropy-decoded.

[0511] Encoding / decoding of motion correction mode utilization information can be performed based on the value of the merge flag. The encoding / decoding timing of the motion correction mode utilization information can be determined based on the encoding / decoding timing of the merge flag. For example, when the merge flag is 1 (i.e., when the merge mode is used), entropy encoding / decoding of the motion correction mode utilization information can be performed. In this case, entropy encoding / decoding of the motion correction mode utilization information can be performed after entropy encoding / decoding of the merge flag.

[0512] Conversely, encoding / decoding of the merge flag can be performed based on the value of the motion correction mode utilization information. For example, when the motion correction mode utilization information is 1 (i.e., when the motion correction mode is used), entropy encoding / decoding of the merge flag can be performed. In this case, entropy encoding / decoding of the motion correction mode utilization information can be performed before entropy encoding / decoding of the merge flag.

[0513] The motion correction mode can be encoded / decoded using information based on a specific motion compensation mode. For example, when an affine motion compensation mode is not used, the motion correction mode can be entropy encoded / decoded using information.

[0514] The motion vector derivation mode flag of the decoder can be encoded / decoded based on the value of the motion correction mode utilization information. That is, when the motion correction mode utilization information is 0 (i.e., when the motion correction mode is not used), the motion vector derivation mode flag on the decoder side can be entropy encoded / entropy decoded.

[0515] Furthermore, motion correction mode utilization information can be encoded / decoded based on motion correction mode utilization information from one or more neighboring blocks of the target block. For example, one or more neighboring blocks of the target block may include one or more spatially adjacent blocks and / or temporally adjacent blocks. One or more spatially adjacent blocks may include blocks located to the left of the target block and / or blocks located above the target block.

[0516] Furthermore, when motion correction mode utilization information is not transmitted via signal, it can be derived based on motion correction mode utilization information of one or more neighboring blocks of the encoded / decoded target block. One or more neighboring blocks of the encoded / decoded target block may include one or more spatially adjacent blocks and / or temporally adjacent blocks. One or more spatially adjacent blocks may include blocks located to the left of the encoded / decoded target block and / or blocks located above the encoded / decoded target block.

[0517] When at least one of the blocks spatially adjacent to the target block being encoded / decoded uses a skip mode, the motion correction mode can utilize information for entropy encoding / decoding.

[0518] When at least one of the blocks spatially adjacent to the target block being encoded / decoded uses a merge mode, the motion correction mode can be entropy encoded / decoded using information.

[0519] When at least one of the blocks spatially adjacent to the target block being encoded / decoded uses an inter-frame mode, the motion correction mode can utilize information for entropy encoding / entropy decoding.

[0520] Bypass mode can be used to entropy encode / decode information from motion correction modes.

[0521] When using motion correction mode and skip mode together, entropy encoding / decoding of the residual signal is not required.

[0522] When used together with motion correction mode and merge mode, the residual signal can be entropy encoded / entropy decoded.

[0523] In motion correction mode, only a portion of the residual signal can be entropy encoded / decoded for use. In this case, this portion of the residual signal can be a DC quantization level (DC transform coefficient).

[0524] When using motion correction mode, entropy encoding / decoding of information other than that used by the motion correction mode is not required. Here, the information other than that used by the motion correction mode can be at least one piece of information about motion compensation.

[0525] The motion correction modes related to motion information correction have been described above. The following section will describe the motion correction candidates derived from these motion correction modes.

[0526] Motion correction candidates can be motion information including at least one of the following: derived motion vectors derived through motion correction modes, derived reference frames, and derived inter-frame prediction indicators. The encoder / decoder can add motion correction candidates as skip candidates to the skip candidate list or as merge candidates to the merge candidate list in skip or merge modes.

[0527] An example of adding motion correction candidates to the skip / merge candidate list will be described below.

[0528] If a candidate for skipping / merging that is identical to the motion correction candidate already exists in the skip / merge candidate list, the motion correction candidate may not be added to the skip / merge candidate list.

[0529] Motion correction candidates can be derived earlier than space skip / merge candidates, and motion correction candidates can be added to the skip / merge candidate list.

[0530] Motion correction candidates can be derived earlier than spatial skip / merge candidates derived from a specific location, and these motion correction candidates can then be added to the skip / merge candidate list. Here, the specific location can be... Figure 10 At least one of positions A1, B1, B0, A0, and B2.

[0531] Furthermore, a motion correction candidate can be derived earlier than at least one of the time skip / merge candidates, combined merge candidates, and merge candidates with predetermined motion information values, and the motion correction candidate can be added to the skip / merge candidate list.

[0532] The encoder / decoder can identify motion information, including at least one of the following: a corrected motion vector obtained through motion information correction via motion correction mode, a corrected reference frame, and a corrected inter-frame prediction indicator, as motion correction candidates. The encoder / decoder can add motion correction candidates as skip candidates to the skip candidate list in skip mode or as merge candidates to the merge candidate list in merge mode.

[0533] Motion correction mode can be used instead of skip mode. In other words, the image can be encoded / decoded using motion correction mode instead of skip mode. Motion correction mode can also be used instead of merge mode. In other words, the image can be encoded / decoded using motion correction mode instead of merge mode.

[0534] At least one of the overlapping block motion compensation mode, local illuminance compensation mode, and bidirectional optical flow mode can be applied to the final prediction block generated using the motion correction mode.

[0535] Furthermore, in motion correction mode, the motion information correction method can be applied to only one piece of motion information or a part of motion information, without generating a list of motion information candidates.

[0536] The conditions for performing motion information correction in Motion Correction Mode (MRM) will be described below.

[0537] Whether to perform motion information correction can be determined based on the frame sequence count (POC) of the reference frame for the motion vector.

[0538] For example, a motion information correction method can be performed when the following two motion vectors exist: a first motion vector indicating a reference frame whose frame sequence count is smaller than the frame sequence count of the encoded / decoded target frame; and a second motion vector indicating a reference frame whose frame sequence count is larger than the frame sequence count of the encoded / decoded target frame.

[0539] Alternatively, when there are two motion vectors of a reference frame whose frame sequence count is smaller than the frame sequence count of the encoded / decoded target frame, a motion information correction method may be performed.

[0540] Further, alternatively, when there are two motion vectors of a reference frame whose frame sequence count is greater than the frame sequence count of the encoded / decoded target frame, a motion information correction method may be performed.

[0541] Motion information correction can be performed on the encoded / decoded target block when at least one of the affine motion compensation mode, decoder motion vector derivation mode, and local illumination compensation mode is not used.

[0542] Motion information correction can be performed when the difference between the frame sequence count of the encoded / decoded target frame and the frame sequence count of the first reference frame, or the difference between the frame sequence count of the encoded / decoded target frame and the frame sequence count of the second reference frame, is less than N (N is an integer greater than or equal to 0). In this case, the first reference frame may refer to the reference frame indicated by the first motion vector, and the second reference frame may refer to the reference frame indicated by the second motion vector.

[0543] Motion information correction can be performed based on a first motion vector and a second motion vector, which serve as the target for motion information correction.

[0544] For example, motion information correction may not be performed when the first motion vector and the second motion vector are the same, and when the reference frame indicated by the first motion vector and the reference frame indicated by the second motion vector are the same. In other words, motion information correction may be performed only when the first motion information and the second motion information are different from each other and / or when the reference frames indicated by the first motion information and the second motion information are different from each other. Furthermore, motion information correction may not be performed when the first motion vector and the second motion vector are the same. In other words, motion information correction may be performed only when the first motion vector and the second motion vector are different from each other.

[0545] Optionally, motion information correction may be performed only when the reference frame indicated by the first motion vector is the same as the reference frame indicated by the second motion vector. Conversely, motion information correction may be performed only when the reference frame indicated by the first motion vector is different from the reference frame indicated by the second motion vector.

[0546] As another example, motion information correction can be performed on motion vectors whose values ​​are not (0,0). Here, motion information correction can be performed when the inter-view prediction indicator indicates bidirectional and when the values ​​of both the first and second motion vectors are not (0,0). Furthermore, motion information correction can be omitted when the inter-view prediction indicator indicates N directions (N being an integer equal to or greater than 2) and when the values ​​of all or a predetermined number of N motion vectors are not (0,0).

[0547] Motion information correction can only be performed when the inter-frame prediction indicator indicates a specific number of directions. For example, motion information correction can only be performed when the inter-frame prediction indicator for skip / merge candidates indicates two-way.

[0548] Motion information correction may be performed only when the reference frame index of the reference frame indicated by the first motion vector is 0 and the reference frame index of the reference frame indicated by the second motion vector is 0. Alternatively, motion information correction may be performed only when the reference frame index of the reference frame indicated by each motion vector has a specific value.

[0549] Motion information correction can be performed only on at least one of the spatial skip / merge candidates, temporal skip / merge candidates, combined skip / merge candidates, and skip / merge candidates with predetermined motion information values.

[0550] The target block for encoding / decoding can be divided into sub-blocks, and motion information correction can be performed based on each sub-block.

[0551] On the other hand, when the encoded / decoded target block has motion information or motion vectors for each sub-block, motion information correction can be performed based on each sub-block to improve encoding efficiency. It is possible that all sub-blocks of the encoded / decoded target block have different motion information or motion vectors, or have the same motion information or motion vectors. Therefore, motion information correction can only be performed when all sub-blocks of the encoded / decoded target block have different motion information or motion vectors, or have the same motion information or motion vectors.

[0552] When the encoded / decoded target block has motion information or motion vectors based on each sub-block, motion information correction can be omitted to reduce computational complexity. Motion information correction can only be performed when all sub-blocks of the encoded / decoded target block have the same motion information or motion vectors to reduce computational complexity.

[0553] Motion information correction can be performed based on each sample point or each block.

[0554] When encoding / decoding target blocks using N-directional prediction (such as three-directional and four-directional prediction), motion information correction can be performed by calculating a template using N motion vectors (such as three-directional and four-directional motion vectors). In this case, M is an integer equal to or greater than 3.

[0555] When bilateral template matching is used as a motion information correction method, new motion information can be generated by scaling the previously derived motion information.

[0556] For example, when there is only one motion information (i.e., the first motion information) or only the first motion vector in spatially adjacent neighboring blocks or temporally adjacent neighboring blocks, scaling is performed based on the first motion information or the first motion vector to generate the second motion information or the second motion vector, and then motion information correction can be performed.

[0557] Similarly, when there is only one motion information (i.e., the second motion information) or only the second motion vector in spatially adjacent neighboring blocks or temporally adjacent neighboring blocks, scaling is performed based on the second motion information or the second motion vector to generate the first motion information or the first motion vector, and then motion information correction can be performed.

[0558] As another example, when the reference frame indicated by the initial motion vector differs from the reference frame indicated by the modified motion vector candidate, the initial motion vector is scaled based on the Frame Order Count (POC) value of each reference frame to obtain the modified motion vector candidate. Here, the modified motion vector candidate represents the motion vector indicating the region to which a search is performed to find the modified motion vector. To obtain the modified motion vector, bilateral template matching can be performed by comparing the distortion value of the position indicated by the modified motion vector with the distortion value of the position indicated by the initial motion vector.

[0559] On the other hand, the corrected motion vector can be calculated by applying motion information correction to at least one of the motion vectors of the luminance component and the two motion vectors of the chrominance component.

[0560] On the other hand, motion information correction is performed on all or part of the reference frames included in the reference frame list, and the reference frame that presents the minimum distortion value after the correction process can then be derived as the corrected reference frame.

[0561] The same conditions for performing motion information correction can be applied to motion information correction for the decoder-side motion vector derivation mode.

[0562] Figure 16 This is a flowchart illustrating an image decoding method according to yet another embodiment of the present invention.

[0563] Reference Figure 16 The decoder can deduce motion correction candidates from at least one of the following: motion information of spatially adjacent neighboring blocks, motion information of temporally adjacent neighboring blocks, predefined motion information, and motion information that appears most frequently in the reference frame (S1601).

[0564] In this scenario, motion correction candidates can be derived in a predetermined order: motion information of spatially adjacent neighboring blocks, motion information of temporally adjacent neighboring blocks, predefined motion information, and the order of motion information appearing with the highest probability within the reference frame. The predetermined order may represent the order of motion information of spatially adjacent neighboring blocks, motion information of temporally adjacent neighboring blocks, and the order of predefined motion information. Here, the predefined motion information may include the zero vector.

[0565] Temporally adjacent neighboring blocks can be included in a reference screen selected based on the reference screen index of spatially adjacent neighboring blocks.

[0566] Next, motion information correction can be performed on the previously derived motion correction candidates (S1602).

[0567] In this case, motion information correction can be performed by applying bilateral template matching to the motion vectors present in the derived motion correction candidates.

[0568] Here, bilateral template matching may include the following steps: generating a bilateral template by using a motion vector selected from the derived motion correction candidates as an initial motion vector; and correcting the initial motion vector by comparing the values ​​of the samples in the bilateral template with the values ​​of the reconstructed reference samples in the reference image indicated by the initial motion vector.

[0569] Here, the steps to correct the initial motion vector can be performed recursively.

[0570] The initial motion vector can be a bidirectional predicted motion vector that is a non-zero motion vector, and it is selected from the derived motion correction candidates. Here, if there is no bidirectional predicted motion vector that is a non-zero vector among the derived motion correction candidates, the initial motion vector can be set to a zero vector.

[0571] On the other hand, the steps to correct the initial motion vector include the following steps: searching for the motion vector of the region with the smallest distortion in the reference image compared to the corresponding region in the bilateral template; and setting the found motion vector as the correction value of the initial motion vector.

[0572] Here, the step of searching for motion vectors can be performed within a limited search range in the reference image.

[0573] In this case, the limited search range can be set to a predetermined area demarcated based on integer pixels, and the step of searching for motion vectors can be performed based on sub-pixels within the predetermined search range demarcated based on integer pixels.

[0574] On one hand, bilateral template matching can be performed based on integer pixels and subpixels. In this case, a step of searching for motion vectors can be performed to search for motion vectors based on subpixels within a predetermined search range demarcated based on integer pixels.

[0575] Meanwhile, when the block does not correspond to any of the unidirectional prediction merging candidates, local illumination compensation mode, and affine motion compensation mode, the step of performing motion information correction on the derived motion correction candidate can be executed.

[0576] Next, a prediction block for the encoding / decoding target block (i.e., the current block) is generated by using motion correction candidates obtained from motion information correction (S1603).

[0577] The image decoding method may further include the following steps: decoding the motion correction mode information performed before step S1601; and determining the motion correction mode based on the decoded motion correction mode information. Step S1601 may only be executed if, as a result of the determination step, the current block is determined to correspond to the motion correction mode.

[0578] Here, the decision to perform the step of decoding motion correction mode information can be based on either the skip flag or the merge flag.

[0579] In the presence of multiple spatially adjacent neighboring blocks, a step of deriving motion correction candidates can be performed, such that motion information is first derived from spatially adjacent neighboring blocks with bidirectional predicted motion vectors, and then motion information is derived from spatially adjacent neighboring blocks with unidirectional predicted motion vectors.

[0580] The image decoding method according to the present invention has been described above. Each step of the image decoding method described above can be performed in the same manner as the image encoding method.

[0581] The above embodiments can be performed in the same way in both the encoder and decoder.

[0582] The order in which the above embodiments are applied may differ between the encoder and the decoder, or the order in which the above embodiments are applied may be the same in the encoder and the decoder.

[0583] The above embodiments can be performed on each luminance signal and chrominance signal, or the above embodiments can be performed on both luminance and chrominance signals in the same way.

[0584] The block shape of the above embodiments of the present invention can be square or non-square.

[0585] The above embodiments of the present invention can be applied based on the size of at least one of the coding block, prediction block, transform block, block, current block, coding unit, prediction unit, transform unit, unit, and current unit. Here, size can be defined as the minimum size or maximum size, or both, for which the above embodiments are applied, or it can be defined as a fixed size to which the above embodiments are applied. Furthermore, in the above embodiments, a first embodiment can be applied to a first size, and a second embodiment can be applied to a second size. In other words, the above embodiments can be applied in combination based on sizes. Furthermore, the above embodiments can be applied when the size is equal to or greater than the minimum size and equal to or less than the maximum size. In other words, the above embodiments can be applied when the block size is included within a specific range.

[0586] For example, the above embodiments can be applied when the size of the current block is 8×8 or larger. For example, the above embodiments can be applied when the size of the current block is 4×4 or larger. For example, the above embodiments can be applied when the size of the current block is 16×16 or larger. For example, the above embodiments can be applied when the size of the current block is equal to or greater than 16×16 and equal to or less than 64×64.

[0587] The above embodiments of the present invention can be applied according to time layers. To identify the time layer to which the above embodiments can be applied, they can be applied to a specific time layer identified by a corresponding identifier. Here, the identifier can be defined as the lowest or highest layer to which the above embodiments can be applied, or both, or it can be defined as indicating a specific layer to which the embodiment is applied. Furthermore, a fixed time layer to which the embodiments are applied can be defined.

[0588] For example, the above embodiments can be applied when the time layer of the current image is the lowest layer. For example, the above embodiments can be applied when the time layer identifier of the current image is 1. For example, the above embodiments can be applied when the time layer of the current image is the highest layer.

[0589] The strip types to which the above embodiments of the present invention are applied can be defined, and the above embodiments can be applied according to the corresponding strip types.

[0590] The above embodiments of the invention can also be applied when the motion vector has at least one of 16 pel units, 8 pel units, 4 pel units, an integer pel unit, 1 / 8 pel unit, 1 / 16 pel unit, 1 / 32 pel unit, and 1 / 64 pel unit. The motion vector can be selectively used for each pixel unit.

[0591] In the above embodiments, the method is described based on a flowchart having a series of steps or units. However, the present invention is not limited to the order of these steps, but some steps may be performed simultaneously with other steps or in a different order. Furthermore, those skilled in the art should understand that the steps in the flowchart are not mutually exclusive, and other steps may be added to the flowchart or some steps may be deleted from the flowchart without affecting the scope of the present invention.

[0592] The embodiments include various aspects of the examples. Not all possible combinations of the aspects may be described, but those skilled in the art will recognize different combinations. Therefore, the invention may include all substitutions, modifications, and alterations within the scope of the claims.

[0593] Embodiments of the present invention can be implemented in the form of program instructions, which can be executed by various computer components and recorded in a computer-readable recording medium. The computer-readable recording medium may individually include program instructions, data files, data structures, etc., or may include a combination of program instructions, data files, data structures, etc. The program instructions recorded in the computer-readable recording medium may be specifically designed and constructed for the present invention or are well known to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic recording media (such as hard disks, floppy disks, and magnetic tapes), optical data storage media (such as CD-ROMs or DVD-ROMs), magneto-optical media (such as floppy disks), and hardware devices (such as read-only memory (ROM), random access memory (RAM), flash memory, etc.) specifically configured to store and implement program instructions. Examples of program instructions include not only machine language code formatted by a compiler but also high-level language code that can be implemented by a computer using an interpreter. The hardware device may be configured to be operated by one or more software modules or vice versa to implement the processing according to the present invention.

[0594] This disclosure also provides the following configurations:

[0595] 1. An image decoding method, comprising:

[0596] Motion correction candidates are derived from motion information of spatially neighboring blocks, motion information of temporally neighboring blocks, predefined motion information, and motion information that appears most frequently in the reference frame.

[0597] Perform motion information correction on the derived motion correction candidates; and

[0598] The predicted block for the current block is generated by using motion correction candidates that have already undergone motion information correction.

[0599] 2. The method of Scheme 1, wherein motion information candidates are derived sequentially from motion information of spatially neighboring blocks, motion information of temporally neighboring blocks, defined motion information, and motion information that appears most frequently in the reference frame in a predetermined order.

[0600] 3. The method as described in Scheme 2, wherein the predetermined order represents the order of motion information of spatially neighboring blocks, motion information of temporally neighboring blocks, and defined motion information.

[0601] 4. The method as described in Scheme 1, wherein the step of performing motion information correction includes: applying bilateral template matching to the motion vectors present in the derived motion correction candidates.

[0602] 5. The method as described in Scheme 4, wherein the bilateral template matching includes:

[0603] A two-sided template is generated by using motion vectors present in the derived motion correction candidates as initial motion vectors; and

[0604] The initial motion vector is corrected by comparing the sample points within the bilateral template with the reconstructed sample points within the reference image, which are indicated by the initial motion vector.

[0605] 6. The method as described in Scheme 5, wherein the initial motion vector is a bidirectional predicted motion vector as a non-zero vector in the derived motion correction candidate.

[0606] 7. The method as described in Scheme 6, wherein when there is no bidirectional predicted motion vector as a non-zero vector in the derived motion correction candidates, the initial motion vector is set to a zero vector.

[0607] 8. The method of Scheme 1, wherein the temporal neighbor block is included in the reference screen selected by the reference screen index of the spatial neighbor block.

[0608] 9. The method as described in Scheme 4, wherein the bilateral template matching is performed based on integer pixels and subpixels.

[0609] 10. The method as described in Scheme 5, wherein the step of correcting the initial motion vector includes:

[0610] The search indicates the motion vector of a region in the reference frame, wherein the region has minimal distortion relative to a corresponding region in the bilateral template; and

[0611] The found motion vector is set as a correction value for the initial motion vector.

[0612] 11. The method of embodiment 10, wherein the step of searching the initial motion vector includes:

[0613] The search scope is limited within the reference screen.

[0614] 12. The method of claim 11, wherein the search range of the restriction is set to a predetermined range based on integer pixel delimitation.

[0615] 13. The method as described in Scheme 12, wherein the step of searching the initial motion vector includes:

[0616] Motion vectors are searched within the predetermined range based on sub-pixels.

[0617] 14. The method of embodiment 10, wherein the correction of the initial motion vector is performed recursively.

[0618] 15. The method as described in Scheme 1, wherein when the current block does not correspond to the unidirectional prediction merging candidate, the local illumination compensation mode, and the affine motion compensation mode, motion information correction is performed on the derived motion correction candidate.

[0619] 16. The method of Scheme 1, further comprising:

[0620] The motion correction mode is decoded using information; and

[0621] Motion correction patterns are determined based on information obtained from the decoded motion correction patterns, wherein motion correction candidates are derived within the motion correction patterns.

[0622] 17. The method of claim 16, wherein the step of decoding the motion correction mode using information includes:

[0623] Whether to decode the motion correction mode using information is determined based on a skip flag or a merge flag.

[0624] 18. The method of Scheme 1, wherein, when multiple spatial neighbor blocks exist, the step of deriving the motion information candidate includes: deriving motion information from a spatial neighbor block having a bidirectional predicted motion vector, and subsequently deriving motion information from a spatial neighbor block having a unidirectional predicted motion vector.

[0625] 19. An image coding method, comprising:

[0626] Motion correction candidates are derived from motion information of spatially neighboring blocks, motion information of temporally neighboring blocks, predefined motion information, and motion information that appears most frequently in the reference frame.

[0627] Perform motion information correction on the derived motion correction candidates; and

[0628] The predicted block for the current block is generated by using motion correction candidates that have already undergone motion information correction.

[0629] 20. A non-transitory storage medium comprising a bitstream generated by an image encoding method, the method comprising:

[0630] Motion correction candidates are derived from motion information of spatially neighboring blocks, motion information of temporally neighboring blocks, predefined motion information, and motion information that appears most frequently in the reference frame.

[0631] Perform motion information correction on the derived motion correction candidates; and

[0632] The predicted block for the current block is generated by using motion correction candidates that have already undergone motion information correction.

[0633] Although the invention has been described with respect to specific items (such as detailed elements) and limited embodiments and drawings, these are provided only to aid in a more complete understanding of the invention, and the invention is not limited to the embodiments described above. Those skilled in the art will understand that various modifications and changes can be made to the above description.

[0634] Therefore, the spirit of the present invention should not be limited to the above embodiments, and the entire scope of the appended claims and their equivalents shall fall within the scope and spirit of the present invention.

[0635] Industrial applicability

[0636] This invention can be applied to devices used for encoding / decoding images.

Claims

1. A method for decoding an image, the method comprising: Whether a merge mode is applied to the current block is determined based on the merge flags decoded from the bitstream; When the merging mode is applied to the current block, a merging candidate list for the current block is generated, the merging candidate list including at least one of spatial merging candidates derived from the motion information of spatially neighboring blocks and temporal merging candidates derived from the motion information of temporally neighboring blocks; Update the merge candidate list of the current block, the updated merge candidate list including a new merge candidate generated by using two merge candidates in the merge candidate list; Select a merge candidate from the updated merge candidate list; The first and second initial motion vectors of the current block are derived from the merge candidates; The first corrected motion vector and the second corrected motion vector are obtained by performing motion information correction; The predicted block of the current block is generated using the first corrected motion vector and the second corrected motion vector; as well as The reconstructed block is obtained based on the predicted block.

2. The method according to claim 1, wherein, The motion information correction is performed only when inter-frame prediction for the current block is performed bidirectionally and the directions of the L0 reference frame and the L1 reference frame are different from each other.

3. The method according to claim 1, wherein, The first modified motion vector is obtained by modifying the first initial motion vector, and the second modified motion vector is obtained by modifying the second initial motion vector. Specifically, the difference between the first prediction block specified by the first initial motion vector and the second prediction block specified by the second initial motion vector, and the SAD, are compared with a threshold to determine whether to correct the first initial motion vector and the second initial motion vector.

4. The method according to claim 3, wherein, Correcting the first initial motion vector includes: Determine the optimal integer position among multiple integer positions within the search range, and Determine the optimal fractional position around the optimal integer position.

5. The method according to claim 4, wherein, The first motion vector difference, representing the difference between the first corrected motion vector and the first initial motion vector, is composed of an integer component corresponding to the optimal integer position and a fractional component corresponding to the optimal fractional position.

6. The method according to claim 4, wherein, The optimal score position is determined based on the SAD at the optimal integer position and the SAD at the neighboring integer positions adjacent to the optimal integer position.

7. The method according to claim 4, wherein, The optimal score position is searched only within the search range.

8. The method according to claim 1, wherein, Without decoding the information indicating whether the motion information correction is applied to the current block, determine whether the motion information correction is applied to the current block.

9. A method for encoding an image, the method comprising: When a merge mode is applied to the current block, a merge candidate list for the current block is generated, the merge candidate list including at least one of spatial merge candidates derived from the motion information of spatially neighboring blocks and temporal merge candidates derived from the motion information of temporally neighboring blocks; Update the merge candidate list of the current block, the updated merge candidate list including a new merge candidate generated by using two merge candidates in the merge candidate list; Select a merge candidate from the updated merge candidate list; The first and second initial motion vectors of the current block are derived from the merge candidates; The first corrected motion vector and the second corrected motion vector are obtained by performing motion information correction; The predicted block of the current block is generated using the first corrected motion vector and the second corrected motion vector; Encode the merge flag that indicates whether the merge mode is applied to the current block; as well as The residual block is obtained based on the prediction block.

10. A non-transitory storage medium comprising a bitstream generated by an image encoding method, the image encoding method comprising: When a merge mode is applied to the current block, a merge candidate list for the current block is generated, the merge candidate list including at least one of spatial merge candidates derived from the motion information of spatially neighboring blocks and temporal merge candidates derived from the motion information of temporally neighboring blocks; Update the merge candidate list of the current block, the updated merge candidate list including a new merge candidate generated by using two merge candidates in the merge candidate list; Select a merge candidate from the updated merge candidate list; The first and second initial motion vectors of the current block are derived from the merge candidates; The first corrected motion vector and the second corrected motion vector are obtained by performing motion information correction; The predicted block of the current block is generated using the first corrected motion vector and the second corrected motion vector; Encode the merge flag that indicates whether the merge mode is applied to the current block; as well as The residual block is obtained based on the prediction block.

Images