Image encoding / decoding method and device, and recording medium storing bitstream

Abstract

To improve the efficiency of image encoding and decoding. [Solution] An image decoding method according to one embodiment includes the steps of: creating a merge candidate list for a current block, the merge candidate list including at least one of motion information of spatial surrounding blocks, motion information of temporal surrounding blocks, and predefined motion information; determining a motion compensation candidate from the merge candidate list; determining motion information of the current block by performing motion information compensation on the determined motion compensation candidate; and generating a predicted block for the current block using the motion information of the current block, wherein the motion information compensation is performed only if the current block is not in a unidirectional prediction mode and the first prediction direction of the current block and the second prediction direction of the current block are different from each other.

Description

[Technical Field]

[0001] The present invention relates to an image encoding / decoding method and apparatus, and a recording medium storing a bitstream. More specifically, the present invention relates to an image encoding / decoding method and apparatus that corrects motion information to perform motion compensation. [Background technology]

[0002] Recently, demand for high-resolution, high-quality images, such as HD (High Definition) images and UHD (Ultra High Definition) images, is increasing in various application fields. As image data becomes higher in resolution and quality, the amount of data increases relatively compared to conventional image data. Therefore, when image data is transmitted over conventional media such as wired or wireless broadband lines or stored using conventional storage media, transmission and storage costs increase. To solve the problems that arise with the increase in resolution and quality of image data, highly efficient image encoding / decoding technologies for images with higher resolution and quality are required.

[0003] There are various image compression techniques, such as inter-frame (inter) prediction techniques that predict pixel values ​​contained in a current picture from pictures before or after the current picture, intra-frame (intra) prediction techniques that predict pixel values ​​contained in a current picture using pixel information within the current picture, transformation and quantization techniques that compress the energy of residual signals, and entropy coding techniques that assign short codes to values ​​that occur frequently and long codes to values ​​that occur less frequently.By using these image compression techniques, image data can be effectively compressed and transmitted or stored.

[0004] Conventional image encoding / decoding methods and devices perform motion compensation based on motion information of spatially / temporally adjacent blocks in a block to be encoded / decoded, and therefore there is a limit to how much the encoding efficiency can be improved. Summary of the Invention [Problem to be solved by the invention]

[0005] SUMMARY OF THE INVENTION An object of the present invention is to provide a method and apparatus for correcting motion information to perform motion compensation. [Means for solving the problem]

[0006] The image decoding method according to the present invention may include the steps of deriving a motion compensation candidate from at least one of motion information of spatial surrounding blocks, motion information of temporal surrounding blocks, predefined motion information, and the most common motion information in a reference image, performing motion information compensation on the induced motion compensation candidate, and generating a predicted block of a current block using the motion compensation candidate after the motion information compensation.

[0007] In the image decoding method, the step of deriving the motion compensation candidate may derive the motion compensation candidate from at least one of motion information of spatial surrounding blocks, motion information of temporal surrounding blocks, predefined motion information, and motion information that is most frequently present in the reference image according to a predetermined order.

[0008] In the image decoding method, the predetermined order may be the order of motion information of spatially neighboring blocks, motion information of temporally neighboring blocks, and predefined motion information.

[0009] In the image decoding method, the step of performing motion information compensation may perform motion information compensation by applying bilateral template matching to the motion vectors included in the induced motion compensation candidates.

[0010] In the image decoding method, the bilateral template matching may include a step of generating a bilateral template using a motion vector included in the induced motion compensation candidate as an initial motion vector, and a step of correcting the initial motion vector by comparing samples in the bilateral template with reconstructed samples in a reference image indicated by the initial motion vector.

[0011] In the image decoding method, the initial motion vector may be a bidirectionally predicted motion vector that is not a zero vector among the derived motion compensation candidates.

[0012] In the image decoding method, if there is no bidirectional predictive motion vector that is not a zero vector among the derived motion compensation candidates, the initial motion vector may be set to a zero vector.

[0013] In the image decoding method, the temporal neighboring blocks may be included in a reference image selected based on a reference image index of the spatial neighboring blocks.

[0014] In the image decoding method, the bilateral template matching can be performed in integer pixel units and sub-pixel units.

[0015] In the image decoding method, the step of correcting the initial motion vector may include the steps of searching for a motion vector that indicates an area in the reference image that exhibits minimum distortion with both templates, and setting the searched motion vector as a correction value for the initial motion vector.

[0016] In the image decoding method, the step of searching for the motion vector can search within a limited search area within the reference image.

[0017] In the image decoding method, the restricted search area can be set to a predetermined range in integer pixel units.

[0018] In the image decoding method, the step of searching for the motion vector can search for the motion vector in sub-pixel units within the predetermined range of the integer pixel units.

[0019] In the image decoding method, the step of correcting the initial motion vector may be performed recursively.

[0020] In the image decoding method, the step of performing motion information compensation on the guided motion compensation candidate may be performed when the current block does not correspond to a unidirectional prediction merge candidate, a regional illumination compensation mode, or an affine motion compensation mode.

[0021] The image decoding method may further include a step of decoding motion compensation mode usage information and a step of determining a motion compensation mode based on the decoded motion compensation mode usage information, and the step of deriving the motion compensation candidate may be performed when the motion compensation mode is selected.

[0022] In the image decoding method, the step of decoding the motion compensation mode use information can determine whether to decode the motion compensation mode use information based on a skip flag or a merge flag.

[0023] In the image decoding method, when there are multiple spatial neighboring blocks, the step of deriving the motion compensation candidate may first derive motion information from spatial neighboring blocks having bidirectional predictive motion vectors, and then derive motion information from spatial neighboring blocks having unidirectional predictive motion vectors.

[0024] The image encoding method according to the present invention may include the steps of deriving a motion compensation candidate from at least one of motion information of spatial surrounding blocks, motion information of temporal surrounding blocks, predefined motion information, and the most common motion information in a reference image, performing motion information compensation on the induced motion compensation candidate, and generating a predicted block of a current block using the motion compensation candidate after the motion information compensation.

[0025] The non-transitory storage medium including the bitstream according to the present invention may include a bitstream generated by an image encoding method including the steps of deriving a motion compensation candidate from at least one of motion information of spatial surrounding blocks, motion information of temporal surrounding blocks, predefined motion information, and the motion information that is most prevalent in a reference image, performing motion information compensation on the derived motion compensation candidate, and generating a predicted block of a current block using the motion compensation candidate after the motion information compensation. [Effects of the Invention]

[0026] According to the present invention, it is possible to provide an image encoding / decoding method and apparatus with improved compression efficiency.

[0027] According to the present invention, it is possible to improve the efficiency of encoding and decoding images.

[0028] According to the present invention, the computational complexity of image encoders and decoders can be reduced. [Brief explanation of the drawings]

[0029] [Figure 1] 1 is a block diagram showing the configuration of an embodiment of an encoding device to which the present invention is applied. [Figure 2] 1 is a block diagram showing the configuration of an embodiment of a decoding device to which the present invention is applied. [Figure 3] FIG. 2 is a schematic diagram showing a division structure of an image when encoding and decoding the image. [Figure 4]FIG. 1 is a diagram illustrating an embodiment of an inter-prediction process. [Figure 5] 1 is a flowchart illustrating an image encoding method according to an embodiment of the present invention. [Figure 6] 2 is a flowchart illustrating an image decoding method according to an embodiment of the present invention. [Figure 7] 10 is a flowchart illustrating an image encoding method according to another embodiment of the present invention. [Figure 8] 10 is a flowchart illustrating an image decoding method according to another embodiment of the present invention. [Figure 9] 10A and 10B are diagrams illustrating an example of deriving spatial motion vector candidates for a current block. [Figure 10] 10A and 10B are diagrams illustrating an example of deriving temporal motion vector candidates for a current block. [Figure 11] FIG. 10 is a diagram illustrating an example in which a spatial merge candidate is added to a merge candidate list. [Figure 12] FIG. 10 is a diagram illustrating an example in which a temporal merge candidate is added to a merge candidate list. [Figure 13] FIG. 10 is a diagram for explaining bilateral template matching. [Figure 14] FIG. 10 is a diagram showing an area for searching a motion vector corrected by bilateral template matching. [Figure 15] FIG. 10 is a diagram showing an area for searching a motion vector corrected by bilateral template matching. [Figure 16] 2 is a flowchart illustrating an image decoding method according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0030] Because the present invention is susceptible to various modifications and can have various embodiments, specific embodiments are illustrated in the drawings and described in detail in the detailed description. However, this does not limit the present invention to the specific embodiments, but rather encompasses all modifications, equivalents, and alternatives within the spirit and technical scope of the present invention. In the drawings, like reference numerals indicate the same or similar functions throughout the various aspects. The shape and size of elements in the drawings may be exaggerated for clarity. The detailed description of exemplary embodiments below refers to the accompanying drawings, which show specific embodiments by way of example. These embodiments are described in sufficient detail to enable those skilled in the art to practice the embodiments. It should be understood that various embodiments, although different from one another, are not necessarily mutually exclusive. For example, specific shapes, structures, and characteristics described herein in connection with one embodiment can be implemented in various embodiments without departing from the spirit and scope of the present invention. It should also be understood that the location or arrangement of individual components within each disclosed embodiment can be changed without departing from the spirit and scope of the embodiment. Accordingly, the following detailed description is not to be taken in a limiting sense, and the scope of the exemplary embodiments is limited only by the appended claims, if properly recited, and to the full scope of equivalents to which those claims are entitled.

[0031] In the present invention, the terms "first," "second," etc. may be used to describe various elements, but these elements should not be limited by these terms. These terms are used only to distinguish one element from another. For example, a first element can be called a second element, and similarly, a second element can be called a first element, without departing from the scope of the present invention. The term "and / or" includes a combination of multiple related listed items or any of multiple related listed items.

[0032] When a component of the present invention is said to be "coupled" or "connected" to another component, it should be understood that the component may be directly coupled or connected to the other component, but that there may be other components between them. In contrast, when a component is said to be "directly coupled" or "directly connected" to another component, it should be understood that there are no other components between them.

[0033] The components shown in the embodiments of the present invention are illustrated independently to show different characteristic functions, and do not mean that each component is composed of separate hardware or a single software unit. That is, each component is included in the list of components for the convenience of explanation, and at least two of the components may be combined to form a single component, or one component may be divided into multiple components to perform a function. Such integrated and separated embodiments of each component are also included in the scope of the present invention as long as they do not deviate from the essence of the present invention.

[0034] The terms used in the present invention are merely used to describe specific embodiments and do not limit the present invention. A singular expression includes a plural expression unless the context clearly dictates otherwise. In the present invention, terms such as "comprise" or "have" specify the presence of features, numbers, steps, operations, components, parts, or combinations thereof described in the specification, and should be understood not to preclude the presence or additional possibility of one or more other features, numbers, steps, operations, components, parts, or combinations thereof. In other words, in the present invention, a description that a specific configuration "comprises" does not exclude configurations other than the specified configuration, but means that additional configurations may be included within the scope of the implementation of the present invention or the technical idea of ​​the present invention.

[0035] Some components of the present invention may not be essential components that perform essential functions in the present invention, but may be optional components simply for improving performance. The present invention can be realized by including only components that are essential for achieving the essence of the present invention, excluding components used simply for improving performance, and a structure including only essential components excluding optional components used simply for improving performance is also included in the scope of the present invention.

[0036] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. In describing the embodiments of this specification, if it is determined that a detailed description of related known configurations or functions may obscure the gist of this specification, the detailed description will be omitted, and the same reference numerals will be used for the same components in the drawings, and duplicate descriptions of the same components will be omitted.

[0037] Hereinafter, an image may refer to either a single picture constituting a video or the video itself. For example, "encoding and / or decoding of an image" may mean "encoding and / or decoding of a video" or "encoding and / or decoding of any one of the images constituting a video."

[0038] In the following, the terms "motion picture" and "video" may be used interchangeably or interchangeably.

[0039] Hereinafter, a target image may be a target image to be coded, which is a target of coding, and / or a target image to be decoded, which is a target of decoding. The target image may also be an input image input to an encoding device or an input image input to a decoding device. Here, the target image may have the same meaning as a current image.

[0040] In the following, the terms "image", "picture", "frame" and "screen" may be used interchangeably or interchangeably.

[0041] Hereinafter, a target block may be a target block to be coded and / or a target block to be decoded. Also, a target block may be a current block to be coded and / or decoded. For example, the terms "target block" and "current block" may be used interchangeably.

[0042] In the following, the terms "block" and "unit" may be used interchangeably or "block" may refer to a specific unit.

[0043] In the following, the terms "region" and "segment" may be used interchangeably.

[0044] In the following, a specific signal may be a signal indicating a specific block. For example, an original signal may be a signal indicating a current block. A prediction signal may be a signal indicating a predicted block. A residual signal may be a signal indicating a residual block.

[0045] In an embodiment, each of specific information, data, flags, indexes, elements, attributes, etc. can have a value. A value of "0" for information, data, flags, indexes, elements, attributes, etc. can indicate logical false or a first predefined value. In other words, the values ​​of "0", false, logical false, and the first predefined value may be used interchangeably. A value of "1" for information, data, flags, indexes, elements, attributes, etc. can indicate logical true or a second predefined value. In other words, the values ​​of "1", true, logical true, and the second predefined value may be used interchangeably.

[0046] When a variable such as i or j is used to indicate a row, column, or index, the value of i may be an integer greater than or equal to 0, or may be an integer greater than or equal to 1. In other words, in an embodiment, rows, columns, and indices may be counted from 0 or from 1.

[0047] Terminology Encoder: A device that performs encoding. In other words, it can refer to a coding device.

[0048] Decoder: A device that performs decoding. In other words, it can refer to a decoding device.

[0049] Block: An MxN array of samples. Here, M and N may represent positive integer values, and a block may generally represent a two-dimensional sample array. A block may represent a unit. A current block may represent a block to be coded during coding, or a block to be decoded during decoding. The current block may be at least one of a coding block, a predicted block, a residual block, and a transform block.

[0050] Sample: The basic unit that makes up a block. Bit depth (B d ) from 0 to 2 depending on Bd It can be expressed as a value up to -1. In the present invention, a sample can be used synonymously with a picture element or pixel. That is, a sample, a picture element, and a pixel can have the same meaning.

[0051] Unit: This may refer to a unit of image encoding and decoding. In image encoding and decoding, a unit may be a region into which an image is divided. Also, when an image is divided into smaller units for encoding or decoding, a unit may refer to the divided units. That is, an image may be divided into multiple units. In image encoding and decoding, predefined processing may be performed on each unit. A unit may be further divided into subunits having a smaller size than the unit. Depending on the function, a unit may refer to a block, macroblock, coding tree unit, coding tree block, coding unit, coding block, prediction unit, prediction block, residual unit, residual block, transform unit, transform block, etc. Furthermore, a unit may refer to a luma component block, a corresponding chroma component block, and syntax elements for each block in order to distinguish it from a block. The unit may have various sizes and shapes, and in particular, the shape of the unit may include geometric shapes that can be expressed two-dimensionally, such as a square, a rectangle, a trapezoid, a triangle, a pentagon, etc. Furthermore, the unit information may include at least one of a unit type indicating a coding unit, a prediction unit, a residual unit, a transform unit, etc., a unit size, a unit depth, an encoding and decoding order of the unit, etc.

[0052] Coding tree unit: Consists of two chrominance (Cb, Cr) coding tree blocks associated with one luminance (Y) coding tree block. It can also refer to the blocks and the syntax elements for each block. Each coding tree unit can be divided using one or more division methods, such as a quad tree, a binary tree, or a ternary tree, to form subunits such as a coding unit, a prediction unit, or a transform unit. Like division of an input image, it can be used as a term to indicate a sample block that serves as a processing unit in the image decoding / encoding process. Here, quad tree can refer to a quarternary tree.

[0053] Coding Tree Block: A term that can be used to refer to any of the Y coding tree block, Cb coding tree block, and Cr coding tree block.

[0054] Neighbor block: This may refer to a block adjacent to a current block. The block adjacent to the current block may refer to a block bordering the current block or a block located within a predetermined distance from the current block. The neighbor block may refer to a block adjacent to a vertex of the current block. Here, the block adjacent to a vertex of the current block may be a block adjacent to a horizontal neighboring block of the current block and a block adjacent to a vertical neighboring block of the current block, or a block adjacent to a horizontal neighboring block of the current block and a block adjacent to a vertical neighboring block of the current block. The neighbor block may also refer to a restored neighboring block.

[0055] Reconstructed Neighbor Block: This may refer to a neighboring block that has already been coded or decoded spatially / temporally around the current block. In this case, the reconstructed neighboring block may refer to a reconstructed neighboring unit. The reconstructed spatial neighboring block may be a block in the current picture that has already been reconstructed through coding and / or decoding. The reconstructed temporal neighboring block may be a reconstructed block or its neighboring block at a position corresponding to the current block of the current picture in the reference image.

[0056] Unit depth: This can refer to the degree to which a unit is divided. The top node in a tree structure can correspond to the first undivided unit. The top node is also called the root node. The top node can have the minimum depth value. In this case, the top node can have a depth of level 0. A node with a depth of level 1 can indicate a unit generated by dividing the first unit once. A node with a depth of level 2 can indicate a unit generated by dividing the first unit twice. A node with a depth of level n can indicate a unit generated by dividing the first unit n times. A leaf node is the lowest node and can be a node that cannot be divided any further. The depth of a leaf node can be the maximum level. For example, a predefined value for the maximum level can be 3. The root node has the shallowest depth, and the leaf node has the deepest depth. When units are represented in a tree structure, the level at which a unit exists can refer to the unit depth.

[0057] Bitstream: Can mean a string of bits containing coded image information.

[0058] Parameter Set: This corresponds to header information among structures within a bitstream. A parameter set may include at least one of a video parameter set, a sequence parameter set, a picture parameter set, and an adaptation parameter set. A parameter set may also include slice header and tile header information.

[0059] Parsing: This can refer to entropy decoding a bitstream to determine the values ​​of syntax elements, or it can refer to entropy decoding itself.

[0060] Symbol: A symbol may refer to at least one of a syntax element of a unit to be coded / decoded, a coding parameter, a value of a transform coefficient, etc. Also, a symbol may refer to an object of entropy coding or a result of entropy decoding.

[0061] Prediction Mode: This may be information indicating a mode in which coding / decoding is performed using intra-picture prediction or a mode in which coding / decoding is performed using inter-picture prediction.

[0062] Prediction unit: This may refer to a basic unit when performing prediction, such as inter prediction, intra prediction, inter compensation, intra compensation, motion compensation, etc. One prediction unit may be divided into multiple partitions or multiple sub-prediction units having smaller sizes. Multiple partitions may also be basic units for performing prediction or compensation. Partitions generated by dividing a prediction unit may also be prediction units.

[0063] Prediction Unit Partition: This may refer to the form in which a prediction unit is divided.

[0064] Reference Picture List: This can refer to a list containing one or more reference pictures used for inter-frame prediction or motion compensation. Types of reference picture lists include LC (List Combined), L0 (List 0), L1 (List 1), L2 (List 2), and L3 (List 3). Inter-frame prediction can use one or more reference picture lists.

[0065] Inter Prediction Indicator: This may refer to the inter prediction direction of the current block (unidirectional prediction, bidirectional / bi-predictive / double prediction, etc.), the number of reference images used when generating a predicted block of the current block, or the number of predicted blocks used when performing inter prediction or motion compensation on the current block.

[0066] Prediction list utilization flag: Indicates whether a prediction block is generated using at least one reference picture in a particular reference picture list. The prediction list utilization flag can be used to derive an inter-picture prediction indicator, and conversely, the inter-picture prediction indicator can be used to derive the prediction list utilization flag. For example, when the prediction list utilization flag indicates a first value "0", it can indicate that a prediction block is not generated using a reference picture in the reference picture list, and when it indicates a second value "1", it can indicate that a prediction block can be generated using the reference picture list.

[0067] Reference Picture Index: This can refer to an index that points to a particular reference picture in a reference picture list.

[0068] Reference Picture: This can refer to an image that a specific block references for inter-frame prediction or motion compensation. Alternatively, the reference picture can be an image including a reference block that a current block references for inter-frame prediction or motion compensation. Hereinafter, the terms "reference picture" and "reference image" may be used interchangeably or interchangeably.

[0069] Motion Vector: A two-dimensional vector used for inter-frame prediction or motion compensation. A motion vector may indicate an offset between a current block to be coded / decoded and a reference block. For example, (mvX, mvY) may indicate a motion vector. mvX may indicate the horizontal component, and mvY may indicate the vertical component.

[0070] Search Range: A search range may be a two-dimensional area in which a search for a motion vector is performed in inter prediction. For example, the size of the search range may be MxN, where M and N may each be a positive integer.

[0071] Motion Vector Candidate: This can refer to a block that is a prediction candidate when predicting a motion vector, or the motion vector of that block. Also, the motion vector candidate may be included in a motion vector candidate list.

[0072] Motion Vector Candidate List: This may refer to a list composed of one or more motion vector candidates. Motion Vector Candidate Index: This can refer to an indicator of a motion vector candidate in a motion vector candidate list. It may also be an index of a motion vector predictor.

[0073] Motion Information: This can refer to information including at least one of a motion vector, a reference image index, an inter-frame prediction indicator, a prediction list utilization flag, reference image list information, a reference image, a motion vector candidate, a motion vector candidate index, a merge candidate, a margin index, etc.

[0074] Merge Candidate List: This may refer to a list composed of one or more merge candidates.

[0075] Merge Candidate: This can refer to a spatial merge candidate, a temporal merge candidate, a combined merge candidate, a combined bi-predictive merge candidate, a zero merge candidate, etc. The merge candidate can include motion information such as an inter-picture prediction indicator, a reference image index for each list, a motion vector, a prediction list utilization flag, and an inter-picture prediction indicator.

[0076] Merge Index: This may refer to an indicator that indicates a merge candidate in a merge candidate list. The merge index may also indicate a block that has guided the merge candidate among reconstructed blocks that are spatially / temporally adjacent to the current block. The merge index may also indicate at least one of the motion information items of the merge candidate.

[0077] Transform unit: A basic unit for encoding / decoding a residual signal, such as transform, inverse transform, quantization, inverse quantization, and transform coefficient encoding / decoding. One transform unit can be divided into a plurality of sub-transform units with smaller sizes. Here, the transform / inverse transform can include at least one of a primary transform / inverse transform and a secondary transform / inverse transform.

[0078] Scaling: This can refer to the process of multiplying quantized levels by a factor. Transform coefficients can be generated as a result of scaling the quantized levels. Scaling can also be called dequantization.

[0079] Quantization Parameter: This can refer to a value used when generating a quantized level using a transform coefficient in quantization. Alternatively, it can refer to a value used when generating a transform coefficient by scaling a quantized level in inverse quantization. The quantization parameter can be a value mapped to a quantization step size.

[0080] Delta Quantization Parameter: This may refer to a difference value between a predicted quantization parameter and a quantization parameter of a unit to be coded / decoded.

[0081] Scan: This can refer to a method of sorting the order of coefficients in a unit, block, or matrix. For example, sorting a two-dimensional array into a one-dimensional array is called a scan. Alternatively, sorting a one-dimensional array into a two-dimensional array can also be called a scan or inverse scan.

[0082] Transform coefficient: This can refer to a coefficient value generated after performing a transform in an encoder. It can also refer to a coefficient value generated after performing at least one of entropy decoding and inverse quantization in a decoder. A quantization level or a quantized transform coefficient level obtained by applying quantization to a transform coefficient or a residual signal can also be included in the meaning of a transform coefficient.

[0083] Quantized Level: This can refer to a value generated by quantizing a transform coefficient or a residual signal in an encoder. Alternatively, it can refer to a value to be subjected to inverse quantization before inverse quantization in a decoder. Similarly, the quantized transform coefficient level, which is the result of transform and quantization, can also be included in the meaning of the quantized level.

[0084] Non-zero Transform Coefficient: This can refer to a transform coefficient whose magnitude is not 0, or a transform coefficient level whose magnitude is not 0, or a quantized level.

[0085] Quantization Matrix: A matrix used in the quantization or dequantization process to improve the subjective or objective image quality. A quantization matrix can also be called a scaling list.

[0086] Quantization Matrix Coefficient: This can refer to each element in a quantization matrix. The quantization matrix coefficient can also be called a matrix coefficient.

[0087] Default Matrix: This may refer to a predetermined quantization matrix that is predefined in the encoder and decoder.

[0088] Non-default Matrix: This can refer to a quantization matrix that is not predefined in the encoder and decoder but is signaled by the user.

[0089] Statistical value: A statistical value for at least one of a variable, coding parameter, constant, etc. having a specific value that can be calculated may be at least one of the average value, weighted average value, weighted sum value, minimum value, maximum value, mode value, median value, and interpolated value of the specific value.

[0090] FIG. 1 is a block diagram showing the configuration of an embodiment of an encoding device to which the present invention is applied.

[0091] The encoding device 100 may be an encoder, a video encoding device, or an image encoding device. A video may include one or more images. The encoding device 100 may sequentially encode one or more images.

[0092] Referring to FIG. 1, the encoding device 100 may include a motion prediction unit 111, a motion compensation unit 112, an intra prediction unit 120, a switch 115, a subtractor 125, a transform unit 130, a quantization unit 140, an entropy encoding unit 150, an inverse quantization unit 160, an inverse transform unit 170, an adder 175, a filter unit 180, and a reference picture buffer 190.

[0093] The encoding device 100 may perform encoding on an input image in intra mode and / or inter mode. The encoding device 100 may also generate a bitstream including encoded information through encoding on the input image and output the generated bitstream. The generated bitstream may be stored in a computer-readable recording medium or streamed via a wired / wireless transmission medium. When an intra mode is used as a prediction mode, the switch 115 may switch to intra mode. When an inter mode is used as a prediction mode, the switch 115 may switch to inter mode. Here, the intra mode may refer to an intra-frame prediction mode, and the inter mode may refer to an inter-frame prediction mode. The encoding device 100 may generate a prediction block for an input block of the input image. After the prediction block is generated, the encoding device 100 may also encode a residual block using a residual between the input block and the prediction block. The input image may also be referred to as a current image currently being encoded. The input block may also be referred to as a current block currently being encoded or a block to be encoded.

[0094] When the prediction mode is the intra mode, the intra prediction unit 120 can also use samples of already coded / decoded blocks surrounding the current block as reference samples. The intra prediction unit 120 can perform spatial prediction on the current block using the reference samples and generate predicted samples for the input block through the spatial prediction. Here, intra prediction can mean intra-frame prediction.

[0095] When the prediction mode is inter mode, the motion prediction unit 111 may search for an area that best matches the input block from a reference image during the motion prediction process, and may derive a motion vector using the searched area. In this case, the search area may be used as the area. The reference image may be stored in the reference picture buffer 190. Here, when encoding / decoding of the reference image is processed, the reference image may be stored in the reference picture buffer 190.

[0096] The motion compensation unit 112 may generate a prediction block for the current block by performing motion compensation using the motion vector, where inter prediction may refer to inter prediction or motion compensation.

[0097] The motion prediction unit 111 and the motion compensation unit 112 may generate a prediction block by applying an interpolation filter to a part of a reference image when the value of a motion vector does not have an integer value. To perform inter-frame prediction or motion compensation, it may be determined based on a coding unit whether the motion prediction and motion compensation method of a prediction unit included in the corresponding coding unit is a skip mode, a merge mode, an advanced motion vector prediction (AMVP) mode, or a current picture reference mode, and inter-frame prediction or motion compensation may be performed according to each mode.

[0098] The subtractor 125 may generate a residual block using the difference between the input block and the predicted block. The residual block may also be referred to as a residual signal. The residual signal may refer to the difference between the original signal and the predicted signal. Alternatively, the residual signal may be a signal generated by transforming, quantizing, or transforming and quantizing the difference between the original signal and the predicted signal. The residual block may be a block-by-block residual signal.

[0099] The transform unit 130 may perform a transform on the residual block to generate transform coefficients and output the generated transform coefficients. Here, the transform coefficients may be coefficient values ​​generated by performing a transform on the residual block. When a transform skip mode is applied, the transform unit 130 may skip transforming the residual block.

[0100] By applying quantization to the transform coefficients or the residual signal, quantized levels can be generated, which may also be referred to as transform coefficients in the following embodiments.

[0101] The quantization unit 140 may generate quantization levels by quantizing the transform coefficients or the residual signal based on the quantization parameter, and may output the generated quantization levels. In this case, the quantization unit 140 may quantize the transform coefficients using a quantization matrix.

[0102] The entropy coding unit 150 may generate a bitstream by performing entropy coding based on a probability distribution on values ​​calculated by the quantization unit 140 or coding parameter values ​​calculated during the coding process, and may output the bitstream. The entropy coding unit 150 may perform entropy coding on information about image samples and information for decoding the image. For example, the information for decoding the image may include syntax elements.

[0103] When entropy coding is applied, fewer bits are assigned to symbols with higher occurrence probabilities and more bits are assigned to symbols with lower occurrence probabilities to represent the symbols, thereby reducing the size of the bit string for the symbol to be coded. The entropy coding unit 150 can use coding methods such as exponential Golomb, context-adaptive variable length coding (CAVLC), and context-adaptive binary arithmetic coding (CABAC) for entropy coding. For example, the entropy coding unit 150 can perform entropy coding using a variable length coding / code (VLC) table. The entropy coding unit 150 can also derive a binarization method for the target symbol and a probability model for the target symbol / bin, and then perform arithmetic coding using the derived binarization method, probability model, and context model.

[0104] The entropy coding unit 150 can convert two-dimensional block-form coefficients into one-dimensional vectors through a transform coefficient scanning method in order to code the transform coefficient levels (quantization levels).

[0105] Coding parameters can include not only information (flags, indexes, etc.) that is coded by an encoder and signaled to a decoder, like syntax elements, but also information that is induced during the coding or decoding process, and can refer to information required when coding or decoding an image. For example, unit / block size, unit / block depth, unit / block division information, unit / block type, unit / block division structure, whether it is quadtree division, whether it is binary tree division, direction of binary tree division (horizontal or vertical), type of binary tree division (symmetric or asymmetric), whether it is ternary tree division, direction of ternary tree division (horizontal or vertical), type of ternary tree division (symmetric or asymmetric), whether it is hybrid tree division, direction of hybrid tree division (horizontal or vertical), type of hybrid tree division (symmetric or asymmetric), division tree of hybrid tree (binary tree or ternary tree), prediction mode (intra prediction or inter prediction), intra luminance prediction mode / direction, intra chrominance prediction mode / direction, intra division information, inter division information, coding block division flag, prediction block division flag, transform block division flag, reference sample filtering method, reference sample filter tap, reference sample filter coefficient, prediction Block filtering method, prediction block filter taps, prediction block filter coefficients, prediction block boundary filtering method, prediction block boundary filter taps, prediction block boundary filter coefficients, intra-frame prediction mode, inter-frame prediction mode, motion information, motion vector, motion vector difference, reference image index, inter-frame prediction direction, inter-frame prediction indicator, prediction list utilization flag, reference image list, reference image, motion vector prediction index, motion vector prediction candidate, motion vector candidate list, whether merge mode is used or not, merge index, merge candidate, merge candidate list, whether skip mode is used or not, interpolation filter type, interpolation filter taps, interpolation filter coefficients, motion vector size, motion vector representation accuracy, transform type, transform size, information on whether primary transform is used or not, information on whether secondary transform is used or not, primary transform index, secondary transform index, information on whether residual signal is used or not, coded block patternPattern), coded block flag, coded block flag, quantization parameter, residual quantization parameter, quantization matrix, whether intra-frame loop filter is applied, intra-frame loop filter coefficient, intra-frame loop filter tap, intra-frame loop filter shape / form, whether deblocking filter is applied, deblocking filter coefficient, deblocking filter tap, deblocking filter strength, deblocking filter shape / form, whether adaptive sample offset is applied, adaptive sample offset value, adaptive sample offset category, type of adaptive sample offset, whether adaptive loop filter is applied, adaptive loop filter coefficient, adaptive loop filter tap, adaptive loop filter shape / form, binarization / de-binarization method, context model determination method, context model update method, whether regular mode is executed, whether bypass mode is executed, context bin, bypass bin, important coefficient, flag, last important coefficient flag, coefficient group unit coding flag, last important coefficient position, flag indicating whether coefficient value is greater than 1, function The encoding parameters may include at least one value or combination of a flag indicating whether a numerical value is greater than 2, a flag indicating whether a coefficient value is greater than 3, residual coefficient value information, sign information, reconstructed luma sample, reconstructed chroma sample, residual luma sample, residual chroma sample, luma transform coefficient, chroma transform coefficient, luma quantization level, chroma quantization level, transform coefficient level scanning method, size of decoder side motion vector search area, shape of decoder side motion vector search area, number of decoder side motion vector searches, CTU size information, minimum block size information, maximum block size information, maximum block depth information, minimum block depth information, image display / output order, slice identification information, slice type, slice division information, tile identification information, tile type, tile division information, picture type, input sample bit depth, reconstructed sample bit depth, residual sample bit depth, transform coefficient bit depth, quantization level bit depth, information on the luma signal, and information on the chroma signal.

[0106] Here, signaling a flag or index can mean that an encoder entropy encodes the corresponding flag or index and includes it in a bitstream, and a decoder entropy decodes the corresponding flag or index from the bitstream.

[0107] When the encoding device 100 performs encoding using inter prediction, the encoded current image can be used as a reference image for other images to be processed later. Thus, the encoding device 100 can further restore or decode the encoded current image and store the restored or decoded image in the reference picture buffer 190 as a reference image.

[0108] The quantization levels may be dequantized by the inverse quantization unit 160 and inverse transformed by the inverse transform unit 170. The inverse quantized and / or inverse transformed coefficients may be combined with a prediction block via an adder 175. A reconstructed block may be generated by combining the inverse quantized and / or inverse transformed coefficients with the prediction block. Here, the inverse quantized and / or inverse transformed coefficients refer to coefficients that have undergone at least one of inverse quantization and inverse transformation, and may refer to a reconstructed residual block.

[0109] The reconstructed blocks may pass through a filter unit 180. The filter unit 180 may apply at least one of a deblocking filter, a sample adaptive offset (SAO), an adaptive loop filter (ALF), etc. to the reconstructed samples, the reconstructed blocks, or the reconstructed images. The filter unit 180 is also referred to as an in-loop filter.

[0110] The deblocking filter can remove block artifacts that occur at boundaries between blocks. To determine whether to perform the deblocking filter, it can be determined whether to apply the deblocking filter to the current block based on samples included in several columns or rows included in the block. When applying the deblocking filter to the block, different filters can be applied depending on the required deblocking filtering strength.

[0111] To compensate for encoding errors using sample adaptive offset, an appropriate offset value can be added to a sample value. Sample adaptive offset can correct the offset between a deblocked image and an original image on a sample-by-sample basis. After dividing the samples included in an image into a certain number of regions, the region to be offset is determined and the offset is applied to the corresponding region, or the offset is applied taking into account edge information of each sample.

[0112] The adaptive loop filter can perform filtering based on a value obtained by comparing a restored image with an original image. After dividing samples included in an image into predetermined groups, a filter to be applied to each group can be determined, and differential filtering can be performed for each group. Information related to whether to apply an adaptive loop filter can be signaled for each coding unit (CU), and the shape and filter coefficients of the adaptive loop filter applied can vary depending on each block.

[0113] The reconstructed blocks or reconstructed images that have passed through the filter unit 180 may be stored in the reference picture buffer 190. The reconstructed blocks that have passed through the filter unit 180 may be part of a reference image. In other words, the reference image may be a reconstructed image made up of the reconstructed blocks that have passed through the filter 180. The stored reference image may be used for subsequent inter prediction or motion compensation.

[0114] FIG. 2 is a block diagram showing the configuration of an embodiment of a decoding device to which the present invention is applied.

[0115] The decoding device 200 may be a decoder, a video decoding device or an image decoding device.

[0116] Referring to FIG. 2, the decoding device 200 may include an entropy decoding unit 210, an inverse quantization unit 220, an inverse transform unit 230, an intra prediction unit 240, a motion compensation unit 250, an adder 255, a filter unit 260, and a reference picture buffer 270.

[0117] The decoding apparatus 200 may receive a bitstream output from the encoding apparatus 100. The decoding apparatus 200 may receive a bitstream stored in a computer-readable recording medium or a bitstream streamed via a wired / wireless transmission medium. The decoding apparatus 200 may perform decoding on the bitstream in an intra mode or an inter mode. The decoding apparatus 200 may also generate a reconstructed image or a decoded image through decoding and output the reconstructed image or the decoded image.

[0118] If the prediction mode used for decoding is an intra mode, the switch can be changed to intra. If the prediction mode used for decoding is an inter mode, the switch can be changed to inter.

[0119] The decoding apparatus 200 may decode an input bitstream, obtain a reconstructed residual block, and generate a prediction block. After obtaining the reconstructed residual block and the prediction block, the decoding apparatus 200 may generate a reconstructed block to be decoded by adding the reconstructed residual block and the prediction block. The block to be decoded may also be referred to as a current block.

[0120] The entropy decoding unit 210 may generate symbols by performing entropy decoding based on a probability distribution on the bitstream. The generated symbols may include symbols in the form of quantization levels. Here, the entropy decoding method may be the inverse process of the entropy encoding method described above.

[0121] The entropy decoding unit 210 can convert one-dimensional vector form coefficients into two-dimensional block form coefficients according to a scanning method of the transform coefficients in order to decode the transform coefficient levels (quantization levels).

[0122] The quantization levels can be inversely quantized by the inverse quantization unit 220 and inversely transformed by the inverse transform unit 230. The quantization levels are the result of inverse quantization and / or inverse transformation and can be generated as reconstructed residual blocks. In this case, the inverse quantization unit 220 can apply a quantization matrix to the quantization levels.

[0123] When intra mode is used, the intra prediction unit 240 can generate a predicted block by performing spatial prediction on the current block using sample values ​​of already decoded blocks surrounding the block to be decoded.

[0124] When the inter mode is used, the motion compensation unit 250 may generate a prediction block by performing motion compensation on the current block using a motion vector and a reference image stored in the reference picture buffer 270. When the value of the motion vector does not have an integer value, the motion compensation unit 250 may generate a prediction block by applying an interpolation filter to a portion of the reference image. To perform motion compensation, it may determine, based on a coding unit, whether the motion compensation method of a prediction unit included in the corresponding coding unit is skip mode, merge mode, AMVP mode, or current picture reference mode, and perform motion compensation according to each mode.

[0125] The adder 225 may generate a reconstructed block by adding the reconstructed residual block and the prediction block. The filter unit 260 may apply at least one of a deblocking filter, a sample adaptive offset, and an adaptive loop filter to the reconstructed block or the reconstructed image. The filter unit 260 may output the reconstructed image. The reconstructed block or the reconstructed image is stored in the reference picture buffer 270 and can be used for inter prediction. The reconstructed block that has passed through the filter unit 260 may be part of a reference image. In other words, the reference image may be a reconstructed image made up of the reconstructed block that has passed through the filter unit 260. The stored reference image can be used for inter prediction or motion compensation thereafter.

[0126] 3 is a schematic diagram showing the division structure of an image when encoding and decoding the image, which shows an embodiment in which one unit is divided into multiple sub-units.

[0127] In order to efficiently divide an image, a coding unit (CU) can be used in encoding and decoding. The coding unit can be used as a basic unit for image encoding / decoding. Furthermore, when encoding / decoding an image, the coding unit can be used as a unit in which an intra prediction mode and an inter prediction mode are distinguished. The coding unit may be a basic unit used for prediction, transform, quantization, inverse transform, inverse quantization, or encoding / decoding of transform coefficients.

[0128] Referring to FIG. 3, an image 300 is sequentially divided into largest coding units (LCUs), and a division structure is determined for each LCU. Here, LCU can be used interchangeably with coding tree unit (CTU). Division of a unit may refer to division of blocks corresponding to the unit. Block division information may include information about the depth of the unit. The depth information may indicate the number of times and / or the degree to which the unit is divided. One unit may be hierarchically divided into multiple subunits using depth information based on a tree structure. In other words, a unit and subunits generated by dividing the unit may correspond to a node and its child nodes, respectively. Each divided subunit may have depth information. The depth information indicates the size of a CU and may be stored for each CU. Since the unit depth indicates the number of times and / or the degree to which the unit is divided, the subunit division information may also include information about the size of the subunit.

[0129] The partition structure may refer to the distribution of coding units (CUs) within the CTU 310. This distribution may be determined by whether or not to partition one CU into multiple CUs (a positive integer equal to or greater than 2, including 2, 4, 8, 16, etc.). The width and height of a CU generated by partitioning may be half the width and half the height of the CU before partitioning, respectively, or may be smaller than the width and height of the CU before partitioning depending on the number of partitions. A CU may be recursively partitioned into multiple CUs. Recursive partitioning may reduce at least one of the width and height of the partitioned CU compared to at least one of the width and height of the CU before partitioning. CU partitioning may be performed recursively to a predefined depth or up to a predefined size. For example, the depth of a CTU may be 0, and the depth of a smallest coding unit (SCU) may be a predefined maximum depth. Here, as described above, a CTU may be a coding unit having the largest coding unit size, and an SCU may be a coding unit having the smallest coding unit size. Splitting begins with a CTU 310, and the depth of the CU increases by 1 each time the width and / or height of the CU decreases due to splitting. For example, for each depth, an unsplit CU may have a size of 2N×2N. Furthermore, for split CUs, a 2N×2N CU may be split into four N×N CUs. The size of N may be halved each time the depth increases by 1.

[0130] In addition, information on whether a CU is divided can be expressed through division information of the CU. The division information may be 1-bit information. All CUs except for SCUs may include division information. For example, if the value of the division information is a first value, the CU may not be divided, and if the value of the division information is a second value, the CU may be divided.

[0131] Referring to Figure 3, a CTU with depth 0 may be 64x64 blocks. 0 may be the minimum depth. A SCU with depth 3 may be 8x8 blocks. 3 may be the maximum depth. CUs with 32x32 blocks and 16x16 blocks may be represented with depth 1 and depth 2, respectively.

[0132] For example, when one coding unit is divided into four coding units, the width and height of the four divided coding units may be half the size of the width and height of the coding unit before division. As an example, when a 32x32 coding unit is divided into four coding units, each of the four divided coding units may have a size of 16x16. When one coding unit is divided into four coding units, it can be said that the coding units are divided into a quad-tree (quad-tree partition).

[0133] For example, when one coding unit is divided into two coding units, the width or height of the two divided coding units may be half the width or height of the coding unit before division. As an example, when a 32x32 coding unit is vertically divided into two coding units, each of the divided coding units may have a size of 16x32. As an example, when an 8x32 coding unit is horizontally divided into two coding units, each of the divided coding units may have a size of 8x16. When one coding unit is divided into two coding units, it can be said that the coding unit is divided into a binary tree (binary-tree partition).

[0134] For example, when one coding unit is divided into three coding units, the coding unit can be divided into three coding units by dividing the width or height of the coding unit in a ratio of 1:2:1 before division. As an example, when a 16x32 coding unit is divided horizontally into three coding units, the three divided coding units may have sizes of 16x8, 16x16, and 16x8 from the top. As an example, when a 32x32 coding unit is divided vertically into three coding units, the three divided coding units may have sizes of 8x32, 16x32, and 8x32 from the left. When one coding unit is divided into three coding units, it can be said that the coding unit is divided into a ternary-tree shape (ternary-tree partition).

[0135] The CTU 320 in FIG. 3 is an example of a CTU to which quadtree division, binary tree division, and ternary tree division are all applied.

[0136] As described above, at least one of quadtree partitioning, binary tree partitioning, and ternary tree partitioning can be applied to partition a CTU. Each partitioning method can be applied based on a predetermined priority. For example, quadtree partitioning can be applied preferentially to a CTU. A coding unit that cannot be further partitioned into quadtrees can correspond to a leaf node of a quadtree. A coding unit that corresponds to a leaf node of a quadtree can become a root node of a binary tree and / or a ternary tree. That is, a coding unit that corresponds to a leaf node of a quadtree can be partitioned into binary trees, ternary trees, or cannot be further partitioned. In this case, quadtree partitioning is not performed again on a coding unit generated by partitioning a coding unit that corresponds to a leaf node of a quadtree into binary trees or ternary trees, thereby effectively partitioning blocks and / or signaling partition information.

[0137] The division of the coding unit corresponding to each node of the quadtree can be signaled using quad division information. Quad division information having a first value (e.g., "1") can indicate that the corresponding coding unit is quadtree divided. Quad division information having a second value (e.g., "0") can indicate that the corresponding coding unit is not quadtree divided. The quad division information can be a flag having a predetermined length (e.g., 1 bit).

[0138] There may be no priority between binary tree division and ternary tree division. That is, a coding unit corresponding to a leaf node of a quadtree can be divided into either a binary tree or a ternary tree. Furthermore, a coding unit generated by binary tree division or ternary tree division can be divided into either a binary tree or a ternary tree again, or cannot be divided further.

[0139] A partition in which there is no priority between binary tree partitioning and ternary tree partitioning may be called a multi-type tree partition. That is, a coding unit corresponding to a leaf node of a quadtree may become the root node of a multi-type tree. The partition of a coding unit corresponding to each node of a multi-type tree may be signaled using at least one of information on whether the multi-type tree is partitioned, split direction information, and split tree information. For the partition of a coding unit corresponding to each node of the multi-type tree, information on whether the multi-type tree is partitioned, split direction information, and split tree information may be signaled sequentially.

[0140] The hybrid tree split or not information having a first value (e.g., "1") may indicate that the coding unit is hybrid tree split. The hybrid tree split or not information having a second value (e.g., "0") may indicate that the coding unit is not hybrid tree split.

[0141] When a coding unit corresponding to each node of a hybrid tree is split into hybrid trees, the coding unit may further include split direction information. The split direction information may indicate the split direction of the hybrid tree split. Split direction information having a first value (e.g., "1") may indicate that the coding unit is split vertically. Split direction information having a second value (e.g., "0") may indicate that the coding unit is split horizontally.

[0142] When a coding unit corresponding to each node of a hybrid tree is hybrid tree split, the coding unit may further include splitting tree information. The splitting tree information may indicate a tree used for hybrid tree splitting. Splitting tree information having a first value (e.g., "1") may indicate that the coding unit is binary tree split. Splitting tree information having a second value (e.g., "0") may indicate that the coding unit is ternary tree split.

[0143] The information on whether or not a partition has occurred, the partition tree information, and the partition direction information may each be a flag having a predetermined length (for example, 1 bit).

[0144] At least one of quad partition information, information on whether a hybrid tree is split, split direction information, and split tree information can be entropy coded / decoded. For entropy coding / decoding of the information, information on neighboring coding units adjacent to the current coding unit can be used. For example, the partitioning pattern (splitting pattern, partition tree, and / or split direction) of the left coding unit and / or the upper coding unit is likely to be similar to the partitioning pattern of the current coding unit. Therefore, context information for entropy coding / decoding of information on the current coding unit can be derived based on information on neighboring coding units. In this case, the information on neighboring coding units may include at least one of quad partition information of the current coding unit, information on whether a hybrid tree is split, split direction information, and split tree information.

[0145] In another embodiment, binary tree partitioning may be performed preferentially between binary tree partitioning and ternary tree partitioning. That is, binary tree partitioning may be applied first, and a coding unit corresponding to a leaf node of the binary tree may be set as the root node of the ternary tree. In this case, quadtree partitioning and binary tree partitioning may not be performed on a coding unit corresponding to a node of the ternary tree.

[0146] A coding unit that is not further divided by quadtree partitioning, binary tree partitioning, and / or ternary tree partitioning may become a unit of coding, prediction, and / or transformation. That is, a coding unit may not be further divided for prediction and / or transformation. Therefore, a partition structure, partition information, etc. for dividing a coding unit into prediction units and / or transform units may not exist in the bitstream.

[0147] However, if the size of a coding unit, which is the division unit, is larger than the size of the largest transform block, the coding unit can be recursively divided until it becomes equal to or smaller than the size of the largest transform block. For example, if the size of a coding unit is 64x64 and the size of the largest transform block is 32x32, the coding unit can be divided into four 32x32 blocks for transformation. For example, if the size of a coding unit is 32x64 and the size of the largest transform block is 32x32, the coding unit can be divided into two 32x32 blocks for transformation. In this case, whether or not the coding unit is divided for transformation is not separately signaled, but can be determined by comparing the width or height of the coding unit with the width or height of the largest transform block. For example, if the width of a coding unit is larger than the width of the largest transform block, the coding unit can be divided vertically in half. Also, if the height of a coding unit is larger than the height of the largest transform block, the coding unit can be divided horizontally in half.

[0148] Information regarding the maximum and / or minimum size of a coding unit and information regarding the maximum and / or minimum size of a transform block can be signaled or determined at a higher level of the coding unit. The higher level may be, for example, the sequence level, the picture level, the slice level, etc. For example, the minimum size of a coding unit may be determined to be 4x4. For example, the maximum size of a transform block may be determined to be 64x64. For example, the minimum size of a transform block may be determined to be 4x4.

[0149] Information regarding the minimum size of a coding unit corresponding to a leaf node of a quadtree (minimum size of a quadtree) and / or information regarding the maximum depth from the root node of a hybrid tree to a leaf node (maximum depth of a hybrid tree) can be signaled or determined at a higher level of the coding unit. The higher level may be, for example, a sequence level, a picture level, a slice level, etc. The information regarding the minimum size of a quadtree and / or information regarding the maximum depth of the hybrid tree can be signaled or determined for each of an intra-screen slice and an inter-screen slice.

[0150] Difference information regarding the size of the CTU and the maximum size of the transform block can be signaled or determined at a higher level of the coding unit. The higher level may be, for example, the sequence level, the picture level, or the slice level. Information regarding the maximum size of the coding unit corresponding to each node of the binary tree (maximum size of the binary tree) can be determined based on the size of the coding tree unit and the difference information. The maximum size of the coding unit corresponding to each node of the ternary tree (maximum size of the ternary tree) may have different values ​​depending on the type of slice. For example, in the case of an intra-screen slice, the maximum size of the ternary tree may be 32x32. Also, in the case of an inter-screen slice, the maximum size of the ternary tree may be 128x128. For example, the minimum size of the coding unit corresponding to each node of the binary tree (minimum size of the binary tree) and / or the minimum size of the coding unit corresponding to each node of the ternary tree (minimum size of the ternary tree) can be set as the minimum size of the coding block.

[0151] As another example, the maximum size of a binary tree and / or the maximum size of a ternary tree can be signaled or determined at the slice level, and the minimum size of a binary tree and / or the minimum size of a ternary tree can be signaled or determined at the slice level.

[0152] Depending on the size and depth information of the various blocks mentioned above, quad split information, information on whether it is a composite tree split, split tree information and / or split direction information may or may not be present in the bitstream.

[0153] For example, if the size of a coding unit is not larger than the minimum size of a quadtree, the coding unit does not include quad partition information, and the quad partition information can be inferred to a second value.

[0154] For example, if the size (width and height) of a coding unit corresponding to a node of a hybrid tree is larger than the maximum size (width and height) of a binary tree and / or the maximum size (width and height) of a ternary tree, the coding unit may not be split into a binary tree and / or a ternary tree. Therefore, information on whether the hybrid tree is split or not is not signaled and can be inferred from a second value.

[0155] Alternatively, if the size (width and height) of a coding unit corresponding to a node of a hybrid tree is the same as the minimum size (width and height) of a binary tree, or if the size (width and height) of a coding unit is the same as twice the minimum size (width and height) of a ternary tree, the coding unit may not be partitioned into binary trees and / or ternary trees. Therefore, information on whether the hybrid tree is to be partitioned is not signaled and can be inferred from a second value. This is because, if the coding unit is partitioned into binary trees and / or ternary trees, coding units smaller than the minimum size of a binary tree and / or the minimum size of a ternary tree are generated.

[0156] Alternatively, if the depth of a coding unit corresponding to a node of the hybrid tree in the hybrid tree is equal to the maximum depth of the hybrid tree, the coding unit may not be split into a binary tree and / or a ternary tree, and thus, information on whether the hybrid tree is split or not may not be signaled and may be inferred from a second value.

[0157] Alternatively, information on whether the hybrid tree is split can be signaled only if at least one of vertical binary tree splitting, horizontal binary tree splitting, vertical ternary tree splitting, and horizontal ternary tree splitting is possible for a coding unit corresponding to a node of the hybrid tree. Otherwise, the coding unit may not be split into a binary tree and / or a ternary tree. Therefore, information on whether the hybrid tree is split can be inferred as a second value without being signaled.

[0158] Alternatively, the split direction information may be signaled only if both vertical binary tree splitting and horizontal binary tree splitting are possible for a coding unit corresponding to a node of a hybrid tree, or both vertical ternary tree splitting and horizontal ternary tree splitting are possible. Otherwise, the split direction information may not be signaled and may be inferred to a value indicating the possible split direction.

[0159] Alternatively, the split tree information may be signaled only if both vertical binary tree splitting and vertical ternary tree splitting are possible for a coding unit corresponding to a node of a hybrid tree, or both horizontal binary tree splitting and horizontal ternary tree splitting are possible. Otherwise, the split tree information may not be signaled and may be inferred to a value indicating a tree that can be split.

[0160] FIG. 4 is a diagram for explaining an embodiment of the inter-prediction process.

[0161] The rectangles shown in Fig. 4 may represent images. The arrows in Fig. 4 may represent prediction directions. Each image may be classified into an I-picture (Intra Picture), a P-picture (Predictive Picture), a B-picture (Bi-predictive Picture), etc. depending on the encoding type.

[0162] An I picture can be encoded / decoded using intra prediction without inter prediction. A P picture can be encoded / decoded using inter prediction using reference images existing in only one direction (e.g., forward or backward). A B picture can be encoded / decoded using inter prediction using reference images existing in both directions (e.g., forward and backward). A B picture can also be encoded / decoded using inter prediction using reference images existing in both directions, or using inter prediction using reference images existing in one of the forward and backward directions. Here, the bidirectional direction can be forward and backward. When inter prediction is used, an encoder can perform inter prediction or motion compensation, and a decoder can perform corresponding motion compensation.

[0163] Inter prediction according to the embodiment will be specifically described below.

[0164] Inter-prediction or motion compensation can be performed using reference images and motion information.

[0165] Motion information for the current block can be derived during inter prediction by each of the encoding device 100 and the decoding device 200. The motion information can be derived using motion information of reconstructed neighboring blocks, motion information of a collocated block (col block), and / or a block adjacent to the collocated block. The collocated block may be a block corresponding to the spatial position of the current block in an already reconstructed collocated picture (col picture). Here, the collocated picture may be one picture among at least one reference picture included in the reference picture list.

[0166] The method of deriving motion information may vary depending on the prediction mode of the current block. For example, prediction modes applicable to inter-picture prediction may include AMVP mode, merge mode, skip mode, current picture reference mode, etc. Here, merge mode may also be referred to as motion merge mode.

[0167] For example, when AMVP is applied as a prediction mode, at least one of the motion vector of the reconstructed neighboring block, the motion vector of the co-located block, the motion vector of the block adjacent to the co-located block, and the (0,0) motion vector may be determined as a motion vector candidate to generate a motion vector candidate list. Motion vector candidates may be derived using the generated motion vector candidate list. Motion information of the current block may be determined based on the derived motion vector candidates. Here, the motion vector of the co-located block or the motion vector of the block adjacent to the co-located block may be referred to as a temporal motion vector candidate, and the motion vector of the reconstructed neighboring block may be referred to as a spatial motion vector candidate.

[0168] The encoding apparatus 100 may calculate a motion vector difference (MVD) between a motion vector of a current block and a motion vector candidate and entropy code the MVD. The encoding apparatus 100 may also entropy code a motion vector candidate index to generate a bitstream. The motion vector candidate index may indicate an optimal motion vector candidate selected from among the motion vector candidates included in a motion vector candidate list. The decoding apparatus 200 may entropy decode the motion vector candidate index from the bitstream and select a motion vector candidate for a current block from among the motion vector candidates included in the motion vector candidate list using the entropy-decoded motion vector candidate index. The decoding apparatus 200 may also derive a motion vector for a current block using the sum of the entropy-decoded MVD and the motion vector candidate.

[0169] The bitstream may include a reference image index indicating a reference image, etc. The reference image index may be entropy coded and signaled from the coding device 100 to the decoding device 200 via the bitstream. The decoding device 200 may generate a prediction block for the block to be decoded based on the derived motion vector and reference image index information.

[0170] Another example of a motion information derivation method is a merge mode. The merge mode may refer to the merging of motions for multiple blocks. The merge mode may refer to a mode in which motion information of a current block is derived from motion information of neighboring blocks. When the merge mode is applied, a merge candidate list may be generated using motion information of reconstructed neighboring blocks and / or motion information of co-located blocks. The motion information may include at least one of 1) a motion vector, 2) a reference image index, and 3) an inter-frame prediction indicator. The prediction indicator may be unidirectional (L0 prediction, L1 prediction) or bidirectional.

[0171] The merge candidate list may indicate a list in which motion information is stored. The motion information stored in the merge candidate list may be at least one of motion information of neighboring blocks adjacent to the current block (spatial merge candidate), motion information of a block collocated with the current block in a reference image (temporal merge candidate), new motion information generated by combining motion information already present in the merge candidate list, and a zero merge candidate.

[0172] The encoding apparatus 100 may entropy-encode at least one of a merge flag and a merge index to generate a bitstream and then signal the bitstream to the decoding apparatus 200. The merge flag may be information indicating whether to perform a merge mode for each block, and the merge index may be information indicating which block among neighboring blocks adjacent to the current block to merge with. For example, the neighboring blocks of the current block may include at least one of a left neighboring block, an upper neighboring block, and a temporal neighboring block of the current block.

[0173] The skip mode may be a mode in which motion information of a neighboring block is applied to a current block as is. When the skip mode is used, the encoding apparatus 100 may entropy encode information about which block's motion information is to be used as the motion information of the current block and signal the information to the decoding apparatus 200 via a bitstream. In this case, the encoding apparatus 100 may not need to signal syntax elements related to at least one of motion vector differential information, coded block flags, and transform coefficient levels (quantization levels) to the decoding apparatus 200.

[0174] The current picture reference mode may refer to a prediction mode using an already reconstructed region in the current picture to which the current block belongs. In this case, a vector may be defined to identify the already reconstructed region. Whether the current block is encoded in the current picture reference mode may be determined using the reference image index of the current block. A flag or index indicating whether the current block is a block encoded in the current picture reference mode may be signaled, or may be inferred using the reference image index of the current block. If the current block is encoded in the current picture reference mode, the current picture may be added to a fixed position or an arbitrary position in the reference image list for the current block. The fixed position may be, for example, a position where the reference image index is 0 or the last position. If the current picture is added to an arbitrary position in the reference image list, a separate reference image index indicating the arbitrary position may be signaled.

[0175] Based on the above, the image encoding / decoding method according to the present invention will be described in detail.

[0176] FIG. 5 is a flowchart showing an image encoding method according to an embodiment of the present invention, and FIG. 6 is a flowchart showing an image decoding method according to an embodiment of the present invention.

[0177] 5, the encoding apparatus may derive motion vector candidates (S501) and generate a motion vector candidate list based on the induced motion vector candidates (S502). Once the motion vector candidate list is generated, the generated motion vector candidate list may be used to determine a motion vector (S503), and motion compensation may be performed using the motion vector (S504). Thereafter, the encoding apparatus may entropy encode information related to motion compensation (S505).

[0178] 6, the decoding apparatus entropy decodes information about motion compensation received from the encoding apparatus (S601) and can derive motion vector candidates (S602). The decoding apparatus then generates a motion vector candidate list based on the induced motion vector candidates (S603) and can determine a motion vector using the generated motion vector candidate list (S604). Thereafter, the decoding apparatus can perform motion compensation using the motion vector (S605).

[0179] FIG. 7 is a flowchart illustrating an image encoding method according to another embodiment of the present invention, and FIG. 8 is a flowchart illustrating an image decoding method according to another embodiment of the present invention.

[0180] 7, the encoding device may derive merge candidates (S701) and generate a merge candidate list based on the derived merge candidates. Once the merge candidate list is generated, motion information may be determined using the generated merge candidate list (S702), and motion compensation for the current frame may be performed using the determined motion information (S703). Thereafter, the encoding device may entropy encode information related to motion compensation (S704).

[0181] 8, the decoding device may entropy decode information related to motion compensation received from the encoding device (S801), derive merge candidates (S802), and generate a merge candidate list based on the derived merge candidates. Once the merge candidate list is generated, the decoding device may determine motion information of the current block using the generated merge candidate list (S803). Thereafter, the decoding device may perform motion compensation using the motion information (S804).

[0182] Here, FIGS. 5 and 6 may be an example in which the AMVP mode described in FIG. 4 is applied, and FIGS. 7 and 8 may be an example in which the merge mode described in FIG. 4 is applied.

[0183] 5 and 6 will be described first, followed by a description of the steps shown in Figures 7 and 8. However, the motion compensation execution steps (S504, S605, S703, S804) and the entropy encoding / decoding steps (S505, S601, S704, S801) will be described together.

[0184] Each step shown in FIGS. 5 and 6 will now be described in detail.

[0185] First, the steps of deriving a motion vector will be specifically described (S501, S602).

[0186] The motion vector candidates for the current block may include at least one of spatial motion vector candidates or temporal motion vector candidates.

[0187] The spatial motion vector of the current block can be derived from reconstructed blocks surrounding the current block. For example, the motion vectors of reconstructed blocks surrounding the current block can be determined as spatial motion vector candidates for the current block.

[0188] FIG. 9 is a diagram illustrating an example of deriving spatial motion vector candidates for the current block.

[0189] Referring to FIG. 9, spatial motion vector candidates for a current block X can be derived from neighboring blocks neighboring the current block X. Here, the neighboring blocks neighboring the current block may include at least one of block B1 adjacent to the upper side of the current block, block A1 adjacent to the left side of the current block, block B0 adjacent to the upper right corner of the current block, block B2 adjacent to the upper left corner of the current block, and block A0 adjacent to the lower left corner of the current block. Meanwhile, the neighboring blocks neighboring the current block may be square or non-square in shape. If a motion vector exists in a neighboring block neighboring the current block, the motion vector of the neighboring block may be determined as a spatial motion vector candidate for the current block. Whether a motion vector of a neighboring block exists or whether the motion vector of the neighboring block can be used as a spatial motion vector candidate for the current block may be determined based on whether a neighboring block exists or whether the neighboring block is coded using inter-frame prediction. In this case, whether a motion vector of a neighboring block exists or whether the motion vector of a neighboring block can be used as a spatial motion vector candidate for the current block may be determined based on a predetermined priority. For example, in the example shown in FIG. 9, the availability of motion vectors can be determined in the order of blocks at positions A0, A1, B0, B1, and B2.

[0190] When a reference image of a current block and a reference image of a neighboring block having a motion vector are different, a scaled motion vector of the neighboring block may be determined as a spatial motion vector candidate for the current block. Here, the scaling may be performed based on at least one of the distance between the current image and the reference image referenced by the current block and the distance between the current image and the reference image referenced by the neighboring block. For example, a spatial motion vector candidate for the current block may be derived by scaling the motion vector of the neighboring block according to the ratio of the distance between the current image and the reference image referenced by the current block and the distance between the current image and the reference image referenced by the neighboring block.

[0191] On the other hand, if the reference image index of the current block is different from the reference image index of the neighboring block having the motion vector, a scaled motion vector of the neighboring block may be determined as a spatial motion vector candidate for the current block. In this case, scaling may be performed based on at least one of the distance between the current image and the reference image referenced by the current block and the distance between the current image and the reference image referenced by the neighboring block.

[0192] In relation to scaling, the motion vector of a neighboring block may be determined as a spatial motion vector candidate by scaling it based on a reference image indicated by a reference image index having a predefined value. In this case, the predefined value may be a positive integer including 0. As an example, a spatial motion vector candidate for the current block may be derived by scaling the motion vector of the neighboring block according to the ratio of the distance between the current image and the reference image of the current block indicated by the reference image index having a predefined value and the distance between the current image and the reference image of the neighboring block having a predefined value.

[0193] Also, spatial motion vector candidates for the current block can be derived based on at least one of the coding parameters of the current block.

[0194] The temporal motion vector candidates for the current block can be derived from the reconstructed blocks included in the co-located picture of the current image, where the co-located picture may be an image that has been coded / decoded before the current image and has a different temporal order from the current image.

[0195] FIG. 10 is a diagram illustrating an example of deriving temporal motion vector candidates for the current block.

[0196] 10, in a collocated picture of the current image, a temporal motion vector candidate for the current block can be derived from a block including an external position of a block corresponding to the same spatial position as the current block X, or a block including an internal position of a block corresponding to the same spatial position as the current block X. Here, the temporal motion vector candidate may refer to the motion vector of the collocated block. As an example, the temporal motion vector candidate for the current block X can be derived from block H adjacent to the lower left corner of block C corresponding to the same spatial position as the current block, or block C3 including the center point of block C. Block H or block C3 used to derive the temporal motion vector candidate for the current block can be referred to as a "collocated block."

[0197] Also, at least one of a temporal motion vector candidate, a corresponding location image, a corresponding location block, a prediction list utilization flag, and a reference image index may be derived based on at least one of the coding parameters.

[0198] When the distance between a current image including the current block and a reference image of the current block is different from the distance between a corresponding position image including the corresponding position block and a reference image of the corresponding position block, a temporal motion vector candidate for the current block can be obtained by scaling the motion vector of the corresponding position block. Here, the scaling can be performed based on at least one of the distance between the current image and the reference image referenced by the current block and the distance between the corresponding position image and the reference image referenced by the corresponding position block. For example, the temporal motion vector candidate for the current block can be derived by scaling the motion vector of the corresponding position block according to the ratio of the distance between the current image and the reference image referenced by the current block and the distance between the corresponding position image and the reference image referenced by the corresponding position block.

[0199] Next, the steps of generating a motion vector candidate list based on the guided motion vector candidates will be described (S502, S503).

[0200] Generating the motion vector candidate list may include adding or removing motion vector candidates from the motion vector candidate list, and adding combined motion vector candidates to the motion vector candidate list.

[0201] Considering the step of adding or removing the induced motion vector candidate to the motion vector candidate list, the encoding device and the decoding device can add the induced motion vector candidate to the motion vector candidate list in the induced order of the motion vector candidate.

[0202] It is assumed that the motion vector candidate list mvpListLX means the motion vector candidate list corresponding to the reference image lists L0, L1, L2, and L3. For example, the motion vector candidate list corresponding to the reference image list L0 can be called mvpListL0.

[0203] A motion vector having a predetermined value other than the spatial motion vector candidate and the temporal motion vector candidate may be added to the motion vector candidate list. For example, if the number of motion vector candidates included in the motion vector list is less than the maximum number of motion vector candidates, a motion vector with a value of 0 may be added to the motion vector candidate list.

[0204] Next, the step of adding the combined motion vector candidate to the motion vector candidate list will be described.

[0205] If the number of motion vector candidates included in the motion vector candidate list is less than the maximum number of motion vector candidates, a combined motion vector candidate may be added to the motion vector candidate list using at least one of the motion vector candidates included in the motion vector candidate list. For example, a combined motion vector candidate may be generated using at least one of the spatial motion vector candidate, the temporal motion vector candidate, and the zero motion vector candidate included in the motion vector candidate list, and the combined motion vector candidate may be included in the motion vector candidate list.

[0206] Alternatively, a combined motion vector candidate may be generated based on at least one of the coding parameters, or the combined motion vector candidate may be added to a motion vector candidate list based on at least one of the coding parameters.

[0207] Next, the steps of determining a predicted motion vector from the motion vector candidate list will be described (S503, S604).

[0208] Among the motion vector candidates included in the motion vector candidate list, the motion vector candidate indicated by the motion vector candidate index can be determined as the predicted motion vector for the current block.

[0209] The encoding device can calculate the difference between a motion vector and a predicted motion vector to calculate a motion vector differential value, and the decoding device can calculate a motion vector by combining the predicted motion vector and the motion vector differential value.

[0210] Here, motion information refinement can be applied to any of the motion vector candidates included in the motion vector candidate list, the predicted motion vector, or the motion vector calculated by combining the predicted motion vector and a motion vector differential value. The motion information refinement will be described in detail later.

[0211] Meanwhile, the steps of performing motion compensation (S504, S605) in Figures 5 and 6 and the steps of entropy encoding / decoding information related to motion compensation (S505, S601) will be integrated with the steps of performing motion compensation (S703, S804) and the steps of entropy encoding / decoding (S704, S801) in Figures 7 and 8 and will be described later.

[0212] Each step shown in FIGS. 7 and 8 will now be described in detail.

[0213] First, the steps for deriving merge candidates will be specifically described (S701, 802).

[0214] The merge candidates for the current block may include at least one of a spatial merge candidate, a temporal merge candidate, or an additional merge candidate. Here, inducing a spatial merge candidate may mean inducing a spatial merge candidate and adding it to a merge candidate list.

[0215] 9, spatial merge candidates for the current block can be derived from neighboring blocks adjacent to the current block X. The neighboring blocks adjacent to the current block X can include at least one of block B1 adjacent to the upper side of the current block, block A1 adjacent to the left side of the current block, block B0 adjacent to the upper right corner of the current block, block B2 adjacent to the upper left corner of the current block, and block A0 adjacent to the lower left corner of the current block.

[0216] In order to derive spatial merge candidates for the current block, it may be determined whether neighboring blocks adjacent to the current block can be used to derive spatial merge candidates for the current block. In this case, whether neighboring blocks adjacent to the current block can be used to derive spatial merge candidates for the current block may be determined based on a predetermined priority. For example, in the example shown in FIG. 9, spatial merge candidate deriving availability may be determined for blocks located at positions A1, B1, B0, A0, and B2 in this order. The spatial merge candidates determined based on the availability determination order may be added sequentially to a merge candidate list for the current block.

[0217] FIG. 11 is a diagram illustrating an example in which a spatial merge candidate is added to a merge candidate list.

[0218] Referring to FIG. 11, when four spatial merge candidates are derived from neighboring blocks at positions A1, B0, A0, and B2, the derived spatial merge candidates can be added sequentially to the merge candidate list.

[0219] The spatial merge candidates may also be derived based on at least one of the coding parameters.

[0220] Here, the motion information of the spatial merge candidate can have not only L0 and L1 motion information but also three or more motion information such as L2, L3, etc. Here, the reference image list can include at least one of L0, L1, L2, L3, etc.

[0221] Next, a method for deriving temporal merge candidates for the current block will be described.

[0222] The temporal merge candidates for the current block can be derived from the reconstructed blocks included in the co-located picture of the current image, where the co-located picture may be an image that has been encoded / decoded before the current image and has a different temporal order from the current image.

[0223] Inducing a temporal merge candidate may mean inducing a temporal merge candidate and adding it to a merge candidate list.

[0224] 10, in a collocated picture of the current image, a temporal merge candidate for the current block may be derived from a block including an outer position of a block corresponding to the same spatial position as the current block X, or a block including an inner position of a block corresponding to the same spatial position as the current block X. Here, the temporal merge candidate may refer to motion information of the collocated block. As an example, a temporal merge candidate for the current block X may be derived from block H adjacent to the lower left corner of block C corresponding to the same spatial position as the current block, or block C3 including the center point of block C. Block H or block C3 used to derive a temporal merge candidate for the current block may be referred to as a "collocated block."

[0225] If a temporal merge candidate for the current block can be derived from block H, which includes an external position of block C, block H can be set as the corresponding position block of the current block. In this case, the temporal merge candidate for the current block can be derived based on motion information of block H. On the other hand, if a temporal merge candidate for the current block cannot be derived from block H, block C3, which includes an internal position of block C, can be set as the corresponding position block of the current block. In this case, the temporal merge candidate for the current block can be derived based on motion information of block C3. If a temporal merge candidate for the current block cannot be derived from blocks H and C3 (e.g., when both blocks H and C3 are intra-coded), a temporal merge candidate for the current block may not be derived or may be derived from a block at a position different from blocks H and C3.

[0226] Alternatively, the temporal merge candidates for the current block may be derived from multiple blocks in the corresponding position image. For example, multiple temporal merge candidates for the current block may be derived from block H and block C3.

[0227] FIG. 12 is a diagram illustrating an example in which a temporal merge candidate is added to the merge candidate list.

[0228] Referring to FIG. 12, if one temporal merge candidate is derived from the corresponding position block of the H1 position, the derived temporal merge candidate can be added to the merge candidate list.

[0229] If the distance between the current image including the current block and the reference image of the current block is different from the distance between the corresponding position image including the corresponding position block and the reference image of the corresponding position block, the motion vector of the temporal merge candidate for the current block can be obtained by scaling the motion vector of the corresponding position block. Here, the scaling can be performed based on at least one of the distance between the current image and the reference image referenced by the current block and the distance between the corresponding position image and the reference image referenced by the corresponding position block. For example, the motion vector of the temporal merge candidate for the current block can be derived by scaling the motion vector of the corresponding position block according to the ratio of the distance between the current image and the reference image referenced by the current block and the distance between the corresponding position image and the reference image referenced by the corresponding position block.

[0230] In addition, at least one of a temporal merge candidate, a corresponding position image, a corresponding position block, a prediction list utilization flag, and a reference image index can be derived based on at least one of the coding parameters of the current block, the surrounding block, or the corresponding position block.

[0231] After deriving at least one of spatial merge candidates and temporal merge candidates, the derived merge candidates may be added to the merge candidate list in the order in which they were derived, thereby generating a merge candidate list.

[0232] Next, a method for deriving additional merge candidates for the current block will be described.

[0233] The additional merge candidate may refer to at least one of a modified spatial merge candidate, a modified temporal merge candidate, a combined merge candidate, and a merge candidate having a predetermined motion information value. Here, inducing the additional merge candidate may refer to inducing the additional merge candidate and adding it to the merge candidate list.

[0234] The modified spatial merge candidate may refer to a merge candidate obtained by modifying at least one of the motion information of the derived spatial merge candidate.

[0235] A modified temporal merge candidate may refer to a merge candidate in which at least one of the motion information of the derived temporal merge candidate is modified.

[0236] A combined merge candidate may refer to a merge candidate derived by combining at least one motion information of a spatial merge candidate, a temporal merge candidate, a modified spatial merge candidate, a modified temporal merge candidate, a combined merge candidate, and a merge candidate having a predetermined motion information value that exists in the merge candidate list.

[0237] Alternatively, a combined merge candidate may refer to a spatial merge candidate and a temporal merge candidate derived from a block that is not present in the merge candidate list but can derive at least one of a spatial merge candidate and a temporal merge candidate, and a merge candidate derived by combining motion information of at least one of a modified spatial merge candidate, a modified temporal merge candidate, a combined merge candidate, and a merge candidate having a predetermined motion information value that is generated based on these.

[0238] Alternatively, the decoder can derive combined merging candidates using motion information entropy decoded from the bitstream, and the motion information used to derive combined merging candidates in the encoder can be entropy coded into the bitstream.

[0239] The combined merge candidate may refer to a combined bi-predictive merge candidate. The combined bi-predictive merge candidate may refer to a merge candidate that uses bi-prediction and has L0 motion information and L1 motion information.

[0240] A merge candidate with a predetermined motion information value may refer to a zero merge candidate with a motion vector of (0, 0). Alternatively, a merge candidate with a predetermined motion information value may be set to use the same value in the encoding device and the decoding device.

[0241] At least one of a modified spatial merge candidate, a modified temporal merge candidate, a combined merge candidate, and a merge candidate with a predetermined motion information value may be derived or generated based on at least one of the coding parameters of the current block, the surrounding block, or the correspondingly located block. Also, at least one of the modified spatial merge candidate, the modified temporal merge candidate, the combined merge candidate, and a merge candidate with a predetermined motion information value may be added to a merge candidate list based on at least one of the coding parameters of the current block, the surrounding block, or the correspondingly located block.

[0242] Meanwhile, the size of the merge candidate list can be determined based on the coding parameters of the current block, the neighboring blocks, or the corresponding position block, and the size can be changed based on the coding parameters.

[0243] Next, the steps of determining the motion information of the current block using the generated merge candidate list will be described in detail (S702, S803).

[0244] The encoder can determine a merge candidate to be used for motion compensation from among the merge candidates in the merge candidate list through motion estimation, and encode a merge candidate index (merge_idx) indicating the determined merge candidate into the bitstream.

[0245] Meanwhile, to generate a predicted block, the encoder may select a merge candidate from the merge candidate list based on the merge candidate index, determine motion information for the current block, and perform motion compensation based on the determined motion information to generate a predicted block for the current block.

[0246] The decoder may decode a merge candidate index in the bitstream and determine a merge candidate in the merge candidate list indicated by the merge candidate index. The determined merge candidate may be determined as motion information of the current block. The determined motion information is used for motion compensation of the current block. In this case, motion compensation may have the same meaning as inter prediction.

[0247] Meanwhile, motion information compensation may be applied to one of the merge candidates included in the merge candidate list or the motion information determined from the merge candidate list based on the merge candidate index. The motion information compensation will be described in detail later.

[0248] Next, the steps of performing motion compensation using the motion vector or motion information will be described (S504, S605, S703, S804).

[0249] The encoding device and the decoding device can calculate a motion vector using the predicted motion vector and the motion vector differential value. Once the motion vector is calculated, the calculated motion vector can be used to perform inter-frame prediction or motion compensation (S504, S605).

[0250] Meanwhile, the encoding apparatus and the decoding apparatus may perform inter prediction or motion compensation using the determined motion information (S703, S804), where the current block may have motion information of the determined merge candidate.

[0251] The current block can have a minimum of 1 to a maximum of N motion vectors based on the prediction direction. Using the motion vectors, a minimum of 1 to a maximum of N prediction blocks can be generated to derive the final prediction block of the current block.

[0252] For example, if the current block has one motion vector, a predicted block generated using the motion vector (or motion information) may be determined as the final predicted block of the current block.

[0253] On the other hand, if the current block has multiple motion vectors (or motion information), multiple prediction blocks can be generated using the multiple motion vectors (or motion information), and the final prediction block of the current block can be determined based on a weighted sum of the multiple prediction blocks. Reference images including each of the multiple prediction blocks indicated by the multiple motion vectors (or motion information) can be included in different reference image lists or can be included in the same reference image list.

[0254] As an example, multiple predictive blocks can be generated based on at least one of spatial motion vector candidates, temporal motion vector candidates, motion vectors with predetermined values, or combined motion vector candidates, and the final predictive block of the current block can be determined based on a weighted sum of the multiple predictive blocks.

[0255] As another example, a plurality of prediction blocks may be generated based on motion vector candidates indicated by previously set motion vector candidate indexes, and a final prediction block of the current block may be determined based on a weighted sum of the plurality of prediction blocks. Alternatively, a plurality of prediction blocks may be generated based on motion vector candidates existing within a range of previously set motion vector candidate indexes, and a final prediction block of the current block may be determined based on a weighted sum of the plurality of prediction blocks.

[0256] The weight applied to each prediction block may be 1 / N (where N is the number of generated prediction blocks) and may have an equal value. For example, if two prediction blocks are generated, the weight applied to each prediction block may be 1 / 2, if three prediction blocks are generated, the weight applied to each prediction block may be 1 / 3, and if four prediction blocks are generated, the weight applied to each prediction block may be 1 / 4. Alternatively, a different weight may be assigned to each prediction block to determine the final prediction block of the current block.

[0257] The weights do not have to have fixed values ​​for each prediction block, but can also have variable values ​​for each prediction block. In this case, the weights applied to each prediction block may be the same or different. For example, when two prediction blocks are generated, the weights applied to the two prediction blocks may be variable values ​​for each block, such as (1 / 2, 1 / 2), (1 / 3, 2 / 3), (1 / 4, 3 / 4), (2 / 5, 3 / 5), (3 / 8, 5 / 8), etc. Meanwhile, the weights may be positive real numbers or negative real numbers. For example, they may include negative real numbers, such as (-1 / 2, 3 / 2), (-1 / 3, 4 / 3), (-1 / 4, 5 / 4), etc.

[0258] Meanwhile, in order to apply variable weights, one or more weight information for the current block may be signaled via a bitstream. The weight information may be signaled for each prediction block or for each reference image. Multiple prediction blocks may share one weight information.

[0259] The encoding device and the decoding device may determine whether to use a predicted motion vector (or motion information) based on the prediction block list utilization flag. As an example, if the prediction block list utilization flag for each reference image list indicates a first value '1', it indicates that the encoding device and the decoding device can use the predicted motion vector of the current block to perform inter prediction or motion compensation, and if it indicates a second value '0', it indicates that the encoding device and the decoding device do not perform inter prediction or motion compensation using the predicted motion vector of the current block. Meanwhile, the first value of the prediction block list utilization flag may be set to 0, and the second value may be set to 1. The following Equations 3 to 5 respectively show an example of generating a final predicted block for the current block when the inter prediction indicator of the current block is PRED_BI, PRED_TRI, and PRED_QUAD and the prediction direction for each reference image list is unidirectional.

[0260]

number

[0261]

number

[0262]

number

[0263] In Equations 1 to 3, P_BI, P_TRI, and P_QUAD represent the final predicted block of the current block, and LX (X=0, 1, 2, 3) may represent a reference image list. WF_LX represents a weight value of a predicted block generated using LX, and OFFSET_LX represents an offset value for a predicted block generated using LX. P_LX represents a predicted block generated using a motion vector (or motion information) for LX of the current block. RF represents a rounding factor and may be set to 0, a positive number, or a negative number. The LX reference picture list may include at least one of long-term reference pictures, reference pictures not subjected to a deblocking filter, reference pictures not subjected to a sample adaptive offset, reference pictures not subjected to an adaptive loop filter, reference pictures subjected to only a deblocking filter and an adaptive offset, reference pictures subjected to only a deblocking filter and an adaptive loop filter, reference pictures subjected to only a sample adaptive offset and an adaptive loop filter, and reference pictures subjected to a deblocking filter, a sample adaptive offset, and an adaptive loop filter. In this case, the LX reference picture list may be at least one of an L2 reference picture list and an L3 reference picture list.

[0264] Even when there are multiple prediction directions for a given reference image list, a final prediction block for a current block can be obtained based on a weighted sum of prediction blocks, in which case weights applied to prediction blocks derived from the same reference image list may have the same value or different values.

[0265] At least one of the weights WF_LX and offsets OFFSET_LX for the plurality of prediction blocks may be an entropy coded / decoded coding parameter. As another example, the weights and offsets may be derived from coded / decoded neighboring blocks around the current block. Here, the neighboring blocks around the current block may include at least one of blocks used to derive spatial motion vector candidates for the current block or blocks used to derive temporal motion vector candidates for the current block.

[0266] As another example, the weight and offset may be determined based on the display order (POC) of the current picture and each reference picture. In this case, the greater the distance between the current picture and the reference picture, the smaller the weight or offset may be set, and the closer the distance between the current picture and the reference picture, the larger the weight or offset may be set. For example, if the difference in POC between the current picture and the L0 reference picture is 2, the weight value applied to the predicted block generated with reference to the L0 reference picture may be set to 1 / 3, whereas if the difference in POC between the current picture and the L0 reference picture is 1, the weight value applied to the predicted block generated with reference to the L0 reference picture may be set to 2 / 3. As illustrated above, the weight or offset value may be inversely proportional to the difference in display order between the current picture and the reference picture. As another example, the weight or offset value may be proportional to the difference in display order between the current picture and the reference picture.

[0267] As another example, at least one of the weights or offsets may be entropy coded / decoded based on at least one of the coding parameters, and a weighted sum of the prediction block may be calculated based on at least one of the coding parameters.

[0268] The weighted sum of multiple predicted blocks can be applied only to a portion of the predicted block. Here, the portion of the predicted block may be a boundary region within the predicted block. As described above, in order to apply the weighted sum only to a portion of the predicted block, the weighted sum can be performed in units of sub-blocks of the predicted block.

[0269] For sub-blocks of smaller block sizes within a block of the block size indicated by the region information, inter-picture prediction or motion compensation can be performed using the same prediction block or the same final prediction block.

[0270] Furthermore, in a sub-block at a deeper block depth within a block at a block depth indicated by the region information, inter-picture prediction or motion compensation can be performed using the same prediction block or the same final prediction block.

[0271] In addition, when calculating a weighted sum of a predicted block using motion vector prediction, the weighted sum can be calculated using at least one motion vector candidate present in the motion vector candidate list and can be used as the final predicted block of the current block.

[0272] For example, a prediction block can be generated using only spatial motion vector candidates, a weighted sum of the prediction blocks can be calculated, and the calculated weighted sum can be used as the final prediction block for the current block.

[0273] For example, a prediction block may be generated using spatial motion vector candidates and temporal motion vector candidates, a weighted sum of the prediction blocks may be calculated, and the calculated weighted sum may be used as the final prediction block of the current block.

[0274] For example, a prediction block can be generated using only the combined motion vector candidates, a weighted sum of the prediction blocks can be calculated, and the calculated weighted sum can be used as the final prediction block for the current block.

[0275] For example, a prediction block can be generated using only motion vector candidates having a specific motion vector candidate index, a weighted sum of the prediction blocks can be calculated, and the calculated weighted sum can be used as the final prediction block for the current block.

[0276] For example, a prediction block can be generated using only motion vector candidates that are within a specific range of motion vector candidate indexes, a weighted sum of the prediction blocks can be calculated, and the calculated weighted sum can be used as the final prediction block for the current block.

[0277] In addition, when calculating a weighted sum of predicted blocks using the merge mode, the weighted sum can be calculated using at least one merge candidate present in the merge candidate list and can be used as the final predicted block of the current block.

[0278] For example, a prediction block can be generated using only spatial merge candidates, a weighted sum of the prediction blocks can be calculated, and the calculated weighted sum can be used as the final prediction block for the current block.

[0279] For example, a prediction block may be generated using spatial merge candidates and temporal merge candidates, a weighted sum of the prediction blocks may be calculated, and the calculated weighted sum may be used as the final prediction block of the current block.

[0280] For example, a prediction block can be generated using only the combined merging candidates, a weighted sum of the prediction blocks can be calculated, and the calculated weighted sum can be used as the final prediction block for the current block.

[0281] For example, a prediction block may be generated using only merge candidates with a particular merge candidate index, a weighted sum of the prediction blocks may be calculated, and the calculated weighted sum may be used as the final prediction block for the current block.

[0282] For example, a prediction block can be generated using only merge candidates that are within a specific range of merge candidate indices, a weighted sum of the prediction blocks can be calculated, and the calculated weighted sum can be used as the final prediction block for the current block.

[0283] The encoder and decoder can perform motion compensation using motion vectors / information of a current block. At this time, a final predicted block, which is a result of motion compensation, can be generated using at least one predicted block. Here, the current block can refer to at least one of a current coding block and a current prediction block.

[0284] A final predicted block can be generated by performing overlapped block motion compensation on a region corresponding to a boundary within the current block.

[0285] The area corresponding to the boundary within the current block may be an area within the current block adjacent to the boundary of a neighboring block of the current block. Here, the area corresponding to the boundary within the current block may include at least one of an upper boundary area, a left boundary area, a lower boundary area, a right boundary area, an upper right corner area, a lower right corner area, an upper left corner area, and a lower left corner area of ​​the current block. In addition, the area corresponding to the boundary within the current block may be an area corresponding to a portion within a predicted block of the current block.

[0286] The overlap block motion compensation may mean performing motion compensation by calculating a weighted sum of a predicted block area corresponding to a boundary within the current block and a predicted block generated using motion information of a block encoded / decoded adjacent to the current block.

[0287] The weighted sum may be performed in units of sub-blocks after dividing the current block into a number of sub-blocks. That is, motion compensation may be performed in units of sub-blocks using motion information of blocks coded / decoded adjacent to the current block. In this case, the sub-block may refer to a lower block.

[0288] In addition, the weighted sum may be calculated using a first predicted block generated on a sub-block basis using motion information of the current block and a second predicted block generated using motion information of neighboring sub-blocks spatially adjacent to the current block. Here, "using motion information" may mean deriving motion information. The first predicted block may refer to a predicted block generated using motion information of a sub-block to be coded / decoded within the current block. The second predicted block may refer to a predicted block generated using motion information of a neighboring sub-block spatially adjacent to the sub-block to be coded / decoded within the current block.

[0289] The final predicted block can be generated using a weighted sum of the first predicted block and the second predicted block, i.e., overlapped block motion compensation can generate the final predicted block using motion information of other blocks in addition to the motion information of the current block.

[0290] In addition, when at least one of Advanced Motion Vector Prediction (AMVP), merge mode, affine motion compensation mode, decoder motion vector guidance mode, adaptive motion vector resolution mode, regional illumination compensation mode, and bidirectional optical flow mode is selected, the current prediction block can be divided into sub-blocks and then overlap block motion compensation can be performed for each sub-block.

[0291] Here, in the case of merge mode, overlapped block motion compensation can be performed on at least one of an advanced temporal motion vector predictor (ATMVP) candidate and a spatial-temporal motion vector predictor (STMVP) candidate.

[0292] Next, the process of entropy coding / decoding information related to motion compensation will be described in detail (S505, S601, S704, S801).

[0293] The encoding device entropy encodes information related to motion compensation via a bitstream, and the decoding device can entropy decode the information related to motion compensation included in the bitstream. Here, the information regarding the motion compensation to be entropy coded / decoded may include at least one of an inter prediction indicator (inter_pred_idc), a reference image index (ref_idx_l0, ref_idx_l1, ref_idx_l2, ref_idx_l3), a motion vector candidate index (mvp_l0_idx, mvp_l1_idx, mvp_l2_idx, mvp_l3_idx), a motion vector difference, skip mode usage information (cu_skip_flag), merge mode usage information (merge_flag), merge index information (merge_index), weight values ​​(wf_l0, wf_l1, wf_l2, wf_l3), and offset values ​​(offset_l0, offset_l1, offset_l2, offset_l3).

[0294] The inter prediction indicator may indicate at least one of the inter prediction direction or the number of prediction directions of the current block when the current block is encoded / decoded using inter prediction. For example, the inter prediction indicator may indicate unidirectional prediction or multiple directional prediction such as bidirectional prediction, three-way prediction, or four-way prediction. The inter prediction indicator may indicate the number of reference images used when the current block generates a predicted block. Alternatively, one reference image may be used for multiple directional predictions. In this case, N (N>M) directional predictions can be performed using M reference images. The inter prediction indicator may also indicate the number of prediction blocks used when performing inter prediction or motion compensation on the current block.

[0295] The reference image indicator may indicate one-way (PRED_LX), bi-predictive or bidirectional (PRED_BI), three-way (PRED_TRI), four-way (PRED_QUAD) or more directions depending on the number of prediction directions of the current block.

[0296] The prediction list utilization flag indicates whether or not the reference image list is used to generate a prediction block.

[0297] For example, when the prediction list utilization flag indicates a first value “1”, it indicates that a prediction block can be generated using the corresponding reference image list, and when the prediction list utilization flag indicates a second value “0”, it indicates that a prediction block is not generated using the corresponding reference image list. Here, the first value of the prediction list utilization flag may be set to 0, and the second value may be set to 1.

[0298] That is, when the prediction list utilization flag indicates a first value, a prediction block of the current block can be generated using motion information corresponding to a corresponding reference image list.

[0299] The reference image index can identify the reference image from each reference image list that the current block refers to. For each reference image list, one or more reference image indexes can be entropy coded / decoded. The current block can perform motion compensation using one or more reference image indexes.

[0300] The motion vector candidate index indicates a motion vector candidate for the current block from a motion vector candidate list generated for each reference image list or each reference image index. At least one motion vector candidate index can be entropy coded / decoded for each motion vector candidate list. The current block can be motion compensated using at least one motion vector candidate index.

[0301] A motion vector differential indicates a difference between a motion vector and a predicted motion vector. One or more motion vector differentials can be entropy coded / decoded for a reference image list or a motion vector candidate list generated for each reference image index for a current block. The current block can be motion compensated using one or more motion vector differentials.

[0302] The skip mode use information (cu_skip_flag) indicates the use of the skip mode when it has a first value of 1, and does not indicate the use of the skip mode when it has a second value of 0. Based on the skip mode use information, motion compensation of the current block can be performed using the skip mode.

[0303] The merge mode use information (merge_flag) may indicate the use of the merge mode when it has a first value of 1, and may not indicate the use of the merge mode when it has a second value of 0. Motion compensation of the current block may be performed using the merge mode based on the information on whether the merge mode is used.

[0304] The merge index information (merge_index) may refer to information indicating a merge candidate in a merge candidate list.

[0305] Also, merge index information may refer to information on a merge index.

[0306] In addition, the merge index information may indicate a block that induces a merge candidate among reconstructed blocks that are spatially / temporally adjacent to the current block.

[0307] Furthermore, the merge index information may indicate at least one of the motion information possessed by the merge candidate. For example, if the merge index information has a first value of 0, it may indicate the first merge candidate in the merge candidate list. If the merge index information has a second value of 1, it may indicate the second merge candidate in the merge candidate list. If the merge index information has a third value of 2, it may indicate the third merge candidate in the merge candidate list. Similarly, if the merge index information has a fourth to Nth value, it may indicate a merge candidate corresponding to the value based on the order in the merge candidate list. Here, N may represent a positive integer including 0.

[0308] Based on the merge mode index information, the current block can be motion compensated using the merge mode.

[0309] When two or more prediction blocks are generated during motion compensation for a current block, a final prediction block for the current block can be generated through a weighted sum of the respective prediction blocks. During the weighted sum operation, at least one of a weight and an offset can be applied to each prediction block. The weighted sum factor, such as a weight (weighting factor) or an offset, used in the weighted sum operation can be entropy coded / decoded for at least one of the reference image list, reference image, motion vector candidate index, motion vector difference, motion vector, skip mode use / non-use information, merge mode use / non-use information, and merge index information, or for at least one of the reference image list, reference image, motion vector candidate index, motion vector difference, motion vector, skip mode use / non-use information, merge mode use / non-use information, and merge index information. In addition, the weighted sum factor of each prediction block can be entropy coded / decoded based on an inter-frame prediction indicator. Here, the weighted sum factor can include at least one of a weight and an offset.

[0310] Information related to motion compensation may be entropy coded / decoded on a block-by-block basis, or may be entropy coded / decoded at a higher level. For example, information related to motion compensation may be entropy coded / decoded on a block-by-block basis, such as a CTU, CU, or PU, or may be entropy coded / decoded at a higher level, such as a video parameter set, a sequence parameter set, a picture parameter set, an adaptation parameter set, or a slice header.

[0311] The information regarding motion compensation may be entropy coded / decoded based on an information difference value regarding motion compensation indicating a difference value between the information regarding motion compensation and an information prediction value regarding motion compensation.

[0312] Instead of entropy coding / decoding information about motion compensation of the current block, information about motion compensation of blocks coded / decoded around the current block can also be used as information about motion compensation of the current block.

[0313] Also, at least one of the information relating to the motion compensation can be derived based on at least one of the coding parameters.

[0314] Furthermore, at least one of the pieces of information relating to motion compensation can be entropy decoded from a bitstream based on at least one of the coding parameters, and at least one of the pieces of information relating to motion compensation can be entropy coded into a bitstream based on at least one of the coding parameters.

[0315] The information related to motion compensation may further include at least one of a motion vector, a motion vector candidate, a motion vector candidate index, a motion vector difference, a motion vector prediction value, skip mode use / non-use information (skip_flag), merge mode use / non-use information (merge_flag), merge index information (merge_index), motion vector resolution information, overlapped block motion compensation information, local illumination compensation information, affine motion compensation information, decoder-side motion vector derivation information, and bi-directional optical flow information. Here, decoder motion vector derivation may mean pattern matched motion vector derivation.

[0316] The motion vector resolution information may be information indicating whether a specific resolution is used for at least one of a motion vector and a motion vector differential value. Here, the resolution may refer to precision. The specific resolution may be set to at least one of 16-pel units, 8-pel units, 4-pel units, integer-pel units, 1 / 2-pel units, 1 / 4-pel units, 1 / 8-pel units, 1 / 16-pel units, 1 / 32-pel units, and 1 / 64-pel units.

[0317] The overlap block motion compensation information may be information indicating whether or not to calculate a weighted sum of predicted blocks of the current block by further using motion vectors of neighboring blocks spatially adjacent to the current block when motion compensating the current block.

[0318] The regional illumination compensation information may be information indicating whether at least one of a weight value and an offset value is applied when generating a predicted block of a current block, where at least one of the weight value and the offset value may be a value calculated based on a reference block.

[0319] The affine motion compensation information may be information indicating whether an affine motion model is used during motion compensation for a current block. Here, the affine motion model may be a model that divides one block into multiple sub-blocks using multiple parameters and calculates motion vectors of the divided sub-blocks using a representative motion vector.

[0320] The decoder motion vector guidance information may be information indicating whether a motion vector required for motion compensation is guided and used in the decoder. Based on the decoder motion vector guidance information, information about the motion vector may not be entropy coded / decoded. Furthermore, if the decoder motion vector guidance information indicates that a motion vector is guided and used in the decoder, information about the merge mode may be entropy coded / decoded. That is, the decoder motion vector guidance information may indicate whether the decoder uses the merge mode.

[0321] The bidirectional optical flow information may be information indicating whether to calibrate a motion vector in a pixel unit or a sub-block unit to perform motion compensation. Based on the bidirectional optical flow information, the pixel unit or sub-block unit motion vector may not be entropy coded / decoded. Here, the motion vector calibration may be to change a block unit motion vector to a pixel unit or sub-block unit motion vector value.

[0322] The current block can be motion compensated using at least one of the information related to motion compensation, and can be entropy coded / decoded using at least one of the information related to motion compensation.

[0323] When entropy coding / decoding information related to motion compensation, binarization methods such as truncated Rice binarization, K-th order Exp_Golomb binarization, K-th order Exp_Golomb binarization, fixed-length binarization, unary binarization, or truncated unary binarization can be used.

[0324] When entropy encoding / decoding information regarding motion compensation, a context model can be determined using at least one of information regarding motion compensation of neighboring blocks around the current block or area information of the neighboring blocks, information regarding previously encoded / decoded motion compensation or area information previously encoded / decoded, information regarding the depth of the current block, and information regarding the size of the current block.

[0325] In addition, when entropy encoding / decoding information regarding motion compensation, entropy encoding / decoding can also be performed using at least one of information regarding motion compensation of surrounding blocks, information regarding previously encoded / decoded motion compensation, information regarding the depth of the current block, and information regarding the size of the current block as a predicted value for information regarding motion compensation of the current block.

[0326] Hereinafter, motion information refinement will be described in detail with reference to FIGS. 13 to 15. Herein, motion information refinement may refer to the correction of at least one piece of motion information. That is, information to be subjected to motion information refinement may be at least one piece of information included in the motion information, such as a motion vector, a reference image index, a reference image, an inter-frame prediction indicator, a prediction list utilization flag, a weight (weighting factor), and an offset. In this case, motion information refinement may refer to the correction of one or more values ​​of information included in the motion information, such as a motion vector, a reference image index, a reference image, an inter-frame prediction indicator, a prediction list utilization flag, a weight, and an offset.

[0327] Furthermore, the information to be subjected to the motion information correction may be at least one of the information included in the coding parameters, and in this case, the motion information correction may mean that one or more values ​​of the information included in the coding parameters are corrected.

[0328] After the motion information correction, corrected motion information can be calculated. The motion information corrected by the motion information correction method can be used for motion compensation of the block to be coded / decoded (S504 in FIG. 5, S605 in FIG. 6, S703 in FIG. 7, and S804 in FIG. 8).

[0329] Regarding the step of performing motion information correction, the motion information correction may be performed before the motion compensation step, i.e., before the motion compensation process of the block to be coded / decoded is performed, a motion information correction method may be performed to calculate corrected motion information, and the motion compensation process may be performed using the corrected motion information.

[0330] For example, in the case of the AMVP mode, motion information refinement may be performed in the step of determining a motion vector (S503 in FIG. 5, S604 in FIG. 6). In this case, motion information refinement may be applied to any of the motion vector candidates included in the motion vector candidate list, the predicted motion vector, or the motion vector calculated by combining the predicted motion vector and the motion vector difference.

[0331] As another example, in the case of merge mode, motion information compensation may be performed in the step of determining motion information (S702 in FIG. 7, S803 in FIG. 8). In this case, motion information compensation may be applied to either the merge candidates included in the merge candidate list or the motion information determined in the merge candidate list based on the merge candidate index.

[0332] Meanwhile, in the case of skip mode, motion information correction may be performed, in which case motion information correction may be applied to either a skip candidate included in the skip mode candidate list or motion information determined in the skip candidate list based on the skip index.

[0333] Furthermore, with regard to the step of performing motion information correction, the motion information correction may be performed within a motion information compensation step (S504 of FIG. 5, S605 of FIG. 6, S703 of FIG. 7, and S804 of FIG. 8). That is, a motion information correction method may be performed within the motion compensation process of a block to be coded / decoded to calculate corrected motion information, and the motion compensation process may be performed using the corrected motion information.

[0334] For example, the encoder / decoder may generate a prediction block based on motion information determined in a motion vector determination step or a motion information determination step (S503 of FIG. 5, S604 of FIG. 6, S702 of FIG. 7, and S803 of FIG. 8), and calculate corrected motion information by correcting the motion information using the generated prediction block. Here, the encoder / decoder may generate a final prediction block using the corrected motion information.

[0335] Meanwhile, the motion information can be corrected based on the same rule in the encoder and decoder. Since the motion information is corrected based on the same rule in the encoder and decoder, it is not necessary to entropy code / decode information on whether or not motion information correction is used.

[0336] Hereinafter, bilateral template matching, which is one embodiment of motion information correction, will be described with reference to FIGS.

[0337] Bi-directional template matching is an embodiment of a correction method for a motion vector in motion information. At least one of two motion vectors for bi-directional prediction can be corrected using bi-directional template matching. Here, the bi-directional template may refer to a predicted block calculated by weighting a predicted block generated using two motion vectors in bi-directional prediction. The bi-directional template may refer to bi-prediction.

[0338] Specifically, when bidirectional template matching is performed on a bidirectionally predicted block, at least one of the first motion vector corresponding to reference image list 0 and the second motion vector corresponding to reference image list 1 can be corrected. In this case, the corrected motion vector can have a motion vector value different from the motion vector before correction.

[0339] FIG. 13 is a diagram for explaining bilateral template matching.

[0340] Referring to FIG. 13, bilateral template matching may include steps of: 1) generating a bilateral template using an initial motion vector; and 2) correcting the motion vector by comparing samples in the bilateral template with reconstructed samples in a reference image.

[0341] 1) Generating both templates The encoder / decoder may generate a bidirectional template using a predictive block generated via a first motion vector corresponding to reference image list 0 and a second motion vector corresponding to reference image list 1. The encoder / decoder may also generate a bidirectional template using a predictive block generated via a first motion vector corresponding to reference image list 0 and a second motion vector corresponding to reference image list 0. The encoder / decoder may also generate a bidirectional template using a predictive block generated via a first motion vector corresponding to reference image list 1 and a second motion vector corresponding to reference image list 1. Here, the bidirectional template may be generated by weighting the predictive block, and weights used in the weighting sum may be 0.5:0.5.

[0342] Meanwhile, the motion vector used to generate the bidirectional template may be referred to as an initial motion vector, which may refer to a motion vector calculated by at least one of a merge mode, an AMVP mode, a skip mode, etc.

[0343] 2) Motion vector correction The encoder / decoder can compare the values ​​of the samples in both templates with the values ​​of the reconstructed samples in the reference images to correct each motion vector corresponding to each reference image list.

[0344] The motion vector that indicates a position in the reference image of reference image list 0 that shows the least distortion with both templates can be determined as the corrected first motion vector, and the motion vector that indicates a position in the reference image of reference image list 1 that shows the least distortion with both templates can be determined as the corrected second motion vector.

[0345] Specifically, if the distortion value between the region in the reference image indicated by the initial first motion vector and both templates is greater than the distortion value between the region in the reference image indicated by the corrected first motion vector and both templates, the first motion vector can be corrected. Similarly, if the distortion value between the region in the reference image indicated by the initial second motion vector and both templates is greater than the distortion value between the region in the reference image indicated by the corrected second motion vector and both templates, the second motion vector can be corrected.

[0346] In this case, when comparing the values ​​of samples in both templates with the restored samples in the reference image, the encoder / decoder may determine, as the compensated motion vector, a motion vector indicating the position of the restored sample in the reference image that shows the minimum distortion according to a sample-to-sample distortion calculation method such as SAD (Sum of Absolute Difference), SATD (Sum of Absolute Transformed Difference), SSE (Sum of Squared Error), MSE (Mean of Squared Error), MR-SAD (Mean Removed SAD), etc. In this case, the distortion may be calculated for at least one of a luminance component and a chrominance component.

[0347] On the other hand, if the SAD value between the regions in the reference image indicated by the initial first motion vector and the initial second motion vector is greater than a predetermined value, bilateral template matching can be performed to correct the initial first motion vector and the initial second motion vector. Conversely, if the SAD value between the regions in the reference image indicated by the initial first motion vector and the initial second motion vector is smaller than a predefined value, bilateral template matching does not need to be performed.

[0348] Also, in the case of bidirectional prediction, if the initial first motion vector value corresponding to the first prediction direction (e.g., the L0 prediction direction) is the same as the corrected first motion vector value to which motion information correction has been applied, motion information correction does not need to be performed on the initial second motion vector corresponding to the second prediction direction (e.g., the L1 prediction direction).

[0349] Alternatively, two-way template matching can be performed recursively.

[0350] Specifically, a second bidirectional template may be generated using the corrected first motion vector and the corrected second motion vector, and then bidirectional template matching may be performed using the generated second bidirectional template to re-correct the corrected first motion vector and the corrected second motion vector. In this case, the method of determining the re-corrected motion vector through bidirectional template matching may be repeated up to M times. Here, M is a positive integer (e.g., 2), and may be a fixed value predetermined by the encoder / decoder or a variable value coded and signaled by the encoder. Alternatively, M may be determined based on the size of the block to be coded / decoded.

[0351] As an example, if the height or width of the block to be coded / decoded is less than 8, M can be set to 2.

[0352] Also, if the height and width of the block to be coded / decoded are 8, M can be set to 2.

[0353] Also, if the width of the block to be coded / decoded is 8 and the height is 16, or vice versa, M can be set to 4.

[0354] If not, M can be set to 8. In this case, bilateral template matching can be performed four times for the first reference image (e.g., L0 reference image) and four times for the second reference image (e.g., L1 reference image).

[0355] Meanwhile, if bilateral template matching is performed recursively in subpixel units, bilateral template matching can be performed recursively in integer pixel units.

[0356] The corrected motion vector generated as a result of the bidirectional template matching can be used for motion compensation of the current block by replacing the initial motion vector, and the bidirectional template matching can be performed in the encoder and decoder according to the same rules.

[0357] In bilateral template matching, the corrected motion vector can be searched within a limited area of ​​the reference image.

[0358] 14 and 15 are diagrams showing areas (hereinafter referred to as "search areas") in which a motion vector corrected by bilateral template matching is searched.

[0359] 14, the search region may be defined in integer pixel units across and down to a range of −M to +N pixels, where M and N may be positive integers.

[0360] The search area may be defined in sub-pel units in the horizontal and vertical directions within a range of -O to +P sub-pixels, where O and P may be fractional values. For example, the sub-pel unit may mean 1 / 2 pixel, 1 / 4 pixel, 1 / 8 pixel, 1 / 16 pixel, 1 / 32 pixel, etc. O and P may also have positive integer values ​​to represent the sub-pels.

[0361] When searching for a motion vector corrected in sub-pixel units, the search targets for the motion vector are the sub-pixels spatially adjacent to the integer pixel indicated by the motion vector in integer pixel units. In Figure 14, the sub-pixel units are shown as an example of half pixel units.

[0362] Furthermore, the sub-pixel search region can be restricted to reduce memory access bandwidth, for example, so that the sub-pixel search region is included in the integer pixel search region.

[0363] FIG. 15 is a diagram showing an embodiment in which a sub-pixel search region is restricted to be included in an integer pixel search region.

[0364] 15, the shaded subpixels indicate subpixels that are not included in the search region of the integer pixel unit, and the search for the corrected motion vector can be restricted to the subpixel unit. That is, by restricting the search for the corrected motion vector to only the subpixel unit not shaded, and by not further fetching integer pixels for generating the shaded subpixels from memory, the memory access bandwidth can be reduced.

[0365] Meanwhile, the search area of ​​integer units may include at least one of a center point, an upper point, a lower point, a left point, and a right point.

[0366] Here, at least one point may be further searched based on the distortion values ​​of the upper point, lower point, left point, and right point. Here, the at least one point may be at least one of the upper left point, lower left point, upper right point, and lower right point. In this case, a sub-pixel search region may be set using at least one of the center point, upper point, lower point, left point, right point, upper left point, lower left point, upper right point, and lower right point. For example, a sub-pixel search region may be set using the center point, upper point, lower support point, left support point, and right point.

[0367] Meanwhile, the shape of the search area may be a shape that can be expressed in two dimensions based on the pixel indicated by the initial motion vector, such as a square, a rectangle, a diamond / rhombus, a cross, etc. The shape of the search area may be a fixed shape predetermined by the encoder / decoder, or may be specified by information coded and signaled by the encoder.

[0368] In addition, the range of motion vector values ​​for at least one of the initial motion vector and the corrected motion vector can be limited so that the search area does not exceed the boundary of the image by more than a certain range. The restriction of the range of motion vector values ​​can be achieved by clipping.

[0369] In this case, the range of motion vector values ​​may be determined as a predetermined range by the encoder and decoder. Here, the predetermined range may refer to a threshold for limiting the motion vector values, and the threshold may be realized as at least one of a minimum value and a maximum value. Alternatively, the range of motion vector values ​​may be a variable range that is coded and signaled by the encoder.

[0370] On the other hand, it is possible to restrict the boundary of the image to at least one of the integer pixel search region and the sub-pixel search region so that the boundary does not exceed a certain range.

[0371] On the other hand, after the motion information correction in integer pixel units has been performed, the motion information correction in sub-pixel units can be permitted.

[0372] When using the bidirectional template matching described above, motion vectors can be improved in the decoder without transmitting additional syntax elements. Meanwhile, bidirectional template matching can be applied to bidirectional prediction merge mode or decoder motion vector guidance mode.

[0373] According to one embodiment, bidirectional template matching may be performed if the current block does not correspond to at least one of a unidirectional prediction merge candidate, a regional illumination compensation mode, an affine motion compensation mode, and a sub-CU merge mode.

[0374] A motion-refined mode related to motion information refinement will be described below. Here, the motion-refined mode may refer to a prediction mode for deriving motion information used in inter prediction. As an example, the motion-refined mode may be a type of merge mode based on motion information refinement using the above-mentioned bidirectional template matching.

[0375] The motion compensation mode allows efficient coding of areas with no or little motion within the image.

[0376] The motion compensation mode is a motion information derivation method based on the motion information compensation described above, and each of the above-described embodiments of motion information compensation can be applied to the motion compensation mode.

[0377] Meanwhile, whether a motion compensation mode is available or applied may be determined based on information signaled through a bitstream. The information may be signaled for at least one unit selected from the group consisting of a video, a sequence, a picture, a slice, a tile, a coding tree unit, a coding unit, a prediction unit, a transform unit, and a block. Whether a motion compensation mode is available or applied may indicate whether a motion compensation mode is available or applied to a corresponding unit.

[0378] In the motion compensation mode, 1) the step of deriving motion information and 2) the step of generating compensated motion information by applying motion information compensation to the induced motion information may be performed. Here, the motion information compensation may include the above-mentioned bidirectional template matching. The motion information compensated in the motion compensation mode may be used for motion compensation of the current block to be coded / decoded.

[0379] Meanwhile, the motion information targeted for the motion compensation mode may include at least one of a motion vector, a reference image, an inter-frame indicator, a reference image index, a prediction list utilization flag, a weight, and an offset.

[0380] 1) Induction of movement information In the motion compensation mode, motion information may be derived from at least one of motion information of spatially adjacent neighboring blocks, motion information of temporally adjacent neighboring blocks, predefined motion information, or motion information that is most frequently present in a reference image. Here, the derived motion information may be stored in a motion compensation mode list as a motion compensation candidate. Motion information derived from motion information of spatially adjacent neighboring blocks may be a spatial motion compensation candidate, and motion information derived from motion information of temporally adjacent neighboring blocks may be a temporal motion compensation candidate.

[0381] A method for deriving a motion vector from the motion information will now be described in detail.

[0382] In the motion compensation mode, the motion vector can be derived from at least one of the motion vectors of spatially adjacent surrounding blocks, the motion vectors of temporally adjacent surrounding blocks, predefined motion vectors, or the most common motion vector in the reference image.

[0383] A. Spatially adjacent peripheral blocks In the motion compensation mode, a motion vector can be derived from the motion vector of at least one of spatially adjacent neighboring blocks, which may include the neighboring blocks of FIG.

[0384] On the other hand, in the motion compensation mode, a motion vector can be derived from the motion vectors of spatially adjacent blocks that satisfy at least one of the following conditions:

[0385] - Blocks whose coding block flag is 0 (e.g., blocks where no residual signal exists)

[0386] -Blocks that fall under merge skip mode

[0387] A block whose motion vector difference value is 0 (e.g., MVD=(0,0)) among neighboring blocks spatially adjacent to the current block

[0388] -Blocks that correspond to merge mode

[0389] - Blocks with inter-frame prediction indicator PRED_LX (X is a positive integer including 0)

[0390] - Blocks whose inter-screen prediction indicator is PRED_BI, PRED_TRI, or PRED_QUAD

[0391] On the other hand, in the motion compensation mode, the motion vector can be derived from the motion vector of at least one of the spatial skip / merge candidates.

[0392] As an example, if the reference image of a spatial skip / merge candidate is not the same as the reference image of the block to be encoded / decoded (if the image sequence count values ​​are different from each other), the motion vector derived from the motion vector of the spatially adjacent surrounding block can be scaled and derived based on the image sequence count value of the image.

[0393] B. Temporally adjacent peripheral blocks In the motion compensation mode, a motion vector can be derived from the motion vector of at least one of the temporally adjacent neighboring blocks, where the temporally adjacent neighboring blocks can include a correspondingly positioned block in the reference image and a correspondingly positioned block in the correspondingly positioned image.

[0394] Here, the corresponding block in the reference image may be at least one of the following blocks:

[0395] - The block in the reference image at the corresponding position (0,0) which is the upper left position of the block to be coded / decoded

[0396] - Corresponding position block in the reference image (C3 in Figure 10)

[0397] - Corresponding position block in the reference image (C1 in Figure 10)

[0398] - Corresponding position block in the reference image (H in Figure 10)

[0399] Meanwhile, the corresponding position block in the corresponding position image may be at least one of the following blocks:

[0400] - The corresponding position block in the corresponding position image corresponding to (0,0), which is the upper left position of the block to be coded / decoded

[0401] - Corresponding position block in the corresponding position image (C3 in Figure 10)

[0402] - Corresponding position block in the corresponding position image (C1 in Figure 10)

[0403] - Corresponding position block in the corresponding position image (H in Figure 10)

[0404] On the other hand, in the motion compensation mode, a motion vector can be derived from the motion vector of a block that satisfies at least one of the following conditions among the temporally adjacent blocks.

[0405] - Blocks inside / outside the corresponding position block whose coding block flag is 0 (e.g., blocks without residual signals)

[0406] - Blocks that fall within / outside the corresponding position block and are in merge skip mode

[0407] - Blocks inside / outside the corresponding position block whose motion vector difference value is 0 (e.g., MVD=(0,0))

[0408] - Blocks that correspond to merge mode among blocks inside / outside the corresponding position block

[0409] Among the blocks inside / outside the corresponding position block, the block whose inter-frame prediction indicator is PRED_LX (X is a positive integer including 0)

[0410] Among the blocks inside / outside the corresponding position block, the block whose inter-screen prediction indicator is PRED_BI, PRED_TRI, or PRED_QUAD

[0411] On the other hand, in the motion compensation mode, the motion vector can be derived from the motion vector of at least one of the time skip / merge candidates.

[0412] As an example, if the reference image or corresponding position image of a time skip / merge candidate is not the same as the reference image of the block to be encoded / decoded (if the image sequence count values ​​are different from each other), the motion vector derived from the motion vector of the temporally adjacent surrounding block can be scaled and derived based on the image sequence count value of the image.

[0413] Meanwhile, the reference image index indicating the corresponding position image and the reference image including the temporally adjacent neighboring blocks can be derived from the reference image index of the spatially adjacent neighboring blocks.

[0414] C. Predefined motion vectors In the motion compensation mode, the (0,0) motion vector (zero vector) can be induced as the motion vector.

[0415] D. The most common motion vector in the reference image In the motion compensation mode, the most common motion vector in the reference image can be derived.

[0416] In this case, the motion vectors are sorted in descending order of frequency in the reference image, and a maximum of L motion vectors can be derived in the sorted order, where L may be a positive integer. In this case, L may be a fixed value predetermined by the encoder / decoder, or a variable value coded and signaled by the encoder.

[0417] Meanwhile, in the motion compensation mode, the motion vectors may be derived in the above-described order (A→B→C→D). In this case, the motion vectors may be derived until a maximum of M motion vectors are derived. Furthermore, if a derived motion vector exists after the motion vectors are derived in the above-described order, the motion vector deriving may be stopped and a maximum of M motion vectors may be used as motion vectors for the motion compensation mode. Here, M is a positive integer, and M may be a fixed value predetermined in the encoder / decoder or a variable value coded and signaled by the encoder. Here, when there are multiple spatially adjacent neighboring blocks, a bidirectionally predicted motion vector may be derived first and inserted into the motion compensation mode list, and then a unidirectionally predicted motion vector may be derived and inserted into the motion compensation mode list.

[0418] The motion vector derivation procedure in the above-mentioned motion compensation mode can be set as follows.

[0419] For example, the motion vectors can be derived in the order of the motion vector of a spatially adjacent neighboring block, the motion vector of a temporally adjacent neighboring block, and the (0,0) motion vector.

[0420] As another example, the motion vectors may be derived in the order of the motion vector of a temporally adjacent neighboring block, the motion vector of a spatially adjacent neighboring block, and the (0,0) motion vector.

[0421] Here, the motion vectors of spatially adjacent surrounding blocks can be induced in the order of induction of spatial skip / merge candidates, and the motion vectors of temporally adjacent surrounding blocks can be induced in the order of induction of temporal skip / merge candidates.

[0422] On the other hand, the motion compensation mode can be used only if the motion vector derived by the above method does not have the value (0,0).

[0423] As an example, when bidirectional prediction is used in motion compensation mode, the induced first and second motion vectors can be used as motion vectors for motion compensation mode if they do not all have the value (0,0).

[0424] When N-directional prediction (N is an integer greater than or equal to 2) is used in motion compensation mode, the derived N motion vectors can be used as motion vectors in motion compensation mode if they do not all have the value (0,0).

[0425] Alternatively, in N-directional prediction (N is an integer greater than or equal to 2), if some of the induced motion vectors do not have the value (0,0), they can be used as motion vectors in motion compensation mode.

[0426] On the other hand, when bidirectional prediction is used in motion compensation mode, if the induced first motion vector and second motion vector are not the same, either one or both of the induced first motion vector and second motion vector can be used as motion vectors in motion compensation mode.

[0427] Meanwhile, only when the inter prediction indicator of a spatially adjacent block or a temporally adjacent block is at least one of bidirectional prediction, tridirectional prediction, and quaddirectional prediction, can the induced motion vector be used as the motion vector for the motion compensation mode. That is, when the inter prediction indicator of a spatially adjacent block or a temporally adjacent block is not unidirectional prediction, the induced motion vector can be used as the motion vector for the motion compensation mode.

[0428] In addition, if the inter-frame prediction indicator of a spatially adjacent neighboring block or a temporally adjacent neighboring block is unidirectional prediction, the inter-frame prediction indicator is changed to bidirectional prediction, and the sign of the motion vector for the unidirectional prediction is reversed, thereby deriving a motion vector for unidirectional prediction opposite to the prediction direction corresponding to the unidirectional prediction and using it as a motion vector for a motion compensation mode.

[0429] For example, if the inter prediction indicator of a spatially adjacent block or a temporally adjacent block indicates L0 unidirectional prediction and the motion vector is (-4, 6), the inter prediction indicator is changed to bidirectional prediction, and the motion vector for L1 unidirectional prediction is induced to (4, -6), so that the two motion vectors can be used as motion vectors for the motion compensation mode. In this case, the reference picture for L1 unidirectional prediction can be determined to be the same as or proportional to the distance between the reference picture for L0 unidirectional prediction and the current picture.

[0430] On the other hand, in a motion compensation mode using N-directional prediction (N is an integer equal to or greater than 2), if the number of induced motion vectors is less than N, motion vectors based on already induced motion vectors can be generated to derive N motion vectors. Specifically, motion vectors can be generated by scaling already induced motion vectors based on the POC of the current image and / or reference image.

[0431] As an example, when bidirectional prediction is used in motion compensation mode and only an induced first motion vector exists, the first motion vector is scaled based on a reference image in reference image list 1 to generate a second motion vector, and the already induced first motion vector and the generated second motion vector can be used as motion vectors in motion compensation mode.

[0432] As another example, when bidirectional prediction is used in motion compensation mode and only the induced second motion vector exists, the second motion vector is scaled based on a reference image in reference image list 0 to generate a first motion vector, and the already induced second motion vector and the generated first motion vector can be used as motion vectors in motion compensation mode.

[0433] A method for deriving a reference image from motion information will be described in detail below.

[0434] In the motion compensation mode, the reference image can be derived using at least one of the following methods:

[0435] As an example, among the reference images included in reference image list 0 and reference image list 1, a reference image having a reference image index of 0 can be guided as a reference image for the motion compensation mode. If the reference images having a reference image index of 0 among the reference images included in reference image list 0 and reference image list 1 are the same, a reference image included in reference image list 1 that is not the same as the reference image having a reference image index of 0 in reference image list 0 can be guided as a reference image for the motion compensation mode. Furthermore, if the reference images having a reference image index of 0 among the reference images included in reference image list 0 and reference image list 1 are the same, a reference image included in reference image list 0 that is not the same as the reference image having a reference image index of 0 in reference image list 1 can be guided as a reference image for the motion compensation mode.

[0436] Meanwhile, as described above, the reference image index 0 can be used to derive the reference image in the motion compensation mode. However, the present invention is not limited to this, and the reference image in the motion compensation mode can be derived using a reference image index other than 0.

[0437] The method of deriving a reference picture in the motion compensation mode using a non-zero reference picture index is as follows.

[0438] For example, the reference picture index may be determined as the median value of reference picture indexes at at least one of spatially adjacent positions A1, B1, B0, A0, and B2 of the block to be coded / decoded. Here, various statistical values ​​such as minimum, maximum, average, weighted average, and mode may be used instead of the median. The reference picture indicated by the determined reference picture index may be guided as a reference picture for the motion compensation mode. In this case, the reference picture index may be determined for each reference picture list using the above method.

[0439] On the other hand, if a block exists only at position A1 and the block is in inter prediction mode, the reference image index can be determined by the reference image index at position A1.

[0440] Furthermore, if a block exists only at position B1 and the block is in inter prediction mode, the reference image can be determined by the reference image index at position B1.

[0441] Furthermore, if a block exists only at the B0 position and the block is in inter prediction mode, the reference image can be determined by the reference image index at the B0 position.

[0442] Furthermore, if a block exists only at the A0 position and the block is in inter prediction mode, the reference image can be determined by the reference image index at the A0 position.

[0443] Furthermore, if a block exists only at position B2 and the block is in inter prediction mode, the reference image can be determined by the reference image index at position B2.

[0444] A corrected reference image can be calculated by applying motion information correction to the reference image derived by the above-described method, and the corrected reference image can be used as a reference image in the motion compensation mode.

[0445] Meanwhile, in the above-described embodiment, two reference image lists are assumed. However, the number of reference image lists is not limited to this, and N reference image lists may exist. Here, N is an integer greater than or equal to 2. Even when N reference image lists exist, the same reference image guiding based on the reference image identity determination as described above may be performed. For example, N mutually different reference images may be induced.

[0446] In another embodiment, among the reference images included in the reference image list i (i = 0...N, N is an integer greater than or equal to 1), the reference image whose difference value of the image sequence count from the image to be encoded / decoded is the smallest can be derived as the reference image for the motion compensation mode.

[0447] Alternatively, among the reference pictures included in the reference picture list i (i = 0...N, N is an integer greater than or equal to 1), the reference picture having the smallest difference value of the picture sequence count from the picture to be encoded / decoded and / or the smallest temporal layer identifier value can be induced as the reference picture for the motion compensation mode.

[0448] As another embodiment, the reference images of spatially adjacent surrounding blocks can be derived as reference images for the motion compensation mode.

[0449] As another embodiment, the reference images of the temporally adjacent surrounding blocks can be derived as reference images for the motion compensation mode.

[0450] In another embodiment, at least one reference image among the skip / merge candidates can be guided as a reference image for the motion compensation mode.

[0451] In another embodiment, after the motion information compensation method has been performed on all or some of the reference images in the reference image list, the reference image showing the least distortion can be derived as the reference image for the motion compensation mode.

[0452] A method for deriving the inter prediction indicator from the motion information will be described in detail below.

[0453] In motion compensated mode, the inter prediction indicator can be derived using at least one of the following methods:

[0454] As an example, in the motion compensation mode, the inter prediction indicator can be fixed and used for any one of unidirectional prediction, bidirectional prediction, tridirectional prediction, quaternary prediction, and N-directional prediction.

[0455] As another example, the inter prediction indicator of a spatially adjacent neighboring block can be derived as the inter prediction indicator for the motion compensated mode.

[0456] As another example, the inter prediction indicator of a temporally adjacent neighboring block can be derived as the inter prediction indicator for the motion compensated mode.

[0457] As another example, the inter prediction indicator of at least one of the skip / merge candidates may be derived as an inter prediction indicator of a motion compensated mode.

[0458] As another example, an inter prediction indicator may be derived as an inter prediction indicator for a motion compensation mode based on an available reference image among the reference images derived by the above-described method. Specifically, if one reference image is available, the inter prediction indicator may be derived for unidirectional prediction. Alternatively, if two reference images are available, the inter prediction indicator may be derived for bidirectional prediction.

[0459] The motion information compensation method may be used to calculate a compensated inter prediction indicator from the induced inter prediction indicator, and the compensated inter prediction indicator may be used as the inter prediction indicator in the motion compensation mode.

[0460] 2) Correction of motion information The encoder / decoder may calculate corrected motion information by applying motion information correction to the motion information derived by the above-described method, and use the corrected motion information as motion information for a motion compensation mode. Specifically, motion compensation may be performed by generating a predicted block using at least one of a corrected motion vector, a corrected reference picture, and a corrected inter-frame prediction indicator. Similarly, a motion information correction method may be performed on at least one of information included in the motion information, such as the derived reference picture index, a prediction list utilization flag, a weight (weighting factor), and an offset, and motion compensation may be performed by generating a predicted block using at least one of the corrected reference picture index, the corrected prediction list utilization flag, the corrected weight, and the corrected offset. Here, the motion information correction method may include the bidirectional template matching. In this case, motion compensation may be performed by calculating a weighted sum based on the picture sequence count between the current picture to be coded / decoded and each reference picture to generate a final predicted block.

[0461] Meanwhile, when motion information correction is applied to the derived motion vector, the derived motion vector can be used to determine an initial motion search position, and a corrected motion vector can be determined based on the initial motion search position.

[0462] Meanwhile, the encoder / decoder may perform motion information compensation on up to M pieces of induced motion information. Here, M may be a fixed value predetermined by the encoder / decoder, or a variable value coded and signaled by the encoder. For example, at least one of the first motion information, the second motion information, ..., the Nth motion information may be compensated. In this case, N may be an integer equal to or greater than 4.

[0463] In addition, the induced motion information may be used as motion information for a motion compensation mode without applying the above-described motion information compensation to all or part of the induced motion information. Specifically, a prediction block may be generated using at least one of an induced motion vector, an induced reference image, and an induced inter-frame prediction indicator, and motion compensation may be performed.

[0464] On the other hand, in the motion compensation mode, motion information compensation (for example, bilateral template matching) can be selectively applied.

[0465] According to an embodiment, motion information compensation may be performed when the current block does not correspond to any of the unidirectional prediction merge candidate, the regional illumination compensation mode, the affine motion compensation mode, and the sub-CU merge mode.

[0466] Furthermore, the motion information correction is performed by calculating the difference value (POC) between the first reference image corresponding to the first prediction direction (for example, the L0 prediction direction) and the encoding / decoding target image order counter. ref0 -POC curr ), and the difference value (POC) between the second reference picture corresponding to the second prediction direction (for example, the L1 prediction direction) and the coding / decoding target picture order counter ref1 -POC curr ) has a negative integer value.

[0467] Furthermore, motion information correction can be performed when the first prediction direction and the second prediction direction are different from each other.

[0468] The encoder / decoder may perform bidirectional template matching on at least one of the motion information derived by the above-described method, generate a prediction block using one motion information with the minimum distortion, and perform motion compensation. In this case, the one motion information may include at least one of L0 motion information and L1 motion information.

[0469] In addition, the encoder / decoder can perform bidirectional template matching on at least one candidate in the skip candidate list or merge candidate list, and generate a prediction block using one candidate with the smallest distortion to perform motion compensation.

[0470] The encoder / decoder may also perform bidirectional template matching on at least one of the motion information derived by the above-described method, calculate a weighted sum of M prediction blocks generated using the M motion information with the minimum distortion, and use the resulting prediction block as a prediction block for the current block to be encoded / decoded, where M may be a positive integer greater than or equal to 2.

[0471] The encoder / decoder may also perform bidirectional template matching on at least one candidate in the skip candidate list or the merge candidate list, calculate a weighted sum of M predicted blocks generated using the M candidates with the minimum distortion, and use the resulting block as a predicted block for the current block to be encoded / decoded, where M may be a positive integer greater than or equal to 2.

[0472] Furthermore, the encoder / decoder entropy encodes / decodes skip indexes or merge indexes for unidirectional prediction candidates in the skip candidate list or merge candidate list, but for M or more bidirectional prediction candidates, bidirectional template matching can be used to generate a predicted block using at least one candidate with the minimum distortion, and then perform motion compensation. In this case, one flag or one index can be entropy encoded / decoded to indicate M or more bidirectional prediction candidates. That is, a skip index or merge index is assigned to each unidirectional prediction candidate, but one skip index or merge index can be assigned to M or more bidirectional prediction candidates. For M or more bidirectional prediction candidates, bidirectional template matching can be used to determine at least one candidate with the minimum distortion, so it is not necessary to assign a skip index or merge index to each bidirectional prediction candidate.

[0473] The use of the motion compensation mode can be determined based on motion compensation mode use information, which can be entropy coded / decoded using at least one of flag information and index information.

[0474] The motion compensation mode usage information may be encoded / decoded based on the value of the skip flag. The encoding / decoding time of the motion compensation mode usage information may be determined based on the encoding / decoding time of the skip flag. For example, the motion compensation mode usage information may be entropy encoded / decoded when the skip flag is 1 (when the skip mode is used). In this case, the motion compensation mode usage information may be entropy encoded / decoded after the skip flag.

[0475] Conversely, the skip flag may be coded / decoded based on the value of the motion compensation mode usage information. For example, if the motion compensation mode usage information is 1 (when the motion compensation mode is used), the skip flag may be entropy coded / decoded. In this case, the motion compensation mode usage information may be entropy coded / decoded before the skip flag.

[0476] Furthermore, the motion compensation mode usage information may be encoded / decoded based on the value of the merge flag. The time at which the motion compensation mode usage information is encoded / decoded may be determined based on the time at which the merge flag is encoded / decoded. For example, the motion compensation mode usage information may be entropy encoded / decoded when the merge flag is 1 (when the merge mode is used). In this case, the motion compensation mode usage information may be entropy encoded / decoded after the merge flag.

[0477] Conversely, the merge flag may be encoded / decoded based on the value of the motion compensation mode usage information. For example, if the motion compensation mode usage information is 1 (when the motion compensation mode is used), the merge flag may be entropy encoded / decoded. In this case, the motion compensation mode usage information may be entropy encoded / decoded before the merge flag.

[0478] The motion compensation mode usage information may be coded / decoded based on a particular motion compensation mode, for example, the motion compensation mode usage information may be entropy coded / decoded when an affine motion compensation mode is not used.

[0479] The decoder motion vector guidance mode flag may be coded / decoded based on the value of the motion compensation mode usage information. For example, if the motion compensation mode usage information is 0 (motion compensation mode is not used), the decoder motion vector guidance mode flag may be entropy coded / decoded.

[0480] Furthermore, the motion compensation mode usage information may be encoded / decoded based on the motion compensation mode usage information of one or more neighboring blocks of the current block. For example, the one or more neighboring blocks may include one or more spatially adjacent blocks and / or one or more temporally adjacent blocks. The one or more spatially adjacent blocks may include a block adjacent to the left and / or a block adjacent to the top.

[0481] Furthermore, if the motion compensation mode usage information is not signaled, the motion compensation mode usage information can be derived based on the motion compensation mode usage information of one or more neighboring blocks of the current block to be coded / decoded. Here, the one or more neighboring blocks may include one or more spatially adjacent blocks and / or one or more temporally adjacent blocks. The one or more spatially adjacent blocks may include a block adjacent to the left and / or a block adjacent to the top.

[0482] The motion compensation mode usage information can be entropy coded / decoded when at least one of the blocks spatially adjacent to the current block to be coded / decoded uses the skip mode.

[0483] Furthermore, the motion compensation mode use information can be entropy coded / decoded when at least one of the blocks spatially adjacent to the block to be coded / decoded uses the merge mode.

[0484] Furthermore, the motion compensation mode usage information can be entropy coded / decoded when at least one of the blocks spatially adjacent to the current block to be coded / decoded is in the inter-frame mode.

[0485] The motion compensation mode usage information can be entropy coded / decoded in bypass mode.

[0486] Also, when the motion compensation mode is used in the form of a skip mode, the residual signal may not be entropy coded / decoded.

[0487] Also, when a motion compensation mode is used in the form of a merge mode, the residual signal can be entropy coded / decoded.

[0488] Also, in the motion compensation mode, only a portion of the residual signal can be entropy coded / decoded and used, where the portion of the residual signal can be a DC quantization level (DC transform coefficient).

[0489] Also, when the motion compensation mode is used, other information other than the motion compensation mode usage information may not be entropy coded / decoded, where the other information may be at least one of information related to motion compensation.

[0490] The motion-refined mode relating to motion information correction has been described above. Below, motion correction candidates induced by the motion-refined mode will be described.

[0491] The motion compensation candidate may refer to motion information including at least one of a motion vector induced through a motion compensation mode, a reference image induced, and an inter-frame prediction indicator induced. The encoder / decoder may add the motion compensation candidate to a skip candidate list or a merge candidate list as a skip candidate or a merge candidate in a skip mode or a merge mode, respectively.

[0492] An embodiment in which motion compensation candidates are added to the skip / merge candidate list is as follows.

[0493] If a skip / merge candidate that is the same as a motion compensation candidate exists in the skip / merge candidate list, the motion compensation candidate may not be added to the skip / merge candidate list. Also, if a skip / merge candidate that is the same as a motion compensation candidate exists in the skip / merge candidate list, the skip / merge candidate may not be added to the skip / merge candidate list.

[0494] Motion compensation candidates can be guided ahead of spatial skip / merge candidates and added to the skip / merge candidate list.

[0495] On the other hand, a motion compensation candidate can be added to the skip / merge candidate list by being derived before a spatial skip / merge candidate derived at a specific location, where the specific location can be at least one of the A1, B1, B0, A0, and B2 locations in FIG.

[0496] Additionally, the motion compensation candidate may be added to the skip / merge candidate list prior to at least one of the time skip / merge candidate, the combined merge candidate, and the merge candidate having a predetermined motion information value.

[0497] The encoder / decoder may determine, as a motion compensation candidate, motion information including at least one of a motion vector compensated by motion information compensation in the motion compensation mode, a compensated reference image, and a compensated inter prediction indicator. As described above, the motion compensation candidate may be added to a skip candidate list or a merge candidate list as a skip candidate or a merge candidate in the skip mode or the merge mode, respectively.

[0498] Meanwhile, the motion compensation mode can be used instead of the skip mode, i.e., the motion compensation mode can be used to encode / decode an image instead of the skip mode. Also, the motion compensation mode can be used instead of the merge mode, i.e., the motion compensation mode can be used to encode / decode an image instead of the merge mode.

[0499] At least one of an overlapped block motion compensation mode, a regional illumination compensation mode, and a bidirectional optical flow mode can be applied to the final predicted block generated using the motion compensation mode.

[0500] In the motion compensation mode, a list of motion information candidates is not generated, and the motion information compensation method can be applied to only one or some of the motion information.

[0501] The conditions under which motion information refinement is performed in the motion-refined mode (MRM) will be described below.

[0502] Whether or not to perform motion information compensation can be determined based on the picture order count (POC) of the reference picture of the motion vector.

[0503] As an example, the motion information correction method can be performed when there are both motion vectors pointing to reference images with image order counts smaller than the image to be encoded / decoded based on the image order count, and motion vectors pointing to reference images with image order counts larger than the image to be encoded / decoded based on the image order count.

[0504] Furthermore, the motion information compensation method can be performed when there are two motion vectors pointing to reference pictures having a picture order count smaller than that of the current picture to be coded / decoded based on the picture order count.

[0505] The motion information compensation method can also be performed when there are two motion vectors pointing to reference pictures with picture order counts greater than the current picture to be coded / decoded based on the picture order count.

[0506] Motion information compensation may be performed on a block to be coded / decoded when at least one of the affine motion compensation mode, the decoder motion vector guidance mode, and the regional illumination compensation mode is not used.

[0507] Motion information compensation may be performed when the difference between the image order count of the current image to be coded / decoded and the image order count of the first reference image, or the difference between the image order count of the current image to be coded / decoded and the image order count of the second reference image, is smaller than N (N is an integer equal to or greater than 0). In this case, the first reference image may refer to the reference image indicated by the first motion vector, and the second reference image may refer to the reference image indicated by the second motion vector.

[0508] Whether or not to perform motion information correction can be determined based on the first and second motion vectors that are targets of motion information correction.

[0509] For example, if the first motion vector is the same as the second motion vector and the reference image indicated by the first motion vector is the same as the reference image indicated by the second motion vector, motion information correction may not be performed. That is, motion information correction may be performed only when the first motion information and the second motion information are different from each other and / or when the reference images indicated by each of the first and second motion information are different. Also, if the first motion vector is the same as the second motion vector, motion information correction may not be performed. That is, motion information correction may be performed only when the first motion vector and the second motion vector are different from each other.

[0510] As another example, the motion information correction method may be performed only when the reference image indicated by the first motion vector and the reference image indicated by the second motion vector are the same, or conversely, the motion information correction may be performed only when the reference image indicated by the first motion vector and the reference image indicated by the second motion vector are different from each other.

[0511] As another example, motion information correction may be performed on motion vectors whose motion vector values ​​are not (0,0). Here, if the inter prediction indicator is bidirectional and the first motion vector and the second motion vector do not all have the value (0,0), motion information correction may be performed. Also, if the inter prediction indicator is N-directional (N is an integer equal to or greater than 2), motion information correction may be performed if all or a predetermined number or more of the N motion vectors do not have the value (0,0).

[0512] The motion information compensation may be performed only if the inter prediction indicator is directional. For example, the motion information compensation may be performed only if the inter prediction indicator of the skip / merge candidate is bidirectional.

[0513] The motion information correction may be performed only when the reference image index of the reference image indicated by the first motion vector is 0 and the reference image index of the reference image indicated by the second motion vector is 0. Alternatively, the motion information correction may be performed only when the reference image index of the reference image indicated by each motion vector is a specific value.

[0514] The motion information correction may be performed only on at least one of the spatial skip / merge candidates, the temporal skip / merge candidates, the combined skip / merge candidates, and the skip / merge candidates having a predetermined motion information value.

[0515] The motion information correction may be performed in units of sub-blocks after dividing the current block to be coded / decoded into units of sub-blocks.

[0516] On the other hand, if the block to be coded / decoded has motion information or motion vectors in sub-block units, motion information correction may be performed in sub-block units to improve coding efficiency. In this case, the sub-blocks of the block to be coded / decoded may have different motion information or motion vectors, or may all have the same motion information or motion vector. Here, motion information correction may be performed only when the sub-blocks of the block to be coded / decoded have different motion information or motion vectors.

[0517] Furthermore, when a block to be coded / decoded has motion information or motion vectors in units of subblocks, motion information correction may not be performed on the motion information or motion vectors in units of subblocks in order to reduce computational complexity. Alternatively, motion information correction may be performed only when all subblocks of the block to be coded / decoded have the same motion information or motion vectors in order to reduce computational complexity.

[0518] Meanwhile, the motion information compensation method can be performed in at least one unit of a sample unit and a block unit.

[0519] When the block to be coded / decoded is N-directionally predicted, such as three-directionally predicted or four-directionally predicted, a template may be calculated using N motion vectors, such as three motion vectors or four motion vectors, to perform motion information correction, where N may be a positive integer equal to or greater than 3.

[0520] When bilateral template matching is used for motion information correction, new motion information can be generated by scaling already derived motion information.

[0521] As an example, if there is only one first motion information or one first motion vector in a spatially adjacent neighboring block or a temporally adjacent neighboring block, motion information correction can be performed by scaling based on the first motion information or the first motion vector to generate second motion information or the second motion vector.

[0522] Similarly, if there is only one second motion information or one second motion vector in a spatially adjacent neighboring block or a temporally adjacent neighboring block, motion information correction can be performed by scaling based on the second motion information or the second motion vector to generate the first motion information or the first motion vector.

[0523] As another example, when a reference image indicated by an initial motion vector and a reference image indicated by a corrected motion vector candidate are different from each other, a corrected motion vector candidate can be calculated by scaling the initial motion vector based on the image sequence count value of each reference image. Here, the corrected motion vector candidate refers to a motion vector indicating an area in which the corrected motion vector is searched, and bilateral template matching can be performed by comparing distortion values ​​at positions indicated by other corrected motion vector candidates and the initial motion vector.

[0524] Meanwhile, a corrected motion vector can be calculated by using motion information correction for at least one of one luminance component motion vector and two chrominance component motion vectors.

[0525] Meanwhile, after performing motion information correction on all or some of the reference pictures in the reference picture list, the reference picture showing the smallest distortion can be selected as the corrected reference picture.

[0526] The above-described conditions for performing motion information compensation can also be applied to motion information compensation in a decoder-side motion vector derivation mode.

[0527] FIG. 16 is a flowchart illustrating an image decoding method according to an embodiment of the present invention.

[0528] Referring to Figure 16, the decoder can derive motion compensation candidates from at least one of spatial surrounding block motion information, temporal surrounding block motion information, predefined motion information, and the most commonly present motion information in the reference image (S1601).

[0529] In this case, motion compensation candidates may be derived from at least one of motion information of spatial neighboring blocks, motion information of temporal neighboring blocks, predefined motion information, and motion information that is most frequently present in the reference image according to a preset order. Here, the preset order may be the order of motion information of spatial neighboring blocks, motion information of temporal neighboring blocks, and predefined motion information. Here, the predefined motion information may include a zero vector.

[0530] On the other hand, the temporally surrounding blocks may be included in the reference picture selected based on the reference picture index of the spatially surrounding blocks.

[0531] Next, motion information correction can be performed on the guided motion correction candidates (S1602).

[0532] In this case, the motion information correction can be corrected by applying bilateral template matching to the motion vectors included in the guided motion correction candidates.

[0533] Here, the bilateral template matching may include a step of generating a bilateral template using a motion vector included in the derived motion correction candidate as an initial motion vector, and a step of correcting the initial motion vector by comparing samples in the bilateral template with reconstructed samples in a reference image indicated by the initial motion vector.

[0534] Here, the step of correcting the initial motion vector may be performed recursively.

[0535] In addition, the initial motion vector may be a bidirectionally predicted motion vector that is not a zero vector among the guided motion compensation candidates. Here, if there is no bidirectionally predicted motion vector that is not a zero vector among the guided motion compensation candidates, the initial motion vector may be set to a zero vector.

[0536] On the other hand, the step of correcting the initial motion vector may include the steps of searching for a motion vector that points to an area in both the template and the reference image that exhibits the least distortion, and setting the searched motion vector as the correction value of the initial motion vector.

[0537] Here, the step of searching for a motion vector can search within a limited search area in the reference image.

[0538] In this case, the limited search area can be set to a predetermined range in integer pixel units, and the step of searching for a motion vector can search for a motion vector in sub-pixel units within the predetermined range in integer pixel units.

[0539] On the other hand, the bidirectional template matching may be performed in integer pixel units and sub-pixel units, in which case the step of searching for a motion vector may search for a motion vector in sub-pixel units within a predetermined range of integer pixel units.

[0540] Meanwhile, the step of performing motion information compensation on the guided motion compensation candidate may be performed when the block does not correspond to a unidirectional prediction merge candidate, a regional illumination compensation mode, or an affine motion compensation mode.

[0541] Next, a predicted block of the current block can be generated using the motion compensation candidate for which motion information compensation has been performed (S1603).

[0542] The image decoding method may further include, before step S1601, decoding motion compensation mode usage information and determining a motion compensation mode based on the decoded motion compensation mode usage information. Based on the determination result, step S1601 may be performed only if the current block is in the motion compensation mode.

[0543] Here, whether to perform the step of decoding the motion compensation mode usage information can be determined based on the skip flag or the merge flag.

[0544] On the other hand, if there are multiple spatial surrounding blocks, the step of deriving motion compensation candidates can first derive motion information from spatial surrounding blocks having bidirectional predictive motion vectors, and then derive motion information from spatial surrounding blocks having unidirectional predictive motion vectors.

[0545] The image decoding method according to the present invention has been described above. Each step of the image decoding method described above can be similarly performed in an image coding method.

[0546] These embodiments may be implemented in a similar manner in the encoder and decoder.

[0547] The order of applying the above embodiments may be different between the encoder and the decoder, or may be the same between the encoder and the decoder.

[0548] The above embodiment can be performed for each of the luminance and color difference signals, and the above embodiment can be performed for each of the luminance and color difference signals in the same manner.

[0549] The shape of the blocks to which the embodiments of the present invention are applied can have a square or non-square shape.

[0550] The embodiments of the present invention may be applied depending on the size of at least one of a coding block, a prediction block, a transform block, a block, a current block, a coding unit, a prediction unit, a transform unit, a unit, and a current unit. The size may be defined as a minimum size and / or a maximum size for applying the embodiments, or as a fixed size for applying the embodiments. The first embodiment may be applied to a first size, and the second embodiment may be applied to a second size. That is, the embodiments may be applied in combination depending on the size. The embodiments of the present invention may be applied only when the block size is equal to or greater than the minimum size and equal to or less than the maximum size. That is, the embodiments may be applied only when the block size is within a certain range.

[0551] For example, the above embodiment is applicable only when the size of the current block is 8x8 or larger. For example, the above embodiment is applicable only when the size of the current block is 4x4. For example, the above embodiment is applicable only when the size of the current block is 16x16 or smaller. For example, the above embodiment is applicable only when the size of the current block is 16x16 or larger and 64x64 or smaller.

[0552] An embodiment of the present invention may be applied according to a temporal layer. A separate identifier may be signaled to identify a temporal layer to which the embodiment is applicable, and the embodiment may be applied to the temporal layer identified by the identifier. The identifier may be defined as the lowest and / or highest layer to which the embodiment is applicable, or may be defined to indicate a specific layer to which the embodiment is applied. A fixed temporal layer to which the embodiment is applied may also be defined.

[0553] For example, the above embodiment is applicable only when the temporal layer of the current image is the lowest layer, for example, the above embodiment is applicable only when the temporal layer identifier of the current image is 1 or greater, or for example, the above embodiment is applicable only when the temporal layer of the current image is the highest layer.

[0554] Slice types to which the embodiments of the present invention are applied are defined, and the embodiments of the present invention can be applied according to the slice types.

[0555] The above-described embodiments of the present invention can also be applied when the motion vector has at least one of the following units: 16-pel, 8-pel, 4-pel, integer-pel, 1 / 2-pel, 1 / 4-pel, 1 / 8-pel, 1 / 16-pel, 1 / 32-pel, and 1 / 64-pel. Also, the motion vector can be selectively used for each pixel unit.

[0556] In the above-described embodiments, the methods are described as a series of steps or units and based on flowcharts, but the present invention is not limited to the order of these steps, and some steps may occur in a different order or simultaneously with other steps than those described above. Furthermore, those skilled in the art will understand that the steps shown in the flowcharts are not exclusive, and other steps may be included, or one or more steps in the flowcharts may be deleted without affecting the scope of the present invention.

[0557] The above-described embodiments include examples of various aspects. It is not possible to describe all possible combinations for illustrating the various aspects, but a person skilled in the art will recognize that other combinations are possible. Therefore, it can be said that the present invention includes all various alterations, modifications, and variations that fall within the scope of the following claims.

[0558] The above-described embodiments of the present invention may be embodied in the form of program instructions that can be executed by various computer components and stored on a computer-readable storage medium. The computer-readable storage medium may include, alone or in combination, program instructions, data files, data structures, and the like. The program instructions stored on the computer-readable storage medium may be those specially designed and constructed for the present invention, or those known and available to those skilled in the computer software arts. Examples of computer-readable storage media include magnetic media such as hard disks, floppy disks, and magnetic tape; optical media such as CD-ROMs and DVDs; magneto-optical media such as floptical disks; and hardware devices specially configured to store and execute program instructions, such as ROM, RAM, and flash memory. Examples of program instructions include not only machine language code produced by a compiler, but also high-level language code that can be executed by a computer using an interpreter, etc. The hardware devices may be configured to operate as one or more software modules to perform the processes of the present invention, or vice versa.

[0559] Although the present invention has been described above using specific details such as specific components, limited embodiments, and drawings, these are provided merely to facilitate a more comprehensive understanding of the present invention, and the present invention is not limited to the above embodiments. Those skilled in the art will be able to make various modifications and variations from these descriptions.

[0560] Therefore, the concept of the present invention should not be limited to the above-described embodiments, but should be considered to fall within the scope of the concept of the present invention, including not only the scope of the claims described below, but also all modifications equivalent to or equivalent to the scope of these claims. [Industrial Applicability]

[0561] The present invention can be used in devices for encoding / decoding images.

Claims

1. An image decoding method, comprising: creating a merge candidate list for the current block, the merge candidate list including at least one of spatial merge candidates derived based on motion information of spatial surrounding blocks, temporal merge candidates derived based on motion information of temporal surrounding blocks, and additional merge candidates with zero motion vectors; determining a merge candidate from the list of merge candidates; deriving a first initial motion vector and a second initial motion vector for the current block from the merging candidate; obtaining a first corrected motion vector and a second corrected motion vector by performing motion information correction on the first initial motion vector and the second initial motion vector derived from the merge candidate; generating a prediction block of the current block using the first corrected motion vector and the second corrected motion vector; Including, When the current block is not in a unidirectional prediction mode and the direction of the L0 reference picture is different from the direction of the L1 reference picture, the motion information compensation is performed; whether to correct the first initial motion vector and the second initial motion vector is determined by comparing a sum of differences (SAD) between a first prediction block specified by the first initial motion vector and a second prediction block specified by the second initial motion vector with a threshold; if the SAD between the first prediction block and the second prediction block is less than the threshold, the first initial motion vector and the second initial motion vector are not corrected; Compensating the first initial motion vector includes: determining a best integer position from among a plurality of integer positions within a search area; determining optimal fractional positions around the optimal integer positions; Including, Even if a fractional position is adjacent to an integer position within the search area, if the fractional position is located outside the search area, the fractional position is not determined as the optimal fractional position. Image decoding method.

2. 1. An image encoding method, comprising: creating a merge candidate list for the current block, the merge candidate list including at least one of spatial merge candidates derived based on motion information of spatial surrounding blocks, temporal merge candidates derived based on motion information of temporal surrounding blocks, and additional merge candidates with zero motion vectors; determining a merge candidate from the list of merge candidates; deriving a first initial motion vector and a second initial motion vector for the current block from the merging candidate; obtaining a first corrected motion vector and a second corrected motion vector by performing motion information correction on the first initial motion vector and the second initial motion vector derived from the merge candidate; generating a prediction block of the current block using the first corrected motion vector and the second corrected motion vector; Including, When the current block is not in a unidirectional prediction mode and a first prediction direction and a second prediction direction of inter prediction are different from each other, the motion information compensation is performed; whether to correct the first initial motion vector and the second initial motion vector is determined by comparing a sum of differences (SAD) between a first prediction block specified by the first initial motion vector and a second prediction block specified by the second initial motion vector with a threshold; if the SAD between the first prediction block and the second prediction block is less than the threshold, the first initial motion vector and the second initial motion vector are not corrected; The correction of the first initial motion vector includes: determining an optimal integer position from among a plurality of integer positions within a search area; determining optimal fractional positions around the optimal integer positions; Including, Even if a fractional position is adjacent to an integer position within the search area, if the fractional position is located outside the search area, the fractional position is not determined as the optimal fractional position. Image encoding method.

3. 1. A non-transitory storage medium comprising instructions for transmitting a bitstream generated by encoding an image, the non-transitory storage medium comprising: The instruction: encoding the image to generate a bitstream; transmitting the bitstream; Including, Encoding the image to generate the bitstream includes: creating a merge candidate list for the current block, the merge candidate list including at least one of spatial merge candidates derived based on motion information of spatial surrounding blocks, temporal merge candidates derived based on motion information of temporal surrounding blocks, and additional merge candidates with zero motion vectors; determining a merge candidate from the list of merge candidates; deriving a first initial motion vector and a second initial motion vector for the current block from the merging candidate; obtaining a first corrected motion vector and a second corrected motion vector by performing motion information correction on the first initial motion vector and the second initial motion vector derived from the merge candidate; generating a prediction block of the current block using the first corrected motion vector and the second corrected motion vector; Including, When the current block is not in a unidirectional prediction mode and a first prediction direction and a second prediction direction of inter prediction are different from each other, the motion information compensation is performed; whether to correct the first initial motion vector and the second initial motion vector is determined by comparing a sum of differences (SAD) between a first prediction block specified by the first initial motion vector and a second prediction block specified by the second initial motion vector with a threshold; if the SAD between the first prediction block and the second prediction block is less than the threshold, the first initial motion vector and the second initial motion vector are not corrected; The correction of the first initial motion vector includes: determining an optimal integer position from among a plurality of integer positions within a search area; determining optimal fractional positions around the optimal integer positions; Including, Even if a fractional position is adjacent to an integer position within the search area, if the fractional position is located outside the search area, the fractional position is not determined as the optimal fractional position. storage medium.

4. 2. The method of claim 1, wherein a first motion vector differential representing a difference between the first corrective motion vector and the first initial motion vector is composed of an integer component corresponding to the optimal integer position and a fractional component corresponding to the optimal fractional position.

Images