Image decoding method and image encoding method

Abstract

An image decoding method and an image encoding method are provided that correct motion vectors to improve image encoding / decoding efficiency. [Solution] An image decoding method includes the steps of: deriving an initial motion vector of a first prediction direction and an initial motion vector of a second prediction direction from a merge candidate list of a current block; deriving a search area of ​​the first prediction direction and a search area of ​​the second prediction direction in sub-block units of the current block based on the initial motion vector of the first prediction direction and the initial motion vector of the second prediction direction, respectively; and deriving a corrected motion vector of the first prediction direction and a corrected motion vector of the second prediction direction in sub-block units of the current block based on the search area of ​​the first prediction direction and the search area of ​​the second prediction direction, wherein the first prediction direction is different from the second prediction direction.

Description

[Technical field]

[0001] The present invention relates to an image encoding / decoding method and device, and a recording medium storing a bitstream. More specifically, the present invention relates to an image encoding / decoding method and device using motion vector correction. [Background technology]

[0002] Recently, the demand for high-resolution, high-quality images, for example, HD (High Definition) images and UHD (Ultra High Definition) images, is increasing in various application fields. As the resolution and quality of image data increases, the amount of data increases relatively compared to conventional image data. Therefore, when the image data is transmitted using conventional media such as wired or wireless broadband lines or stored using conventional storage media, the transmission and storage costs increase. In order to solve the problems that arise with the increase in the resolution and quality of image data, a highly efficient image encoding / decoding technology for images with higher resolution and quality is required.

[0003] There are various image compression techniques, such as inter prediction techniques that predict pixel values ​​contained in a current picture from pictures before or after the current picture, intra prediction techniques that predict pixel values ​​contained in a current picture using pixel information in the current picture, transformation and quantization techniques for compressing the energy of residual signals, and entropy coding techniques that assign short codes to values ​​that occur frequently and long codes to values ​​that occur less frequently. By using these image compression techniques, image data can be effectively compressed and transmitted or stored. Summary of the Invention [Problem to be solved by the invention]

[0004] The present invention can provide an image decoding / encoding method and device that corrects motion vectors to improve image encoding / decoding efficiency. [Means for solving the problem]

[0005] The image decoding method according to the present invention may include a step of deriving an initial motion vector from a merge candidate list of a current block, a step of deriving a corrected motion vector using the initial motion vector, and a step of generating a prediction block of the current block using the corrected motion vector.

[0006] In the image decoding method, the step of deriving the initial motion vector can derive the initial motion vector from a merge candidate indicated by merge index information.

[0007] In the image decoding method, when the current block is a block capable of bi-prediction and only motion information of one prediction direction exists in the merge candidate indicated by the merge index information, the step of deriving the initial motion vector may involve mirroring or scaling the motion vector of the existing motion information to derive an initial motion vector for the non-existent prediction direction.

[0008] In the image decoding method, the step of deriving the corrected motion vector may include searching for a motion vector indicating a template that minimizes a difference between the current block and surrounding templates within a search area of ​​a reference picture indicated by the initial motion vector, and deriving the motion vector indicating the template that minimizes the difference as the corrected motion vector.

[0009] In the image decoding method, the predetermined search area may be determined to be an area having at least one of a cross, a square, a diamond, and a hexagon shape centered on a pixel position of a reference picture indicated by the initial motion vector.

[0010] In the image decoding method, a motion vector indicating a template that minimizes a difference between neighboring templates of a current block within a search area of ​​a reference picture indicated by the initial motion vector can be searched based on at least one of a distortion value between templates and a cost value for the motion vector.

[0011] In the image decoding method, the step of deriving the corrected motion vector can derive the corrected motion vector based on at least one predictive block indicated by the initial motion vector.

[0012] In the image decoding method, the step of deriving the corrected motion vector may include, when the current block is a bidirectional prediction block, setting a weighted sum of a first prediction block indicated by a first prediction direction initial motion vector and a second prediction block indicated by a second prediction direction initial motion vector as a template, setting a search area of ​​at least one reference picture indicated by at least one of the first prediction direction initial motion vector and the second prediction direction initial motion vector, and deriving a motion vector indicating a third prediction block that minimizes a difference from the template in the search area as the corrected motion vector. Here, when a first search area indicated by the first prediction direction initial motion vector and a second search area indicated by the second prediction direction initial motion vector are set, the corrected motion vector may be derived using motion vectors indicating respective prediction blocks that minimize a difference from the template in the first search area and the second search area.

[0013] In the image decoding method, if either the first prediction direction initial motion vector or the second prediction direction initial motion vector does not exist, the non-existent initial motion vector can be derived by mirroring or scaling an existing initial motion vector.

[0014] In the image decoding method, the current block may be a sub-block of a block to be decoded.

[0015] The image encoding method according to the present invention may include a step of deriving an initial motion vector from a merge candidate list of a current block, a step of deriving a corrected motion vector using the initial motion vector, and a step of generating a prediction block of the current block using the corrected motion vector.

[0016] In the image coding method, the step of deriving the initial motion vector can derive the initial motion vector from a merging candidate indicated by merge index information.

[0017] In the image encoding method, the step of deriving the initial motion vector may, when the current block is a block capable of bi-prediction and only motion information of one prediction direction exists in the merge candidate indicated by the merge index information, derive an initial motion vector for the non-existent prediction direction by mirroring or scaling the motion vector of the existing motion information.

[0018] In the image encoding method, the step of deriving the corrected motion vector may include searching for a motion vector indicating a template that minimizes a difference between the current block and a surrounding template within a search area of ​​a reference picture indicated by the initial motion vector, and deriving the motion vector indicating the template that minimizes the difference as the corrected motion vector.

[0019] In the image coding method, the predetermined search area may be determined to be an area having at least one of a cross, a square, a diamond, and a hexagon shape centered on a pixel position of the reference picture indicated by the initial motion vector.

[0020] In the image encoding method, a motion vector indicating a template that minimizes a difference between neighboring templates of a current block within a search area of ​​a reference picture indicated by the initial motion vector can be searched based on at least one of a distortion value between the templates and a cost value for the motion vector.

[0021] In the image coding method, the step of deriving the corrected motion vector can derive the corrected motion vector based on at least one predictive block indicated by the initial motion vector.

[0022] In the image encoding method, the step of deriving the corrected motion vector may include, when the current block is a bidirectional predictive block, setting a weighted sum of a first predictive block indicated by a first predictive direction initial motion vector and a second predictive block indicated by a second predictive direction initial motion vector as a template, setting a search area of ​​a reference picture indicated by the first predictive direction initial motion vector or the second predictive direction initial motion vector, and deriving a motion vector indicating a third predictive block that minimizes a difference from the template within the search area as a corrected motion vector.

[0023] In the image encoding method, if either the first prediction direction initial motion vector or the second prediction direction initial motion vector does not exist, the non-existent initial motion vector can be derived by mirroring or scaling an existing initial motion vector.

[0024] In the image coding method, the current block may be a sub-block of a block to be decoded.

[0025] The recording medium according to the present invention can store a bitstream generated by an image decoding method including the steps of: deriving an initial motion vector from a merge candidate list of a current block; deriving a corrected motion vector using the initial motion vector; and generating a prediction block of the current block using the corrected motion vector. Effect of the Invention

[0026] According to the present invention, it is possible to provide an image encoding / decoding method and device with improved compression efficiency.

[0027] According to the present invention, it is possible to improve the efficiency of image encoding and decoding.

[0028] According to the invention, the computational complexity of image encoders and decoders can be reduced. [Brief description of the drawings]

[0029] [Figure 1] 1 is a block diagram showing a configuration of an embodiment of an encoding device to which the present invention is applied. [Diagram 2] 1 is a block diagram showing a configuration of an embodiment of a decoding device to which the present invention is applied. [Diagram 3] FIG. 2 is a schematic diagram showing an image division structure when encoding and decoding an image. [Figure 4] FIG. 1 is a diagram for explaining an embodiment of an inter-prediction process. [Diagram 5] 2 is a flowchart illustrating an image encoding method according to an embodiment of the present invention. [Figure 6] 2 is a flowchart illustrating an image decoding method according to an embodiment of the present invention. [Figure 7] 13A and 13B are diagrams illustrating an example of deriving spatial merge candidates and temporal merge candidates for a current block. [Figure 8] FIG. 13 illustrates spatial and temporal neighboring blocks for constructing an additional merge candidate list. [Figure 9] FIG. 13 is a diagram for explaining the configuration of an initial motion vector list. [Figure 10] FIG. 13 is a diagram for explaining the configuration of an initial motion vector list. [Figure 11] FIG. 11 is a diagram for explaining a method of deriving an initial motion vector using merge index information. [Figure 12] FIG. 13 is a diagram for explaining motion vector mirroring. [Figure 13] FIG. 13 is a diagram for explaining motion vector scaling. [Figure 14] 13 is a diagram illustrating an example of deriving an additional motion vector when there is only one motion vector corresponding to the L0 or L1 list in a bi-predictable block. FIG. [Figure 15] 11 is a diagram illustrating an example of selecting a merge candidate having bidirectional motion information as an initial motion vector in a bi-predictable block. FIG. [Figure 16] FIG. 2 is a diagram for explaining a template according to an embodiment of the present invention. [Figure 17] FIG. 2 is a diagram for explaining a template according to an embodiment of the present invention. [Figure 18] FIG. 2 is a diagram for explaining a template according to an embodiment of the present invention. [Figure 19] FIG. 2 is a diagram for explaining a template according to an embodiment of the present invention. [Figure 20] FIG. 2 is a diagram for explaining a template according to an embodiment of the present invention. [Figure 21] FIG. 2 is a diagram for explaining a template according to an embodiment of the present invention. [Figure 22] FIG. 13 is a diagram for explaining an example of correcting an initial motion vector using a template. [Diagram 23] 11 is a diagram illustrating an example of correcting an initial motion vector using a prediction block of a current block. [Figure 24] 11 is a diagram illustrating an example of correcting an initial motion vector using a prediction block of a current block. [Diagram 25] 11 is a diagram illustrating an example of correcting an initial motion vector using a prediction block of a current block. [Figure 26] FIG. 2 is a diagram for explaining a sub-block; [Figure 27] FIG. 2 is a diagram for explaining a sub-block; [Figure 28] 11 is a diagram for explaining an example of setting an initial motion vector for motion vector correction in sub-block units; FIG. [Figure 29] 1 is a flowchart illustrating an image decoding method according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0030] Since the present invention can be modified in various ways and can have various embodiments, a specific embodiment is illustrated in the drawings and described in detail in the detailed description. However, this does not limit the present invention to a specific embodiment, and it should be understood that it includes all modifications, equivalents, or alternatives within the spirit and technical scope of the present invention. In the drawings, similar reference symbols indicate the same or similar functions in various aspects. The shape and size of elements in the drawings may be exaggerated for more clear explanation. The detailed description of the exemplary embodiments described below refers to the attached drawings showing specific embodiments as examples. These embodiments are described in sufficient detail to enable those skilled in the art to implement the embodiments. It should be understood that the various embodiments are different from each other, but are not necessarily mutually exclusive. For example, the specific shapes, structures, and characteristics described herein can be realized in various embodiments without departing from the spirit and scope of the present invention in relation to one embodiment. It should also be understood that the position or arrangement of individual components in each disclosed embodiment can be changed without departing from the spirit and scope of the embodiment. Accordingly, the following detailed description is not to be taken in a limiting sense, and the scope of the example embodiments is limited only by the appended claims, if properly recited, and to the full extent of equivalents to which those claims are entitled.

[0031] In the present invention, the terms "first", "second", etc. may be used to describe various components, but these components should not be limited by the above terms. These terms are used only to distinguish one component from another. For example, a first component can be named a second component, and similarly, a second component can be named a first component, without departing from the scope of the present invention. The term "and / or" includes any combination of two or more related listed items or two or more related listed items.

[0032] When a component of the present invention is described as being "coupled" or "connected" to another component, it should be understood that the component may be directly coupled or connected to the other component, but there may be other components between them. In contrast, when a component is described as being "directly coupled" or "directly connected" to another component, it should be understood that there are no other components between them.

[0033] The components shown in the embodiments of the present invention are illustrated independently to show different characteristic functions, and do not mean that each component is composed of separate hardware or a single software unit. That is, each component is included in each component for convenience of explanation, and at least two of the components can be combined to form a single component, or each component can be divided into multiple components to perform a function, and such integrated and separated embodiments of each component are also included in the scope of the present invention as long as they do not deviate from the essence of the present invention.

[0034] The terms used in the present invention are merely used to describe certain embodiments and are not intended to limit the present invention. A singular expression includes a plural expression unless the context clearly indicates otherwise. In the present invention, the terms "include" or "have" specify the presence of features, numbers, steps, operations, components, parts, or combinations thereof described in the specification, and should be understood not to preclude the presence or additional possibility of one or more other features, numbers, steps, operations, components, parts, or combinations thereof. In other words, in the present invention, the content described as "including" a specific configuration does not exclude configurations other than the corresponding configuration, and means that additional configurations may be included in the implementation of the present invention or the technical idea of ​​the present invention.

[0035] Some components of the present invention may not be essential components performing essential functions in the present invention, but may be optional components merely for improving performance. The present invention can be realized by including only components essential for realizing the essence of the present invention, excluding components merely used for improving performance, and a structure including only essential components, excluding optional components merely used for improving performance, is also included in the scope of the present invention.

[0036] Hereinafter, the embodiments of the present invention will be described in detail with reference to the drawings. In describing the embodiments of the present specification, if it is determined that a detailed description of related known configurations or functions may obscure the gist of the specification, the detailed description will be omitted, and the same reference numerals will be used for the same components in the drawings, and duplicate descriptions of the same components will be omitted.

[0037] In the following, an image may refer to one picture constituting a video, or may refer to the video itself. For example, "encoding and / or decoding an image" may mean "encoding and / or decoding a video" or "encoding and / or decoding one of the images constituting a video." Here, a picture may have the same meaning as an image.

[0038] <Terminology> Encoder: A device that performs encoding.

[0039] Decoder: A device that performs decoding.

[0040] Block: An M×N array of samples. Here, M and N are positive integers, and a block may generally refer to a two-dimensional sample array. A block may refer to a unit. A current block may refer to a block to be coded that is a target for coding during coding, or a block to be decoded that is a target for decoding during decoding. In addition, the current block may be at least one of a coding block, a predicted block, a residual block, and a transform block.

[0041] Sample: The basic unit that makes up a block. d ) 0 to 2 depending on Bd It can be expressed as a value up to -1. In the present invention, a sample can be used synonymously with a picture element or pixel.

[0042] Unit: refers to a unit of image encoding and decoding. In image encoding and decoding, a unit may be an area into which an image is divided. Also, when an image is divided into smaller units for encoding or decoding, a unit may refer to the divided units. In image encoding and decoding, a predefined process may be performed for each unit. A unit may be further divided into subunits having a smaller size than the unit. Depending on the function, a unit may refer to a block, a macroblock, a coding tree unit, a coding tree block, a coding unit, a coding block, a prediction unit, a prediction block, a residual unit, a residual block, a transform unit, a transform block, etc. Also, a unit may mean including a luma component block, a corresponding chroma component block, and syntax elements for each block in order to distinguish and indicate the unit from a block. The unit may have various sizes and shapes, and in particular, the shape of the unit may include geometric shapes that can be expressed two-dimensionally, such as a rectangle, a square, a trapezoid, a triangle, a pentagon, etc. Also, the unit information may include at least one of a type of unit indicating a coding unit, a prediction unit, a residual unit, a transform unit, etc., a size of the unit, a depth of the unit, an encoding and decoding order of the unit, etc.

[0043] Coding Tree Unit: Consists of two chrominance (Cb, Cr) coding tree blocks associated with one luminance (Y) coding tree block. It can also mean including the blocks and syntax elements for each block. Each coding tree unit can be divided using one or more division methods such as a quad tree or a binary tree to form subunits such as a coding unit, a prediction unit, and a transform unit. It can be used as a term to indicate pixel blocks that are processing units in the process of decoding / encoding an image, such as division of an input image.

[0044] Coding Tree Block: A term that can be used to refer to any of the Y coding tree block, the Cb coding tree block, and the Cr coding tree block.

[0045] Neighbor block: means a block adjacent to a current block. The block adjacent to the current block may mean a block bordering the current block or a block located within a predetermined distance from the current block. The neighbor block may mean a block adjacent to a vertex of the current block. Here, the block adjacent to a vertex of the current block may be a neighbor block adjacent to the horizontal side and a block adjacent to the vertical side of the current block, or a block adjacent to the vertical side and a block adjacent to the horizontal side of the current block. The neighbor block may also mean a restored neighbor block.

[0046] Reconstructed Neighbor Block: refers to a neighboring block that has already been encoded or decoded spatially / temporally around a current block. In this case, the reconstructed neighboring block may refer to a reconstructed neighboring unit. The reconstructed spatial neighboring block may be a block in the current picture that has already been reconstructed through encoding and / or decoding. The reconstructed temporal neighboring block may be a reconstructed block or a neighboring block at the same position as the current block of the current picture in the reference picture.

[0047] Unit Depth: It means the degree to which a unit is divided. In a tree structure, the root node has the shallowest depth and the leaf node has the deepest depth. Also, when a unit is expressed in a tree structure, the level at which the unit exists can mean the unit depth.

[0048] Bitstream: A string of bits that contains encoded image information.

[0049] Parameter Set: This corresponds to header information among structures in a bitstream. At least one of a video parameter set, a sequence parameter set, a picture parameter set, and an adaptation parameter set may be included in the parameter set. The parameter set may also include slice header and tile header information.

[0050] Parsing: This can mean entropy decoding a bitstream to determine the values ​​of syntax elements, or it can mean entropy decoding itself.

[0051] Symbol: may refer to at least one of a syntax element of a unit to be coded / decoded, a coding parameter, a value of a transform coefficient, etc. Also, a symbol may refer to an object of entropy coding or a result of entropy decoding.

[0052] Prediction unit: A basic unit for performing prediction such as inter prediction, intra prediction, inter compensation, intra compensation, motion compensation, etc. One prediction unit may be divided into a plurality of partitions or sub-prediction units having smaller sizes.

[0053] Prediction Unit Partition: This refers to the shape in which a prediction unit is divided.

[0054] Reference Picture List: A list containing one or more reference pictures used for inter-frame prediction or motion compensation. There are various types of reference picture lists, such as LC (List Combined), L0 (List 0), L1 (List 1), L2 (List 2), and L3 (List 3). Inter-frame prediction can use one or more reference picture lists.

[0055] Inter Prediction Indicator: This may refer to the inter prediction direction (unidirectional prediction, bidirectional prediction, etc.) of the current block, or the number of reference images used when generating a prediction block of the current block, or the number of prediction blocks used when performing inter prediction or motion compensation on the current block.

[0056] Reference Picture Index: An index that points to a particular reference picture from a reference picture list.

[0057] Reference Picture: This may refer to a picture that a particular block references for inter-frame prediction or motion compensation.

[0058] Motion Vector: A two-dimensional vector used in inter-frame prediction or motion compensation, which can indicate an offset between a picture to be coded / decoded and a reference picture. For example, (mvX, mvY) can indicate a motion vector, where mvX indicates the horizontal component and mvY indicates the vertical component.

[0059] Motion Vector Candidate: A block that is a prediction candidate when predicting a motion vector, or a motion vector of the block. The motion vector candidate may be included in a motion vector candidate list.

[0060] Motion Vector Candidate List: This may refer to a list constructed using motion vector candidates.

[0061] Motion Vector Candidate Index: An indicator that indicates a motion vector candidate in a motion vector candidate list. It can also be called an index of a motion vector predictor.

[0062] Motion Information: This can refer to information including at least one of a motion vector, a reference image index, an inter prediction indicator, reference image list information, a reference image, a motion vector candidate, a motion vector candidate index, a merge candidate, a margin index, etc.

[0063] Merge Candidate List: means a list constructed using merge candidates.

[0064] Merge Candidate: refers to a spatial merge candidate, a temporal merge candidate, a combined merge candidate, a combined bi-predictive merge candidate, a zero merge candidate, etc. The merge candidate may include motion information such as an inter-screen prediction indicator, a reference image index for each list, and a motion vector.

[0065] Merge Index: Information indicating a merge candidate in a merge candidate list. The merge index may indicate a block from which the merge candidate is derived among blocks reconstructed so as to be adjacent to the current block spatially / temporally. The merge index may also indicate at least one of the motion information items held by the merge candidate.

[0066] Transform Unit: A basic unit for performing residual signal coding / decoding such as transform, inverse transform, quantization, inverse quantization, and transform coefficient coding / decoding. One transform unit can be divided into multiple transform units of smaller size.

[0067] Scaling: refers to the process of multiplying transform coefficient levels by a factor. Transform coefficients can be generated as a result of scaling on transform coefficient levels. Scaling can also be called dequantization.

[0068] Quantization Parameter: This can mean a value used when generating a transform coefficient level for a transform coefficient in quantization. Or, it can mean a value used when generating a transform coefficient by scaling a transform coefficient level in inverse quantization. The quantization parameter can be a value mapped to a quantization step size.

[0069] Delta Quantization Parameter: This refers to the difference between a predicted quantization parameter and a quantization parameter of a unit to be coded / decoded.

[0070] Scan: A method for sorting the order of coefficients in a block or matrix. For example, sorting a two-dimensional array into a one-dimensional array is called scanning. Or, sorting a one-dimensional array into a two-dimensional array can also be called scanning or inverse scanning.

[0071] Transform Coefficient: A transform coefficient refers to a coefficient value generated after performing a transform in an encoder. It can also refer to a coefficient value generated after performing at least one of entropy decoding and inverse quantization in a decoder. A quantization level or a quantized transform coefficient level obtained by applying quantization to a transform coefficient or a residual signal can also be included in the meaning of a transform coefficient.

[0072] Quantized Level: A value generated by quantizing a transform coefficient or a residual signal in an encoder. Alternatively, it can refer to a value to be inversely quantized before inverse quantization in a decoder. Similarly, a quantized transform coefficient level, which is a result of transform and quantization, can also be included in the meaning of the quantized level.

[0073] Non-zero transform coefficient: A transform coefficient whose magnitude is not zero, or a transform coefficient level whose magnitude is not zero.

[0074] Quantization Matrix: A matrix used in the quantization or inverse quantization process to improve the subjective or objective image quality of an image. A quantization matrix can also be called a scaling list.

[0075] Quantization Matrix Coefficient: Each element in a quantization matrix. The quantization matrix coefficient can also be called a matrix coefficient.

[0076] Default Matrix: A default quantization matrix that is predefined in the encoder and decoder.

[0077] Non-default Matrix: This refers to a quantization matrix that is not predefined in the encoder and decoder, but is signaled by the user.

[0078] FIG. 1 is a block diagram showing the configuration of an embodiment of an encoding device to which the present invention is applied.

[0079] The encoding device 100 may be an encoder, a video encoding device, or an image encoding device. A video may include one or more images. The encoding device 100 may sequentially encode one or more images.

[0080] Referring to FIG. 1, the encoding device 100 may include a motion prediction unit 111, a motion compensation unit 112, an intra prediction unit 120, a switch 115, a subtractor 125, a transform unit 130, a quantization unit 140, an entropy encoding unit 150, an inverse quantization unit 160, an inverse transform unit 170, an adder 175, a filter unit 180, and a reference picture buffer 190.

[0081] The encoding device 100 may perform encoding on an input image in an intra mode and / or an inter mode. The encoding device 100 may generate a bitstream through encoding on the input image and output the generated bitstream. The generated bitstream may be stored in a computer-readable recording medium or streamed via a wired / wireless transmission medium. When an intra mode is used as a prediction mode, the switch 115 may switch to intra, and when an inter mode is used as a prediction mode, the switch 115 may switch to inter. Here, the intra mode may mean an intra-frame prediction mode, and the inter mode may mean an inter-frame prediction mode. The encoding device 100 may generate a prediction block for an input block of an input image. After the prediction block is generated, the encoding device 100 may encode a residual between the input block and the prediction block. The input image may be referred to as a current image currently being encoded. The input block may be referred to as a current block currently being encoded or a block to be encoded.

[0082] When the prediction mode is an intra mode, the intra prediction unit 120 may also use pixel values ​​of already encoded / decoded blocks around the current block as reference pixels. The intra prediction unit 120 may perform spatial prediction using the reference pixels and generate a prediction sample for an input block through the spatial prediction. Here, intra prediction may mean intra-frame prediction.

[0083] When the prediction mode is an inter mode, the motion prediction unit 111 can search for an area that best matches the input block in the reference image during the motion prediction process, and can derive a motion vector using the searched area. The reference image can be stored in the reference picture buffer 190.

[0084] The motion compensation unit 112 may generate a prediction block by performing motion compensation using a motion vector. Here, inter prediction may refer to inter-frame prediction or motion compensation.

[0085] The motion prediction unit 111 and the motion compensation unit 112 may generate a prediction block by applying an interpolation filter to a part of a reference image when a motion vector value does not have an integer value. In order to perform inter-picture prediction or motion compensation, it may be determined based on a coding unit whether a motion prediction and motion compensation method of a prediction unit included in the corresponding coding unit is a skip mode, a merge mode, an advanced motion vector prediction (AMVP) mode, or a current picture reference mode, and inter-picture prediction or motion compensation may be performed according to each mode.

[0086] The subtractor 125 may generate a residual block using the difference between the input block and the prediction block. The residual block may also be referred to as a residual signal. The residual signal may mean a difference between an original signal and a prediction signal. Alternatively, the residual signal may be a signal generated by transforming, quantizing, or transforming and quantizing the difference between the original signal and the prediction signal. The residual block may be a block-by-block residual signal.

[0087] The transform unit 130 may perform a transform on the residual block to generate a transform coefficient, and may output the transform coefficient. Here, the transform coefficient may be a coefficient value generated by performing a transform on the residual block. When a transform skip mode is applied, the transform unit 130 may skip a transform on the residual block.

[0088] By applying quantization to the transform coefficients or the residual signal, quantized levels can be generated. In the following, in the embodiments, the quantized levels may also be referred to as transform coefficients.

[0089] The quantization unit 140 may generate quantization levels by quantizing the transform coefficients or the residual signal based on the quantization parameter, and may output the quantization levels. In this case, the quantization unit 140 may quantize the transform coefficients using a quantization matrix.

[0090] The entropy coding unit 150 may generate a bitstream by performing entropy coding according to a probability distribution on a value calculated by the quantization unit 140 or a coding parameter value calculated during the coding process, and may output the bitstream. The entropy coding unit 150 may perform entropy coding on information regarding pixels of an image and information for decoding the image. For example, the information for decoding the image may include syntax elements, etc.

[0091] When entropy coding is applied, a smaller number of bits are assigned to symbols having a higher occurrence probability and a larger number of bits are assigned to symbols having a lower occurrence probability to represent the symbols, thereby reducing the size of a bit string for a symbol to be coded. The entropy coding unit 150 can use coding methods such as exponential Golomb, CAVLC (Context-Adaptive Variable Length Coding), and CABAC (Context-Adaptive Binary Arithmetic Coding) for entropy coding. For example, the entropy coding unit 150 can perform entropy coding using a variable length coding / code (VLC) table. In addition, the entropy coding unit 150 can derive a binarization method for a target symbol and a probability model for a target symbol / bin, and then perform arithmetic coding using the derived binarization method, probability model, and context model.

[0092] The entropy coding unit 150 can change two-dimensional block shape coefficients into one-dimensional vectors through a transform coefficient scanning method in order to code the transform coefficient levels.

[0093] Coding parameters can include not only information (flags, indexes, etc.) that is coded by an encoder and signaled to a decoder, like syntax elements, but also information derived during the coding or decoding process, and can refer to information required when coding or decoding an image. For example, unit / block size, unit / block depth, unit / block division information, unit / block division structure, whether or not it is a quadtree division, whether or not it is a binary tree division, the direction of the binary tree division (horizontal or vertical), the type of binary tree division (symmetric division or asymmetric division), intra-screen prediction mode / direction, reference sample filtering method, prediction block filtering method, prediction block filter tap, prediction block filter coefficient, inter-screen prediction mode, motion information, motion vector, reference image index, inter-screen prediction direction, inter-screen prediction indicator, reference image list, reference image, motion vector prediction candidate, motion vector candidate list, whether or not merge mode is used, merge candidate, merge candidate list, whether or not skip mode is used, type of interpolation filter, interpolation filter tap, interpolation filter coefficient, size of motion vector, accuracy of motion vector representation, type of transformation, transformation size, information on whether or not primary transformation is used, information on whether or not secondary transformation is used, primary transformation index, secondary transformation index, information on whether or not residual signal is used, coded block pattern, coded block flagThe coding parameters may include at least one value or combination form of information regarding a regular mode, a bypass mode, a context bin, a bypass bin, a transform coefficient, a transform coefficient level, a transform coefficient level scanning method, an image display / output order, slice identification information, a slice type, slice partition information, a tile identification information, a tile type, a tile partition information, a picture type, a bit depth, and information regarding a luminance signal or a chrominance signal.

[0094] Here, signaling a flag or index can mean that an encoder entropy encodes the corresponding flag or index and includes it in a bitstream, and can mean that a decoder entropy decodes the corresponding flag or index from the bitstream.

[0095] When the encoding device 100 performs encoding using inter prediction, the encoded current image can be used as a reference image for other images to be processed later. Thus, the encoding device 100 can further restore or decode the encoded current image and store the restored or decoded image as a reference image.

[0096] The quantization levels may be dequantized by the inverse quantization unit 160 and inverse transformed by the inverse transform unit 170. The inverse quantized and / or inverse transformed coefficients may be combined with the prediction block via an adder 175. A reconstructed block may be generated by combining the inverse quantized and / or inverse transformed coefficients with the prediction block. Here, the inverse quantized and / or inverse transformed coefficients refer to coefficients that have been subjected to at least one of inverse quantization and inverse transformation, and may refer to a reconstructed residual block.

[0097] The reconstructed block may pass through a filter unit 180. The filter unit 180 may apply at least one of a deblocking filter, a sample adaptive offset (SAO), an adaptive loop filter (ALF), etc. to the reconstructed block or the reconstructed image. The filter unit 180 may also be referred to as an in-loop filter.

[0098] The deblocking filter can remove block noise that occurs at the boundary between blocks. In order to determine whether to perform the deblocking filter, it can be determined whether to apply the deblocking filter to the current block based on pixels included in several columns or rows included in the block. When applying the deblocking filter to the block, different filters can be applied depending on the required deblocking filtering strength.

[0099] In order to compensate for coding errors using sample adaptive offset, an appropriate offset value can be added to pixel values. Sample adaptive offset can correct the offset between a deblocked image and an original image on a pixel-by-pixel basis. After dividing the pixels included in an image into a certain number of regions, a region to be offset is determined and an offset is applied to the corresponding region, or an offset is applied by considering edge information of each pixel.

[0100] The adaptive loop filter can perform filtering based on a value obtained by comparing a restored image with an original image. After dividing pixels included in an image into predetermined groups, a filter to be applied to the group is determined, and filtering can be performed differentially for each group. Information related to whether or not to apply an adaptive loop filter can be signaled for each coding unit (CU), and the shape and filter coefficients of the adaptive loop filter to be applied may differ depending on each block.

[0101] The reconstructed block or image that has passed through the filter unit 180 can be stored in a reference picture buffer 190. Fig. 2 is a block diagram showing a configuration according to an embodiment of a decoding device to which the present invention is applied.

[0102] The decoding device 200 may be a decoder, a video decoding device, or an image decoding device.

[0103] Referring to FIG. 2, the decoding device 200 may include an entropy decoding unit 210, an inverse quantization unit 220, an inverse transform unit 230, an intra prediction unit 240, a motion compensation unit 250, an adder 255, a filter unit 260, and a reference picture buffer 270.

[0104] The decoding device 200 may receive a bitstream output from the encoding device 100. The decoding device 200 may receive a bitstream stored in a computer-readable recording medium or a bitstream streamed via a wired / wireless transmission medium. The decoding device 200 may perform decoding on the bitstream in an intra mode or an inter mode. In addition, the decoding device 200 may generate a restored image or a decoded image through decoding, and may output the restored image or the decoded image.

[0105] If the prediction mode used for decoding is an intra mode, the switch can be changed to intra. If the prediction mode used for decoding is an inter mode, the switch can be changed to inter.

[0106] The decoding device 200 may decode an input bitstream, obtain a reconstructed residual block, and generate a prediction block. When the reconstructed residual block and the prediction block are obtained, the decoding device 200 may generate a reconstructed block to be decoded by adding the reconstructed residual block and the prediction block. The block to be decoded may also be referred to as a current block.

[0107] The entropy decoder 210 may generate symbols by performing entropy decoding based on a probability distribution for the bitstream. The generated symbols may include symbols in the form of quantization levels. Here, the entropy decoding method may be a reverse process of the above-mentioned entropy encoding method.

[0108] The entropy decoder 210 may convert one-dimensional vector form coefficients into two-dimensional block form coefficients according to a scanning method of the transform coefficients in order to decode the transform coefficient levels.

[0109] The quantization levels may be inverse quantized by the inverse quantization unit 220 and inverse transformed by the inverse transform unit 230. The quantization levels may be generated as reconstructed residual blocks resulting from the inverse quantization and / or inverse transform. In this case, the inverse quantization unit 220 may apply a quantization matrix to the quantization levels.

[0110] When the intra mode is used, the intra prediction unit 240 can generate a prediction block by performing spatial prediction using pixel values ​​of already decoded blocks surrounding the current block to be decoded.

[0111] When the inter mode is used, the motion compensation unit 250 may generate a prediction block by performing motion compensation using a motion vector and a reference image stored in the reference picture buffer 270. When a value of a motion vector does not have an integer value, the motion compensation unit 250 may generate a prediction block by applying an interpolation filter to a part of a reference image. In order to perform motion compensation, it may be determined based on a coding unit whether a motion compensation method of a prediction unit included in the corresponding coding unit is a skip mode, a merge mode, an AMVP mode, or a current picture reference mode, and motion compensation may be performed according to each mode.

[0112] The adder 255 may add the reconstructed residual block and the prediction block to generate a reconstructed block. The filter unit 260 may apply at least one of a deblocking filter, a sample adaptive offset, and an adaptive loop filter to the reconstructed block or the reconstructed image. The filter unit 260 may output the reconstructed image. The reconstructed block or the reconstructed image may be stored in the reference picture buffer 270 and used for inter prediction.

[0113] 3 is a schematic diagram showing an image division structure when encoding and decoding an image, which illustrates an embodiment in which one unit is divided into multiple sub-units.

[0114] In order to efficiently divide an image, a coding unit (CU) can be used in encoding and decoding. The coding unit can be used as a basic unit for image encoding / decoding. In addition, the coding unit can be used as a unit for distinguishing between intra-frame mode and inter-frame mode when encoding / decoding an image. The coding unit can be a basic unit used for prediction, transformation, quantization, inverse transformation, inverse quantization, or encoding / decoding of transform coefficients.

[0115] Referring to FIG. 3, an image 300 is sequentially divided into Largest Coding Unit (LCU) units, and a division structure is determined for each LCU. Here, LCU can be used synonymously with Coding Tree Unit (CTU). Division of a unit may mean division of a block corresponding to the unit. Block division information may include information regarding the depth of the unit. The depth information may indicate the number of times and / or the degree to which the unit is divided. One unit may be hierarchically divided with the depth information based on a tree structure. Each divided subunit may have depth information. The depth information is information indicating the size of a CU, and may be stored for each CU.

[0116] The partition structure may refer to the distribution of coding units (CUs) within the LCU 310. Such a distribution may be determined by whether or not one CU is partitioned into a plurality of CUs (a positive integer equal to or greater than 2, including 2, 4, 8, 16, etc.). The width and height of a CU generated by partitioning may be half the width and half the height of the CU before partitioning, respectively, or may have a size smaller than the width and height of the CU before partitioning depending on the number of partitions. A CU may be partitioned recursively into a plurality of CUs. The partitioning of a CU may be performed recursively to a predefined depth or a predefined size. For example, the depth of an LCU may be 0, and the depth of a smallest coding unit (SCU) may be a predefined maximum depth. Here, as described above, an LCU may be a coding unit having a maximum coding unit size, and an SCU may be a coding unit having a minimum coding unit size. Partitioning starts from the LCU 310, and the depth of the CU increases by 1 each time the width and / or height of the CU decreases due to partitioning.

[0117] In addition, information on whether a CU is divided can be expressed through division information of the CU. The division information can be 1-bit information. All CUs except for SCUs can include the division information. For example, if the value of the division information is a first value, the CU may not be divided, and if the value of the division information is a second value, the CU may be divided.

[0118] Referring to Figure 3, an LCU with depth 0 may be a 64x64 block. 0 may be the minimum depth. An SCU with depth 3 may be an 8x8 block. 3 may be the maximum depth. CUs with 32x32 blocks and 16x16 blocks may be represented with depth 1 and depth 2, respectively.

[0119] For example, when one coding unit is divided into four coding units, the width and height of the four divided coding units may be half the size of the width and height of the coding unit before division. As an example, when a coding unit of 32×32 size is divided into four coding units, each of the divided four coding units may have a size of 16×16. When one coding unit is divided into four coding units, it can be said that the coding unit is divided in a quad-tree shape.

[0120] For example, when one coding unit is divided into two coding units, the width or height of the two divided coding units may be half the size of the width or height of the coding unit before the division. As an example, when a coding unit of 32×32 size is vertically divided into two coding units, each of the divided coding units may have a size of 16×32. When one coding unit is divided into two coding units, it can be said that the coding unit is divided in a binary-tree shape. The LCU 310 in FIG. 3 is an example of an LCU to which both quad-tree division and binary-tree division are applied.

[0121] FIG. 4 is a diagram for explaining an embodiment of an inter-prediction process.

[0122] The rectangles shown in Fig. 4 may represent pictures. The arrows in Fig. 4 may represent prediction directions. Each picture may be classified into an I-picture (Intra Picture), a P-picture (Predictive Picture), a B-picture (Bi-predictive Picture), etc., depending on the coding type.

[0123] An I picture can be coded via intra prediction without inter prediction. A P picture can be coded via inter prediction using only reference pictures existing in one direction (e.g., forward or backward). A B picture can be coded via inter prediction using reference pictures existing in both directions (e.g., forward and backward). Here, when inter prediction is used, an encoder can perform inter prediction or motion compensation, and a decoder can perform corresponding motion compensation.

[0124] Next, inter-prediction according to the embodiment will be specifically described.

[0125] Inter prediction or motion compensation can be performed using reference pictures and motion information.

[0126] Motion information for the current block may be derived during inter prediction by each of the encoding device 100 and the decoding device 200. The motion information may be derived using motion information of reconstructed neighboring blocks, motion information of a collocated block (col block) and / or a block adjacent to the collocated block. The collocated block may be a block corresponding to a spatial position of the current block in an already reconstructed collocated picture (col picture). Here, the collocated picture may be one of at least one reference picture included in a reference picture list.

[0127] The motion information is derived in a different manner depending on the prediction mode of the current block. For example, the prediction mode applied for inter-picture prediction may be an AMVP mode, a merge mode, a skip mode, a current picture reference mode, etc. Here, the merge mode may be called a motion merge mode.

[0128] For example, when AMVP is applied as a prediction mode, at least one of the motion vector of the restored surrounding block, the motion vector of the co-located block, the motion vector of the block adjacent to the co-located block, and the (0,0) motion vector may be determined as a motion vector candidate to generate a motion vector candidate list. A motion vector candidate may be derived using the generated motion vector candidate list. Motion information of the current block may be determined based on the derived motion vector candidate. Here, the motion vector of the co-located block or the motion vector of the block adjacent to the co-located block may be referred to as a temporal motion vector candidate, and the motion vector of the restored surrounding block may be referred to as a spatial motion vector candidate.

[0129] The encoding device 100 may calculate a motion vector difference (MVD) between a motion vector of a current block and a motion vector candidate, and may entropy code the MVD. The encoding device 100 may also entropy code a motion vector candidate index to generate a bitstream. The motion vector candidate index may indicate an optimal motion vector candidate selected from among the motion vector candidates included in a motion vector candidate list. The decoding device 200 may entropy decode the motion vector candidate index from the bitstream, and may select a motion vector candidate for a block to be decoded from among the motion vector candidates included in the motion vector candidate list, using the entropy decoded motion vector candidate index. The decoding device 200 may also derive a motion vector for a block to be decoded, using a sum of the entropy decoded MVD and the motion vector candidate.

[0130] The bitstream may include a reference image index indicating a reference picture, etc. The reference image index may be entropy coded and signaled from the coding device 100 to the decoding device 200 via the bitstream. The decoding device 200 may generate a prediction block for a block to be decoded based on the derived motion vector and reference image index information.

[0131] Another example of a motion information derivation method is a merge mode. The merge mode may mean a merging of motions for multiple blocks. The merge mode may mean a mode in which motion information of a current block is derived from motion information of neighboring blocks. When the merge mode is applied, a merge candidate list may be generated using motion information of reconstructed neighboring blocks and / or motion information of co-located blocks. The motion information may include at least one of 1) a motion vector, 2) a reference image index, and 3) an inter-frame prediction indicator. The prediction indicator may be unidirectional (L0 prediction, L1 prediction) or bidirectional.

[0132] The merge candidate list may indicate a list in which motion information is stored. The motion information stored in the merge candidate list may be at least one of motion information of neighboring blocks adjacent to the current block (spatial merge candidate), motion information of a block collocated with the current block in a reference image (temporal merge candidate), new motion information generated by combining motion information already present in the merge candidate list, and a zero merge candidate.

[0133] The encoding apparatus 100 may generate a bitstream by entropy coding at least one of a merge flag and a merge index, and then signal the bitstream to the decoding apparatus 200. The merge flag may be information indicating whether a merge mode is performed for each block, and the merge index may be information regarding which block among the neighboring blocks neighboring the current block is to be merged with. For example, the neighboring blocks of the current block may include at least one of a left neighboring block, an upper neighboring block, and a temporal neighboring block of the current block.

[0134] The skip mode may be a mode in which motion information of a neighboring block is applied to a current block as is. When the skip mode is used, the encoding device 100 may entropy-encode information on which block's motion information is to be used as the motion information of the current block, and signal the information to the decoding device 200 via a bitstream. In this case, the encoding device 100 may not need to signal syntax elements related to at least one of motion vector differential information, coded block flags, and transform coefficient levels to the decoding device 200.

[0135] The current picture reference mode may refer to a prediction mode using an already reconstructed region in the current picture to which the current block belongs. In this case, a vector may be defined to identify the already reconstructed region. Whether the current block is encoded in the current picture reference mode may be encoded using a reference image index of the current block. A flag or index indicating whether the current block is a block encoded in the current picture reference mode may be signaled, or may be inferred using the reference image index of the current block. If the current block is encoded in the current picture reference mode, the current picture may be added to a fixed position or an arbitrary position in a reference image list for the current block. The fixed position may be, for example, a position where the reference image index is 0 or the last position. If the current picture is added to an arbitrary position in the reference image list, a separate reference image index indicating the arbitrary position may be signaled.

[0136] Based on the above, the image encoding / decoding method according to the present invention will be considered in detail.

[0137] FIG. 5 is a flowchart showing an image encoding method according to one embodiment of the present invention, and FIG. 6 is a flowchart showing an image decoding method according to one embodiment of the present invention.

[0138] 5, the encoding apparatus may derive merge candidates (S501) and generate a merge candidate list based on the derived merge candidates. When the merge candidate list is generated, the generated merge candidate list may be used to determine motion information (S502), and the determined motion information may be used to perform motion compensation of the current block (S503). Thereafter, the encoding apparatus may entropy code information related to motion compensation (S504).

[0139] 6, the decoding apparatus may entropy decode information related to motion compensation received from the encoding apparatus (S601), derive merge candidates (S602), and generate a merge candidate list based on the derived merge candidates. When the merge candidate list is generated, the decoding apparatus may determine motion information of a current block using the generated merge candidate list (S603). Thereafter, the decoding apparatus may perform motion compensation using the motion information (S604).

[0140] Here, FIG. 5 and FIG. 6 may be an example in which the merge mode described in FIG. 4 is applied.

[0141] Each of the steps illustrated in Figures 5 and 6 will now be considered in detail.

[0142] First, the steps of deriving merging candidates will be specifically described (S501, S602).

[0143] The merge candidates for the current block may include at least one of a spatial merge candidate, a temporal merge candidate, or an additional merge candidate, where deriving a spatial merge candidate may mean deriving a spatial merge candidate and adding it to a merge candidate list.

[0144] FIG. 7 is a diagram for explaining an example of deriving spatial merge candidates and temporal merge candidates for the current block.

[0145] 7, spatial merging candidates for a current block may be derived from neighboring blocks adjacent to the current block X. The neighboring blocks adjacent to the current block may include at least one of block B1 adjacent to the upper side of the current block, block A1 adjacent to the left side of the current block, block B0 adjacent to the upper right corner of the current block, block B2 adjacent to the upper left corner of the current block, and block A0 adjacent to the lower left corner of the current block.

[0146] In order to derive spatial merge candidates for a current block, it may be determined whether neighboring blocks adjacent to the current block can be used to derive spatial merge candidates for the current block. In this case, it may be determined based on a predetermined priority order whether neighboring blocks adjacent to the current block can be used to derive spatial merge candidates for the current block. As an example, in the example shown in FIG. 7, spatial merge candidate deriving availability may be determined in the order of blocks at positions A1, B1, B0, A0, and B2. The spatial merge candidates determined based on the availability determination order may be added sequentially to a merge candidate list for the current block.

[0147] As an example, when four spatial merge candidates are derived from the neighboring blocks at positions A1, B0, A0, and B2, the derived spatial merge candidates can be added sequentially to the merge candidate list.

[0148] The spatial merging candidates may also be derived based on at least one of the coding parameters.

[0149] Here, the motion information of the spatial merge candidate can have not only L0 and L1 motion information, but also three or more motion information such as L2, L3, etc. Here, the reference image list can include at least one of L0, L1, L2, L3, etc.

[0150] Next, a method for deriving temporal merging candidates for the current block will be described.

[0151] The temporal merge candidates for the current block can be derived from the reconstructed blocks included in the co-located picture of the current image, where the co-located picture may be an image that has been encoded / decoded before the current image and has a different temporal order from the current image.

[0152] Deriving temporal merge candidates can mean deriving and adding temporal merge candidates to a merge candidate list.

[0153] Referring to FIG. 7, in a collocated picture of the current image, a temporal merge candidate for the current block may be derived from a block including an external position of a block corresponding to the same spatial position as the current block X, or a block including an internal position of a block corresponding to the same spatial position as the current block X. Here, the temporal merge candidate may refer to motion information of the collocated block. As an example, a temporal merge candidate for the current block X may be derived from a block H adjacent to the lower right corner of a block C corresponding to the same spatial position as the current block, or a block C3 including the center point of the block C. The block H or block C3 used to derive a temporal merge candidate for the current block may be referred to as a "collocated block".

[0154] If a temporal merge candidate for the current block can be derived from block H including an external position of block C, block H can be set as the corresponding position block of the current block. In this case, the temporal merge candidate for the current block can be derived based on the motion information of block H. On the other hand, if a temporal merge candidate for the current block cannot be derived from block H, block C3 including an internal position of block C can be set as the corresponding position block of the current block. In this case, the temporal merge candidate for the current block can be derived based on the motion information of block C3. If a temporal merge candidate for the current block cannot be derived from block H and block C3 (e.g., when both block H and block C3 are intra-coded), a temporal merge candidate for the current block may not be derived or may be derived from a block at a position different from block H and block C3.

[0155] As another example, the temporal merging candidates for the current block may be derived from multiple blocks in the corresponding position image. As an example, multiple temporal merging candidates for the current block may be derived from block H and block C3.

[0156] When the distance between the current image including the current block and the reference image of the current block is different from the distance between the corresponding position image including the corresponding position block and the reference image of the corresponding position block, the motion vector of the temporal merge candidate of the current block can be obtained by scaling the motion vector of the corresponding position block. Here, the scaling can be performed based on at least one of the distance between the current image and the reference image referred to by the current block and the distance between the corresponding position image and the reference image referred to by the corresponding position block. As an example, the motion vector of the temporal merge candidate of the current block can be derived by scaling the motion vector of the corresponding position block according to the ratio of the distance between the current image and the reference image referred to by the current block and the distance between the corresponding position image and the reference image referred to by the corresponding position block.

[0157] In addition, at least one of a temporal merge candidate, a corresponding position image, a corresponding position block, a prediction list utilization flag, and a reference image index may be derived based on at least one of the encoding parameters of the current block, the surrounding block, or the corresponding position block.

[0158] After deriving at least one of the spatial merge candidates and the temporal merge candidates, the derived merge candidates may be added to a merge candidate list in order to generate a merge candidate list.

[0159] Next, a method for deriving additional merging candidates for the current block will be described.

[0160] The additional merge candidate may mean at least one of a modified spatial merge candidate, a modified temporal merge candidate, a combined merge candidate, and a merge candidate having a predetermined motion information value. Here, deriving the additional merge candidate may mean deriving the additional merge candidate and adding it to a merge candidate list.

[0161] A modified spatial merge candidate may refer to a merge candidate in which at least one of the motion information of the derived spatial merge candidate is modified.

[0162] A modified temporal merge candidate may refer to a merge candidate in which at least one of the motion information of a derived temporal merge candidate is modified.

[0163] A combined merge candidate may refer to a merge candidate derived by combining at least one motion information of a spatial merge candidate, a temporal merge candidate, a modified spatial merge candidate, a modified temporal merge candidate, a combined merge candidate, and a merge candidate having a predetermined motion information value present in a merge candidate list.

[0164] Alternatively, a combined merge candidate may refer to a merge candidate derived by combining a spatial merge candidate and a derived temporal merge candidate derived from a block from which at least one of a spatial merge candidate and a temporal merge candidate can be derived, which are not present in the merge candidate list, and at least one motion information of a modified spatial merge candidate, a modified temporal merge candidate, a combined merge candidate, and a merge candidate having a predetermined motion information value generated based on the spatial merge candidate and a derived temporal merge candidate.

[0165] Alternatively, the decoder may derive the combined merging candidates using motion information entropy decoded from the bitstream, and the motion information used to derive the combined merging candidates in the encoder may be entropy coded into the bitstream.

[0166] The combined merge candidate may refer to a combined bi-predictive merge candidate. The combined bi-predictive merge candidate may refer to a merge candidate that uses bi-prediction and has L0 motion information and L1 motion information.

[0167] A merge candidate having a certain motion information value may mean a zero merge candidate whose motion vector is (0, 0). Meanwhile, a merge candidate having a certain motion information value may be set to use the same value in the encoding device and the decoding device.

[0168] At least one of the modified spatial merge candidates, the modified temporal merge candidates, the combined merge candidates, and the merge candidates with the predetermined motion information values ​​may be derived or generated based on at least one of the coding parameters of the current block, the surrounding blocks, or the correspondingly located blocks, and at least one of the modified spatial merge candidates, the modified temporal merge candidates, the combined merge candidates, and the merge candidates with the predetermined motion information values ​​may be added to a merge candidate list based on at least one of the coding parameters of the current block, the surrounding blocks, or the correspondingly located blocks.

[0169] Meanwhile, the size of the merge candidate list can be determined based on the coding parameters of the current block, the neighboring blocks, or the corresponding position block, and the size can be changed based on the coding parameters.

[0170] Meanwhile, in the present invention, a corrected motion vector derived using a motion vector correction method can be used when deriving merging candidates, which will be described in detail later with reference to FIGS.

[0171] Next, the steps of determining the motion information of the current block using the generated merge candidate list will be specifically described (S502, S603).

[0172] The encoder can determine a merge candidate to be used for motion compensation from among the merge candidates in the merge candidate list through motion estimation, and encode a merge candidate index (merge_idx) indicating the determined merge candidate into the bitstream.

[0173] Meanwhile, in order to generate a predicted block, the encoder may select a merge candidate from the merge candidate list based on the above-mentioned merge candidate index, and may determine motion information of the current block. Here, the encoder may perform motion compensation based on the determined motion information to generate a predicted block of the current block.

[0174] The decoder may decode a merge candidate index in a bitstream and determine a merge candidate in a merge candidate list indicated by the merge candidate index. The determined merge candidate may be determined as motion information of a current block. The determined motion information is used for motion compensation of the current block. In this case, the motion compensation may have the same meaning as inter prediction.

[0175] Next, the steps of performing motion compensation using the determined motion information will be described (S503, S604).

[0176] The encoding apparatus and the decoding apparatus may perform inter prediction or motion compensation using the determined motion information (S503, S604), where the current block may have motion information of the determined merging candidate.

[0177] The current block can have a minimum of 1 to a maximum of N motion vectors based on the prediction direction. The motion vectors can be used to generate a minimum of 1 to a maximum of N prediction blocks to derive the final prediction block of the current block.

[0178] For example, if the current block has one motion vector, a prediction block generated using the motion vector (or motion information) may be determined as the final prediction block of the current block.

[0179] On the other hand, when the current block has multiple motion vectors (or motion information), multiple prediction blocks are generated using the multiple motion vectors (or motion information), and the final prediction block of the current block can be determined based on a weighted sum of the multiple prediction blocks. Reference images including each of the multiple prediction blocks indicated by the multiple motion vectors (or motion information) may be included in different reference image lists or may be included in the same reference image list.

[0180] As an example, multiple predictive blocks can be generated based on at least one of a spatial motion vector candidate, a temporal motion vector candidate, a motion vector having a predetermined value, or a combined motion vector candidate, and a final predictive block for the current block can be determined based on a weighted sum of the multiple predictive blocks.

[0181] As another example, a plurality of prediction blocks may be generated based on motion vector candidates indicated by a previously set motion vector candidate index, and a final prediction block of the current block may be determined based on a weighted sum of the plurality of prediction blocks. Also, a plurality of prediction blocks may be generated based on motion vector candidates existing within a range of previously set motion vector candidate indexes, and a final prediction block of the current block may be determined based on a weighted sum of the plurality of prediction blocks.

[0182] The weight applied to each prediction block may have an equal value of 1 / N (where N is the number of generated prediction blocks). For example, if two prediction blocks are generated, the weight applied to each prediction block may be 1 / 2, if three prediction blocks are generated, the weight applied to each prediction block may be 1 / 3, and if four prediction blocks are generated, the weight applied to each prediction block may be 1 / 4. Alternatively, a different weight may be applied to each prediction block to determine the final prediction block of the current block.

[0183] The weights do not have to have fixed values ​​for each prediction block, but may have variable values ​​for each prediction block. In this case, the weights applied to each prediction block may be the same or different. For example, when two prediction blocks are generated, the weights applied to the two prediction blocks may be variable values ​​for each block, such as (1 / 2, 1 / 2), (1 / 3, 2 / 3), (1 / 4, 3 / 4), (2 / 5, 3 / 5), (3 / 8, 5 / 8), etc. Meanwhile, the weights may be positive real values ​​or negative real values. For example, they may include negative real values, such as (-1 / 2, 3 / 2), (-1 / 3, 4 / 3), (-1 / 4, 5 / 4), etc.

[0184] Meanwhile, in order to apply variable weights, one or more weight information for the current block may be signaled through a bitstream. The weight information may be signaled for each prediction block or may be signaled for each reference image. A plurality of prediction blocks may share one weight information.

[0185] The encoding device and the decoding device may determine whether to use a predicted motion vector (or motion information) based on a prediction block list utilization flag. As an example, when the prediction block list utilization flag for each reference image list indicates a first value "1", the encoding device and the decoding device may use a predicted motion vector of the current block to perform inter-picture prediction or motion compensation, and when the prediction block list utilization flag indicates a second value "0", the encoding device and the decoding device may not perform inter-picture prediction or motion compensation using the predicted motion vector of the current block. Meanwhile, the first value of the prediction block list utilization flag may be set to 0, and the second value may be set to 1. The following Equations 1 to 3 respectively show an example of generating a final predicted block of the current block when the inter-picture prediction indicator of the current block is PRED_BI, PRED_TRI, and PRED_QUAD, and the prediction direction for each reference image list is one direction.

number

number

number

[0186] In Equations 1 to 3, P_BI, P_TRI, and P_QUAD indicate the final predicted block of the current block, and LX (X=0, 1, 2, 3) may indicate a reference image list. WF_LX may indicate a weight value of a predicted block generated using LX, and OFFSET_LX may indicate an offset value for a predicted block generated using LX. P_LX refers to a predicted block generated using a motion vector (or motion information) for LX of the current block. RF refers to a rounding factor, and may be set to 0, a positive number, or a negative number. The LX reference image list may include at least one of a long-term reference image, a reference image not subjected to a deblocking filter, a reference image not subjected to a sample adaptive offset, a reference image not subjected to an adaptive loop filter, a reference image only subjected to a deblocking filter and an adaptive offset, a reference image only subjected to a deblocking filter and an adaptive loop filter, a reference image only subjected to a sample adaptive offset and an adaptive loop filter, and a reference image all subjected to a deblocking filter, a sample adaptive offset, and an adaptive loop filter. In this case, the LX reference image list may be at least one of an L2 reference image list and an L3 reference image list.

[0187] Even when there are multiple prediction directions for a given reference image list, a final prediction block for a current block can be obtained based on a weighted sum of prediction blocks, where weights applied to prediction blocks derived from the same reference image list may have the same value or different values.

[0188] At least one of the weights WF_LX and offsets OFFSET_LX for the multiple prediction blocks may be an entropy coded / decoded coding parameter. As another example, the weights and offsets may be derived from coded / decoded neighboring blocks around the current block. Here, the neighboring blocks around the current block may include at least one of blocks used to derive spatial motion vector candidates for the current block or blocks used to derive temporal motion vector candidates for the current block.

[0189] As another example, the weight and offset may be determined based on the display order (POC) of the current image and each reference image. In this case, the weight or offset may be set to a smaller value as the distance between the current image and the reference image increases, and the weight or offset may be set to a larger value as the distance between the current image and the reference image decreases. As an example, when the difference between the POC of the current image and the L0 reference image is 2, the weight value applied to the predicted block generated with reference to the L0 reference image may be set to 1 / 3, whereas when the difference between the POC of the current image and the L0 reference image is 1, the weight value applied to the predicted block generated with reference to the L0 reference image may be set to 2 / 3. As exemplified above, the weight or offset value may have an inversely proportional relationship with the difference in display order between the current image and the reference image. As another example, the weight or offset value may be set to be proportional to the difference in display order between the current image and the reference image.

[0190] As another example, at least one of the weights or offsets may be entropy coded / decoded based on at least one of the coding parameters, and a weighted sum of the prediction block may be calculated based on at least one of the coding parameters.

[0191] The weighted sum of multiple predicted blocks can be applied only to a part of the area in the predicted block. Here, the part of the area may be an area corresponding to a boundary in the predicted block. As described above, in order to apply the weighted sum only to a part of the area, the weighted sum can be performed in sub-block units of the predicted block.

[0192] In a sub-block of a smaller block size within a block of a block size indicated by the region information, inter-picture prediction or motion compensation can be performed using the same prediction block or the same final prediction block.

[0193] Furthermore, in a sub-block at a deeper block depth within a block at a block depth indicated by the region information, inter-picture prediction or motion compensation can be performed using the same prediction block or the same final prediction block.

[0194] In addition, when calculating a weighted sum of a predicted block using motion vector prediction, the weighted sum can be calculated using at least one motion vector candidate present in the motion vector candidate list and used as the final predicted block of the current block.

[0195] For example, a prediction block can be generated using only spatial motion vector candidates, a weighted sum of the prediction blocks can be calculated, and the calculated weighted sum can be used as the final prediction block for the current block.

[0196] For example, a prediction block may be generated using spatial and temporal motion vector candidates, a weighted sum of the prediction blocks may be calculated, and the calculated weighted sum may be used as the final prediction block of the current block.

[0197] For example, a prediction block can be generated using only the combined motion vector candidates, a weighted sum of the prediction blocks can be calculated, and the calculated weighted sum can be used as the final prediction block for the current block.

[0198] For example, a prediction block can be generated using only motion vector candidates having a particular motion vector candidate index, a weighted sum of the prediction blocks can be calculated, and the calculated weighted sum can be used as the final prediction block for the current block.

[0199] For example, a prediction block can be generated using only motion vector candidates that are within a particular range of motion vector candidate indexes, a weighted sum of the prediction blocks can be calculated, and the calculated weighted sum can be used as the final prediction block for the current block.

[0200] In addition, when calculating a weighted sum of predicted blocks using the merge mode, the weighted sum may be calculated using at least one merge candidate present in the merge candidate list and may be used as the final predicted block of the current block.

[0201] For example, a prediction block can be generated with only spatial merging candidates, a weighted sum of the prediction blocks can be calculated, and the calculated weighted sum can be used as the final prediction block for the current block.

[0202] For example, a prediction block may be generated using spatial and temporal merge candidates, a weighted sum of the prediction blocks may be calculated, and the calculated weighted sum may be used as the final prediction block of the current block.

[0203] For example, a prediction block can be generated with only the combined merging candidates, a weighted sum of the prediction blocks can be calculated, and the calculated weighted sum can be used as the final prediction block for the current block.

[0204] For example, a prediction block may be generated with only merging candidates having a particular merging candidate index, a weighted sum of the prediction blocks may be calculated, and the calculated weighted sum may be used as the final prediction block for the current block.

[0205] For example, a prediction block may be generated with only merging candidates that are within a certain range of merging candidate indexes, a weighted sum of the prediction blocks may be calculated, and the calculated weighted sum may be used as the final prediction block for the current block.

[0206] In the encoder and decoder, motion compensation can be performed using motion vectors / information of a current block. In this case, a final predicted block, which is a result of motion compensation, can be generated using at least one predicted block. Here, the current block can mean at least one of a current coding block and a current prediction block.

[0207] Next, the process of entropy coding / decoding information related to motion compensation will be described in detail (S504, S601).

[0208] The encoding device entropy codes information related to motion compensation via a bitstream, and the decoding device entropy decodes the information related to motion compensation included in the bitstream. Here, the information related to motion compensation to be entropy coded / decoded includes an inter prediction indicator (inter_pred_idc), reference image indexes (ref_idx_l0, ref_idx_l1, ref_idx_l2, ref_idx_l3), motion vector candidate indexes (mvp_l0_idx, mvp_l1_idx, mvp_l2_idx, mvp_l3_idx), motion vector difference (motion vector The flag may include at least one of the following: information on whether skip mode is being used (cu_skip_flag), information on whether merge mode is being used (merge_flag), merge index information (merge_index), information on whether motion vector correction based on merge mode is being used (fruc_flag), weight values ​​(wf_l0, wf_l1, wf_l2, wf_l3), and offset values ​​(offset_l0, offset_l1, offset_l2, offset_l3).

[0209] The inter prediction indicator may indicate at least one of the inter prediction direction or the number of prediction directions of the current block when the current block is encoded / decoded by inter prediction. For example, the inter prediction indicator may indicate one-way prediction or a plurality of directional predictions such as bidirectional prediction, three-way prediction, or four-way prediction. The inter prediction indicator may indicate the number of reference images used when the current block generates a predicted block. Or, one reference image may be used for a plurality of directional predictions. In this case, N (N>M) directional predictions may be performed using M reference images. The inter prediction indicator may also indicate the number of prediction blocks used when performing inter prediction or motion compensation on the current block.

[0210] The reference picture indicator may indicate one direction (PRED_LX), two directions (PRED_BI), three directions (PRED_TRI), four directions (PRED_QUAD) or more directions depending on the number of prediction directions of the current block.

[0211] A prediction list utilization flag indicates whether or not the reference image list is used to generate a prediction block.

[0212] The reference image index can specify a reference image from each reference image list to which the current block refers. For each reference image list, one or more reference image indexes can be entropy coded / decoded. The current block can perform motion compensation using one or more reference image indexes.

[0213] The motion vector candidate index indicates a motion vector candidate for a current block from a motion vector candidate list generated for each reference image list or each reference image index. At least one motion vector candidate index can be entropy coded / decoded for each motion vector candidate list. The current block can perform motion compensation using at least one motion vector candidate index.

[0214] A motion vector differential indicates a difference value between a motion vector and a predicted motion vector. For a current block, one or more motion vector differentials can be entropy coded / decoded for a reference image list or a motion vector candidate list generated for each reference image index. The current block can perform motion compensation using one or more motion vector differentials.

[0215] The skip mode use information (cu_skip_flag) indicates the use of the skip mode when it has a first value of 1, and does not indicate the use of the skip mode when it has a second value of 0. Based on the skip mode use information, motion compensation of the current block can be performed using the skip mode.

[0216] The merge mode use information (merge_flag) indicates the use of the merge mode when it has a first value of 1, and does not indicate the use of the merge mode when it has a second value of 0. Based on the information on whether the merge mode is used, motion compensation for the current block can be performed using the merge mode.

[0217] The merge index information (merge_index) may refer to information indicating a merge candidate in a merge candidate list.

[0218] Also, the merge index information may refer to information on a merge index.

[0219] In addition, the merge index information may indicate a block from which a merge candidate is derived among reconstructed blocks that are spatially / temporally adjacent to the current block.

[0220] Also, the merge index information may indicate at least one of the motion information held by the merge candidates. For example, the merge index information may indicate the first merge candidate in the merge candidate list when it has a first value of 0, may indicate the second merge candidate in the merge candidate list when it has a second value of 1, and may indicate the third merge candidate in the merge candidate list when it has a third value of 2. Similarly, when it has a fourth to Nth value, it may indicate a merge candidate corresponding to the value based on the order in the merge candidate list. Here, N may represent a positive integer including 0.

[0221] Based on the merge mode index information, motion compensation of the current block can be performed using the merge mode.

[0222] The merge mode based motion vector compensation use information (fruc_flag) may indicate information indicating whether or not to use the merge mode based motion vector compensation. Entropy coding / decoding may be performed in units of at least one of a coding block and a prediction block.

[0223] For example, the use / non-use information of merge mode-based motion vector correction indicates the use of a merge mode-based motion vector when it has a first value of 1, and does not indicate the use of a merge mode-based motion vector when it has a second value of 0. Here, the first value and the second value can be changed.

[0224] However, without being limited thereto, it may be determined whether to use motion vector compensation based on other inter prediction modes (e.g., skip mode, AMVP mode, etc.) already defined in the encoder / decoder. That is, even if fruc_flag is not explicitly signaled, motion vector compensation may be selectively used under certain conditions.

[0225] On the other hand, the information on whether or not the merge mode-based motion vector correction is used (fruc_falg) can be entropy decoded when the information on whether or not the merge mode is used (merge_flag) has the first value indicating the use of the merge mode.

[0226] When two or more prediction blocks are generated during motion compensation for the current block, a final prediction block for the current block can be generated through a weighted sum of each prediction block. At least one of a weight and an offset can be applied to each prediction block during the weighted sum operation. The weighted sum factor used in the weighted sum operation, such as a weight factor or an offset, can be entropy coded / decoded for at least one of the reference image list, the reference image, the motion vector candidate index, the motion vector difference, the motion vector, the skip mode use / non-use information, the merge mode use / non-use information, and the merge index information, or at least one or more of them. In addition, the weighted sum factor of each prediction block can be entropy coded / decoded based on an inter-frame prediction indicator. Here, the weighted sum factor can include at least one of a weight value and an offset.

[0227] When information regarding motion compensation in a CTU is entropy coded / decoded, motion compensation can be performed on all or some of the blocks in the CTU using the information regarding motion compensation, depending on the value of the information regarding motion compensation.

[0228] When entropy encoding / decoding information regarding motion compensation from a CTU or a sub-unit of a CTU, the information regarding motion compensation can be entropy encoded / decoded based on at least one of a predetermined block size or a predetermined block depth.

[0229] Here, the information on the predetermined block size or the predetermined block depth can be further entropy coded / decoded, or the information on the predetermined block size or the predetermined block depth can be determined in the encoder and the decoder based on a predetermined value, at least one of the coding parameters, or at least one of the values ​​of other syntax elements.

[0230] Information on motion compensation may be entropy coded / decoded only in blocks having a block size equal to or larger than a predetermined block size, and information on motion compensation may not be entropy coded / decoded in blocks having a block size smaller than the predetermined block size. In this case, sub-blocks in blocks having a block size equal to or larger than the predetermined block size can perform motion compensation based on information on motion compensation entropy coded / decoded in blocks having a block size equal to or larger than the predetermined block size. That is, sub-blocks in blocks having a block size equal to or larger than the predetermined block size can share information on motion compensation including motion vector candidates, motion vector candidate lists, merge candidates, merge candidate lists, etc.

[0231] Information on motion compensation may be entropy coded / decoded only in blocks having a block depth equal to or shallower than a predetermined block depth, and information on motion compensation may not be entropy coded / decoded in blocks having a block depth deeper than the predetermined block depth. In this case, sub-blocks in blocks having a block depth equal to or shallower than a predetermined block depth can perform motion compensation based on information on motion compensation entropy coded / decoded in blocks having a block depth equal to or shallower than the predetermined block depth. In other words, sub-blocks in blocks having a block depth equal to or shallower than a predetermined block depth can share information on motion compensation including motion vector candidates, motion vector candidate lists, merge candidates, merge candidate lists, etc.

[0232] Information related to motion compensation may be entropy coded / decoded on a block-by-block basis, or may be entropy coded / decoded at a higher level. For example, information related to motion compensation may be entropy coded / decoded on a block-by-block basis, such as a CTU, a CU, or a PU, or may be entropy coded / decoded at a higher level, such as a video parameter set, a sequence parameter set, a picture parameter set, an adaptation parameter set, or a slice header.

[0233] The information regarding motion compensation may be entropy coded / decoded based on an information difference value regarding motion compensation indicating a difference value between the information regarding motion compensation and an information prediction value regarding motion compensation.

[0234] Instead of entropy coding / decoding information on motion compensation of the current block, it is also possible to use information on motion compensation of blocks coded / decoded around the current block as information on motion compensation of the current block.

[0235] Also, at least one of the pieces of information relating to the motion compensation can be derived based on at least one of the coding parameters.

[0236] Also, at least one of the pieces of information relating to motion compensation can be entropy decoded from the bitstream based on at least one of the coding parameters. At least one of the pieces of information relating to motion compensation can be entropy coded into the bitstream based on at least one of the coding parameters.

[0237] The information on motion compensation may further include at least one of a motion vector, a motion vector candidate, a motion vector candidate index, a motion vector difference value, a motion vector prediction value, skip mode use / non-use information (skip_flag), merge mode use / non-use information (merge_flag), merge index information (merge_index), motion vector resolution information, overlapped block motion compensation information, local illumination compensation information, affine motion compensation information, decoder-side motion vector derivation information, and bi-directional optical flow information. Here, the decoder motion vector derivation may mean a pattern matched motion vector derivation.

[0238] The motion vector resolution information may be information indicating whether a specific resolution is used for at least one of a motion vector and a motion vector differential value. Here, the resolution may mean precision. The specific resolution may be set to at least one of 16-pel units, 8-pel units, 4-pel units, integer-pel units, 1 / 2-pel units, 1 / 4-pel units, 1 / 8-pel units, 1 / 16-pel units, 1 / 32-pel units, and 1 / 64-pel units.

[0239] The overlapped block motion compensation information may be information indicating whether or not to further use motion vectors of neighboring blocks spatially adjacent to the current block to calculate a weighted sum of predicted blocks of the current block when performing motion compensation for the current block.

[0240] The regional illumination compensation information may be information indicating whether to apply at least one of a weight value and an offset value when generating a predicted block of a current block, where at least one of the weight value and the offset value may be a value calculated based on a reference block.

[0241] The affine motion compensation information may be information indicating whether an affine motion model is used in motion compensation for a current block. Here, the affine motion model may be a model that divides one block into a number of sub-blocks using a number of parameters and calculates motion vectors of the divided sub-blocks using a representative motion vector.

[0242] The decoder motion vector derivation information may be information indicating whether a motion vector required for motion compensation is derived and used by the decoder. Based on the decoder motion vector derivation information, information on the motion vector may not be entropy coded / decoded. And, when the decoder motion vector derivation information indicates that the decoder derives and uses the motion vector, information on the merge mode may be entropy coded / decoded. That is, the decoder motion vector derivation information may indicate whether the decoder uses the merge mode.

[0243] The bidirectional optical flow information may be information indicating whether to perform motion compensation by correcting a motion vector in a pixel unit or a sub-block unit. Based on the bidirectional optical flow information, the pixel unit or sub-block unit motion vector may not be entropy coded / decoded. Here, the motion vector correction may be to change a block unit motion vector to a pixel unit or sub-block unit motion vector value.

[0244] The current block can be motion compensated using at least one of the pieces of information related to motion compensation, and can be entropy coded / decoded using at least one of the pieces of information related to motion compensation.

[0245] When entropy coding / decoding information related to motion compensation, binarization methods such as the truncated Rice binarization method, the K-th order Exp_Golomb binarization method, the constrained K-th order Exp_Golomb binarization method, the fixed-length binarization method, the unary binarization method, or the truncated unary binarization method can be used.

[0246] When entropy encoding / decoding information regarding motion compensation, a context model can be determined using at least one of information regarding motion compensation of neighboring blocks around the current block or area information of the neighboring blocks, information regarding previously encoded / decoded motion compensation or area information previously encoded / decoded, information regarding the depth of the current block, and information regarding the size of the current block.

[0247] In addition, when entropy encoding / decoding information regarding motion compensation, entropy encoding / decoding can also be performed using at least one of information regarding motion compensation of neighboring blocks, information regarding previously encoded / decoded motion compensation, information regarding the depth of the current block, and information regarding the size of the current block as a predicted value for information regarding motion compensation of the current block.

[0248] Hereinafter, a detailed description of the motion vector correction method will be given with reference to FIGS.

[0249] In deriving a motion vector for a current block, the motion vector of the current block may be derived using at least one of a method of deriving a motion vector based on a skip / merge mode and a method of deriving using an improved motion vector prediction mode. Here, deriving a motion vector may include deriving reference image index information of the current block in addition to deriving a motion vector of the current block.

[0250] Next, a motion vector correction method based on the merge mode will be described. However, the following description is not limited to the merge mode, and can also be applied to a motion vector correction method based on another inter prediction mode other than the merge mode.

[0251] The merge mode based motion vector correction method may include at least one of an initial motion vector derivation step, a corrected motion vector deriving step based on the initial motion vector, and a corrected motion vector deriving step on a sub-block basis.

[0252] First, the initial motion vector derivation step will be specifically described.

[0253] As shown in FIG. 7, an initial motion vector for the current block may be derived using merging candidates in a merging candidate list derived from at least one of the spatial and temporal neighboring blocks.

[0254] Meanwhile, a separate merge candidate list can be configured for motion vector compensation based on the merge mode.

[0255] FIG. 8 illustrates spatial and temporal neighboring blocks for constructing a separate merge candidate list.

[0256] Referring to FIG. 8, when constructing a separate merge candidate list, a predetermined number of candidates from spatial surrounding blocks (A0, A1, A2, B0, B1, B2, B3) are included in the separate merge candidate list in the order of A1, B1, B0, A0, B2, B3, A2, and then candidates derived from temporal surrounding blocks, combined bi-predictive merge candidates, and zero merge candidates are included in the separate merge candidate list to derive an initial motion vector.

[0257] Meanwhile, when using the merging candidates in the merging candidate list, an initial motion vector for the current block may be derived using all or some of the merging candidates in the merging candidate list.

[0258] For example, if a merge candidate in the merge candidate list has motion information derived on a sub-block basis, the merge candidate may replace motion information derived from additional spatial and / or temporal surrounding blocks in constructing the merge candidate list, or may not be used as an initial motion vector.

[0259] 9 and 10 are diagrams for explaining the structure of the initial motion vector list.

[0260] When the merge candidate list is constructed as shown in Fig. 9, an initial motion vector list for each list direction can be constructed from merge candidates corresponding to the list direction as shown in Fig. 10. Here, the initial motion vector list can include at least one of a motion vector and a reference image index.

[0261] As an example, the initial motion vector list may use only the motion vectors in the merge candidate list, and the reference image index may use a fixed, predetermined value, where the fixed, predetermined value may be a value signaled from the encoder to the decoder, or a value commonly set in the encoder and the decoder.

[0262] On the other hand, if the reference image index information of a merging candidate in the merging candidate list does not match the specified reference image index information in the initial motion vector list, a motion vector scaled taking into account the temporal distance (or the difference in POC) can be used as the initial motion vector.

[0263] On the other hand, when forming the initial motion vector list, if the same motion vector and reference image index are already included in the list, no addition is required.

[0264] 11 is a diagram for explaining a method of deriving an initial motion vector using merge index information. The motion information of a merge candidate corresponding to the merge index information among the merge candidates in the merge candidate list can be used as the initial motion vector.

[0265] Referring to Figure 11, when the merge index information value corresponds to 3, the L0 list [motion vector: (3, 1), reference image index: 0] and the L1 list [motion vector: (-3, 1), reference image index: 0] corresponding to the fourth candidate can be derived from the merge candidate list as the initial motion vectors for each list direction.

[0266] On the other hand, when the merge index information value corresponds to 4, only the L0 list [motion vector: (6, 2), reference image index: 0] corresponding to the fifth candidate from the merge candidate list can be used as the initial motion vector in the L0 list direction. Here, the merge index information may be index information for the merge mode.

[0267] However, initial motion vector index information used for motion vector correction based on the merge mode can be further defined, which can be entropy coded / decoded for each coding / decoding block unit.

[0268] Meanwhile, when using a merging candidate in the merging candidate list, a motion vector of a merging candidate corresponding to a predetermined number among the merging candidates in the merging candidate list may be set as an initial motion vector without performing entropy decoding on the merging index information, where the predetermined number may be a number commonly set in the encoder and the decoder.

[0269] For example, if the encoder / decoder similarly uses the motion vector corresponding to the first candidate in the merge candidate list as the initial motion vector, the L0 list [motion vector: (3, 1), reference image index: 0] and L1 list [motion vector: (-2, 0), reference image index: 0] corresponding to the first candidate from the merge candidate list in Figure 11 can be used as the initial motion vector in the L0 / L1 list direction.

[0270] Meanwhile, when using a merge candidate in the merge candidate list, if the current block is a bi-predictable block but only one motion information corresponding to the L0 or L1 list is present, motion information derived based on the existing motion information may be used as additional initial motion information. Here, the existing motion information may be motion information of another merge candidate in the same list.

[0271] When deriving additional motion information based on existing motion information as described above, an additional initial motion vector to replace the non-existent motion information can be derived by taking into account at least one of the reference pictures indicated by the existing motion information, the current picture to which the current block belongs, or the temporal distance (or POC difference) between reference pictures in the list for which motion information needs to be generated.

[0272] Specifically, by performing at least one of mirroring and scaling on the existing motion vectors, additional initial motion vectors can be derived to replace the non-existent motion vectors.

[0273] For example, when the value of the merge index information is 2 in FIG. 11, only L0 motion information can be present.

[0274] In this case, an L0 motion vector that is mirrored based on a reference picture in the L1 reference picture list that has the same POC difference as the POC difference between the reference picture indicated by the L0 motion vector and the picture to which the current block belongs can be derived as the L1 list motion vector.

[0275] In addition, among the reference pictures in the L1 list, except for the reference picture having the same POC as the reference picture indicated by the L0 motion vector, the picture with the smallest POC difference from the current picture is derived as the L1 reference picture, and an L0 motion vector scaled taking into account the difference in POC between the L0 reference picture and the current picture, and the difference in POC between the derived L1 reference picture and the current picture, can be derived as the motion vector in the L1 direction.

[0276] FIG. 12 is a diagram for explaining motion vector mirroring.

[0277] Referring to Figure 12, an L0 motion vector mirrored based on a reference picture in an L1 reference picture list having the same POC difference as the POC difference between the reference picture indicated by the L0 motion vector and the picture to which the current block belongs can be derived as an L1 list motion vector.

[0278] Specifically, since the POC of the reference picture indicated by the L0 motion vector is 0 and the POC of the current picture is 4, the POC difference between the two pictures can be calculated as 4. Among the reference pictures in the L1 list direction, there is a picture whose POC difference with the current picture is 4, so the motion vector generated by mirroring the L0 motion vector for that picture and the reference image index of that reference picture can be used as the L1 motion information. Here, the L1 motion vector generated by mirroring the L0 motion vector in the L1 list direction is (3, 1), and the L1 reference image index is 0.

[0279] FIG. 13 is a diagram for explaining the scaling of a motion vector.

[0280] Referring to Figure 13, among the reference pictures in the L1 list, except for the reference picture having the same POC as the reference picture indicated by the L0 motion vector, the picture with the smallest POC difference from the current picture is derived as the L1 reference picture, and the L0 motion vector scaled taking into account the difference in POC between the L0 reference picture and the current picture and the difference in POC between the derived L1 reference picture and the current picture can be derived as the L1 direction motion vector.

[0281] Specifically, except for the reference picture POC2 indicated by the L0 motion vector, a picture POC1 having the smallest POC difference with the current picture can be derived as the L1 reference picture. Then, an L0 motion vector scaled in consideration of the POC difference tb between the L0 reference picture and the current picture and the POC difference td between the derived L1 reference picture and the current picture can be derived as the L1 direction motion vector.

[0282] The motion vector scaling process can be performed as follows.

[0283] td may represent a difference between the image order count for the current picture and the image order count for the L1 reference picture of the current block, and tb may represent a difference between the image order count for the current picture and the image order count for the L0 reference picture of the current block.

[0284] At least one of the td value and the tb value may be adjusted so that the td value or the tb value falls within a range between -128 and 127. In this case, if the td value or the tb value is smaller than -128, the td value or the tb value may be adjusted to -128, and if the td value or the tb value is larger than 127, the td value or the tb value may be adjusted to 127. If the td value or the tb value falls within a range between -128 and 127, the td value or the tb value does not need to be adjusted. That is, the td value or the tb value may be clipped so that the td value or the tb value falls within a certain value range.

[0285] The tx value can be determined by (16384+Abs(td / 2)) / td, where Abs() indicates an absolute value function, and the output value of the function can be the absolute value of the input value.

[0286] The scaling factor DistScaleFactor can be determined as (tb*tx+32)>>6, and the DistScaleFactor value is adjusted to be in the range between -1024 and 1023.

[0287] The scaled motion vector can be determined by Sign(DistScaleFactor*mv)*((Abs(DistScaleFactor*mv)+127)>>8). In this case, Sign() outputs the sign information of a particular value, and if Sign(-1), it can output -. mv can indicate the motion vector value that exists before scaling.

[0288] 14 is a diagram for explaining an example of deriving an additional motion vector when there is only one motion vector corresponding to the L0 or L1 list in a bi-predictable block. When using a merge candidate in a merge candidate list, if a current block is a bi-predictable block and there is only one motion vector corresponding to the L0 or L1 list, an additional motion vector can be derived in the merge candidate list of a prediction direction in which no motion vector exists.

[0289] 14, when the merge index information value is 2, only the L0 motion vector exists. In this case, the nth available motion vector from the L1 list can be used as the initial L1 motion vector of the current block. Here, n may be a value commonly set in the encoder / decoder, or may be a positive integer including 1.

[0290] 15 is a diagram for explaining an example of selecting a merge candidate having bidirectional motion information as an initial motion vector in a bi-predictable block. When using a merge candidate in a merge candidate list, if a current block is a bi-predictable block, a merge candidate having bidirectional motion information in the merge candidate list can be used as an initial motion vector. Here, if there are multiple candidates having bi-directional motion information in the merge candidate list, a candidate having the smallest merge index information can be used.

[0291] Referring to FIG. 15, if the current block corresponds to a bi-predictable block, the motion vector corresponding to merge candidate index 3 having the first available bi-directional motion information can be used as the initial motion vector.

[0292] If there is no reference image that satisfies the above condition, the motion information of a merge candidate selected by the existing merge index information can be derived as an initial motion vector.

[0293] On the other hand, when performing motion compensation based on an initial motion vector derived by the above-mentioned method, if the initial motion vector has a sub-pixel unit motion vector, motion compensation can be performed based on a motion vector rounded to the nearest integer pixel position.

[0294] For example, when having 1 / 16 motion vector precision, the following Equation 4 can be used to round to integer pixel positions: The Shift value depends on the precision and can have a value of 4 for 1 / 16 motion vector precision and a value of 3 for 1 / 8 motion vector precision.

number

[0295] Next, the step of deriving a corrected motion vector based on the derived initial motion vector will be specifically described.

[0296] In the encoder / decoder, the derived initial motion vector can be used to perform motion vector correction on a block-by-block basis.

[0297] In relation to the motion vector correction, the initial motion vector can be corrected using spatial and / or temporal surrounding blocks of the current block. The restored pixels of the spatial and / or temporal surrounding blocks used for the motion vector correction can be defined as a template.

[0298] As an example, the encoder / decoder may construct a template with spatially surrounding restored pixels of the current block and use the template to perform correction on the initial motion vector.

[0299] 16 to 21 are diagrams for explaining a template according to an embodiment of the present invention.

[0300] In constructing the template, as shown in FIG. 16, the template can be constructed using pixels of the upper and / or left neighboring reconstruction blocks of the current block.

[0301] Here, if pixels of the neighboring reconstruction blocks above or to the left of the current block are not available, the template can be constructed only from pixels of the available blocks.

[0302] As an example, as shown in FIG. 17, if the upper peripheral reconstruction block of the current block is not available for a picture boundary, slice boundary, tile boundary, or CTU boundary, a template can be constructed only with pixels of the left peripheral reconstruction block, and if the left peripheral reconstruction block of the current block is not available for a picture boundary, slice boundary, tile boundary, or CTU boundary, a template can be constructed only with pixels of the upper peripheral reconstruction block.

[0303] In addition, when constructing a template, unavailable pixels may be replaced with available pixels to construct the template. Here, the available pixels may be neighboring pixels of the current block, pixels derived by applying an interpolation filter to the neighboring pixels, or pixels having values ​​predefined in the encoder / decoder. Meanwhile, the template may be constructed of predicted pixel values ​​instead of restored pixel values.

[0304] When constructing the template, if the motion vector has a sub-pixel unit, an interpolation filter may be applied to generate a pixel value at the corresponding position. Here, the filter used for the interpolation may be a bi-linear filter, an 8-tap DCT-IF filter, etc. A predetermined method may be used in the encoder / decoder without transmitting information regarding the interpolation filter used to construct the template, and information regarding the type of the interpolation filter may be included in the bitstream and signaled.

[0305] On the other hand, the template may have a size of M×N, where M and N may be positive integers.

[0306] For example, as shown in FIG. 18, if the size (horizontal x vertical) of the current block is 64x32 and the pixel length of the template predefined in the encoder / decoder is 4 pixels, the size of the top template may be 64x4 and the size of the left template may be 4x32.

[0307] Here, the length of the pixel can be calculated according to the accuracy of the motion vector.

[0308] As an example, if the motion vector accuracy is 1 / 4 unit and the block size is 16 pixels, the pixel length of the template can be calculated to be 4 pixels.

[0309] On the other hand, when configuring a template, if the horizontal / vertical sizes of the current block are different as shown in FIG. 19, the size of the upper template and the size of the left template can be made the same.

[0310] On the other hand, when constructing a template, if the template size is M×N, M of the upper template can have a value larger than the horizontal size of the current block, and N can have a pixel length fixed in the encoder / decoder, and N of the right template can have a value larger than the vertical size of the current block, and M can have a pixel length fixed in the encoder / decoder.

[0311] For example, as shown in FIG. 20, if the size of the current block is 64×64 and the pixel length of the template defined in the encoder / decoder is 6 pixels, the size of the top template may be 128×6 and the size of the left template may be 6×128.

[0312] Meanwhile, in constructing a template, the size of the template can be M×N, where M and N may be equal to the horizontal / vertical size of the current block.

[0313] For example, as shown in FIG. 21, if the size of the current block is 64×64, the size of the left template may be 64×64 and the size of the top template may be 64×64.

[0314] Meanwhile, when constructing a template, the size of the template can be M×N, and templates of different sizes can be used according to the size or depth information of the current block.

[0315] For example, if the size of the current block is 128x64, which is greater than 32 in horizontal or vertical size, the size of the top template may be 128x6 and the size of the left template may be 6x64.

[0316] For example, if the size (horizontal x vertical) of the current block is 16x16, which is smaller than the horizontal or vertical size 32, the size of the top template may be 16x2 and the size of the left template may be 2x16.

[0317] Meanwhile, template size information required for constructing a template can be signaled by being included in the bitstream.

[0318] FIG. 22 is a diagram for explaining an example of correcting an initial motion vector using a template.

[0319] Referring to FIG. 22, a motion vector indicating a template that minimizes the error with the surrounding templates of the current block in a search area of ​​a reference picture indicated by an initial motion vector can be searched for, and the motion vector can be determined as a corrected motion vector.

[0320] Here, a template in the reference picture can be generated based on a predefined search area that includes the pixel location pointed to by the initial motion vector.

[0321] The predetermined search area may be applied in different shapes depending on the search method, such as a cross, a square, a diamond, a hexagon, etc., centered on the pixel position indicated by the initial motion vector. Meanwhile, minimizing the error between a template in a reference picture and a neighboring template of the current block may mean minimizing the difference between a template in a reference picture and a neighboring template of the current block.

[0322] In calculating an error between a template in a reference picture and a neighboring template of a current block, the error can be calculated by taking into account at least one of a distortion value and a cost value for a motion vector.

[0323] As an example, if the left and top templates are all possible, the cost value can be calculated as follows:

[0324] Cost value = (distortion value between left template + distortion value between upper template) + (initial motion vector (x, y) - corrected motion vector (x, y)) * weight As an example, if only the top template is possible, the cost value can be calculated as follows:

[0325] Cost value = Distortion value between upper template + (Initial motion vector (x, y) - Corrected motion vector (x, y)) * Weight Here, in the calculation of the cost value, the weight can have a negative number less than 0 or a positive number greater than 0.

[0326] Alternatively, the distortion value can be calculated using at least one of the inter-template SAD and MR-SAD.

number

[0327] Here, C tp (i,j) denotes the left or top template pixel value of the current block, and L tp(i,j) indicates the left or top template pixel value in the reference picture, and deltaDC may indicate (left template average pixel value of current block−left template average pixel value in reference picture) or (top template average pixel value of current block−top template average pixel value in reference picture).

[0328] A new motion vector nMV can be found that points to a template that minimizes the error with the template of the current block within a predetermined search area centered on the pixel position pointed to by the corrected motion vector rMV found based on the initial motion vector.

[0329] Here, if the error between the template of the reference picture obtained from the new motion vector nMV and the template of the current block is greater than the error between the template of the reference picture obtained from the corrected motion vector rMV and the template of the current block, no further correction process is required.

[0330] However, if the error between the reference picture template obtained from the new motion vector nMV and the template of the current block is smaller than the error between the reference picture template obtained from the corrected motion vector rMV and the template of the current block, another new motion vector can be obtained and corrected within a predetermined search area centered on the pixel position indicated by the new motion vector nMV.

[0331] On the other hand, if the error between the template of the reference picture and the template of the current block obtained from the new motion vector nMV is smaller than the error between the template of the reference picture and the template of the current block obtained from the compensated motion vector rMV, the search for the new motion vector can be repeated. In this case, the search for the new motion vector can be limited based on the information on the number of times the motion compensation is performed.

[0332] As an example, if the motion compensation execution count information is 2, the search for a new motion vector can be limited to a total of two times.

[0333] Information on the number of times motion compensation is performed can be signaled in the bitstream.

[0334] On the other hand, the number of times the motion compensation is executed may be a predetermined value that is determined in advance in both the encoder and the decoder.

[0335] On the other hand, when the initial motion vector has motion vectors for N prediction directions such as an L0 motion vector and an L1 motion vector, it is possible to obtain a correction motion vector for the motion vector from each prediction direction.

[0336] Meanwhile, the predetermined search space may be determined based on search space information, where the search space information required for determining the search space may be included in a bitstream and signaled.

[0337] The size of the predetermined search area may be varied depending on coding parameters such as picture type, temporal layer, block size, etc. For example, if the size of the search area of ​​a picture corresponding to the highest layer can have a search area of ​​1 integer pixel size, the size of the search area of ​​a picture corresponding to a lower layer can have a search area of ​​8 integer pixels size.

[0338] 23 to 25 are diagrams illustrating an example of correcting an initial motion vector using a predicted block of a current block. The initial motion vector can be corrected using a predicted block of a current block generated from a motion vector without using the template described in FIG.

[0339] When performing motion compensation using the prediction block, the motion vector can be compensated based on the difference between the prediction blocks in each reference picture pointed to by the initial motion vector.

[0340] Here, the prediction block may be located within a predetermined search range centered on a pixel position indicated by the initial motion vector. A prediction block may be generated from motion vectors derived within the predetermined search range, and a motion vector that minimizes the difference between the prediction blocks may be determined as a corrected motion vector.

[0341] The difference between prediction blocks may mean the difference between multiple prediction blocks generated based on an initial motion vector in the L0 direction (hereinafter referred to as the "L0 initial motion vector") or an initial motion vector in the L1 direction (hereinafter referred to as the "L1 initial motion vector"), or may mean the difference between prediction blocks generated respectively based on the L0 initial motion vector and the L1 initial motion vector.

[0342] As an example, a motion vector that minimizes an error between a predicted block P0 generated from a motion vector moved within a predetermined search range centered on a pixel position in an L0 reference picture indicated by an L0 initial motion vector, and a predicted block P1 generated from an L1 motion vector derived based on the L0 motion vector, can be determined as a corrected motion vector, as shown in Fig. 23. Here, the L1 motion vector may be a motion vector derived by mirroring or scaling the initial L0 motion vector or the L0 motion vector moved within a predetermined search range.

[0343] As another example, as shown in Fig. 24, a motion vector that minimizes an error between a predicted block P1 generated from a motion vector moved within a predetermined search range centered on a corresponding pixel position in an L1 reference picture indicated by an L1 initial motion vector and a predicted block P0 generated from an L0 motion vector derived based on the L1 motion vector may be determined as a corrected motion vector. Here, the L0 motion vector may be a motion vector derived by mirroring or scaling the initial L1 motion vector or the L1 motion vector moved within a predetermined search range.

[0344] Meanwhile, when deriving the Ly direction motion vector from the Lx motion vector, it can be derived by considering at least one of the reference picture indicated by the Lx motion vector, the picture to which the current block belongs, and the temporal distance (or POC difference) between the reference pictures in the Ly direction, where x and y can be positive integers different from each other, including 0.

[0345] As an example of mirroring, an L0 motion vector mirrored based on a reference picture in an L1 reference picture list that has the same POC difference as the POC difference between the reference picture indicated by the L0 motion vector and the picture to which the current block belongs can be derived as an L1 direction motion vector.

[0346] As an example of scaling, an L0 motion vector can be derived as an L1 direction motion vector by scaling the L0 motion vector based on the picture with the smallest POC difference from the current picture, excluding reference pictures in the L1 reference picture list that have the same POC as the reference picture indicated by the L0 motion vector, taking into account the POC difference.

[0347] On the other hand, when performing motion compensation using a predictive block, motion compensation can be performed using a block generated by weighting the sum of a predictive block generated from an L0 initial motion vector and a predictive block generated from an L1 initial motion vector.

[0348] For example, as shown in Fig. 25, a prediction block Ps generated by weighting a prediction block P0 generated from an L0 initial motion vector and a prediction block P1 generated from an L1 initial motion vector can be set as a template. Then, a prediction block Pr0 can be generated by moving the motion vector within a predetermined search range centered on a pixel position indicated by the L0 initial motion vector. Here, a motion vector that minimizes the error between the generated prediction block Pr0 and the template Ps can be determined as a corrected motion vector in the L0 list direction.

[0349] In addition, the motion vector may be moved within a predetermined search range centered on the pixel position indicated by the L1 motion vector to generate a prediction block Pr1. Here, the motion vector that minimizes the error between the generated prediction block Pr1 and the template Ps may be determined as the corrected motion vector in the L1 list direction.

[0350] On the other hand, the predicted block Ps can be calculated as follows.

[0351] Ps=(a1*P0+a2*P1)>>1 Here, a1 and a2 are weights used in the weighted sum. The weights are not limited to fixed values ​​and may be variable values. The weights applied to each prediction block may be the same or different from each other. To apply variable weights, weight information for the block to be decoded may be signaled via the bitstream.

[0352] On the other hand, when the initial motion vector is derived only from L0, only the prediction block P0 generated from the L0 initial motion vector can be used as a template. Then, the motion vector that minimizes the error between the prediction block Pr0 generated by moving the motion vector within a predetermined search range centered on the pixel position indicated by the L0 initial motion vector and the template can be determined as the corrected motion vector in the L0 list direction.

[0353] Meanwhile, in calculating the difference between the predicted blocks of the current block, the difference between the predicted blocks may be calculated taking into consideration at least one of a distortion value and a cost value for a motion vector.

[0354] As an example, when a prediction block (P0, P1) is generated using the L0 initial motion vector, the cost value can be calculated as follows:

[0355] Cost value = distortion value (predicted block (P0) - predicted block (P1)) + (initial L0 motion vector - corrected L0 motion vector) * weight As an example, when a prediction block (P0, P1) is generated using the L1 initial motion vector, the cost value can be calculated as follows:

[0356] Cost value = distortion value (predicted block (P0) - predicted block (P1)) + (initial L1 motion vector - corrected L1 motion vector) * weight Here, in the calculation of the cost value, the weight can have a negative number less than 0 or a positive number greater than 0.

[0357] Meanwhile, in calculating the cost value, the distortion value can be calculated using at least one of SAD, SATD, and MR-SAD between prediction blocks.

[0358] A new motion vector nMV that minimizes the error between prediction blocks can be found within a predetermined search area centered on a pixel position indicated by a corrected motion vector rMV found based on the initial motion vector.

[0359] Here, if the error between the predicted blocks (P0, P1) obtained from the new motion vector nMV is larger than the error between the predicted blocks (P0, P1) obtained from the corrected motion vector rMV, no further correction process is required. However, if the error between the predicted blocks (P0, P1) obtained from the new motion vector nMV is smaller than the error between the predicted blocks (P0, P1) obtained from the corrected motion vector rMV, another new motion vector can be obtained within a predetermined search area centered on the pixel position indicated by the new motion vector nMV for correction.

[0360] On the other hand, if the error between the prediction blocks obtained from the new motion vector is smaller than the error between the prediction blocks obtained from the corrected motion vector, the search for the new motion vector can be repeated. In this case, the search for the new motion vector can be limited based on the information on the number of times the motion correction has been performed.

[0361] As an example, if the motion compensation execution count information is 2, the search for a new motion vector can be limited to a total of two times.

[0362] The information on the number of times the motion compensation is performed can be signaled by being included in a bitstream. Alternatively, the number of times the motion compensation is performed can be a predetermined value that is determined in advance in both the encoder and the decoder.

[0363] As described above, the methods of correcting a motion vector based on an initial motion vector are roughly classified into a method using a template and a method using a predicted block.

[0364] The encoder / decoder can select a motion vector correction method using motion vector correction mode information (MV_REFINE_MODE) that indicates the motion vector correction method.

[0365] As an example, when the motion vector compensation mode information (MV_REFINE_MODE) is a first value "0", the motion vector can be compensated using a neighboring template of the current block. Conversely, when the motion vector compensation mode information (MV_REFINE_MODE) is a second value "1", the motion vector can be compensated using a predicted block obtained from the motion vector. Here, the first value and the second value may be set to 1 and 0, respectively.

[0366] Meanwhile, the motion vector compensation mode information can be signaled by being included in the bitstream.

[0367] On the other hand, the motion vector compensation method can be determined based on the coding parameters.

[0368] For example, if the initial motion vector of the current block has a unidirectional motion vector, the motion vector can be corrected using a surrounding template, and if the initial motion vector of the current block has a bidirectional vector, the motion vector can be corrected using a predicted block signal obtained from the motion vector.

[0369] The encoder / decoder can select a motion vector correction method using coding parameters without using motion vector correction mode information. For example, if only L0 or L1 motion vectors are available for the current block, the motion vector can be corrected using a surrounding template, and if both L0 and L1 motion vectors are available for the current block, the motion vector of each prediction direction can be corrected using a prediction block generated by weighting and summing prediction blocks obtained from the motion vectors as a template. As another example, if only L0 or L1 motion vectors are available for the current block, the motion vector can be corrected using a surrounding template, and if both L0 and L1 motion vectors are available for the current block, the motion vector of each prediction direction can be corrected using a difference between prediction blocks obtained from the motion vectors of each prediction direction.

[0370] On the other hand, when deriving a motion vector for the current block, the initial motion vector can be used as the motion vector for the current block without correction to the initial motion vector.

[0371] Next, the step of deriving a motion vector corrected on a sub-block basis will be specifically described.

[0372] A current block can be divided into one or more sub-blocks.

[0373] For example, the current block may be divided into M*N sub-blocks by dividing the current block horizontally into N blocks and vertically into M blocks, where M and N may each be any integer greater than or equal to 1. M and N may be the same or different integers.

[0374] Alternatively, the current block may be divided into W*H sub-blocks, each of which is made up of W pixels horizontally and H pixels vertically, where W and H may each be any integer greater than or equal to 2. W and H may be the same or different integers.

[0375] The size of the sub-block can be determined by the size of the current block.

[0376] For example, if the size of the current block is less than or equal to K samples, 4×4 sub-blocks may be used, and if the size of the current block is greater than N samples, 8×8 sub-blocks may be used, where K is a positive integer, e.g., 256.

[0377] Meanwhile, the size of the sub-blocks can be a size predefined in the encoder and decoder.

[0378] Meanwhile, the size of the sub-block may be determined based on size information of the sub-block, which may be signaled in a bitstream.

[0379] In addition, the size of the sub-block may be determined by taking into consideration both the size information of the sub-block and the size information of the current block.

[0380] For example, if the size information of the sub-block indicates the depth information of the sub-block, the size of the sub-block can be determined as follows using the horizontal / vertical sizes of the current block and the depth information of the sub-block.

[0381] Subblock size = (average of block length and width >> subblock depth information) For example, as shown in FIG. 26, if the average horizontal / vertical size of a block is 64 and the depth information of a sub-block is 3, the size of the sub-block can be determined to be 8.

[0382] The sub-blocks may be at least one of square and rectangular in shape.

[0383] For example, if the current block is square-shaped, the sub-block may be rectangular-shaped, as shown in Figure 27. If the current block is rectangular-shaped, the sub-block may be square-shaped.

[0384] In the encoder and decoder, an initial motion vector for sub-block-based motion vector correction can be set by at least one of the following methods. (1) An initial motion vector determined on a block-by-block basis or a corrected motion vector can be set as an initial motion vector on a sub-block-by-sub-block basis. (2) A zero motion vector (0, 0) can be set as the initial motion vector for each subblock. (3) A motion vector derived from a block reconstructed in a co-located picture that is a temporal periphery of a sub-block can be set as an initial motion vector for each sub-block. (4) The motion vector derived through the subblock-based merging can be set as the initial value of the subblock-based initial motion vector. (5) The motion vectors of the sub-blocks adjacent to the left and above of the current sub-block can be set as initial motion vectors for each sub-block. As an example, as shown in FIG 28, the motion vectors of the left sub-block (b), the lower left sub-block (a), the upper left sub-block (c), the upper sub-block (d), and the upper right sub-block (f) of the current sub-block (s) can be used as the initial motion vector of the current sub-block.

[0385] Of at least one sub-block unit initial motion vector set by the methods (1) to (5), only the initial motion vector having the same value as the reference image index information of the motion information determined on a block unit basis can be used.

[0386] However, if the motion information determined on a block-by-block basis does not match the reference image index information, a value scaled in consideration of the time distance can be used as the initial motion vector on a sub-block-by-subblock basis.

[0387] When determining an initial motion vector for motion vector correction on a sub-block basis, the reference image index information of the sub-block can be set to the same predetermined value.

[0388] For example, the reference image index information of all sub-blocks can be set to a zero value.

[0389] Therefore, if the reference image index information of the motion information determined on a block-by-block basis and the motion information derived from the periphery does not have a value of 0, the motion vector can be scaled taking into account the temporal distance and then used as the initial motion vector of the sub-block.

[0390] In the encoder and decoder, the derived initial motion vector can be used to perform motion vector correction on a sub-block basis.

[0391] The encoder and decoder can similarly use spatial and / or temporal neighboring blocks of the current block to perform correction of the initial motion vector of the sub-block using at least one of the above-mentioned template-based motion vector correction method and the prediction block-based motion vector correction method.

[0392] Meanwhile, information on whether or not to use motion vector compensation based on a merge mode that can be used in the above-mentioned motion vector compensation method, template size information, search area information, weight information, information on the number of times motion compensation is performed, motion vector compensation mode information, and sub-block size information may be included in information on motion compensation.

[0393] The entropy coding / decoding of information relating to motion compensation has been specifically described in steps S504 and S601 of FIG. 5 and FIG. 6, so a detailed description thereof will be omitted.

[0394] AMVP mode is applied to the current block instead of merge mode, and if there is at least one motion vector candidate, a corrected motion vector candidate can be derived using the generated motion vector candidate list.

[0395] For example, at least one of the motion vector of the reconstructed surrounding block, the motion vector of the co-located block, and the motion vector of the block adjacent to the co-located block is determined as a motion vector candidate, and when a motion vector candidate list is generated, a corrected motion vector can be derived through a motion vector correction method using the template by using the generated motion vector candidate as an initial motion vector.

[0396] Here, when there are N motion vector candidates, the motion vector candidate that minimizes the difference between the template in the reference image and the surrounding templates of the block to be encoded / decoded can be used as the corrected motion vector.

[0397] On the other hand, when using the determined compensated motion vector candidate, if the motion vector value is not the same as the first motion vector candidate in the motion vector candidate list, the determined compensated motion vector candidate can be used as the first candidate in the motion vector candidate list. In this case, the position of the existing motion vector candidate in the list can be increased by one step, and the Nth candidate can be removed from the list.

[0398] For example, when there is only one motion vector candidate determined from the motion vector of the reconstructed surrounding block, the motion vector of the co-located block, and the motion vector of the block adjacent to the co-located block, the motion vector candidate can be used as an initial motion vector to derive a corrected motion vector through the motion vector correction method using the template.

[0399] On the other hand, when using the determined corrected motion vector candidate, if the first motion vector candidate in the motion vector candidate list and the corrected motion vector value are not identical, the corrected motion vector candidate can be used as the second candidate in the motion vector candidate list.

[0400] On the other hand, when constructing the merge candidate list and the motion vector candidate list, if a spatial / temporal candidate block is encoded / decoded using the motion compensation method, the candidate does not need to be included in the list.

[0401] In addition, for blocks encoded / decoded through the motion compensation method, it is not necessary to further apply overlapped block motion compensation (OMBC) or bi-directional optical flow (BIO) motion compensation methods for bidirectional prediction.

[0402] In addition, local illumination compensation information for a block encoded / decoded through the motion compensation method can be derived from a candidate block corresponding to merge index information.

[0403] FIG. 29 is a flowchart showing an image decoding method according to one embodiment of the present invention.

[0404] Referring to FIG. 29, the decoder may derive an initial motion vector from the merge candidate list for the current block (S2910).

[0405] In this case, an initial motion vector can be derived from a merge candidate indicated by the merge index information in the merge candidate list.

[0406] Meanwhile, if the current block is a bi-predictable block and only motion information of one prediction direction exists in the merge candidate indicated by the merge index information, the motion vector of the existing motion information may be mirrored or scaled to be derived as an initial motion vector of the non-existent prediction direction. A detailed description of this has been described above with reference to Figures 12 and 13, and will not be provided here.

[0407] The decoder can then use the initial motion vector to derive a corrected motion vector (S2920).

[0408] Specifically, the step of deriving a corrected motion vector (S2920) may search for a motion vector indicating a template that minimizes the difference between the current block and surrounding templates within a search area of ​​a reference picture indicated by the initial motion vector, and derive the motion vector indicating the template that minimizes the difference as the corrected motion vector.

[0409] Here, the predetermined search area may be determined to be an area having at least one of a cross, a square, a diamond, and a hexagon shape centered on a pixel position of a reference picture indicated by the initial motion vector.

[0410] A motion vector indicating a template that minimizes a difference between neighboring templates of the current block within a search area of ​​a reference picture indicated by the initial motion vector can be searched based on at least one of a distortion value between the templates and a cost value for the motion vector.

[0411] Meanwhile, the step of deriving a corrected motion vector (S2920) may derive the corrected motion vector based on at least one predictive block indicated by the initial motion vector.

[0412] Here, when the current block is a bidirectional prediction block, a weighted sum of a first prediction block indicated by a first prediction direction initial motion vector and a second prediction block indicated by a second prediction direction initial motion vector is set as a template, and a search area of ​​at least one reference picture indicated by at least one of the first prediction direction initial motion vector and the second prediction direction initial motion vector is set, and a motion vector indicating a third prediction block that minimizes a difference from the template within the search area can be derived as a corrected motion vector. Here, when a first search area indicated by the first prediction direction initial motion vector and a second search area indicated by the second prediction direction initial motion vector are set, a corrected motion vector can be derived using motion vectors indicating respective prediction blocks that minimize a difference from the template within the first search area and the second search area.

[0413] On the other hand, if any of the first prediction direction initial motion vector or the second prediction direction initial motion vector does not exist, the non-existent initial motion vector can be derived by mirroring or scaling the existing initial motion vector.

[0414] Then, a prediction block of the current block can be generated using the corrected motion vector (S2930).

[0415] In the above-mentioned image decoding method, the current block may be a sub-block of the block to be decoded, in which case the motion vector may be compensated for in units of sub-blocks.

[0416] These embodiments may be implemented in a similar manner in the encoder and decoder.

[0417] The order of application of the above embodiments may be different between the encoder and the decoder, or may be the same between the encoder and the decoder.

[0418] The above embodiment can be performed for each of the luminance and color difference signals, and the above embodiment can be performed for the luminance and color difference signals in a similar manner.

[0419] The shape of the block to which the embodiment of the present invention is applied may be square or non-square.

[0420] The embodiments of the present invention may be applied according to the size of at least one of a coding block, a prediction block, a transformation block, a block, a current block, a coding unit, a prediction unit, a transformation unit, a unit, and a current unit. The size may be defined as a minimum size and / or a maximum size for applying the embodiments, or may be defined as a fixed size for applying the embodiments. The first embodiment may be applied to a first size, and the second embodiment may be applied to a second size. That is, the embodiments may be applied in a composite manner according to the size. The embodiments of the present invention may be applied only when the block size is equal to or larger than the minimum size and equal to or smaller than the maximum size. That is, the embodiments may be applied only when the block size is within a certain range.

[0421] For example, the embodiment is applicable only if the size of the current block is 8×8 or more. For example, the embodiment is applicable only if the size of the current block is 4×4. For example, the embodiment is applicable only if the size of the current block is 16×16 or less. For example, the embodiment is applicable only if the size of the current block is 16×16 or more and 64×64 or less.

[0422] The embodiments of the present invention may be applied according to a temporal layer. A separate identifier may be signaled to identify a temporal layer to which the embodiments are applicable, and the embodiments may be applied to a temporal layer identified by the identifier. The identifier may be defined as the lowest layer and / or the highest layer to which the embodiments are applicable, or may be defined to indicate a specific layer to which the embodiments are applied. A fixed temporal layer to which the embodiments are applied may also be defined.

[0423] For example, the above embodiment is applicable only when the temporal hierarchy of the current image is the lowest hierarchy, for example, the above embodiment is applicable only when the temporal hierarchy identifier of the current image is equal to or greater than 1, for example, the above embodiment is applicable only when the temporal hierarchy of the current image is the highest hierarchy.

[0424] A slice type to which the embodiment of the present invention is applied is defined, and the embodiment of the present invention can be applied according to the slice type.

[0425] In the above-mentioned embodiments, the methods are described based on the flowcharts in a series of steps or units, but the present invention is not limited to the order of these steps, and some steps may occur in a different order or simultaneously with other steps than those described above. Also, those skilled in the art will understand that the steps shown in the flowcharts are not exclusive, and other steps may be included, or one or more steps of the flowcharts may be deleted without affecting the scope of the present invention.

[0426] The above-described embodiments include examples of various aspects. It is not possible to describe all possible combinations for illustrating the various aspects, but a person having ordinary skill in the art will recognize that other combinations are possible. Therefore, it can be said that the present invention includes all various alterations, modifications, and variations that fall within the scope of the following claims.

[0427] The above-described embodiments of the present invention may be realized in the form of program instructions that can be executed by various computer components and recorded on a computer-readable recording medium. The computer-readable recording medium may include, alone or in combination, program instructions, data files, data structures, and the like. The program instructions recorded on the computer-readable recording medium may be those specially designed and constructed for the present invention, or those known and available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and hardware devices specially configured to store and execute program instructions, such as ROMs, RAMs, and flash memories. Examples of program instructions include not only machine language code produced by a compiler, but also high-level language code that can be executed by a computer using an interpreter, etc. The hardware devices may be configured to operate as one or more software modules to perform the processes of the present invention, and vice versa.

[0428] Although the present invention has been described above by using specific details such as specific components, limited embodiments, and drawings, these are merely provided to facilitate a more general understanding of the present invention, and the present invention is not limited to the above embodiments. A person having ordinary knowledge in the technical field to which the present invention pertains can make various modifications and variations from such descriptions.

[0429] Therefore, the concept of the present invention should not be limited to the above-described embodiments, and all modifications equivalent to or equivalent to the scope of the claims, as well as the claims set forth below, fall within the scope of the concept of the present invention. [Industrial Applicability]

[0430] The present invention can be used in devices for encoding / decoding images.

Claims

1. deriving an initial motion vector for a first prediction direction and an initial motion vector for a second prediction direction from a merging candidate list for the current block; deriving a search area of ​​the first prediction direction and a search area of ​​the second prediction direction in a sub-block unit of the current block based on an initial motion vector of the first prediction direction and an initial motion vector of the second prediction direction, respectively; deriving a corrected motion vector of the first prediction direction and a corrected motion vector of the second prediction direction in a sub-block unit of the current block based on a search area of ​​the first prediction direction and a search area of ​​the second prediction direction; Including, the first prediction direction is different from the second prediction direction; the search area of ​​the first prediction direction is a square area centered on a pixel position of a reference picture indicated by an initial motion vector of the first prediction direction; the search area of ​​the second prediction direction is a square area centered on a pixel position of a reference picture indicated by an initial motion vector of the second prediction direction, The derived corrected motion vector is not added to the merging candidates. Image decoding method.

2. the corrected motion vector of the first prediction direction and the corrected motion vector of the second prediction direction are derived based on a difference between a prediction block in a search area of ​​the first prediction direction and a prediction block in a search area of ​​the second prediction direction. The image decoding method according to claim 1 .

3. The difference is derived based on a sum of absolute differences (SAD). The image decoding method according to claim 2 .

4. At least one of the prediction block in the search area of ​​the first prediction direction and the prediction block in the search area of ​​the second prediction direction is derived by applying a bi-linear filter. The image decoding method according to claim 2 .

5. the corrected motion vector of the second prediction direction is derived based on the corrected motion vector of the first prediction direction. The image decoding method according to claim 1 .

6. a size of the sub-block of the current block is determined based on a size of the current block; The image decoding method according to claim 1 .

7. deriving an initial motion vector for a first prediction direction and an initial motion vector for a second prediction direction from a merging candidate list for the current block; deriving a search area of ​​the first prediction direction and a search area of ​​the second prediction direction in a sub-block unit of the current block based on an initial motion vector of the first prediction direction and an initial motion vector of the second prediction direction, respectively; deriving a corrected motion vector of the first prediction direction and a corrected motion vector of the second prediction direction in a sub-block unit of the current block based on a search area of ​​the first prediction direction and a search area of ​​the second prediction direction; Including, the first prediction direction is different from the second prediction direction; the search area of ​​the first prediction direction is a square area centered on a pixel position of a reference picture indicated by an initial motion vector of the first prediction direction; the search area of ​​the second prediction direction is a square area centered on a pixel position of a reference picture indicated by an initial motion vector of the second prediction direction, The derived corrected motion vector is not added to the merging candidates. Image encoding method.

Images