Image encoding / decoding apparatus and apparatus for transmitting image data
By performing motion compensation in units of sub-blocks in the image encoding/decoding method, using initial motion information and distortion value correction, the problem of limited encoding efficiency in the traditional method is solved, and the compression efficiency and decoding efficiency of image data are improved.
Patent Information
- Application Number
- CN202411300002.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2018-09-18
- Filing Date
- 2019-03-13
- Publication Date
- 2024-11-15
AI Technical Summary
Traditional image encoding/decoding methods perform motion compensation based on blocks, resulting in limited encoding efficiency and decoding efficiency, especially when processing high resolution and high quality image data, with increased costs.
By deducing initial motion information from spatial neighbor blocks, temporal neighbor blocks and predefined motion information, corrected motion information is generated, and motion compensation is performed in units of sub-blocks, information correction is performed using bilateral templates and distortion value calculations.
The encoding efficiency and decoding efficiency of images are improved, the calculation degree of image encoder and decoder is reduced, and more efficient image data compression is achieved.
Smart Images

Figure CN118972614A_ABST
Abstract
Description
[0001] This application is a divisional application of the invention patent application with the original application number 201980019167.3 (International application number: PCT / KR2019 / 002875, application date: March 13, 2019, invention name: Method and device for encoding / decoding an image and a recording medium storing a bit stream). Technical Field
[0002] The present invention relates to an image encoding / decoding method and apparatus and a recording medium storing a bit stream. More particularly, the present invention relates to an image encoding / decoding method and apparatus for performing motion compensation by correcting motion information. Background Art
[0003] Recently, the demand for high-resolution and high-quality images such as high-definition (HD) images and ultra-high-definition (UHD) images has increased in various application fields. However, compared with traditional image data, higher resolution and quality image data has an increasing amount of data. Therefore, when image data is transmitted by using a medium such as a traditional wired and wireless broadband network, or when image data is stored by using a traditional storage medium, the cost of transmission and storage increases. In order to solve these problems occurring as the resolution and quality of image data increase, efficient image encoding / decoding technology is required for higher resolution and higher quality images.
[0004] The image compression technology includes various technologies, including: an inter-frame prediction technology for predicting pixel values included in a current picture from a previous picture or a subsequent picture of the current picture; an intra-frame prediction technology for predicting pixel values included in the current picture by using pixel information in the current picture; a transform and quantization technology for compressing energy of a residual signal; an entropy coding technology for assigning short codes to values with high occurrence frequencies and assigning long codes to values with low occurrence frequencies; etc. Image data can be effectively compressed by using such image compression technology, and the image data can be transmitted or stored.
[0005] Since the conventional methods and apparatuses perform motion compensation for an encoding / decoding target block based on motion information of spatial / temporal neighboring blocks, the conventional image encoding / decoding methods and apparatuses have their limitations in improving encoding efficiency.
[0006] Furthermore, since the conventional methods and apparatuses perform motion compensation in units of blocks, the conventional image encoding / decoding methods and apparatuses have their limitations in improving encoding efficiency. Summary of the invention
[0007] Technical issues
[0008] The present invention provides an encoding / decoding method and a device for performing motion compensation by modifying motion information.
[0009] The present invention provides a coding / decoding method and a device for performing motion compensation in sub-block units.
[0010] Technical Solution
[0011] According to a method for decoding an image of the present invention, the method may include: deriving initial motion information of a current block from at least one of motion information of a spatially neighboring block, motion information of a temporally neighboring block, and predefined motion information; generating corrected motion information by performing motion information correction on the initial motion information; and generating a prediction block of the current block by using the corrected motion information.
[0012] In a method for decoding an image, the step of generating modified motion information includes: generating a bilateral template based on the initial motion information, calculating a distortion value by comparing the bilateral template with a search area within a reference image indicated by the initial motion information, and generating the modified motion information by modifying the initial motion information based on the distortion value.
[0013] In the method for decoding an image, in the step of generating the modified motion information, the modified motion information is generated in units of sub-blocks of the current block.
[0014] In the method of decoding an image, wherein the search area is an area of a square shape and is defined with a pixel range of a predefined integer pixel unit in a horizontal direction and a vertical direction from a pixel indicated by the initial motion information.
[0015] In the method for decoding an image, wherein when the current block has a bi-prediction merge mode, a step of generating modified motion information is performed.
[0016] In the method for decoding an image, the step of generating the modified motion information is performed based on the size of the current block.
[0017] In the method for decoding an image, the step of generating the corrected motion information is performed according to the motion correction mode using information determination.
[0018] In a method for decoding an image, the distortion value is calculated based on at least one of the sum of absolute differences (SAD), the sum of absolute transformed differences (SATD), the sum of square errors (SSE), the mean of squared errors (MSE) and the mean removed SAD (MR-SAD).
[0019] In the method for decoding an image, in the step of generating the modified motion information, the initial motion information is modified within the search area.
[0020] In the method for decoding an image, wherein the search area is cross-shaped.
[0021] In the method for decoding an image, wherein when the current block has a merge mode, a step of generating modified motion information is performed.
[0022] In the method for decoding an image, when the initial motion information is modified, the initial motion information is modified within the search area based on at least one of information on a search direction and information on a search distance.
[0023] According to a method for decoding an image of the present invention, the method may include: determining a co-located image of a current block; determining a co-located block in the co-located image by using at least one motion information of a block spatially adjacent to the current block; deriving merge candidates in units of sub-blocks from sub-blocks obtained by dividing the co-located block; and generating a merge candidate list by using the merge candidates.
[0024] In the method of decoding an image, wherein the step of determining a co-located image of a current block is performed based on information indicating the co-located image, wherein the information indicating the co-located image is signaled in units of at least one of a sequence, a picture, and a slice group.
[0025] In the method for decoding an image, when the size of the current block is equal to or greater than a preset value, the merging candidate in units of sub-blocks is derived.
[0026] In the method for decoding an image, the sub-block has a square shape, wherein the square shape has a preset size.
[0027] According to a method for encoding an image of the present invention, the method may include: determining a co-located image of a current block; determining a co-located block in the co-located image by using at least one motion information of a block spatially adjacent to the current block; deriving merge candidates in units of sub-blocks from sub-blocks obtained by dividing the co-located block; and generating a merge candidate list by using the merge candidates in units of sub-blocks.
[0028] According to a non-temporary storage medium of the present invention, the non-temporary storage medium includes a bit stream generated by using an image encoding method, wherein the image encoding method includes: determining a co-located image of a current block; determining a co-located block in the co-located image by using at least one motion information of a block spatially adjacent to the current block; deriving merge candidates in units of sub-blocks from sub-blocks obtained by dividing the co-located block; and generating a merge candidate list by using the merge candidates in units of sub-blocks.
[0029] Beneficial Effects
[0030] According to the present invention, an image encoding / decoding method and apparatus with improved compression efficiency can be provided.
[0031] According to the present invention, the encoding efficiency and decoding efficiency of an image can be improved.
[0032] According to the present invention, the degree of calculation of texture of an image encoder and an image decoder can be reduced. BRIEF DESCRIPTION OF THE DRAWINGS
[0033] Figure 1 is a block diagram showing the configuration of an encoding device according to an embodiment of the present invention.
[0034] Figure 2 is a block diagram showing the configuration of a decoding device according to an embodiment of the present invention.
[0035] Figure 3 is a diagram schematically showing a partition structure of an image when the image is encoded and decoded.
[0036] Figure 4 is a diagram illustrating an inter-frame prediction process.
[0037] Figure 5 is a flowchart illustrating an image encoding method according to an embodiment of the present invention.
[0038] Figure 6 is a flowchart illustrating an image decoding method according to an embodiment of the present invention.
[0039] Figure 7 is a flowchart illustrating an image encoding method according to another embodiment of the present invention.
[0040] Figure 8 is a flowchart illustrating an image decoding method according to another embodiment of the present invention.
[0041] Fig. 9 is a diagram illustrating an exemplary method of deriving spatial motion vector candidates for a current block.
[0042] Fig.10is a diagram illustrating an exemplary method of deriving a temporal motion vector candidate of a current block.
[0043] Fig.11 is a diagram showing an example in which a spatial merging candidate is added to a merging candidate list.
[0044] Fig.12 is a diagram illustrating an example in which a temporal merge candidate is added to a merge candidate list.
[0045] Fig.13 is a diagram illustrating a method of predicting motion information in units of a first sub-block according to an embodiment of the present invention.
[0046] Fig.14 is a diagram illustrating a method of predicting motion information in units of second sub-blocks according to an embodiment of the present invention.
[0047] Fig.15 is a diagram illustrating bilateral template matching.
[0048] Figures 16 to 21 is a diagram showing an area for searching for a corrected motion vector in bilateral template matching.
[0049] Fig. 22 is a flowchart illustrating an image decoding method according to an embodiment of the present invention.
[0050] Fig.23 is a flowchart illustrating an image decoding method according to an embodiment of the present invention. DETAILED DESCRIPTION
[0051] Various modifications may be made to the present invention, and there are various embodiments, examples of the present invention that will now be provided and described in detail with reference to the accompanying drawings. However, although the exemplary embodiments may be interpreted as including all modifications, equivalents or substitutions within the technical concept and technical scope of the present invention, the present invention is not limited thereto. Similar figure numerals refer to the same or similar functions in various aspects. In the accompanying drawings, the shapes and sizes of the elements may be exaggerated for clarity. In the following detailed description of the present invention, reference is made to the accompanying drawings showing specific embodiments in which the present invention can be practiced by way of illustration. These embodiments are described in sufficient detail to enable those skilled in the art to implement the present disclosure. It should be understood that the various embodiments of the present disclosure are different, but not necessarily mutually exclusive. For example, without departing from the spirit and scope of the present disclosure, the specific features, structures and characteristics described herein in conjunction with one embodiment may be implemented in other embodiments. In addition, it should be understood that the position or arrangement of each element within each disclosed embodiment may be modified without departing from the spirit and scope of the present disclosure. Therefore, the following detailed description should not be considered as limiting, and the scope of the present disclosure is limited only by the full range of the attached claims and the equivalents claimed by the claims, which are properly interpreted.
[0052] The terms "first", "second", etc. used in the specification may be used to describe various components, but the components should not be interpreted as limited to the terms. The terms are only used to distinguish one component from other components. For example, without departing from the scope of the present invention, the "first" component may be referred to as the "second" component, and the "second" component may also be similarly referred to as the "first" component. The term "and / or" includes a combination of multiple items or any one of the multiple terms.
[0053] It will be understood that when an element is simply referred to as being “connected to” or “coupled to” another element in this specification rather than being “directly connected to” or “directly coupled to” another element, it can be “directly connected to” or “directly coupled to” another element, or connected to or coupled to another element with other elements intervening therebetween. Conversely, it will be understood that when an element is referred to as being “directly coupled to” or “directly connected to” another element, there are no intervening elements.
[0054] In addition, the components shown in the embodiments of the present invention are shown independently to represent the characteristic functions that are different from each other. Therefore, this does not mean that each component is constituted with separated hardware or software as a constituent unit. In other words, for convenience, each component includes each of the listed components. Therefore, at least two components in each component can be combined to form a component, or a component can be divided into multiple components to perform each function. Without departing from the essence of the present invention, the embodiments in which each component is combined and the embodiments in which a component is divided are also included in the scope of the present invention.
[0055] The terms used in this specification are only used to describe specific embodiments and are not intended to limit the present invention. Expressions used in the singular form cover expressions in the plural form unless it has a significantly different meaning in the context. In this specification, it is to be understood that terms such as "including", "having" and the like are intended to indicate the existence of features, numbers, steps, actions, elements, parts or combinations thereof disclosed in the specification, and are not intended to exclude the possibility that one or more other features, numbers, steps, actions, elements, parts or combinations thereof may exist or may be added. In other words, when a particular element is referred to as "included in", elements other than the corresponding element are not excluded, but additional elements may be included in embodiments of the present invention or in the scope of the present invention.
[0056] In addition, some components may not be indispensable components for performing the necessary functions of the present invention, but selective components that only improve the performance thereof. The present invention may be realized by only including the indispensable constituent parts for realizing the essence of the present invention except the components for improving the performance. A structure including only the indispensable components except the selective components for only improving the performance is also included in the scope of the present invention.
[0057] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. When describing the exemplary embodiments of the present invention, well-known functions or structures may unnecessarily obscure the understanding of the present invention, so well-known functions or structures will not be described in detail. The same constituent elements in the accompanying drawings are represented by the same reference numerals, and repeated descriptions of the same elements will be omitted.
[0058] Hereinafter, an image may refer to a picture configuring a video, or may refer to a video itself. For example, "encoding or decoding of an image, or both" may refer to "encoding or decoding of a moving picture, or both encoding and decoding", and may refer to "encoding or decoding of one image among a plurality of images of a moving picture, or both encoding and decoding".
[0059] Hereinafter, the terms "moving image" and "video" may be used as the same meaning and replaced with each other.
[0060] Hereinafter, a target image may be an encoding target image as a target of encoding and / or a decoding target image as a target of decoding. In addition, the target image may be an input image input to an encoding device and an input image input to a decoding device. Here, the target image may have the same meaning as the current image.
[0061] Hereinafter, the terms "image", "picture", "frame" and "screen" may be used as the same meaning and replaced with each other.
[0062] Hereinafter, the target block may be an encoding target block as a target of encoding and / or a decoding target block as a target of decoding. In addition, the target block may be a current block as a target of current encoding and / or decoding. For example, the terms "target block" and "current block" may be used as the same meaning and may be replaced with each other.
[0063] Hereinafter, the terms "block" and "unit" may be used as the same meaning and replaced with each other. Alternatively, a "block" may refer to a specific unit.
[0064] In the following, the terms "region" and "segment" may be used interchangeably.
[0065] Hereinafter, a specific signal may be a signal representing a specific block. For example, an original signal may be a signal representing a target block. A prediction signal may be a signal representing a prediction block. A residual signal may be a signal representing a residual block.
[0066] In an embodiment, each of the specific information, data, flags, indexes, elements, attributes, etc. may have a value. The value of the information, data, flags, indexes, elements, and attributes equal to "0" may represent a logical false or a first predefined value. In other words, the value "0", false, logical false, and the first predefined value may be replaced with each other. The value of the information, data, flags, indexes, elements, and attributes equal to "1" may represent a logical true or a second predefined value. In other words, the value "1", true, logical true, and the second predefined value may be replaced with each other.
[0067] When the variable i or j is used to represent a column, row, or index, the value of i may be an integer equal to or greater than 0 or equal to or greater than 1. That is, the column, row, index, etc. may be counted from 0 or may be counted from 1.
[0068] Description of terms
[0069] Encoder: This refers to the device that performs encoding. In other words, it refers to the encoding device.
[0070] Decoder: refers to a device that performs decoding. In other words, it refers to a decoding device.
[0071] Block: is an M×N array of samples. Here, M and N may represent positive integers, and a block may represent an array of samples in a two-dimensional form. A block may refer to a unit. A current block may represent an encoding target block that becomes a target at the time of encoding, or a decoding target block that becomes a target at the time of decoding. In addition, the current block may be at least one of a coding block, a prediction block, a residual block, and a transform block.
[0072] Sample: It is the basic unit of a block. Samples can be expressed from 0 to 2 according to the bit depth (Bd). Bd In the present invention, a sample point may be used as the meaning of a pixel. That is, a sample point, a pel, and a pixel may have the same meaning as each other.
[0073] Unit: may refer to a coding unit and a decoding unit. When encoding and decoding an image, a unit may be a region generated by partitioning a single image. In addition, when a single image is partitioned into subdivided units during encoding or decoding, a unit may represent a subdivided unit. That is, an image may be partitioned into a plurality of units. When encoding and decoding an image, a predetermined process for each unit may be performed. A single unit may be partitioned into sub-units having a size smaller than that of the unit. According to the function, a unit may represent a block, a macroblock, a coding tree unit, a coding tree block, a coding unit, a coding block, a prediction unit, a prediction block, a residual unit, a residual block, a transform unit, a transform block, and the like. In addition, in order to distinguish a unit from a block, a unit may include a luminance component block, a chrominance component block associated with the luminance component block, and a syntax element for each color component block. A unit may have various sizes and forms, and in particular, the form of a unit may be a two-dimensional geometric figure such as a square shape, a rectangular shape, a trapezoidal shape, a triangular shape, a pentagonal shape, and the like. Also, the unit information may include at least one of a unit type indicating a coding unit, a prediction unit, a transformation unit, etc., and a unit size, a unit depth, an order of encoding and decoding of the unit, and the like.
[0074] Coding tree unit: A single coding tree block configured with a luminance component Y and two coding tree blocks associated with chrominance components Cb and Cr. In addition, the coding tree unit may represent a syntax element including a block and each block. Each coding tree unit may be partitioned by using at least one of a quadtree partitioning method, a binary tree partitioning method, and a ternary tree partitioning method to configure lower-level units such as coding units, prediction units, transform units, and the like. The coding tree unit may be used as a term for designating a sample block that becomes a processing unit when encoding / decoding an image as an input image. Here, the quadtree may represent a quaternary tree.
[0075] If the size of the coding block is within a predetermined range, it can be divided only into quadtrees. Here, the predetermined range may be defined as at least one of the maximum size and the minimum size of the coding block that can be divided only into quadtrees. Information indicating the maximum / minimum size of the coding block that allows quadtree type partitioning may be signaled by a bitstream, and the information may be represented in units of at least one of a sequence, a picture parameter, a parallel block group, or a slice. Optionally, the maximum / minimum size of the coding block may be a fixed size pre-set in the codec. For example, if the size of the coding block corresponds to 256×256 to 64×64, it may be divided only into quadtrees. If the size of the coding block is larger than the size of the maximum transform block, it may be partitioned only into quadtrees. In this case, the block to be divided may be at least one of a coding block and a transform block. In this case, information indicating the division of the coding block (e.g., split_flag) may be a flag indicating whether it is divided into a quadtree. When the size of the coding block falls within a predetermined range, it may be divided only into a binary tree or a ternary tree. In this case, the above description of the quadtree may be applied to a binary tree or a ternary tree in the same manner.
[0076] Coding tree block: may be used as a term for designating any one of a Y coding tree block, a Cb coding tree block, and a Cr coding tree block.
[0077] Neighboring block: may refer to a block adjacent to the current block. The block adjacent to the current block may refer to a block that touches the boundary of the current block or a block that is within a predetermined distance from the current block. The neighboring block may refer to a block adjacent to a vertex of the current block. Here, the block adjacent to a vertex of the current block may refer to a block that is vertically adjacent to a neighboring block that is horizontally adjacent to the current block or a block that is horizontally adjacent to a neighboring block that is vertically adjacent to the current block.
[0078] Reconstructed neighboring block: may refer to a neighboring block that is adjacent to the current block and has been encoded or decoded in space / time. Here, the reconstructed neighboring block may refer to a reconstructed neighboring unit. The reconstructed spatial neighboring block may be a block within the current picture that has been reconstructed by encoding or decoding or both encoding and decoding. The reconstructed temporal neighboring block is a block at a position corresponding to the current block of the current picture within the reference image or a neighboring block of the block.
[0079] Unit depth: can represent the degree of partitioning of a unit. In a tree structure, the highest node (root node) can correspond to the first unit that is not partitioned. In addition, the highest node can have a minimum depth value. In this case, the highest node can have a depth of level 0. A node with a depth of level 1 can represent a unit generated by partitioning the first unit once. A node with a depth of level 2 can represent a unit generated by partitioning the first unit twice. A node with a depth of level n can represent a unit generated by partitioning the first unit n times. A leaf node can be the lowest node and a node that cannot be further partitioned. The depth of a leaf node can be the maximum level. For example, the predefined value of the maximum level can be 3. The depth of the root node can be the lowest, and the depth of the leaf node can be the deepest. In addition, when a unit is expressed as a tree structure, the level at which the unit is located can represent the unit depth.
[0080] Bitstream: can represent a stream of bits including coded image information.
[0081] Parameter set: corresponds to header information in the configuration within the bitstream. At least one of a video parameter set, a sequence parameter set, a picture parameter set, and an adaptation parameter set may be included in the parameter set. In addition, the parameter set may include slice header and parallel block header information.
[0082] Parsing: may mean determining the value of a syntax element by performing entropy decoding, or may mean the entropy decoding itself.
[0083] Symbol: can represent at least one of a syntax element, a coding parameter, and a transform coefficient value of a coding / decoding target unit. In addition, the symbol can represent an entropy coding target or an entropy decoding result.
[0084] Prediction mode: may be information indicating a mode of encoding / decoding using intra-frame prediction or a mode of encoding / decoding using inter-frame prediction.
[0085] Prediction unit: may represent a basic unit when performing prediction (such as inter prediction, intra prediction, inter compensation, intra compensation, and motion compensation). A single prediction unit may be partitioned into multiple partitions of smaller size, or may be partitioned into multiple prediction units of lower layers. Multiple partitions may be basic units when performing prediction or compensation. Partitions generated by splitting a prediction unit may also be prediction units.
[0086] Prediction unit partition: may represent a form obtained by partitioning a prediction unit.
[0087] Reference picture list: may refer to a list including one or more reference pictures used for inter prediction or motion compensation. There are multiple types of available reference picture lists, including LC (List Combination), L0 (List 0), L1 (List 1), L2 (List 2), L3 (List 3).
[0088] Inter prediction indicator: may refer to the direction of inter prediction (unidirectional prediction, bidirectional prediction, etc.) of the current block. Optionally, the inter prediction indicator may refer to the number of reference pictures used to generate the prediction block of the current block. Optionally, the inter prediction indicator may refer to the number of prediction blocks used when performing inter prediction or motion compensation on the current block.
[0089] Prediction list utilization flag: indicates whether at least one reference picture in a specific reference picture list is used to generate a prediction block. The prediction list utilization flag may be used to derive the inter prediction indicator, and conversely, the inter prediction indicator may be used to derive the prediction list utilization flag. For example, when the prediction list utilization flag has a first value of zero (0), this indicates that the reference picture in the reference picture list is not used to generate the prediction block. On the other hand, when the prediction list utilization flag has a second value of one (1), this indicates that the reference picture list is used to generate the prediction block.
[0090] Reference picture index: may refer to an index indicating a specific reference picture in a reference picture list.
[0091] Reference picture: may refer to a reference picture referenced by a specific block for the purpose of inter-frame prediction or motion compensation of the specific block. Alternatively, the reference picture may be a picture including a reference block referenced by the current block for inter-frame prediction or motion compensation. Hereinafter, the terms "reference picture" and "reference image" have the same meaning and may be interchanged.
[0092] Motion vector: It can be a two-dimensional vector used for inter-frame prediction or motion compensation. The motion vector can represent the offset between the encoding / decoding target block and the reference block. For example, (mvX, mvY) can represent a motion vector. Here, mvX can represent the horizontal component, and mvY can represent the vertical component.
[0093] Search range: may be a two-dimensional area that is searched to retrieve a motion vector during inter prediction. For example, the size of the search range may be M×N. Here, M and N are both integers.
[0094] Motion vector candidate: may refer to a prediction candidate block or a motion vector of a prediction candidate block when predicting a motion vector. In addition, the motion vector candidate may be included in a motion vector candidate list.
[0095] Motion vector candidate list: may represent a list consisting of one or more motion vector candidates.
[0096] Motion vector candidate index: may represent an indicator indicating a motion vector candidate in a motion vector candidate list. Alternatively, the motion vector candidate index may be an index of a motion vector predictor.
[0097] Motion information: may represent information including at least one of the following items: motion vector, reference picture index, inter prediction indicator, prediction list utilization flag, reference picture list information, reference picture, motion vector candidate, motion vector candidate index, merge candidate and merge index.
[0098] Merge candidate list: may represent a list consisting of one or more merge candidates.
[0099] Merge candidate: may represent spatial merge candidate, temporal merge candidate, combined merge candidate, combined bi-prediction merge candidate or zero merge candidate. Merge candidate may include motion information such as inter prediction indicator, reference picture index for each list, motion vector, prediction list utilization flag and inter prediction indicator.
[0100] Merge index: may represent an indicator indicating a merge candidate in a merge candidate list. Optionally, the merge index may indicate a block from which a merge candidate has been derived in a reconstructed block that is spatially / temporally adjacent to the current block. Optionally, the merge index may indicate at least one piece of motion information of the merge candidate.
[0101] Transform unit: may represent a basic unit when performing encoding / decoding (such as transformation, inverse transformation, quantization, inverse quantization, and transformation coefficient encoding / decoding of a residual signal). A single transform unit may be partitioned into a plurality of lower-level transform units having a smaller size. Here, the transform / inverse transform may include at least one of a primary transform / primary inverse transform and a secondary transform / secondary inverse transform.
[0102] Scaling: may refer to the process of multiplying the level of quantization by a factor. Transform coefficients may be generated by scaling the level of quantization. Scaling may also be referred to as inverse quantization.
[0103] Quantization parameter: may indicate a value used when a transform coefficient is used to generate a quantized level during quantization. The quantization parameter may also indicate a value used when a transform coefficient is generated by scaling the quantized level during inverse quantization. The quantization parameter may be a value mapped on a quantization step size.
[0104] Delta quantization parameter: may represent the difference between the predicted quantization parameter and the quantization parameter of the encoding / decoding target unit.
[0105] Scan: may refer to a method of ordering coefficients within a cell, block, or matrix. For example, changing a two-dimensional matrix of coefficients to a one-dimensional matrix may be referred to as scanning, and changing a one-dimensional matrix of coefficients to a two-dimensional matrix may be referred to as scanning or inverse scanning.
[0106] Transform coefficient: may refer to a coefficient value generated after performing a transform in an encoder. A transform coefficient refers to a coefficient value generated after performing at least one of entropy decoding and inverse quantization in a decoder. A quantization level obtained by quantizing a transform coefficient or a residual signal or a quantized transform coefficient level may also fall within the meaning of a transform coefficient.
[0107] Quantization level: may represent a value generated by quantizing a transform coefficient or a residual signal in an encoder. Alternatively, the quantization level may represent a value that is a target of inverse quantization that undergoes inverse quantization in a decoder. Similarly, the quantized transform coefficient level as a result of transformation and quantization may also fall within the meaning of the quantization level.
[0108] Non-zero transform coefficient: may refer to a transform coefficient having a value other than zero or a transform coefficient level or quantization level having a value other than zero.
[0109] Quantization matrix: may refer to a matrix used in a quantization process or an inverse quantization process performed to improve subjective or objective image quality. A quantization matrix may also be referred to as a scaling list.
[0110] Quantization matrix coefficients: can represent each element in the quantization matrix. Quantization matrix coefficients can also be called matrix coefficients.
[0111] Default matrix: may represent a predetermined quantization matrix predefined in an encoder or a decoder.
[0112] Non-default matrix: may represent a quantization matrix that is not predefined in the encoder or decoder but is signaled by the user.
[0113] Statistical value: The statistical value for at least one of a variable, a coding parameter, a constant value, etc. having a calculable specific value can be one or more of the average value, weighted average value, weighted sum value, minimum value, maximum value, most frequent value, median value, and interpolated value of the corresponding specific value.
[0114] Figure 1 is a block diagram showing a configuration of an encoding device according to an embodiment to which the present invention is applied.
[0115] The encoding device 100 may be an encoder, a video encoding device, or an image encoding device. A video may include at least one image. The encoding device 100 may sequentially encode at least one image.
[0116] Reference Figure 1, the encoding device 100 may include a motion prediction unit 111, a motion compensation unit 112, an intra-frame prediction unit 120, a switch 115, a subtractor 125, a transform unit 130, a quantization unit 140, an entropy encoding unit 150, an inverse quantization unit 160, an inverse transform unit 170, an adder 175, a filter unit 180 and a reference picture buffer 190.
[0117] The encoding device 100 may perform encoding of an input image by using an intra mode or an inter mode or both an intra mode and an inter mode. In addition, the encoding device 100 may generate a bit stream including encoded information by encoding the input image, and output the generated bit stream. The generated bit stream may be stored in a computer-readable recording medium, or may be streamed through a wired / wireless transmission medium. When the intra mode is used as a prediction mode, the switch 115 may be switched to the intra mode. Alternatively, when the inter mode is used as a prediction mode, the switch 115 may be switched to the inter mode. Here, the intra mode may represent an intra prediction mode, and the inter mode may represent an inter prediction mode. The encoding device 100 may generate a prediction block for an input block of the input image. In addition, the encoding device 100 may encode the residual block using the residual of the input block and the prediction block after the prediction block is generated. The input image may be referred to as a current image as a current encoding target. The input block may be referred to as a current block as a current encoding target, or may be referred to as an encoding target block.
[0118] When the prediction mode is the intra mode, the intra prediction unit 120 may use samples of a block that has been encoded / decoded and is adjacent to the current block as reference samples. The intra prediction unit 120 may perform spatial prediction for the current block using the reference samples, or may generate prediction samples of the input block by performing spatial prediction. Here, intra prediction may mean intra-prediction.
[0119] When the prediction mode is the inter mode, the motion prediction unit 111 may retrieve the area that best matches the input block from the reference image when performing motion prediction, and derive the motion vector by using the retrieved area. In this case, the search area may be used as the area. The reference image may be stored in the reference picture buffer 190. Here, when encoding / decoding for the reference image is performed, the reference image may be stored in the reference picture buffer 190.
[0120] The motion compensation unit 112 may generate a prediction block by performing motion compensation on the current block using a motion vector. Here, inter prediction may mean inter-prediction or motion compensation.
[0121] When the value of the motion vector is not an integer, the motion prediction unit 111 and the motion compensation unit 112 may generate a prediction block by applying an interpolation filter to a partial area of a reference picture. In order to perform inter-picture prediction or motion compensation on a coding unit, it may be determined which mode of a skip mode, a merge mode, an advanced motion vector prediction (AMVP) mode, and a current picture reference mode is used for motion prediction and motion compensation of a prediction unit included in the corresponding coding unit. Then, inter-picture prediction or motion compensation may be performed differently according to the determined mode.
[0122] The subtractor 125 may generate a residual block by using the residual of the input block and the prediction block. The residual block may be referred to as a residual signal. The residual signal may represent the difference between the original signal and the prediction signal. In addition, the residual signal may be a signal generated by transforming or quantizing the difference between the original signal and the prediction signal, or by transforming and quantizing. The residual block may be a residual signal of a block unit.
[0123] The transform unit 130 may generate a transform coefficient by performing a transform of the residual block and output the generated transform coefficient. Here, the transform coefficient may be a coefficient value generated by performing a transform of the residual block. When the transform skip mode is applied, the transform unit 130 may skip the transform of the residual block.
[0124] The quantized level may be generated by applying quantization to a transform coefficient or a residual signal. Hereinafter, the quantized level may also be referred to as a transform coefficient in an embodiment.
[0125] The quantization unit 140 may generate a quantization level by quantizing the transform coefficient or the residual signal according to the parameter and output the generated quantization level. Here, the quantization unit 140 may quantize the transform coefficient by using a quantization matrix.
[0126] The entropy encoding unit 150 may generate a bitstream by performing entropy encoding on the value calculated by the quantization unit 140 or the encoding parameter value calculated when encoding is performed according to the probability distribution, and output the generated bitstream. The entropy encoding unit 150 may perform entropy encoding on sample information of the image and information for decoding the image. For example, the information for decoding the image may include a syntax element.
[0127] When entropy coding is applied, symbols are represented so that a smaller number of bits are allocated to symbols with a high chance of generation and a larger number of bits are allocated to symbols with a low chance of generation, and therefore, the size of the bitstream for the symbol to be encoded can be reduced. The entropy coding unit 150 can use a coding method for entropy coding, such as exponential Golomb, context adaptive variable length coding (CAVLC), context adaptive binary arithmetic coding (CABAC), etc. For example, the entropy coding unit 150 can perform entropy coding by using a variable length coding / code (VLC) table. In addition, the entropy coding unit 150 can derive a binarization method of the target symbol and a probability model of the target symbol / binary bit, and perform arithmetic coding by using the derived binarization method and context model.
[0128] In order to encode a transform coefficient level (quantized level), the entropy encoding unit 150 may change a two-dimensional block form coefficient into a one-dimensional vector form by using a transform coefficient scanning method.
[0129] The coding parameters may include information such as syntax elements that are encoded in the encoder and sent to the decoder by signaling (flags, indexes, etc.) and information derived when performing encoding or decoding. The coding parameters may represent information required when encoding or decoding an image. For example, at least one value or combination of the following items may be included in the coding parameters: unit / block residual, unit / block depth, unit / block partition information, unit / block shape, unit / block partition structure, whether to perform quadtree partitioning, whether to perform binary tree partitioning, binary tree partition direction (horizontal or vertical), binary tree partition form (symmetric partitioning or asymmetric partitioning), whether to partition the current coding unit by ternary tree partitioning, ternary tree partition direction (horizontal or vertical), ternary tree partition type (symmetric type or asymmetric type), whether to partition the current coding unit by multi-type tree partitioning, multi-type Partition direction of tree partition (horizontal or vertical), type of multi-type tree partition (symmetric or asymmetric) and tree (binary tree or ternary tree) structure of multi-type tree partition, prediction mode (intra-frame prediction or inter-frame prediction), luminance intra-frame prediction mode / direction, chrominance intra-frame prediction mode / direction, intra-frame partition information, inter-frame partition information, coding block partition flag, prediction block partition flag, transform block partition flag, reference sample filtering method, reference sample filter taps, reference sample filter coefficients, prediction block filtering method, prediction block filter taps, prediction block filter coefficients, prediction block boundary filtering method, prediction block boundary filter taps, prediction block boundary filter coefficients, frame Intra-prediction mode, inter-prediction mode, motion information, motion vector, motion vector difference, reference picture index, inter-prediction angle, inter-prediction indicator, prediction list utilization flag, reference picture list, reference picture, motion vector predictor index, motion vector predictor candidate, motion vector candidate list, whether to use merge mode, merge index, merge candidate, merge candidate list, whether to use skip mode, interpolation filter type, interpolation filter tap, interpolation filter coefficient, motion vector size, representation accuracy of motion vector, transform type, transform size, whether to use the primary (first) transform information, whether to use the secondary transform information, primary transform index, secondary transform index, information on whether a residual signal exists, coding block pattern, coding block flag (CBF), quantization parameter, quantization parameter residual, quantization matrix, whether to apply an intra-loop filter, intra-loop filter coefficients, intra-loop filter taps, intra-loop filter shape / form, whether to apply a deblocking filter, deblocking filter coefficients, deblocking filter taps, deblocking filter strength, deblocking filter shape / form, whether to apply adaptive sample offset, adaptive sample offset value, adaptive sample offset category, adaptive sample offset type, whether to apply an adaptive loop filter, adaptive loop filter coefficients, adaptive loop filter taps, adaptive loop filter shape / form,Binarization / debinarization method, context model determination method, context model update method, whether to execute normal mode, whether to execute bypass mode, context binary bit, bypass binary bit, valid coefficient flag, last valid coefficient flag, encoding flag for unit of coefficient group, position of last valid coefficient, flag for whether coefficient value is greater than 1, flag for whether coefficient value is greater than 2, flag for whether coefficient value is greater than 3, information about remaining coefficient values, sign information, reconstructed luminance samples, reconstructed chrominance samples, residual luminance samples, residual chrominance samples, luminance transform coefficient, chrominance transform coefficient, quantized luminance level, quantized chrominance level, transform coefficient level scanning method, in The size of the motion vector search area on the decoder side, the shape of the motion vector search area on the decoder side, the number of motion vector searches on the decoder side, information about the CTU size, information about the minimum block size, information about the maximum block size, information about the maximum block depth, information about the minimum block depth, image display / output order, slice identification information, slice type, slice partition information, tile identification information, tile type, tile partition information, picture type, bit depth of input samples, bit depth of reconstructed samples, bit depth of residual samples, bit depth of transform coefficients, bit depth of quantization levels, and information about a luminance signal or information about a chrominance signal.
[0130] Here, signaling a flag or an index may mean that the corresponding flag or index is entropy encoded by an encoder and included in a bitstream, and may mean that the corresponding flag or index is entropy decoded from the bitstream by a decoder.
[0131] When the encoding apparatus 100 performs encoding by inter-frame prediction, the encoded current image may be used as a reference image for another image that is subsequently processed. Therefore, the encoding apparatus 100 may reconstruct or decode the encoded current image, or store the reconstructed or decoded image as a reference image in the reference picture buffer 190.
[0132] The quantized level may be dequantized in the dequantization unit 160 or may be inversely transformed in the inverse transform unit 170. The dequantized coefficient or the inverse transform coefficient or both the dequantized coefficient and the inverse transform coefficient may be added to the prediction block by the adder 175. By adding the dequantized coefficient or the inverse transform coefficient or both the dequantized coefficient and the inverse transform coefficient to the prediction block, a reconstructed block may be generated. Here, the dequantized coefficient or the inverse transform coefficient or both the dequantized coefficient and the inverse transform coefficient may represent a coefficient on which at least one of dequantization and inverse transformation is performed, and may represent a reconstructed residual block.
[0133] The reconstructed block may pass through the filter unit 180. The filter unit 180 may apply at least one of a deblocking filter, a sample adaptive offset (SAO), and an adaptive loop filter (ALF) to the reconstructed sample, the reconstructed block, or the reconstructed image. The filter unit 180 may be referred to as an in-loop filter.
[0134] The deblocking filter may remove block distortion generated in the boundary between blocks. In order to determine whether to apply the deblocking filter, it may be determined whether to apply the deblocking filter to the current block based on samples included in a number of rows or columns included in the block. When the deblocking filter is applied to the block, another filter may be applied according to the required deblocking filter strength.
[0135] In order to compensate for the coding error, an appropriate offset value may be added to the sample value by using sample adaptive offset. Sample adaptive offset may correct the offset of the deblocked image and the original image in sample units. A method of partitioning the samples of an image into a predetermined number of regions, determining the regions to which the offset is applied, and applying the offset to the determined regions, or a method of applying the offset in consideration of edge information about each sample may be used.
[0136] The adaptive loop filter may perform filtering based on a comparison result of a filtered reconstructed image with an original image. Samples included in an image may be partitioned into predetermined groups, a filter to be applied to each group may be determined, and differential filtering may be performed for each group. Information on whether ALF is applied may be signaled per coding unit (CU), and the form and coefficient of ALF to be applied to each block may vary.
[0137] The reconstructed block or the reconstructed image that has passed through the filter unit 180 may be stored in the reference picture buffer 190. The reconstructed block processed by the filter unit 180 may be a part of the reference image. That is, the reference image is a reconstructed image composed of the reconstructed blocks processed by the filter unit 180. The stored reference image may be used later in inter-frame prediction or motion compensation.
[0138] Figure 2 is a block diagram showing the configuration of a decoding device according to an embodiment and to which the present invention is applied.
[0139] The decoding device 200 may be a decoder, a video decoding device, or an image decoding device.
[0140] Reference Figure 2 , the decoding device 200 may include an entropy decoding unit 210, an inverse quantization unit 220, an inverse transform unit 230, an intra-frame prediction unit 240, a motion compensation unit 250, an adder 225, a filter unit 260 and a reference picture buffer 270.
[0141] The decoding device 200 may receive a bit stream output from the encoding device 100. The decoding device 200 may receive a bit stream stored in a computer-readable recording medium, or may receive a bit stream streamed through a wired / wireless transmission medium. The decoding device 200 may decode the bit stream by using an intra-frame mode or an inter-frame mode. In addition, the decoding device 200 may generate a reconstructed image or a decoded image generated by decoding, and output the reconstructed image or the decoded image.
[0142] When the prediction mode used in decoding is intra mode, the switch may be switched to intra mode. Alternatively, when the prediction mode used in decoding is inter mode, the switch may be switched to inter mode.
[0143] The decoding device 200 can obtain a reconstructed residual block by decoding the input bit stream and generate a prediction block. When the reconstructed residual block and the prediction block are obtained, the decoding device 200 can generate a reconstructed block that becomes a decoding target by adding the reconstructed residual block to the prediction block. The decoding target block may be referred to as a current block.
[0144] The entropy decoding unit 210 may generate symbols by entropy decoding the bit stream according to the probability distribution. The generated symbols may include symbols in the form of quantized levels. Here, the entropy decoding method may be an inverse process of the above entropy encoding method.
[0145] In order to decode the transform coefficient level (quantized level), the entropy decoding unit 210 may change the unidirectional vector form coefficient into a two-dimensional block form by using a transform coefficient scanning method.
[0146] The quantized level may be inversely quantized in the inverse quantization unit 220 or inversely transformed in the inverse transform unit 230. The quantized level may be a result of inverse quantization or inverse transformation or both inverse quantization and inverse transformation, and may be generated as a reconstructed residual block. Here, the inverse quantization unit 220 may apply a quantization matrix to the quantized level.
[0147] When the intra mode is used, the intra prediction unit 240 may generate a predicted block by performing spatial prediction using sample values of blocks that are adjacent to the decoding target block and have been decoded, with respect to the current block.
[0148] When the inter mode is used, the motion compensation unit 250 may generate a prediction block by performing motion compensation using a motion vector and a reference image stored in the reference picture buffer 270 with respect to the current block.
[0149] The adder 225 can generate a reconstructed block by adding the reconstructed residual block to the prediction block. The filter unit 260 can apply at least one of a deblocking filter, a sample adaptive offset, and an adaptive loop filter to the reconstructed block or the reconstructed image. The filter unit 260 can output the reconstructed image. The reconstructed block or the reconstructed image can be stored in the reference picture buffer 270 and used when performing inter-frame prediction. The reconstructed block processed by the filter unit 260 can be a part of the reference image. That is, the reference image is a reconstructed image composed of the reconstructed blocks processed by the filter unit 260. The stored reference image can be used later in inter-frame prediction or motion compensation.
[0150] Figure 3 is a diagram schematically showing a partition structure of an image when the image is encoded and decoded. Figure 3 An example of partitioning a single unit into a plurality of lower-level units is schematically shown.
[0151] In order to efficiently partition an image, a coding unit (CU) may be used when encoding and decoding. When encoding / decoding an image, a coding unit may be used as a basic unit. In addition, when encoding / decoding an image, a coding unit may be used as a unit for distinguishing an intra prediction mode from an inter prediction mode. A coding unit may be a basic unit for prediction, transformation, quantization, inverse transformation, inverse quantization, or encoding / decoding processing of a transform coefficient.
[0152] Reference Figure 3 , the image 300 is partitioned sequentially by the largest coding unit (LCU), and the LCU unit is determined as a partition structure. Here, the LCU may be used in the same meaning as the coding tree unit (CTU). Unit partitioning may mean partitioning a block associated with the unit. In the block partition information, information about the unit depth may be included. The depth information may indicate the number or degree of partitioning of the unit or both the number and degree. A single unit may be partitioned into a plurality of lower-level units hierarchically associated with the depth information based on a tree structure. In other words, the unit and the lower-level units generated by partitioning the unit may correspond to a node and a child node of the node, respectively. Each of the partitioned lower-level units may have depth information. The depth information may be information indicating the size of a CU and may be stored in each CU. The unit depth indicates the number and / or degree associated with partitioning the unit. Therefore, the partition information of the lower-level unit may include information about the size of the lower-level unit.
[0153] The partition structure may represent the distribution of coding units (CUs) within the LCU 310. Such distribution may be determined according to whether a single CU is partitioned into a plurality of (positive integers equal to or greater than 2, including 2, 4, 8, 16, etc.) CUs. The horizontal size and vertical size of the CU generated by partitioning may be half of the horizontal size and vertical size of the CU before partitioning, respectively, or may have a size smaller than the horizontal size and vertical size before partitioning, respectively, according to the number of partitioning times. The CU may be recursively partitioned into a plurality of CUs. By recursive partitioning, at least one of the height and width of the CU after partitioning may be reduced compared to at least one of the height and width of the CU before partitioning. The partitioning of the CU may be recursively performed until a predefined depth or a predefined size. For example, the depth of the LCU may be 0, and the depth of the smallest coding unit (SCU) may be a predefined maximum depth. Here, as described above, the LCU may be a coding unit having a maximum coding unit size, and the SCU may be a coding unit having a minimum coding unit size. Starting partitioning from the LCU 310, as the horizontal size or vertical size or both the horizontal size and the vertical size of the CU decrease through partitioning, the CU depth increases by 1. For example, for each depth, a non-partitioned CU may have a size of 2N×2N. Also, in the case of a partitioned CU, a CU of size 2N×2N may be partitioned into four CUs of size N×N. The size of N may be reduced to half as the depth increases by 1.
[0154] In addition, information on whether a CU is partitioned may be indicated by using the partition information of the CU. The partition information may be 1-bit information. All CUs except the SCU may include partition information. For example, when the value of the partition information is 1, the CU may not be partitioned, and when the value of the partition information is 2, the CU may be partitioned.
[0155] Reference Figure 3 , an LCU with a depth of 0 may be a 64×64 block. 0 may be the minimum depth. An SCU with a depth of 3 may be an 8×8 block. 3 may be the maximum depth. A CU with a block of 32×32 and a block of 16×16 may be represented as depth 1 and depth 2, respectively.
[0156] For example, when a single coding unit is partitioned into four coding units, the horizontal size and vertical size of the four partitioned coding units may be half the horizontal size and vertical size of the CU before partitioning. In one embodiment, when a coding unit of size 32×32 is partitioned into four coding units, each of the four partitioned coding units may have a size of 16×16. When a single coding unit is partitioned into four coding units, it can be said that the coding unit can be partitioned into a quadtree form.
[0157] For example, when one coding unit is partitioned into two sub-coding units, the horizontal size or vertical size (width or height) of each of the two sub-coding units may be half of the horizontal size or vertical size of the original coding unit. For example, when a coding unit of size 32×32 is partitioned vertically into two sub-coding units, each of the two sub-coding units may have a size of 16×32. For example, when a coding unit of size 8×32 is partitioned horizontally into two sub-coding units, each of the two sub-coding units may have a size of 8×16. When one coding unit is partitioned into two sub-coding units, the coding unit may be said to be partitioned into two or partitioned in a binary tree partition structure.
[0158] For example, when one coding unit is partitioned into three sub-coding units, the horizontal size or the vertical size of the coding unit may be partitioned at a ratio of 1:2:1, thereby generating three sub-coding units having a ratio of the horizontal size or the vertical size of 1:2:1. For example, when a coding unit having a size of 16×32 is partitioned horizontally into three sub-coding units, the three sub-coding units may have sizes of 16×8, 16×16, and 16×8, respectively, in order from the uppermost sub-coding unit to the lowermost sub-coding unit. For example, when a coding unit having a size of 32×32 is partitioned vertically into three sub-coding units, the three sub-coding units may have sizes of 8×32, 16×32, and 8×32, respectively, in order from the left sub-coding unit to the right sub-coding unit. When one coding unit is partitioned into three sub-coding units, the coding unit may be said to be partitioned into three or partitioned in a ternary tree partition structure.
[0159] exist Figure 3 , a coding tree unit (CTU) 320 is an example of a CTU to which a quadtree partition structure, a binary tree partition structure, and a ternary tree partition structure are applied.
[0160] As described above, in order to partition a CTU, at least one of a quadtree partition structure, a binary tree partition structure, and a ternary tree partition structure may be applied. Various tree partition structures may be sequentially applied to a CTU according to a predetermined priority order. For example, a quadtree partition structure may be preferentially applied to a CTU. Coding units that can no longer be partitioned using a quadtree partition structure may correspond to leaf nodes of a quadtree. Coding units corresponding to leaf nodes of a quadtree may be used as root nodes of a binary and / or ternary tree partition structure. That is, coding units corresponding to leaf nodes of a quadtree may be further partitioned by a binary tree partition structure or a ternary tree partition structure, or may not be further partitioned. Therefore, by preventing coding blocks generated by binary tree partitioning or ternary tree partitioning of coding units corresponding to leaf nodes of a quadtree from undergoing further quadtree partitioning, block partitioning may be efficiently performed and / or partition information may be sent using a signal.
[0161] The four partition information may be used to signal the fact that the coding unit corresponding to the node of the quadtree is partitioned. The four partition information having a first value (e.g., '1') may indicate that the current coding unit is partitioned in a quadtree partition structure. The four partition information having a second value (e.g., '0') may indicate that the current coding unit is not partitioned in a quadtree partition structure. The four partition information may be a flag having a predetermined length (e.g., one bit).
[0162] There may be no priority between the binary tree partition and the ternary tree partition. That is, the coding unit corresponding to the leaf node of the quadtree may further undergo any partition of the binary tree partition and the ternary tree partition. In addition, the coding unit generated by the binary tree partition or the ternary tree partition may undergo further binary tree partition or further ternary tree partition, or may not be further partitioned.
[0163] A tree structure in which there is no priority between a binary tree partition and a ternary tree partition is referred to as a multi-type tree structure. A coding unit corresponding to a leaf node of a quadtree may be used as a root node of a multi-type tree. At least one of multi-type tree partition indication information, partition direction information, and partition tree information may be used to signal whether to partition a coding unit corresponding to a node of a multi-type tree. For partitions of coding units corresponding to nodes of a multi-type tree, multi-type tree partition indication information, partition direction, and partition tree information may be sequentially signaled.
[0164] The multi-type tree partition indication information having a first value (eg, '1') may indicate that the current coding unit will undergo multi-type tree partitioning. The multi-type tree partition indication information having a second value (eg, '0') may indicate that the current coding unit will not undergo multi-type tree partitioning.
[0165] When the coding unit corresponding to the node of the multi-type tree is further partitioned according to the multi-type tree partition structure, the coding unit may include partition direction information. The partition direction information may indicate in which direction the current coding unit will be partitioned for the multi-type tree partition. The partition direction information having a first value (e.g., "1") may indicate that the current coding unit will be partitioned vertically. The partition direction information having a second value (e.g., "0") may indicate that the current coding unit will be partitioned horizontally.
[0166] When the coding unit corresponding to the node of the multi-type tree is further partitioned according to the multi-type tree partition structure, the current coding unit may include partition tree information. The partition tree information may indicate a tree partition structure to be used for the partition of the node of the multi-type tree. The partition tree information having a first value (e.g., "1") may indicate that the current coding unit will be partitioned according to the binary tree partition structure. The partition tree information having a second value (e.g., "0") may indicate that the current coding unit will be partitioned according to the ternary tree partition structure.
[0167] The partition indication information, the partition tree information and the partition direction information may all be flags having a predetermined length (eg, one bit).
[0168] At least any one of the quadtree partition indication information, the multi-type tree partition indication information, the partition direction information, and the partition tree information may be entropy encoded / decoded. For entropy encoding / decoding of those types of information, information about neighboring coding units adjacent to the current coding unit may be used. For example, the partition type (partition or non-partition, partition tree, and / or partition direction) of the left neighboring coding unit and / or the upper neighboring coding unit of the current coding unit is likely to be similar to the partition type of the current coding unit. Therefore, context information for entropy encoding / decoding the information about the current coding unit may be derived from the information about the neighboring coding unit. The information about the neighboring coding unit may include at least any one of the quadtree partition information, the multi-type tree partition indication information, the partition direction information, and the partition tree information.
[0169] As another example, among binary tree partitioning and ternary tree partitioning, binary tree partitioning may be preferentially performed. That is, the current coding unit may first undergo binary tree partitioning, and then the coding unit corresponding to the leaf node of the binary tree may be set as the root node for the ternary tree partitioning. In this case, quadtree partitioning or binary tree partitioning may not be performed on the coding unit corresponding to the node of the ternary tree.
[0170] A coding unit that cannot be partitioned according to a quadtree partition structure, a binary tree partition structure, and / or a ternary tree partition structure becomes a basic unit for encoding, prediction, and / or transformation. That is, the coding unit cannot be further partitioned for prediction and / or transformation. Therefore, the partition structure information and partition information for partitioning the coding unit into prediction units and / or transformation units may not be present in the bitstream.
[0171] However, when the size of the coding unit (i.e., the basic unit for partitioning) is larger than the size of the maximum transform block, the coding unit may be recursively partitioned until the size of the coding unit is reduced to be equal to or smaller than the size of the maximum transform block. For example, when the size of the coding unit is 64×64 and when the size of the maximum transform block is 32×32, the coding unit may be partitioned into four 32×32 blocks for transformation. For example, when the size of the coding unit is 32×64 and the size of the maximum transform block is 32×32, the coding unit may be partitioned into two 32×32 blocks for transformation. In this case, the partition of the coding unit for transformation is not separately signaled and may be determined by comparison between the horizontal size or vertical size of the coding unit and the horizontal size or vertical size of the maximum transform block. For example, when the horizontal size (width) of the coding unit is larger than the horizontal size (width) of the maximum transform block, the coding unit may be vertically bisected. For example, when the vertical size (length) of the coding unit is larger than the vertical size (length) of the maximum transform block, the coding unit may be horizontally bisected.
[0172] Information on the maximum size and / or minimum size of the coding unit and information on the maximum size and / or minimum size of the transform block may be sent with a signal or determined at an upper layer of the coding unit. The upper layer may be, for example, a sequence layer, a picture layer, a slice layer, etc. For example, the minimum size of the coding unit may be determined to be 4×4. For example, the maximum size of the transform block may be determined to be 64×64. For example, the minimum size of the transform block may be determined to be 4×4.
[0173] Information on the minimum size of the coding unit corresponding to the leaf node of the quadtree (quadtree minimum size) and / or information on the maximum depth from the root node of the multi-type tree to the leaf node (maximum tree depth of the multi-type tree) may be sent with a signal or determined at an upper layer of the coding unit. For example, the upper layer may be a sequence layer, a picture layer, a slice layer, etc. Information on the minimum size of the quadtree and / or information on the maximum depth of the multi-type tree may be sent with a signal or determined for each of an intra-picture slice and an inter-picture slice.
[0174] The difference information between the size of the CTU and the maximum size of the transform block may be signaled or determined at an upper layer of the coding unit. For example, the upper layer may be a sequence layer, a picture layer, a slice layer, etc. The information of the maximum size of the coding unit corresponding to each node of the binary tree (hereinafter referred to as the maximum size of the binary tree) may be determined based on the size of the coding tree unit and the difference information. The maximum size of the coding unit corresponding to each node of the ternary tree (hereinafter referred to as the maximum size of the ternary tree) may vary according to the type of the slice. For example, for an intra-picture slice, the maximum size of the ternary tree may be 32×32. For example, for an inter-picture slice, the maximum size of the ternary tree may be 128×128. For example, the minimum size of the coding unit corresponding to each node of the binary tree (hereinafter referred to as the minimum size of the binary tree) and / or the minimum size of the coding unit corresponding to each node of the ternary tree (hereinafter referred to as the minimum size of the ternary tree) may be set to the minimum size of the coding block.
[0175] As another example, the maximum size of the binary tree and / or the maximum size of the ternary tree may be signaled or determined at the slice level. Alternatively, the minimum size of the binary tree and / or the minimum size of the ternary tree may be signaled or determined at the slice level.
[0176] According to the sizes and depth information of the above-mentioned various blocks, quad partition information, multi-type tree partition indication information, partition tree information and / or partition direction information may be included in the bitstream or may not be included in the bitstream.
[0177] For example, when the size of the coding unit is not greater than the minimum size of the quadtree, the coding unit does not include the quad partition information. Therefore, the quad partition information can be derived from the second value.
[0178] For example, when the size (horizontal size and vertical size) of the coding unit corresponding to the node of the multi-type tree is greater than the maximum size (horizontal size and vertical size) of the binary tree and / or the maximum size (horizontal size and vertical size) of the ternary tree, the coding unit may not be partitioned into two or three partitions. Therefore, the multi-type tree partition indication information may not be signaled, but may be derived from the second value.
[0179] Optionally, when the size (horizontal size and vertical size) of the coding unit corresponding to the node of the multi-type tree is the same as the maximum size (horizontal size and vertical size) of the binary tree and / or is twice the maximum size (horizontal size and vertical size) of the ternary tree, the coding unit may not be further partitioned into two or three partitions. Therefore, the multi-type tree partition indication information may not be sent with a signal, but may be derived from the second value. This is because when the coding unit is partitioned in a binary tree partition structure and / or a ternary tree partition structure, a coding unit smaller than the minimum size of the binary tree and / or the minimum size of the ternary tree is generated.
[0180] Optionally, when the depth of the coding unit corresponding to the node of the multi-type tree is equal to the maximum depth of the multi-type tree, the coding unit may not be further partitioned into two and / or three partitions. Therefore, the multi-type tree partition indication information may not be sent with a signal, but may be derived from the second value.
[0181] Optionally, only when at least one of vertical binary tree partitioning, horizontal binary tree partitioning, vertical ternary tree partitioning, and horizontal ternary tree partitioning is possible for the coding unit corresponding to the node of the multi-type tree, the multi-type tree partition indication information may be sent by signal. Otherwise, the coding unit may not be partitioned into two and / or three partitions. Therefore, the multi-type tree partition indication information may not be sent by signal, but may be derived from the second value.
[0182] Optionally, partition direction information may be signaled only when both vertical binary tree partitioning and horizontal binary tree partitioning or both vertical ternary tree partitioning and horizontal ternary tree partitioning are possible for the coding unit corresponding to the node of the multi-type tree. Otherwise, the partition direction information may not be signaled, but may be derived from a value indicating a possible partition direction.
[0183] Optionally, partition tree information may be signaled only when both vertical binary tree partitioning and vertical ternary tree partitioning or both horizontal binary tree partitioning and horizontal ternary tree partitioning are possible for a coding unit corresponding to a node of a multi-type tree. Otherwise, the partition tree information may not be signaled but may be derived from a value indicating a possible partition tree structure.
[0184] Figure 4 : is a diagram showing an embodiment of intra-screen prediction processing.
[0185] exist Figure 4 In , a rectangle can represent a picture. Figure 4 In FIG. 1 , the arrow indicates the prediction direction. Pictures can be classified into intra-picture (I picture), predicted picture (P picture) and bi-predicted picture (B picture) according to their coding type.
[0186] I pictures may be encoded by intra prediction without inter-picture prediction. P pictures may be encoded by inter-picture prediction using reference pictures existing in one direction (i.e., forward or reverse) for the current block. B pictures may be encoded by inter-picture prediction using reference pictures existing in two directions (i.e., forward and reverse) for the current block. When inter-picture prediction is used, the encoder may perform inter-picture prediction or motion compensation, and the decoder may perform corresponding motion compensation.
[0187] Next, embodiments of inter-picture prediction will be described in detail.
[0188] Reference pictures and motion information may be used to perform inter-picture prediction or motion compensation.
[0189] The motion information of the current block may be derived during inter-picture prediction by each of the encoding apparatus 100 and the decoding apparatus 200. The motion information of the current block may be derived by using the motion information of a reconstructed neighboring block, a co-located block (also referred to as a col block or a co-located block), and / or motion information of a block adjacent to the co-located block. The co-located block may represent a previously reconstructed block that is spatially located at the same position as the current block within a co-located picture (also referred to as a col picture or a co-located picture). The co-located picture may be one of one or more reference pictures included in a reference picture list.
[0190] The method of deriving motion information of the current block may vary according to the prediction mode of the current block. For example, as a prediction mode for inter-picture prediction, there may be AMVP mode, merge mode, skip mode, current picture reference mode, etc. The merge mode may be referred to as a motion merge mode.
[0191] For example, when AMVP is used as a prediction mode, at least one of the motion vector of the reconstructed neighboring block, the motion vector of the co-located block, the motion vector of the block adjacent to the co-located block, and the (0,0) motion vector may be determined as a motion vector candidate for the current block, and a motion vector candidate list is generated by using the motion vector candidate. The motion vector candidate of the current block may be derived by using the generated motion vector candidate list. The motion information of the current block may be determined based on the derived motion vector candidate. The motion vector of the co-located block or the motion vector of the block adjacent to the co-located block may be referred to as a temporal motion vector candidate, and the motion vector of the reconstructed neighboring block may be referred to as a spatial motion vector candidate.
[0192] The encoding device 100 may calculate a motion vector difference (MVD) between a motion vector of a current block and a motion vector candidate and may perform entropy coding on the motion vector difference (MVD). In addition, the encoding device 100 may perform entropy coding on a motion vector candidate index and generate a bitstream. The motion vector candidate index may indicate an optimal motion vector candidate among motion vector candidates included in a motion vector candidate list. The decoding device may perform entropy decoding on the motion vector candidate index included in the bitstream, and may select a motion vector candidate of a decoding target block from the motion vector candidates included in the motion vector candidate list by using the entropy-decoded motion vector candidate index. In addition, the decoding device 200 may add the entropy-decoded MVD to the motion vector candidate extracted by entropy decoding, thereby deriving a motion vector of the decoding target block.
[0193] The bitstream may include a reference picture index indicating a reference picture. The reference picture index may be entropy-encoded by the encoding apparatus 100 and then signaled as a bitstream to the decoding apparatus 200. The decoding apparatus 200 may generate a prediction block of a decoding target block based on the derived motion vector and the reference picture index information.
[0194] Another example of a method of deriving motion information of a current block may be a merge mode. The merge mode may represent a method of merging motions of multiple blocks. The merge mode may represent a mode of deriving motion information of a current block from motion information of neighboring blocks. When the merge mode is applied, the motion information of reconstructed neighboring blocks and / or the motion information of the same-positioned blocks may be used to generate a merge candidate list. The motion information may include at least one of a motion vector, a reference picture index, and an inter-picture prediction indicator. The prediction indicator may indicate unidirectional prediction (L0 prediction or L1 prediction) or bidirectional prediction (L0 prediction and L1 prediction).
[0195] The merge candidate list may be a list of stored motion information. The motion information included in the merge candidate list may be at least any one of a zero merge candidate and motion information of a neighboring block adjacent to the current block (spatial merge candidate), motion information of a co-located block of the current block included in a reference picture (temporal merge candidate), and new motion information of a combination of motion information present in the merge candidate list.
[0196] The encoding device 100 may generate a bitstream by performing entropy encoding on at least one of a merge flag and a merge index, and may signal the bitstream to the decoding device 200. The merge flag may be information indicating whether a merge mode is performed for each block, and the merge index may be information indicating which neighboring block among neighboring blocks of the current block is a merge target block. For example, the neighboring blocks of the current block may include a left neighboring block on the left side of the current block, an upper neighboring block set above the current block, and a temporal neighboring block temporally adjacent to the current block.
[0197] The skip mode may be a mode in which the motion information of the neighboring blocks is applied to the current block as it is. When the skip mode is applied, the encoding apparatus 100 may perform entropy encoding on information of the fact of which block's motion information is to be used as the motion information of the current block to generate a bitstream, and may signal the bitstream to the decoding apparatus 200. The encoding apparatus 100 may not signal a syntax element regarding at least any one of the motion vector difference information, the coded block flag, and the transform coefficient level to the decoding apparatus 200.
[0198] The current picture reference mode may indicate a prediction mode in which a previously reconstructed region within the current picture to which the current block belongs is used for prediction. Here, a vector may be used to specify the previously reconstructed region. Information indicating whether the current block will be encoded in the current picture reference mode may be encoded by using the reference picture index of the current block. A flag or index indicating whether the current block is a block encoded in the current picture reference mode may be signaled and may be derived based on the reference picture index of the current block. In the case where the current block is encoded in the current picture reference mode, the current picture may be added to the reference picture list for the current block to be located at a fixed position or a random position in the reference picture list. The fixed position may be, for example, a position indicated by a reference picture index 0 or the last position in the list. When the current picture is added to the reference picture list to be located at a random position, a reference picture index indicating the random position may be signaled.
[0199] Based on the above description, an image encoding method and an image decoding method according to an embodiment of the present invention will be described in detail below.
[0200] Figure 5 is a flowchart showing an image encoding method according to an embodiment of the present invention, and Figure 6 is a flowchart illustrating an image decoding method according to an embodiment of the present invention.
[0201] Reference Figure 5 , the encoding device may derive motion vector candidates (step S501), and generate a motion vector candidate list based on the derived motion vector candidates (step S502). After the motion vector candidate list is generated, a motion vector may be determined based on the generated motion vector candidate list (step S503), and motion compensation may be performed based on the determined motion vector (step S504). Subsequently, the encoding device may encode information associated with motion compensation (step S505).
[0202] Reference Figure 6 , the decoding device may perform entropy decoding on the information associated with motion compensation received from the encoding device (step S601), and derive motion vector candidates (step S602). The decoding device may generate a motion vector candidate list based on the derived motion vector candidates (step S603), and determine a motion vector using the generated motion vector candidate list (step S604). Subsequently, the decoding device may perform motion compensation by using the determined motion vector (step S605).
[0203] Figure 7 is a flowchart showing an image encoding method according to another embodiment of the present invention, and Figure 8 is a flowchart illustrating an image decoding method according to another embodiment of the present invention.
[0204] Reference Figure 7 , the encoding device may derive a merge candidate (step S701), and generate a merge candidate list based on the derived merge candidate. After generating the merge candidate list, the encoding device may determine motion information using the generated merge candidate list (step S702), and may use the determined motion information to perform motion compensation on the current block (step S703). Subsequently, the encoding device may perform entropy encoding on information associated with motion compensation (step S704).
[0205] Reference Figure 8 , the decoding device may perform entropy decoding on the information associated with motion compensation received from the encoding device (S801), derive a merge candidate (S802), and generate a merge candidate list based on the derived merge candidate. After generating the merge candidate list, the decoding device may determine the motion information of the current block by using the generated merge candidate list (S803). Subsequently, the decoding device may perform motion compensation using the motion information (S804).
[0206] Figure 5 and Figure 6 Show application Figure 4 The example of AMVP shown in , and Figure 7 and Figure 8 Show application Figure 4 An example of the merge mode shown in .
[0207] In the following, the Figure 5 and Figure 6 Each step in the Figure 7 and Figure 8 However, the motion compensation steps corresponding to S504, S605, S703, and S804 and the entropy encoding / decoding steps corresponding to S505, S601, S704, and S801 will be described together.
[0208] In the following, the following will be described in detail Figure 5 and Figure 6 Each step in .
[0209] First, the steps of deriving motion vector candidates (S501, S602) will be described in detail.
[0210] The motion vector candidates of the current block may include at least one of spatial motion vector candidates and temporal motion vector candidates or both spatial motion vector candidates and temporal motion vector candidates.
[0211] The temporal motion vector of the current block may be derived from a reconstructed block adjacent to the current block. For example, a motion vector of a reconstructed block adjacent to the current block may be determined as a spatial motion vector candidate of the current block.
[0212] Fig. 9 is a diagram illustrating an example of deriving spatial motion vector candidates for a current block.
[0213] Reference Fig. 9 , the spatial motion vector candidate of the current block may be derived from the neighboring blocks adjacent to the current block X. Here, the neighboring blocks adjacent to the current block may include at least one of a block B1 adjacent to the top of the current block, a block A1 adjacent to the left side of the current block, a block B0 adjacent to the upper right corner of the current block, a block B2 adjacent to the upper left corner of the current block, a block A0 adjacent to the lower left corner of the current block, a block A2 adjacent to the upper left side of the current block, and a block B3 adjacent to the upper left side of the current block. In addition, the neighboring blocks adjacent to the current block may include at least one of a block located between B3 and B1 and a block located between A2 and A1.
[0214] In addition, the neighboring block adjacent to the current block may have a square shape or a non-square shape. When a motion vector exists in a neighboring block adjacent to the current block, the motion vector of the neighboring block may be determined as a spatial motion vector candidate for the current block. Whether the motion vector of the neighboring block exists or whether the motion vector of the neighboring block can be used as a spatial motion vector candidate for the current block may be determined based on whether the neighboring block exists or whether the neighboring block is encoded by inter-frame prediction. Here, whether the motion vector of the neighboring block exists or whether the motion vector of the neighboring block can be used as a spatial motion vector candidate for the current block may be determined according to a predetermined priority. In the example, Fig. 9 In the example shown in , the availability may be determined in the order of blocks at positions A0, A1, B0, B1, B2, B3, and A2.
[0215] When a reference image of a current block and a reference image of a neighboring block having a motion vector are different, the motion vector of the neighboring block may be determined as a spatial motion vector candidate of the current block by performing scaling on the motion vector of the neighboring block. Here, scaling may be performed based on at least one of a distance between the current image and a reference image referenced by the current block and a distance between the current image and a reference image referenced by the neighboring block. In one example, the spatial motion vector candidate of the current block may be derived by performing scaling on the motion vector of the neighboring block according to a ratio of a distance between the current image and the reference image referenced by the current block and a distance between the current image and the reference image referenced by the neighboring block.
[0216] In addition, when the reference image index of the current block and the reference image index of the neighboring block having the motion vector are different, the motion vector of the neighboring block may be determined as the spatial motion vector candidate of the current block by performing scaling on the motion vector of the neighboring block. Here, the scaling may be performed based on at least one of the distance between the current image and the reference image referenced by the current block and the distance between the current image and the reference image referenced by the neighboring block.
[0217] Regarding scaling, a spatial motion vector candidate may be determined by performing scaling on a motion vector of a neighboring block based on a reference image indicated by a reference image index having a predefined value. Here, the predefined value may be a positive integer including 0. In one example, by performing scaling on a motion vector of a neighboring block according to a ratio of a distance between a current image and a reference image indicated by a reference image index having a predefined value of the current block and a distance between the current image and a reference image indicated by a reference image index having a predefined value of the neighboring block, a spatial motion vector candidate of the current block may be derived.
[0218] Furthermore, a spatial motion vector candidate of the current block may be derived based on at least one encoding parameter of the current block.
[0219] The temporal motion vector candidate of the current block may be derived from a reconstructed block included in a co-located image (co-located picture) of the current image. Here, the co-located image may be an image that has been encoded / decoded before the current image and may be an image having a different temporal order from the current image.
[0220] Fig.10 is a diagram illustrating an example of deriving a temporal motion vector candidate of a current block.
[0221] Reference Fig.10 , in the co-located image of the current image, the temporal motion vector candidate of the current block may be derived from a block including an outer position of a block associated with the same spatial position of the current block X or from a block including an inner position of a block associated with the same spatial position of the current block X. Here, the temporal motion vector candidate may represent a motion vector of the co-located block. In one example, the temporal motion vector candidate of the current block may be derived from at least one of a block H adjacent to the lower right corner of block C, blocks C1 and C3 including the center point of block C, blocks C0, C4, C5, and C6 adjacent to the inner corner of block C, a block D0 adjacent to the upper right side of block C, and a block D1 adjacent to the lower left side of block C, wherein block C is associated with the same spatial position of the current block. Blocks H, block C1, block C3, block C0, block C4, block C5, block C6, block D0, and block D1 used for deriving the temporal motion vector candidate of the current block are referred to as "co-located blocks (co-located blocks)". When a temporal motion vector candidate is not derived from at least one of the above-described blocks H, C1, C3, C0, C4, C5, C6, D0, and D1 (when intra encoding is performed), a temporal motion vector candidate may be derived by using the remaining blocks excluding the above blocks.
[0222] In addition, at least one of a temporal motion vector candidate, a co-located image, a co-located block, a prediction list utilization flag, and a reference image index may be derived based on at least one encoding parameter.
[0223] When the distance between the current image including the current block and the reference image of the current block and the distance between the co-located image including the co-located block and the reference image of the co-located block are different, the temporal motion vector candidate of the current block may be obtained by performing scaling on the motion vector of the co-located block. Here, the scaling may be performed based on at least one of the distance between the current image and the reference image referenced by the current block and the distance between the co-located image and the reference image referenced by the co-located block. In one example, the temporal motion vector candidate of the current block may be derived by performing scaling on the motion vector of the co-located block according to the ratio of the distance between the current image and the reference image referenced by the current block and the distance between the co-located image and the reference image referenced by the co-located block.
[0224] The additional motion vector candidate of the current block may represent at least one of a modified spatial motion vector candidate, a modified temporal motion vector candidate, a combined motion vector candidate, a candidate having a predetermined motion vector value, and a motion vector candidate in units of sub-blocks. Here, deriving the additional motion vector candidate may mean deriving the additional motion vector candidate and adding it to the motion vector candidate list.
[0225] Here, the modified spatial motion vector candidate may mean a motion vector obtained by modifying the above-mentioned spatial motion vector candidate. Furthermore, the modified temporal motion vector candidate may mean a motion vector obtained by modifying the above-mentioned temporal motion vector candidate. Furthermore, the combined motion vector candidate may mean a motion vector obtained by combining at least one of the motion information of the spatial motion vector candidate, the temporal motion vector candidate, the modified spatial motion vector candidate, the modified temporal motion vector candidate, and the motion vector candidate having a predetermined motion vector value existing in the motion vector candidate list.
[0226] In addition, the combined motion vector candidate may represent a combined dual-prediction motion vector candidate. The combined dual-prediction motion vector candidate may be a motion vector candidate using dual prediction, and may represent a motion vector candidate having an L0 motion vector and an L1 motion vector. In addition, the combined motion vector candidate may be a motion vector candidate having a statistical value of at least one of a spatial motion vector candidate, a temporal motion vector candidate, a modified spatial motion vector candidate, a modified temporal motion vector candidate, and a motion vector candidate having a predetermined motion vector value. Here, the statistical value may represent an average value. In addition, an average value may be calculated for each of the L0 motion vector and the L1 motion vector.
[0227] Also, the motion vector candidate having the predetermined motion information value may represent a zero-merged motion vector having a motion vector (0, 0).
[0228] In addition, at least one of a modified spatial motion vector candidate, a modified temporal motion vector candidate, a combined motion vector candidate, and a candidate having a predetermined motion vector value may be derived for each subblock, and the above-mentioned motion vector derived for each subblock may be referred to as a motion vector candidate in units of subblocks and may be added to a motion vector candidate list. Here, "subblock" may be used to have the same meaning as "lower layer block" or "subblock" and may be used interchangeably.
[0229] Will use later Fig.13 and Fig.14 Motion vector candidates in units of subblocks are described in detail.
[0230] Next, the steps of generating a motion vector candidate list based on the derived motion vector candidates (S502, S603) will be described.
[0231] The step of generating the motion vector candidate list may include a process of adding or removing a motion vector candidate from the motion vector candidate list and a process of adding a combined motion vector candidate to the motion vector candidate list.
[0232] First, a process of adding or removing a derived motion vector candidate to or from a motion vector candidate list will be described. The encoding apparatus and the decoding apparatus may add the derived motion vector candidates to the motion vector candidate list in the order in which the motion vector candidates are derived.
[0233] It is assumed that the motion vector candidate list mvpListLX may represent the motion vector candidate list corresponding to the reference picture lists L0, L1, L2, and L3. That is, the motion vector candidate list corresponding to the reference picture list L0 may be represented by mvpListL0.
[0234] In addition to the spatial motion vector candidates and the temporal motion vector candidates, a motion vector having a predetermined value may be added to the motion vector candidate list. For example, when the number of motion vector candidates in the motion vector list is less than the maximum number of motion vector candidates that can be included in the motion vector list, a motion vector having a value of zero may be added to the motion vector candidate list.
[0235] Next, a process of adding a combined motion vector candidate to a motion vector candidate list will be described.
[0236] When the number of motion vector candidates in the motion vector candidate list is less than the maximum number of motion vector candidates that can be included in the motion vector candidate list, one or more motion vector candidates in the motion vector candidate list are combined to generate one or more combined motion vector candidates, and the generated combined motion vector candidates may be added to the motion vector candidate list. For example, at least one or more of the spatial motion vector candidates, the temporal motion vector candidates, and the zero motion vector candidates included in the motion vector candidate list are used to generate the combined motion vector candidate, and the generated combined motion vector candidate may be added to the motion vector candidate list.
[0237] Alternatively, a combined motion vector candidate may be generated based on at least one of the encoding parameters, and the combined motion vector candidate generated based on at least one of the encoding parameters may be added to the motion vector candidate list.
[0238] Next, the steps of selecting a prediction motion vector of the current block from the motion vector candidate list (S503, S604) will be described below.
[0239] Among the motion vector candidates included in the motion vector candidate list, the motion vector candidate indicated by the motion vector candidate index may be determined as the predicted motion vector of the current block.
[0240] The encoding device may calculate the difference between the motion vector of the current block and the predicted motion vector, thereby generating a motion vector difference. The decoding device may generate the motion vector of the current block by adding the predicted motion vector to the motion vector difference.
[0241] Here, the motion information correction may be applied to any one of a motion vector candidate included in the motion vector candidate list, a predicted motion vector, or a motion vector calculated by adding a predicted motion vector to a motion vector difference. A detailed description of the motion information correction will be described later.
[0242] We will describe it together later Figure 5 and Figure 6 The steps of performing motion compensation (S504, S605) and entropy encoding / decoding information associated with motion compensation (S505, S601) shown in Figure 7 and Figure 8 The steps of performing motion compensation (S703, S804) and entropy encoding / decoding (S704, S801) shown in .
[0243] In the following, the following will be described in detail Figure 7 and 8 Each step shown in .
[0244] First, the steps of deriving merge candidates ( S701 , 802 ) will be described.
[0245] The merge candidate for the current block may include at least one of a spatial merge candidate, a temporal merge candidate, and an additional merge candidate. Here, the expression "derive a spatial merge candidate" refers to a process of deriving a spatial merge candidate and adding the derived merge candidate to a merge candidate list.
[0246] Reference Fig. 9 , a spatial merging candidate of the current block may be derived from the neighboring blocks adjacent to the current block X. The neighboring blocks adjacent to the current block may include at least one of a block B1 adjacent to the top of the current block, a block A1 adjacent to the left side of the current block, a block B0 adjacent to the upper right corner of the current block, a block B2 adjacent to the upper left corner of the current block, a block A0 adjacent to the lower left corner of the current block, a block A2 adjacent to the upper left side of the current block, and a block B3 adjacent to the upper left side of the current block. In addition, the neighboring blocks adjacent to the current block may include at least one of a block located between B3 and B1 and a block located between A2 and A1.
[0247] In order to derive a spatial merge candidate for the current block, it may be determined whether a neighboring block adjacent to the current block can be used to derive a spatial merge candidate for the current block. Here, it may be determined whether a neighboring block adjacent to the current block can be used to derive a spatial merge candidate for the current block according to a predetermined priority. In one example, Fig. 9 In the example shown in , the order of blocks at positions A0, A1, B0, B1, B2, B3, and A2 determines whether the derived spatial merge candidates are available. The spatial merge candidates determined based on the above priority determination can be sequentially added to the merge candidate list of the current block.
[0248] Fig.11 is a diagram illustrating an example of a process of adding a spatial merging candidate to a merging candidate list.
[0249] Reference Fig.11 , four spatial merge candidates are derived from the four neighboring blocks A1, B0, A0, and B2, and the derived spatial merge candidates may be sequentially added to the merge candidate list.
[0250] Optionally, the spatial merging candidate may be derived based on at least one of the encoding parameters.
[0251] Here, the motion information of the spatial merging candidate may include three or more pieces of motion information, wherein the three or more pieces of motion information include L2 motion information and L3 motion information in addition to L0 motion information and L1 motion information. Here, there may be at least one reference picture list, for example, including L0, L1, L2 and L3.
[0252] Next, a method of deriving a temporal merging candidate for the current block will be described.
[0253] A temporal merging candidate of the current block may be derived from a reconstructed block included in a co-located picture of the current picture. The co-located picture may be a picture that is encoded / decoded before the current picture and may be different in temporal order from the current picture.
[0254] The expression “derive a temporal merge candidate” means a process of deriving a temporal merge candidate and adding the derived temporal merge candidate to a merge candidate list.
[0255] Reference Fig.10 , in the co-located image of the current image, the temporal motion vector candidate of the current block may be derived from a block including an outer position of a block associated with the same spatial position of the current block X or from a block including an inner position of a block associated with the same spatial position of the current block X. Here, the temporal motion vector candidate may represent a motion vector of the co-located block. In one example, the temporal motion vector candidate of the current block may be derived from at least one of a block H adjacent to the lower right corner of a block C associated with the same spatial position as the current block, blocks C1 and C3 including the center point of block C, blocks C0, C4, C5, and C6 adjacent to the inner corner of block C, block D0 adjacent to the upper right side of block C, and block D1 adjacent to the lower left side of block C. Blocks H, block C1, block C3, block C0, block C4, block C5, block C6, block D0, and block D1 used to derive the temporal motion vector candidate of the current block are referred to as "co-located blocks (co-located blocks)".
[0256] When a temporal merge candidate of the current block can be derived from a block H including a position arranged outside the block C, the block H is set as a co-located block of the current block. In this case, the temporal merge candidate of the current block can be derived based on the motion information of the block H. On the contrary, when a temporal merge candidate of the current block cannot be derived from the block H, at least one of the blocks C1, C3, C0, C4, C5, C6, D0, and D1 can be set as a co-located block of the current block. When any temporal merge candidate of the current block cannot be derived from either the block H or the block C3 (for example, these blocks are intra-coded blocks), the temporal merge candidate of the current block can be derived from blocks other than these blocks.
[0257] In another example, the temporal merge candidates of the current block may be derived from multiple blocks in the co-located image. In one example, multiple temporal merge candidates of the current block may be derived from block H, block C1, block C3, block C0, block C4, block C5, block C6, block D0, or block D1.
[0258] Fig.12 is a diagram illustrating an example of a process of adding a temporal merge candidate to a merge candidate list.
[0259] Reference Fig.12When a temporal merge candidate is derived from the co-located block at position H1, the derived temporal merge candidate may be added to the merge candidate list.
[0260] When the distance between the current picture including the current block and the reference picture of the current block is different from the distance between the co-located picture including the co-located block and the reference picture of the co-located block, the motion vector of the temporal merge candidate of the current block may be obtained by scaling the motion vector of the co-located block. Here, the scaling of the motion vector may be performed based on at least one of the distance between the current picture and the reference picture of the current block and the distance between the co-located picture and the reference picture of the co-located block. For example, the motion vector of the temporal merge candidate of the current block may be derived by scaling the motion vector of the co-located block according to the ratio of the distance between the current picture and the reference picture of the current block and the distance between the co-located picture and the reference picture of the co-located block.
[0261] Furthermore, at least one of a temporal merging candidate, a co-located picture, a co-located block, a prediction list utilization flag, and a reference picture index may be derived based on at least one of encoding parameters of a current block, a neighboring block, or a co-located block.
[0262] The merge candidate list may be generated by generating at least one merge candidate of the spatial merge candidates and the temporal merge candidates and sequentially adding the derived merge candidates to the merge candidate list in the derived order.
[0263] Next, a method of deriving additional merging candidates for the current block will be described.
[0264] The additional merge candidate may represent at least one of a modified spatial merge candidate, a modified temporal merge candidate, a combined merge candidate, a merge candidate with a predetermined motion information value, and a merge candidate in units of subblocks. Here, deriving the additional merge candidate may mean adding the additional merge candidate to the merge candidate list.
[0265] The modified spatial merge candidate may mean a merge candidate obtained by modifying at least one piece of motion information of the derived spatial merge candidate.
[0266] The modified temporal merge candidate may mean a merge candidate obtained by modifying at least one piece of motion information of the derived temporal merge candidate.
[0267] The combined merge candidate may represent a merge candidate derived by combining the motion information of at least one of the spatial merge candidate, the temporal merge candidate, the modified spatial merge candidate, the modified temporal merge candidate, the combined merge candidate and the merge candidate having a predetermined motion information value present in the merge candidate list.
[0268] Optionally, the combined merge candidate may represent a merge candidate derived by combining the motion information of at least one of the following candidates: a spatial merge candidate and a temporal merge candidate that are not included in the merge candidate list but are derived from blocks from which the spatial merge candidate and the temporal merge candidate can be derived; a modified spatial merge candidate and a modified temporal merge candidate generated based on spatial merge candidates and temporal merge candidates that are not included in the merge candidate list but are derived from blocks from which the spatial merge candidate and the temporal merge candidate can be derived; a combined merge candidate; and a merge candidate with a predetermined motion information value.
[0269] Alternatively, the combined merge candidate may be derived by the decoder by using motion information entropy decoded from the bitstream. Here, the motion information used to derive the combined merge candidate by the decoder may be entropy encoded into the bitstream.
[0270] The combined merge candidate may mean a combined bi-prediction merge candidate. The combined bi-prediction merge candidate may be a merge candidate using bi-prediction and means a merge candidate having L0 motion information and L1 motion information.
[0271] The merge candidate having the predetermined motion information value may mean a zero merge candidate having a motion vector (0, 0). In addition, the merge candidate having the predetermined motion information value may be identically preset to have the same value in the encoding apparatus and the decoding apparatus.
[0272] In addition, at least one of a modified spatial merge candidate, a modified temporal merge candidate, a combined merge candidate and a merge candidate with a predetermined motion information value can be derived for each sub-block, and the motion information derived for each sub-block is called a merge candidate in sub-block units and is added to the merge candidate list.
[0273] Will use later Fig.13 and Fig.14 Merge candidates in units of sub-blocks are described in detail.
[0274] At least one of a modified spatial merge candidate, a modified temporal merge candidate, a combined merge candidate, a merge candidate with a predetermined motion information value, and a merge candidate in units of subblocks may be derived or generated based on at least one of the encoding parameters of the current block, the neighboring block, and the co-located block. In addition, at least one of a modified spatial merge candidate, a modified temporal merge candidate, a combined merge candidate, a merge candidate with a predetermined motion information value, and a merge candidate in units of subblocks may be added to a merge candidate list based on at least one of the encoding parameters of the current block, the neighboring block, and the co-located block.
[0275] In addition, the size of the merge candidate list may be determined based on encoding parameters of the current block, the neighboring block, or the co-located block, and its size may vary based on the encoding parameters.
[0276] Next, the steps ( S702 , S803 ) of determining the motion information of the current block using the generated merge candidate list will be described.
[0277] The encoder may select a merge candidate to be used for motion compensation of the current block from the merge candidate list through motion estimation, and encode a merge candidate index merge_idx indicating the determined merge candidate into a bitstream.
[0278] In order to generate a prediction block of the current block, the encoder may select a merge candidate from the merge candidate list by using the merge candidate index and determine the motion information of the current block. Then, the encoder may perform motion compensation based on the determined motion information, thereby generating a prediction block of the current block.
[0279] The decoder may decode the merge candidate index in the received bitstream and determine the merge candidate included in the merge candidate list and indicated by the merge candidate index. The determined merge candidate may be determined as the motion information of the current block. The determined motion information is used for motion compensation of the current block. Here, the term "motion compensation" may have the same meaning as inter-frame prediction.
[0280] In addition, the motion information correction may be applied to the motion information determined based on the merge candidate index or any one of the merge candidates included in the merge candidate list. A detailed description of the motion information correction will be described later.
[0281] Next, the steps ( S504 , S605 , S703 , S804 ) of performing motion compensation using the motion vector or the motion information will be described.
[0282] The encoding device and the decoding device may calculate the motion vector of the current block by using the predicted motion vector and the motion vector difference. After calculating the motion vector, the encoding device and the decoding device may perform inter-frame prediction or motion compensation using the calculated motion vector (S504, S605).
[0283] The encoding apparatus and the decoding apparatus may perform inter prediction or motion compensation using the determined motion information (S703, S804). Here, the current block may have the determined motion information of the merge candidate.
[0284] According to the prediction direction of the current block, the current block may have one (minimum) to N (maximum) motion vectors. One to N motion vectors may be used to generate one (minimum) to N (maximum) prediction blocks, and a final prediction block may be selected from the generated prediction blocks.
[0285] For example, when the current block has one motion vector, a prediction block generated using the motion vector (or motion information) is determined as a final prediction block of the current block.
[0286] In addition, when the current block has multiple motion vectors (or multiple pieces of motion information), multiple prediction blocks are generated using the multiple motion vectors (or multiple pieces of motion information), and a final prediction block of the current block is determined based on a weighted sum of the multiple prediction blocks. Multiple reference pictures, each including a plurality of prediction blocks respectively indicated by a plurality of motion vectors (or multiple pieces of motion information), may be listed in different reference picture lists or in one reference picture list.
[0287] For example, multiple prediction blocks of the current block may be generated based on at least one of a spatial motion vector candidate, a temporal motion vector candidate, a motion vector having a predetermined value, and a combined motion vector candidate, and then a final prediction block of the current block may be determined based on a weighted sum of the multiple prediction blocks.
[0288] Alternatively, for example, multiple prediction blocks of the current block may be generated based on motion vector candidates indicated by preset motion vector candidate indices, and then the final prediction block of the current block may be determined based on a weighted sum of the multiple prediction blocks. In addition, multiple prediction blocks may be generated based on motion vector candidates indicated by indices within a predetermined motion vector candidate index range, and then the final prediction block of the current block may be determined based on a weighted sum of the multiple prediction blocks.
[0289] The weighting factors for each prediction block may be equally 1 / N (where N is the number of prediction blocks generated). For example, when two prediction blocks are generated, the weighting factor for each prediction block is 1 / 2. Similarly, when three prediction blocks are generated, the weighting factor for each prediction block is 1 / 3. When four prediction blocks are generated, the weighting factor for each prediction block may be 1 / 4. Alternatively, the final prediction block of the current block may be determined in a manner that different weighting factors are applied to each prediction block.
[0290] The weighting factor for the prediction block may not be fixed but variable. The weighting factors for the prediction block may not be equal but different. For example, when two prediction blocks are generated, the weighting factors for the two prediction blocks may be equal, such as (1 / 2, 1 / 2), or may be unequal, such as (1 / 3, 2 / 3), (1 / 4, 3 / 4), (2 / 5, 3 / 5) or (3 / 8, 5 / 8). The weighting factor may be a positive real value or a negative real value. That is, the value of the weighting factor may include a negative real value such as (-1 / 2, 3 / 2), (-1 / 3, 4 / 3) or (-1 / 4, 5 / 4).
[0291] In order to apply a variable weighting factor, one or more weighting factor information for the current block may be signaled via the bitstream. The weighting factor information may be signaled on a prediction block-by-prediction block basis or on a reference picture-by-reference picture basis. Optionally, multiple prediction blocks may share one weighting factor.
[0292] The encoding device and the decoding device may determine whether to use the prediction motion vector (or prediction motion information) based on the prediction block list utilization flag. For example, for each reference picture list, when the prediction block list utilization flag has a first value of one (1), the encoding device and the decoding device may use the prediction motion vector for the current block to perform inter-frame prediction or motion compensation of the current block. However, when the prediction block list utilization flag has a second value of zero (0), the encoding device and the decoding device may not use the prediction motion vector for the current block to perform inter-frame prediction or motion compensation of the current block. The first value and the second value of the prediction block list utilization flag may be set to 0 and 1, respectively, inversely. Expressions 3 to 5 are examples of a method of generating a final prediction block of the current block when the inter-frame prediction indicator of the current block is PRED_BI, PRED_TRI, or PRED_QUAD and when the prediction direction for each reference picture list is unidirectional.
[0293] [Expression 1]
[0294] P_BI=(WF_L0*P_L0+OFFSET_L0+WF_L1*P_L1+OFFSET_L1+RF)>>1
[0295] [Expression 2]
[0296] P_TRI=(WF_L0*P_L0+OFFSET_L0+WF_L1*P_L1+OFFSET_L1+WF_L2*P_L2+OFFSET_L2+RF) / 3
[0297] [Expression 3]
[0298] P_QUAD=(WF_L0*P_L0+OFFSET_L0+WF_L1*P_L1+OFFSET_L1+WF_L2*P_L2+OFFSET_L2+WF_L3*P_L3+OFFSET_L+RF)>>2
[0299] In Expression 1 to Expression 3, each of P_BI, P_TRI, and P_QUAD represents a final prediction block of a current block, and LX (X=0, 1, 2, 3) represents a reference picture list. WF_LX represents the use of LX
[0300] The weighting factor of the prediction block generated by the reference picture list. OFFSET_LX represents the offset value for the prediction block generated using the LX reference picture list. P_LX represents the prediction block generated using the motion vector (or motion information) of the LX reference picture list of the current block. RF represents the rounding factor, and it can be set to 0, a positive integer or a negative integer. The LX reference picture list may include at least one of a long-term reference picture, a reference picture that has not undergone a deblocking filter, a reference picture that has not undergone a sample adaptive offset, a reference picture that has not undergone an adaptive loop filter, a reference picture that has only undergone a deblocking filter and an adaptive loop filter, a reference picture that has undergone a sample adaptive offset and an adaptive loop filter, and a reference picture that has undergone a deblocking filter, a sample adaptive offset, and an adaptive loop filter. In this case, the LX reference picture list may be at least any one of the L2 reference picture list and the L3 reference picture list.
[0301] Even when there are multiple prediction directions for a predetermined reference picture list, the final prediction block of the current block can be obtained based on the weighted sum of the prediction blocks. In this case, the weighting factors for multiple prediction blocks derived using one reference picture list may be equal to or different from each other.
[0302] At least any one of the weighting factors WF_LX or the offset OFFSET_LX of the plurality of prediction blocks may be a coding parameter to be entropy encoded / decoded. Alternatively, for example, the weighting factor and the offset may be derived from a previously encoded / decoded neighboring block adjacent to the current block. Here, the neighboring block adjacent to the current block may include at least one of a block for deriving a spatial motion vector candidate of the current block and a block for deriving a temporal motion vector candidate of the current block.
[0303] Further optionally, for example, the weighting factor and the offset may be determined based on the display order (picture order count (POC)) of the current picture and the POC of each of the reference pictures. In this case, as the distance between the current picture and the reference picture increases, the value of the weighting factor or the offset may decrease. That is, when the current picture and the reference picture are close to each other, a larger value may be set as the weighting factor or the offset. For example, when the difference between the POC of the current picture and the POC of the L0 reference picture is 2, the value of the weighting factor applied to the prediction block generated using the L0 reference picture may be set to 1 / 3. In addition, when the difference between the POC of the current picture and the POC of the L0 reference picture is 1, the value of the weighting factor applied to the prediction block generated using the L0 reference picture may be set to 2 / 3. As described above, the weighting factor or the offset may be inversely proportional to the difference between the display order (POC) of the current picture and the reference picture. Alternatively, the weighting factor or the offset may be proportional to the difference between the display order of the current picture and the reference picture.
[0304] Optionally, for example, at least one of the weighting factor and the offset may be entropy encoded / decoded based on at least one encoding parameter. In addition, a weighted sum of the prediction block may be calculated based on at least one encoding parameter.
[0305] The weighted sum of multiple prediction blocks may be applied only to a partial region of the prediction block. The partial region may be a boundary region adjacent to a boundary of each prediction block. In order to apply the weighted sum only to the partial region as described above, the weighted sum may be calculated sub-block by sub-block in each prediction block.
[0306] In a block having a block size indicated by the region information, inter prediction or motion compensation may be performed on a subblock smaller than the block by using the same prediction block or the same final prediction block.
[0307] In a block having a block depth indicated by the region information, inter prediction or motion compensation may be performed on a subblock having a block depth deeper than that of the block by using the same prediction block or the same final prediction block.
[0308] Also, when a weighted sum of a prediction block is calculated through motion vector prediction, the weighted sum may be calculated using at least one of the motion vector candidates included in the motion vector candidate list, and the calculation result may be used as a final prediction block of the current block.
[0309] For example, a prediction block may be generated using only spatial motion vector candidates, a weighted sum of the prediction blocks may be calculated, and the calculated weighted sum may be used as a final prediction block of the current block.
[0310] For example, a prediction block may be generated using the spatial motion vector candidate and the temporal motion vector candidate, a weighted sum of the prediction blocks may be calculated, and the calculated weighted sum may be used as a final prediction block of the current block.
[0311] For example, a prediction block may be generated using only the combined motion vector candidates, a weighted sum of the prediction blocks may be calculated, and the calculated weighted sum may be used as a final prediction block of the current block.
[0312] For example, a prediction block may be generated using only motion vector candidates indicated by a specific index, a weighted sum of the prediction blocks may be calculated, and the calculated weighted sum may be used as a final prediction block of the current block.
[0313] For example, a prediction block may be generated using only motion vector candidates indicated by indices within a predetermined index range, a weighted sum of the prediction blocks may be calculated, and the calculated weighted sum may be used as a final prediction block of the current block.
[0314] When the weighted sum of the prediction block is calculated using the merge mode, the weighted sum may be calculated using at least one of the merge candidates included in the merge candidate list, and the calculation result may be used as the final prediction block of the current block.
[0315] For example, a prediction block may be generated using only spatial merging candidates, a weighted sum of the prediction blocks may be calculated, and the calculated weighted sum may be used as a final prediction block of the current block.
[0316] For example, a prediction block may be generated using the spatial merging candidates and the temporal merging candidates, a weighted sum of the prediction blocks may be calculated, and the calculated weighted sum may be used as a final prediction block of the current block.
[0317] For example, a prediction block may be generated using only the combined merging candidates, a weighted sum of the prediction blocks may be generated, and the calculated weighted sum may be used as a final prediction block of the current block.
[0318] For example, a prediction block may be generated using only merge candidates indicated by a specific index, a weighted sum of the prediction blocks may be generated, and the calculated weighted sum may be used as a final prediction block of the current block.
[0319] For example, a prediction block may be generated using only merge candidates indicated by indexes within a predetermined index range, a weighted sum of the prediction blocks may be calculated, and the calculated weighted sum may be used as a final prediction block of the current block.
[0320] In the encoder and decoder, motion compensation may be performed using a motion vector or motion information of the current block. At this time, at least one prediction block may be used to determine a final prediction block as a result of motion compensation. Here, the current block may refer to at least one of the current coding block and the current prediction block.
[0321] A final prediction block of the current block may be generated by performing overlapped block motion compensation on a boundary region of the current block.
[0322] The boundary area of the current block may be an area arranged within the current block and adjacent to a boundary between the current block and a neighboring block of the current block. The boundary area of the current block may include at least one of an upper boundary area, a left boundary area, a lower boundary area, a right boundary area, an upper right corner area, a lower right corner area, an upper left corner area, and a lower left corner area. The boundary area of the current block may be an area corresponding to a portion of a prediction block of the current block.
[0323] Overlapped block motion compensation may refer to a process of performing motion compensation by calculating a weighted sum of a prediction block corresponding to a boundary area of a current block and a prediction block generated using motion information of an encoded / decoded block adjacent to the current block.
[0324] The weighted sum calculation may be performed sub-block by sub-block by dividing the current block into a plurality of sub-blocks. That is, motion compensation of the current block may be performed sub-block by sub-block using motion information of encoded / decoded sub-blocks adjacent to the current block. The sub-block may represent a lower layer block of the current block.
[0325] In addition, in the calculation of the weighted sum, a first prediction block generated for each subblock of the current block using the motion information of the current block and a second prediction block generated using the motion information of a neighboring subblock spatially adjacent to the current block may be used. In this case, the expression "using motion information" means "deriving motion information". The first prediction block may refer to a prediction block generated by using the motion information of an encoding / decoding target subblock within the current block. The second prediction block may be a prediction block generated by using the motion information of a neighboring subblock spatially adjacent to the encoding / decoding target subblock within the current block.
[0326] The final prediction block of the current block may be generated using the weighted sum of the first prediction block and the second prediction block. That is, overlapped block motion compensation is to find the final prediction block of the current block using the motion information of the current block and the motion information of another block.
[0327] In addition, when at least one of the advanced motion vector prediction (AMVP) mode, merge mode, affine motion compensation mode, decoder-side motion vector derivation mode, adaptive motion vector resolution mode, local illumination compensation mode, and bidirectional optical flow mode is used, the current block can be divided into multiple sub-blocks, and overlapping block motion compensation can be performed on a sub-block basis.
[0328] When the merge mode is used for motion compensation, overlapped block motion compensation may be performed on at least one of an advanced temporal motion vector predictor (ATMVP) candidate and a spatio-temporal motion vector predictor (STMVP) candidate.
[0329] Next, a process of performing entropy encoding / decoding on information associated with motion compensation ( S505 , S601 , S704 , S801 ) will be described.
[0330] The encoding device may entropy encode the information associated with motion compensation as a bitstream, and the decoding device may decode the information associated with motion compensation included in the bitstream. The information associated with motion compensation as a target of entropy encoding or entropy decoding may include at least one of an inter-frame prediction indicator inter_pred_idc, reference picture indexes ref_idx_l0, ref_idx_l1, ref_idx_l2, and ref_idx_l3, motion vector candidate indexes mvp_l0_idx, mvp_l1_idx, mvp_l2_idx, and mvp_l3_idx, motion vector difference, skip mode use / non-use information cu_skip_flag, merge mode use / non-use information merge_flag, merge index information merge_index, weighting factors wf_l0, wf_l1, wf_l2, and wf_l3, and offset values offset_l0, offset_l1, offset_l2, and offset_l3.
[0331] When the current block is encoded / decoded by inter-frame prediction, the inter-frame prediction indicator may indicate the prediction direction of the inter-frame prediction, the number of prediction directions, or both the prediction direction and the number of prediction directions of the inter-frame prediction. For example, the inter-frame prediction indicator may indicate a unidirectional prediction or a multi-directional prediction, such as a bidirectional prediction, a tri-directional prediction, and a quad-directional prediction. The inter-frame prediction indicator may indicate the number of reference pictures used to generate the prediction block of the current block. Optionally, one reference picture may be used for prediction in multiple directions. In this case, M reference pictures are used to perform prediction in N directions (where N>M). The inter-frame prediction indicator may also indicate the number of prediction blocks used for inter-frame prediction or motion compensation for the current block.
[0332] The reference picture indicator may indicate one direction PRED_LX, two directions PRED_BI, three directions PRED_TRI, four directions PRED_QUAD, or more directions according to the number of prediction directions of the current block.
[0333] The prediction list for a specific reference picture list uses a flag to indicate whether the reference picture list is used to generate a prediction block.
[0334] For example, when the prediction list utilization flag of a specific reference picture list has a first value of one (1), this indicates that the prediction block is generated using the reference picture list. When the prediction list utilization flag has a second value of zero (0), this indicates that the prediction block is not generated using the reference picture list. Here, the first value and the second value of the prediction list utilization flag may be set to 0 and 1, respectively, inversely.
[0335] That is, when the prediction list utilization flag of a specific reference picture list has a first value, the prediction block of the current block may be generated using motion information corresponding to the reference picture list.
[0336] The reference picture index may indicate a specific reference picture present in the reference picture list and referenced by the current block. For each reference picture list, one or more reference picture indexes may be entropy encoded / decoded. One or more reference picture indexes may be used to perform motion compensation on the current block.
[0337] The motion vector candidate index indicates a motion vector candidate of the current block among the motion vector candidates included in the motion vector candidate list prepared for each reference picture list or each reference picture index. At least one or more motion vector candidate indexes may be entropy encoded / decoded for each motion vector candidate list. At least one or more motion vector candidate indexes may be used to perform motion compensation on the current block.
[0338] The motion vector difference represents the difference between the current motion vector and the predicted motion vector. For each of the motion vector candidate lists generated for each reference picture list or each reference picture index of the current block, one or more motion vector differences may be entropy encoded / decoded. One or more motion vector differences may be used to perform motion compensation on the current block.
[0339] Regarding the skip mode use / non-use information cu_skip_flag, when it has a first value of one (1), the skip mode may be used. On the contrary, when it has a second value of zero (0), the skip mode may not be used. Motion compensation for the current block may be performed using the skip mode according to the skip mode use / non-use information.
[0340] Regarding the merge mode use / non-use information merge_flag, when it has a first value of one (1), the merge mode may be used. Conversely, when it has a second value of zero (0), the merge mode may not be used. Motion compensation for the current block may be performed using the merge mode according to the merge mode use / non-use information.
[0341] The merge index information merge_index may represent information indicating a merge candidate within the merge candidate list.
[0342] Optionally, the merge index information may represent information about the merge index.
[0343] In addition, the merge index information may indicate a reconstructed block used for deriving a merge candidate among reconstructed blocks spatially / temporally adjacent to the current block.
[0344] The merge index information may indicate one or more pieces of motion information that a merge candidate has. For example, when the merge index information has a first value of zero (0), the merge index information may indicate a first merge candidate listed as a first entry in a merge candidate list; when the merge index information has a second value of one (1), the merge index information may indicate a second merge candidate listed as a second entry in the merge candidate list; and when the merge index information has a third value of two (2), the merge index information indicates a third merge candidate listed as a third entry in the merge candidate list. Similarly, when the merge index information has values from a fourth value to an Nth value, the merge index information may indicate a merge candidate listed at a position in the merge candidate list in an order according to the values. Here, N may be 0 or a positive integer.
[0345] Motion compensation may be performed on the current block using the merge mode based on the merge mode index information.
[0346] When two or more prediction blocks are generated during motion compensation of the current block, the final prediction block of the current block may be determined based on the weighted sum of the prediction blocks. When calculating the weighted sum, a weighting factor, an offset, or both a weighting factor and an offset may be applied to each prediction block. The weighted sum factor (e.g., a weighting factor and an offset) used to calculate the weighted sum may be entropy encoded / decoded by some of or corresponding to at least one of the following: a reference picture list, a reference picture, a motion vector candidate index, a motion vector difference, a motion vector, skip mode use / non-use information, merge mode use / non-use information, merge index information. In addition, the weighted sum factor for each prediction block may be entropy encoded / decoded based on an inter-frame prediction indicator. The weighted sum factor may include at least one of a weighting factor and an offset.
[0347] The information associated with motion compensation may be entropy encoded / decoded on a block-by-block basis, or may be entropy encoded / decoded in units of higher-level units. For example, the information associated with motion compensation may be entropy encoded / decoded on a block-by-block basis (e.g., CTU-by-CTU, CU-by-CU, or PU-by-PU basis). Alternatively, the information associated with motion compensation may be entropy encoded / decoded in units of higher-level units such as video parameter sets, sequence parameter sets, picture parameter sets, adaptation parameter sets, or slice headers.
[0348] The information associated with motion compensation may be entropy encoded / decoded based on a motion compensation information difference indicating a difference between the information associated with motion compensation and a predicted value of the information associated with motion compensation.
[0349] Information associated with motion compensation of an encoding / decoding block adjacent to a current block may be used as information associated with motion compensation of the current block without entropy encoding / decoding the information associated with motion compensation of the current block.
[0350] At least one piece of information associated with motion compensation may be derived based on at least one encoding parameter.
[0351] The bitstream may be decoded based on the at least one encoding parameter to generate at least one piece of information associated with motion compensation. Conversely, the at least one piece of information associated with motion compensation may be entropy encoded into the bitstream based on the at least one encoding parameter.
[0352] At least one piece of information associated with motion compensation may include at least one of a motion vector, a motion vector candidate, a motion vector candidate index, a motion vector difference, a motion vector predictor, skip mode use / non-use information skip_flag, merge mode use / non-use information merge_flag, merge index information merge_index, motion vector resolution information, overlapped block motion compensation information, local illumination compensation information, affine motion compensation information, decoder-side motion vector derivation information, and bidirectional optical flow information. Here, the decoder-side motion vector derivation may represent pattern matching motion vector derivation.
[0353] The motion vector resolution information may be information indicating which specific resolution is used for at least one of the motion vector and the motion vector difference. Here, the resolution may indicate accuracy. The specific resolution may be set to at least any one of a 16-pixel (16-pel) unit, an 8-pixel (8-pel) unit, a 4-pixel (4-pel) unit, an integer pixel (integer-pel) unit, a 1 / 2-pixel (1 / 2-pel) unit, a 1 / 4-pixel (1 / 4-pel) unit, a 1 / 8-pixel (1 / 8-pel) unit, a 1 / 16-pixel (1 / 16-pel) unit, a 1 / 32-pixel (1 / 32-pel) unit, and a 1 / 64-pixel (1 / 64-pel) unit.
[0354] The overlapped block motion compensation information may be information indicating whether a motion vector of a neighboring block spatially adjacent to the current block is additionally used to calculate a weighted sum of a prediction block of the current block during motion compensation of the current block.
[0355] The local illumination compensation information may be information indicating whether one of a weighting factor and an offset is applied when generating a prediction block of a current block. Here, at least one of the weighting factor and the offset may be a value calculated based on a reference block.
[0356] The affine motion compensation information may be information indicating whether to use an affine motion model for motion compensation of a current block. Here, the affine motion model may be a model that divides a block into a plurality of sub-blocks using a plurality of parameters and calculates motion vectors of the sub-blocks using a representative motion vector.
[0357] The decoder-side motion vector derivation information may be information indicating whether a motion vector required for motion compensation is derived by the decoder and then used in the decoder. According to the decoder-side motion vector derivation information, entropy encoding / decoding of information associated with the motion vector may not be performed. When the decoder-side motion vector derivation information indicates that the motion vector is derived by the decoder and then used in the decoder, entropy encoding / decoding of information associated with the merge mode may be performed. That is, the decoder-side motion vector derivation information may indicate whether the merge mode is used in the decoder.
[0358] The bidirectional optical flow information may be information indicating whether to modify the motion vector pixel by pixel or sub-block by sub-block and then use the modified motion vector for motion compensation. According to the bidirectional optical flow information, the motion vector may not be entropy encoded / decoded pixel by pixel or sub-block by sub-block. Modifying the motion vector means converting the value of the block-based motion vector into the value of the pixel-based motion vector or the value of the sub-block-based motion vector.
[0359] The current block may be motion compensated based on at least one piece of information associated with motion compensation, and the at least one piece of information associated with motion compensation may be entropy encoded / decoded.
[0360] When entropy encoding / decoding information associated with motion compensation, a binarization method such as a truncated Rice binarization method, a K-order exponential Golomb binarization method, a finite K-order exponential Golomb binarization method, a fixed-length binarization method, a unary binarization method, and a truncated unary binarization method can be used.
[0361] When entropy encoding / decoding is performed on information associated with motion compensation, the context model may be determined based on at least one of the following items: information associated with motion information of a neighboring block adjacent to the current block or region information of the neighboring block; previously encoded / decoded information associated with motion compensation or previously encoded / decoded region information; information about the depth of the current block; and information about the size of the current block.
[0362] In addition, when entropy encoding / decoding is performed on information associated with motion compensation, entropy encoding / decoding can be performed by using at least one of information associated with motion compensation of a neighboring block, previously encoded / decoded information associated with motion compensation, information about the depth of the current block, and information about the size of the current block as a prediction value of the information associated with motion compensation of the current block.
[0363] In the following, reference will be made to Fig.13 and Fig.14 Detailed description of subblock-based motion vector candidates or subblock-based merge candidates. Here, subblock-based motion vector candidates and subblock-based merge candidates may be determined by the following method of predicting motion information in subblock units.
[0364] When the current block is divided into a plurality of sub-blocks and motion information is derived in units of sub-blocks, the following two methods of predicting motion information in units of sub-blocks may be used. In the following two methods of predicting motion information in units of sub-blocks, an example of predicting a motion vector is shown, and by using the method of predicting motion information in units of sub-blocks, at least one piece of information included in the motion information and the motion vector may be predicted. In the following examples, for convenience, motion vectors in units of sub-blocks are shown instead of motion information in units of sub-blocks.
[0365] In addition, motion information determined in sub-block units by using at least one of the following methods may be included in the additional merge candidate. In other words, motion information determined in sub-block units may be included in the merge / skip candidate list as a merge candidate in sub-block units. In addition, motion information of a sub-block unit determined by using at least one of the following methods may be included in the additional merge candidate of the sub-block merge candidate list. In other words, motion information determined in sub-block units may be included in the sub-block merge / skip candidate list as a merge candidate in sub-block units.
[0366] In addition, motion information of a sub-block unit determined by using at least one of the following methods may be included in the temporal merge candidate. In other words, motion information determined in sub-block units may be included in the merge / skip candidate list as a temporal merge candidate. In addition, motion information of a sub-block unit determined by using at least one of the following methods may be included in the temporal merge candidate of the sub-block merge candidate list. In other words, motion information determined in sub-block units may be included in the merge / skip candidate list as a temporal merge candidate in sub-block units.
[0367] Furthermore, the motion vector of the subblock unit determined as above may be included in the motion vector candidate list as an additional motion vector candidate (ie, a motion vector candidate in units of subblocks), or may be included in the motion vector candidate list as a temporal motion vector candidate.
[0368] Fig.13 is a diagram illustrating a method of predicting motion information in units of a first subblock according to an embodiment of the present invention.
[0369] The method of predicting motion information in units of the first sub-block may mean a method of predicting motion information of a sub-block in the current block by using motion information of a sub-block existing in a co-located block in a reference image. In other words, this may mean determining motion information of a sub-block obtained by dividing the current block based on motion information of a sub-block obtained by dividing the co-located block. Here, the motion information may include at least one of a motion vector, a reference image index, and an inter-prediction direction.
[0370] The method for predicting motion information in units of first sub-blocks may be performed in two steps. In the first step, at least one of a reference image and a co-located block may be identified, and in the second step, a current block may be divided into a plurality of sub-blocks, and motion information may be determined in units of sub-blocks of the current block from sub-blocks within the co-located block.
[0371] In the first step, at least one of a co-located image and a co-located block may be determined by using motion information of at least one of blocks spatially adjacent to the current block. Here, the co-located image may be an image including the co-located block. Information indicating the co-located image may be information signaled in units of at least one of a sequence, a picture, a slice, a tile group, and a tile, and may be represented in an index form.
[0372] For example, in motion information of a block spatially adjacent to the current block, a motion vector may be determined as a temporal motion vector, and a reference picture index may be determined as an index indicating a co-located picture.
[0373] In addition, at least one of the co-located image and the co-located block may be determined by using at least one encoding parameter having a specific value. The specific value may be a fixed value predefined in the encoder / decoder, or may be a variable value encoded from the encoder and transmitted with a signal.
[0374] For example, when a merge candidate in the merge candidate list has at least two motion vectors and at least two reference image indexes, a difference value may be calculated by comparing the POC of the reference image indicated by the at least two reference image indexes with the POC of the current image, and the reference image having the smallest difference between the POC and the POC of the current image may be determined as the co-located image for predicting motion information in units of sub-blocks. Here, a small difference between the POCs may indicate that the reference image is temporally adjacent to the current image.
[0375] For example, when a merge candidate within the merge candidate list has at least two motion vectors and at least two reference image indexes, a reference image indicated by the reference image index in the L0 direction may be determined as a co-located image for predicting motion information in units of subblocks.
[0376] For example, when a merge candidate within the merge candidate list has at least two motion vectors and at least two reference image indexes, a reference image indicated by the reference image index in the L1 direction may be determined as a co-located image for predicting motion information in units of subblocks.
[0377] For example, a motion vector of one merge candidate in the merge candidate list may be determined as a temporal motion vector indicating a co-located block, and a reference image index indicating a co-located image may be determined as a specific value. The specific value may be a fixed value predefined in the encoder / decoder, or may be a variable value encoded from the encoder and signaled. For example, the specific value may be 0. Here, the merge candidate having a motion vector indicating a co-located block may be a merge candidate having a reference image index that is the same as the POC of the determined co-located image.
[0378] For example, the co-located image may be determined by using a reference image index of a merge candidate in the merge candidate list, and the motion vector may be determined as a specific value. The specific value may be a fixed value predefined in the encoder / decoder, or may be a variable value encoded from the encoder and sent with a signal. For example, the specific value may be (0,0).
[0379] For example, the co-located image and the co-located block may be determined according to motion information included in the first merge candidate in the merge candidate list. The motion vector of the first merge candidate may be determined as a temporal motion vector, and the reference image index of the first merge candidate may be used as an index indicating the co-located image.
[0380] In the second step, the co-located block may be determined by adding the coordinates of the current block to the temporal motion vector determined in the first step. The motion information present at each sub-block obtained by dividing the determined co-located block may be determined as the motion information of each sub-block unit obtained by dividing the current block. Here, the motion information of the sub-block within the current block may be determined from the sub-block spatially located in association with the sub-block unit within the co-located block. Here, by using the determined temporal motion vector, the motion information of the sub-block within the current block may be determined as the motion information of the sub-block within the co-located block associated with at least one of the upper left position, the lower left position, the upper right position, the lower right position, and the center position included in the sub-block within the current block.
[0381] Here, in the motion information of each sub-block unit within the current block, scaling may be performed on the motion vector.
[0382] Here, the merge candidate in the merge candidate list may be motion information derived from a reconstructed block adjacent to the current block, and may represent at least one piece of motion information that may be included in the merge candidate list as a merge candidate. For example, the merge candidate in the merge candidate list may be a motion vector derived from a reconstructed block adjacent to the current block, and may represent a motion vector that may be included in the merge candidate list as a merge candidate.
[0383] Fig.14 is a diagram illustrating a method of predicting motion information in units of second sub-blocks according to an embodiment of the present invention.
[0384] The method of predicting motion information in units of the second sub-block means a method of predicting motion information of a sub-block in the current block by using at least one of motion information of a sub-block obtained by dividing the co-located block and motion information of a sub-block adjacent to the sub-block in the current block. In other words, a calculation result obtained by using at least one of motion information of a sub-block obtained by dividing the co-located block and motion information of a sub-block adjacent to the sub-block of the current block may be determined as motion information of the sub-block obtained by dividing the current block. Here, the calculation result may be a statistical value result of at least one of a median, a minimum, a maximum, an average, a weighted average, and a mode.
[0385] In the method of predicting motion information in units of second sub-blocks, the motion information may be recursively derived in units of sub-blocks within the current block in a specific order. Here, the specific order may be at least one of a raster scan order, a horizontal scan order, a vertical scan order, a zigzag scan order, and a diagonal scan order. The specific order may be a fixed order predefined in an encoder / decoder, or may be an order determined by information determined in an encoder and signaled in a decoder.
[0386] For example, when the current block is divided into N sub-blocks and the current sub-block corresponds to one of the N sub-blocks, it can be determined whether the upper block of the current sub-block is available and whether inter-frame prediction is performed for the upper block. In addition, it can be determined whether the left block of the current sub-block is available and whether inter-frame prediction is performed for the left block. As a result, when the upper block or the left block is available and inter-frame prediction is performed for the upper block or the left block, spatial motion information can be determined from the corresponding block. Temporal motion information can be determined from a co-located block in a co-located image associated with a block at the position of at least one sub-block of the N sub-blocks in the current block. Here, scaling can be performed for the derived spatial motion information or the derived temporal motion information based on at least one encoding parameter of the current block. In addition, the motion information of the current sub-block can be determined by using a statistical value of at least one of the derived spatial motion information and the derived temporal motion information. Here, N can be a positive integer, for example, 4.
[0387] Here, spatial motion information may be derived by using motion information based on at least one neighboring subblock of the current subblock, and temporal motion information may be derived by using motion information based on at least one neighboring subblock of the current subblock.
[0388] Furthermore, the number of subblocks within the current block may be determined based on at least one encoding parameter of the current block.
[0389] For example, the number of sub-blocks may vary according to the size of the current block. When the size of the current block is 32×32, the number of sub-blocks may be 16, and when the size of the current block is 64×64, the number of sub-blocks may be 4.
[0390] For example, the number of sub-blocks may vary according to the shape of the current block. When the shape of the current block is a 16×32 vertically long non-square, the number of sub-blocks may be 8, and when the shape of the current block is a 32×32 square, the number of sub-blocks may be 16.
[0391] Furthermore, a sub-block within the current block may be fixed to have a square shape regardless of the shape of the current block.
[0392] Furthermore, the shape of a subblock within the current block may be identically determined as a block shape of the current block.
[0393] For example, when the current block has a vertically long non-square shape, the shape of the sub-block may be determined to be a vertically long non-square shape.
[0394] The motion information of the sub-block unit determined by using at least one of the above-mentioned methods of predicting motion information in units of the first sub-block and in units of the second sub-block may be included as a merge candidate in the merge candidate list. Here, a merge index may be used to indicate the motion information of the determined sub-block unit.
[0395] In addition, the motion information of the sub-block unit determined by using at least one of the above-mentioned methods of predicting motion information in units of the first sub-block and in units of the second sub-block may be included as a merge candidate in units of sub-blocks in the merge candidate list. Here, a merge index in units of sub-blocks of the merge candidate list may be used to indicate the motion information of the determined sub-block unit.
[0396] In addition, additional encoding information may be used to indicate the use of motion information of a sub-block unit determined by using at least one of the above-mentioned methods of predicting motion information in units of first sub-blocks and in units of second sub-blocks. Here, the encoding information may be sent from the encoder to the decoder by a signal and may be a flag or index information.
[0397] In addition, when the size of the current block including the subblock is equal to or greater than a preset value, motion information prediction of the subblock unit may be applied. Here, the preset value may be 8×8.
[0398] Also, motion information of a sub-block unit may be stored in motion information in units of sub-blocks and later used for inter prediction of a block to be encoded / decoded.
[0399] Furthermore, at least one of adaptive sample offset, deblocking filtering, and adaptive loop filtering may be performed according to motion information of the sub-block unit.
[0400] In addition, in order to reduce memory access bandwidth, the sub-block may have a size unit of M×N. M and N may be positive integers, and M and N may have different values from each other. For example, each of M and N may be 8.
[0401] The size of the subblock may be determined in units of at least one of a sequence, a picture, a slice, a tile group, a tile, and a CTU. In other words, the size of the subblock may be defined in units of at least one of a sequence, a picture, a slice, a tile group, a tile, and a CTU.
[0402] For example, the size of the sub-block may be 4×4 or 8×8 in units of tile groups.
[0403] For example, the size of the sub-block may be 8×8 or 16×16 in units of a picture.
[0404] For example, the size of the sub-block may be 8×8 or 16×16 in units of slices.
[0405] Here, information indicating the size of a subblock in units of a sequence, a picture, a slice, a tile, a tile group, and a CTU may be entropy encoded / decoded.
[0406] In addition, in order to reduce the memory access bandwidth, the motion information of the co-located image may be stored in units of M×N. Here, M and N may be positive integers, and M and N may have different values from each other. For example, each of M and N may be 8. Here, M and N having different values from each other indicates that the motion information of the co-located image may be stored in units of non-squares. When M and N have fixed values within an image, this may indicate that the units of the motion information within the image are the same.
[0407] When the current image is divided into sub-image units such as tiles, strips, etc., the shape of the unit in which the motion information of the co-located image is stored may be determined based on the shape of the corresponding sub-image unit. For example, when the current sub-image has a non-square shape, the unit in which the motion information of the co-located image is stored may have a non-square shape.
[0408] In addition, when the current image is divided into sub-image units such as tiles, strips, etc., the size of the non-square unit in which the motion information of the co-located image is stored may be determined based on the aspect ratio of the horizontal length and the vertical length of the corresponding sub-image. For example, when the aspect ratio of the horizontal length and the vertical length of the current sub-image unit is 2:1, the size of the non-square unit in which the motion information of the reference image is stored may be determined so that the aspect ratio is 2:1.
[0409] The motion information of the co-located image may not be stored in units of M×N, and may be stored in units of coding / decoding blocks. Here, the storage of the motion information in units of coding / decoding blocks may mean that the motion information is stored in units of the size of blocks used when encoding / decoding the image. In other words, the unit in which the motion information within the image is stored may vary.
[0410] Here, the stored motion information of the co-located image may be later used as a temporal motion vector, a temporal merge candidate in another image, or used in at least one of methods of predicting motion information in units of sub-blocks. When the motion information of the co-located image is stored in units of M×N, the motion information stored in units of M×N may be used as the motion information of the current block by using at least one of the coordinates of the current block, the coordinates of the co-located block, and size information of the M×N units.
[0411] The corrected motion information of the sub-block unit can be derived by performing the above method of predicting motion information in units of two sub-blocks, and then performing a motion information correction method to be described later on the determined motion information of the sub-block unit.
[0412] In addition, in the motion correction mode, the corrected motion information of the sub-block unit can be calculated by executing the above-mentioned method of predicting motion information in units of two sub-blocks, and then performing bilateral template matching described later on the determined motion information of the sub-block unit.
[0413] In the following, reference will be made to Figures 15 to 21 The motion information correction method is described in detail. Here, the motion information correction may refer to a process of correcting at least one piece of motion information. That is, the information to be subjected to motion information correction may be at least one of the following motion information: motion vector, reference picture index, reference picture, inter-picture prediction indicator, prediction list utilization flag, weighting factor, offset, etc. The motion information correction refers to correcting the value of at least one of the motion vector, reference picture index, reference picture, inter-picture prediction indicator, prediction list utilization flag, weighting factor, and offset.
[0414] The information on which motion information modification is to be performed may be information such as at least one parameter among various encoding parameters. In this case, the motion information modification means modifying the value of the information such as at least one parameter among the encoding parameters.
[0415] By modifying the motion information, modified motion information is generated. The modified motion information obtained by the motion information modification can be used for motion compensation of the current block to be encoded / decoded (hereinafter also referred to as the encoding / decoding target block) ( Figure 5 S504 in Figure 6 S605 in Figure 7 S703 in Figure 8 S804 in ).
[0416] Regarding the timing for performing motion information correction, motion information correction may be performed before performing motion compensation. That is, before performing motion compensation on the encoding / decoding target block, motion information correction is first performed to generate corrected motion information, and then motion compensation is performed using the generated corrected motion information.
[0417] For example, for the AMVP mode, in the step of determining the motion vector ( Figure 5 S503 and Figure 6 In this case, the motion information correction may be applied to any one of the following information types: a motion vector candidate included in the motion vector candidate list, a predicted motion vector, and a motion vector calculated by adding the predicted motion vector to the motion vector difference.
[0418] For example, for the merge mode, in the step of determining the motion information ( Figure 7 S702 and Figure 8 In this case, the motion information correction may be applied to the merge candidate included in the merge candidate list or the motion information extracted from the merge candidate list based on the merge candidate index.
[0419] On the other hand, even for the skip mode, motion information modification may be performed. In this case, the motion information modification may be applied to the skip candidate included in the skip candidate list or the motion information extracted from the skip candidate list based on the skip index.
[0420] Regarding the timing for performing motion information correction, when performing motion compensation ( Figure 5 S504 in Figure 6 S605 in Figure 7 S703 in Figure 8That is, when motion compensation is performed on the encoding / decoding target block, motion information correction is performed to generate corrected motion information, and motion compensation is performed using the corrected motion information.
[0421] For example, the encoder / decoder may be based on the motion vector determination step or the motion information determination step ( Figure 5 S503 in Figure 6 604 of Figure 7 702 and Figure 8 The prediction block is generated by using the motion information determined in S803 in the above method, and the corrected motion information can be generated by performing motion information correction using the generated prediction block. Here, the encoder / decoder can use the corrected motion information to generate a final prediction block.
[0422] The motion information may be modified on the encoder side and the decoder side according to the same rule. Since the motion information is modified on the encoder side and the decoder side according to the same rule, the "information on the use of motion information modification" (hereinafter referred to as motion information modification utilization information) may not be entropy encoded / decoded.
[0423] In the following, reference is made to Figures 15 to 21 , bilateral template matching as an embodiment of motion information correction will be described.
[0424] Bilateral template matching is an embodiment of a method for correcting a motion vector of motion information. At least one of two motion vectors used for bidirectional prediction may be corrected by using bilateral template matching. Here, bilateral template may refer to a prediction block calculated by performing weighted summation on a prediction block generated by using two motion vectors during bidirectional prediction. Bidirectional may refer to bi-prediction. In addition, at least one of a reference image index, a reference image, an inter-prediction indicator, a prediction list utilization flag, a weighting factor, and an offset included in the motion information may be corrected in a form similar to a motion vector by using a correction method such as bilateral template matching.
[0425] In detail, when bilateral template matching is performed for a bidirectional prediction block, at least one of a first motion vector corresponding to reference image list 0 and a second motion vector corresponding to reference image list 1 may be corrected. Here, the corrected motion vector may have a motion vector value different from that of the motion vector before being corrected. In addition, a search for the corrected motion vector may be performed within a restricted area of a reference image to be described later.
[0426] Fig.15 is a diagram illustrating bilateral template matching.
[0427] Reference Fig.15 , bilateral template matching may include: 1) generating a bilateral template by using an initial motion vector; and 2) revising the motion vector by comparing samples within the bilateral template with reconstructed samples within a reference image.
[0428] 1) Generate bilateral template
[0429] The encoder / decoder may generate a bilateral template by using a prediction block generated by a first motion vector corresponding to reference image list 0 and a second motion vector corresponding to reference image list 1. In addition, the encoder / decoder may generate a bilateral template by using a prediction block generated by a first motion vector corresponding to reference image list 0 and a second motion vector corresponding to reference image list 0. In addition, the encoder / decoder may generate a bilateral template by using a prediction block generated by a first motion vector corresponding to reference image list 1 and a second motion vector corresponding to reference image list 1. Here, the bilateral template may be generated by performing weighted summation on the prediction block, and a weighting factor used in the weighted summation may be 0.5:0.5.
[0430] In addition, the motion vector used to generate the bilateral template may be referred to as an initial motion vector. The initial motion vector may refer to a motion vector derived by a method of deriving motion information by using at least one of a merge mode, an AMVP mode, and a skip mode.
[0431] 2) Motion vector correction
[0432] The encoder / decoder may modify each motion vector corresponding to each reference picture list by comparing the values between samples in the bilateral template and reconstructed samples in the reference picture.
[0433] A motion vector indicating a position of minimum distortion with the bilateral template within a reference image representing reference image list 0 may be determined as a modified first motion vector. In addition, a motion vector indicating a position of minimum distortion with the bilateral template within a reference image representing reference image list 1 may be determined as a modified second motion vector.
[0434] In detail, when the distortion value between the area in the reference image indicated by the initial first motion vector and the bilateral template is greater than the distortion value between the area in the reference image indicated by the modified first motion vector and the bilateral template, the first motion vector may be modified. Similarly, when the distortion value between the area in the reference image indicated by the initial second motion vector and the bilateral template is greater than the distortion value between the area in the reference image indicated by the modified second motion vector and the bilateral template, the second motion vector may be modified.
[0435] Here, when comparing the values between the samples in the bilateral template and the reconstructed samples in the reference image, the encoder / decoder may determine the motion vector indicating the position of the reconstructed sample representing the minimum distortion in the reference image as the corrected motion vector by using a method of calculating the distortion between the samples, such as SAD (Sum of Absolute Difference), SATD (Sum of Absolute Transform Difference), SSE (Sum of Squared Error), MSE (Mean of Squared Error), MR-SAD (Mean SAD-removed), etc. The distortion may be calculated for at least one of the luminance component and the chrominance component.
[0436] Furthermore, when comparing values between samples within the bilateral template and reconstructed samples within the reference image, distortion may be calculated by performing an exclusive OR (XOR) between the samples within the bilateral template and the reconstructed samples within the reference image.
[0437] For example, after performing an XOR on at least one sample point within the bilateral template and at least one reconstructed sample point within the reference image, the distortion may be calculated by summing the results obtained by performing the XOR on each sample point.
[0438] Furthermore, when comparing the values of the samples within the bilateral template and the reconstructed samples within the reference image, distortion may be calculated by reducing the bit depth of each sample by a specific value K. Here, K may have a value from 0 to the bit depth of the sample and may be a positive integer including 0.
[0439] For example, when the bit depth of the samples within the bilateral template and the reconstructed samples within the reference image is 10 bits, the bit depth of the samples within the bilateral template and the reconstructed samples within the reference image can be reduced by 4, and the distortion can be calculated by using a method of calculating the distortion between samples (such as SAD, SATD, SSE, MSE, etc.).
[0440] In addition, when comparing the values of the samples in the bilateral template and the reconstructed samples in the reference image, the reconstructed sample position in the reference image indicating the minimum rate distortion value may be used as the corrected motion vector by a method of calculating the distortion between the samples and a method of calculating the rate of at least one piece of information included in the motion information. Here, the rate may be calculated by performing entropy coding on at least one of the initial motion vector value and the corrected motion vector value. In addition, the rate may be calculated by performing entropy coding on the difference between the initial motion vector value and the corrected motion vector value. Here, the entropy coding may refer to at least one of the binarization methods.
[0441] In addition, when the SAD value between the regions within the reference image indicated by the initial first motion vector and the initial second motion vector is greater than a predefined value, the initial first motion vector and the initial second motion vector may be corrected by performing bilateral template matching. Conversely, when the SAD value between the regions within the reference image indicated by the initial first motion vector and the initial second motion vector is less than a predefined value, bilateral template matching may not be performed.
[0442] In addition, in the case of bidirectional prediction, when the value of the initial first motion vector corresponding to the first prediction direction (for example, the L0 prediction direction) and the value of the corrected first motion vector to which motion information correction is applied are the same, motion information correction may not be performed for the initial second motion vector corresponding to the second prediction direction (for example, the L1 prediction direction).
[0443] Furthermore, bilateral template matching can be performed recursively.
[0444] In detail, after generating a second bilateral template by using the corrected first motion vector and the corrected second motion vector, the corrected first motion vector and the corrected second motion vector may be re-corrected by performing bilateral template matching using the generated second bilateral template. Here, the method of determining the re-corrected motion vector by bilateral template matching may be repeated up to M times. Here, M may be a positive integer (e.g., 2), M may be a fixed value predefined in the encoder / decoder, or may be a variable value encoded from the encoder and transmitted with a signal. Optionally, M may be determined based on the size of the encoding / decoding target block. In addition, the corrected motion vector may be re-corrected within a restricted area of a reference image to be described later.
[0445] In one example, when the height or width of the encoding / decoding target block is less than 8, M may be set to 2.
[0446] In addition, when the height and width of the encoding / decoding target block are 8, M can be set to 2.
[0447] Furthermore, when the width of the encoding / decoding target block is 8 and the height is 16, or vice versa, M may be set to 4.
[0448] In addition, otherwise, M may be set to 8. Here, bilateral template matching may be performed four times with respect to the first reference image (eg, L0 reference image), and bilateral template matching may be performed four times with respect to the second reference image (eg, L1 reference image).
[0449] Furthermore, when bilateral template matching is recursively performed in sub-pixel units, bilateral template matching may be recursively performed in integer pixel units.
[0450] The modified motion vector generated by bilateral template matching can be replaced with the original motion vector and used for motion compensation of the encoding / decoding target block. In addition, bilateral template matching can be performed by the same rules in the encoder and the decoder.
[0451] When performing bilateral template matching, the search for the modified motion vector may be performed within a restricted area of the reference image.
[0452] Figures 16 to 21 2 is a diagram showing an area for searching for a corrected motion vector in bilateral template matching (hereinafter, referred to as a search area). Here, searching for a corrected motion vector in the search area means that the motion vector can be corrected so that the motion vector is corrected to have a predetermined value within the search area.
[0453] Furthermore, at least one of the integer pixel range of the search area and the sub-pixel range of the search area may be a range preset in the encoder and the decoder.
[0454] Reference Figures 16 to 21 , the search area can be defined as a pixel range of -M to +N in the horizontal / vertical direction in units of integer pixels (integer pel). Here, M and N can be positive integers. In addition, the search area can be defined as a sub-pixel range of -O to +P in the horizontal / vertical direction in units of sub-pixels (sub-pel). Here, O and P can be decimal values. In one example, the sub-pixel unit can represent 1 / 2 pixel, 1 / 4 pixel, 1 / 8 pixel, 1 / 16 pixel, 1 / 32 pixel, etc. In addition, O and P can have integer values to represent sub-pixels.
[0455] When searching for the corrected motion vector is performed in sub-pixel units, neighboring sub-pixels spatially adjacent to the integer pixel indicated by the motion vector in integer pixel units may become motion vector search targets. Figures 16 to 21 In FIG. 1 , the sub-pixel unit shows an example of a 1 / 2 pixel unit.
[0456] Furthermore, within the search area, the pixel range in the horizontal direction and the pixel range in the vertical direction in units of integer pixels may be different.
[0457] For example, in order to reduce the memory access bandwidth, in the search area, the pixel range in the vertical direction of the integer pixel unit may be determined to be smaller than the pixel range in the horizontal direction. For example, the pixel range in the vertical direction of the integer pixel unit may be determined to be -1 to +1, and the pixel range in the horizontal direction of the integer pixel unit may be determined to be -2 to +2.
[0458] Furthermore, in the search area, a sub-pixel range in a horizontal direction and a sub-pixel range in a vertical direction of a sub-pixel unit may be different from each other.
[0459] For example, in order to reduce the memory access bandwidth, in the search area, the sub-pixel range in the vertical direction of the sub-pixel unit may be determined to be smaller than the sub-pixel range in the horizontal direction. For example, the sub-pixel range in the vertical direction may be determined to be -1 / 4 to +1 / 4, and the sub-pixel range in the horizontal direction may be determined to be -1 / 2 to +1 / 2.
[0460] In addition, in order to reduce the memory access bandwidth, the search area of the sub-pixel unit may be limited. For example, the search area of the sub-pixel unit may be limited to be included in the search area of the integer pixel unit. Therefore, the search for the corrected motion vector may be performed for at least one sub-pixel unit belonging to the search area of the integer pixel unit.
[0461] Fig.17 : is a diagram illustrating an embodiment in which a search area of a sub-pixel unit is limited to be included in a search area of an integer-pixel unit.
[0462] Reference Fig.17 , the hatched sub-pixels represent sub-pixels not included in the search area of the integer pixel unit, and the search for the corrected motion vector may be limited to the corresponding sub-pixel unit. In other words, the search for the corrected motion vector may be limited to be performed in the non-hatched sub-pixel unit, and integer pixels for generating the hatched sub-pixels are not additionally loaded from the memory, so that the memory access bandwidth may be reduced.
[0463] Furthermore, the search area of the integer unit may include at least one of a center point, an upper point, a lower point, a left point, and a right point.
[0464] Here, the search for at least one point may be further performed based on the distortion values of the center point, the upper point, the lower point, the left point, and the right point. Here, the at least one point may be at least one of the upper left point, the lower left point, the upper right point, and the lower right point. Here, a search area of sub-pixel unit may be set in at least one of the center point, the upper point, the lower point, the left point, the right point, the upper left point, the lower left point, the upper right point, and the lower right point. In one example, a search area of sub-pixel unit may be set in the center point, the upper point, the lower point, the left point, and the right point.
[0465] In addition, the shape of the search area can be a graphic shape that can be represented in two dimensions based on the pixels indicated by the initial motion vector, such as a square, a rectangle, a rhombus / rhombus, and a cross. The search area may have a fixed shape predefined in the encoder / decoder, or the shape of the search area may be specified by information encoded from the encoder and sent with a signal.
[0466] In addition, the range of motion vector values of at least one of the initial motion vector and the modified motion vector may be limited so that the search area does not exceed a certain range of image boundaries. Limiting the range of motion vector values may be achieved by clipping.
[0467] Here, the range of motion vector values may be determined as a range predetermined in the encoder and the decoder. Here, the predetermined range may represent a threshold value for limiting the motion vector value, and the threshold value may be implemented by at least one of a minimum value and a maximum value. In addition, the range of motion vector values may be a variable range encoded from the encoder and sent with a signal.
[0468] Furthermore, an image boundary of at least one of the search area of the integer pixel unit and the search area of the sub-pixel unit may be limited so that the image boundary may not exceed a predetermined range.
[0469] In addition, after performing motion information modification in integer pixel units, motion information modification in sub-pixel units may be allowed.
[0470] In addition, information on whether to perform a search for a modified motion vector in integer pixel units may be entropy encoded / decoded according to at least one of a video parameter set, a sequence parameter set, a picture parameter set, an adaptation parameter set, a tile group header, a CTU, and a block. Whether to perform a search for a motion vector in integer pixel units may be determined in an encoder / decoder based on the information on whether to perform a search for a modified motion vector in integer pixel units.
[0471] In addition, information on whether to search for the modified motion vector in sub-pixel units may be entropy encoded / decoded according to at least one of a video parameter set, a sequence parameter set, a picture parameter set, an adaptation parameter set, a tile group header, a CTU, and a block. Whether to search for the motion vector in sub-pixel units may be determined in the encoder / decoder based on the information on whether to search for the modified motion vector in sub-pixel units.
[0472] In addition, information of a search area of at least one of a pixel range of an integer pixel unit and a sub-pixel range of a sub-pixel unit may be entropy encoded / decoded per at least one of a video parameter set, a sequence parameter set, a picture parameter set, an adaptation parameter set, a tile group header, a CTU, and a block. Motion vector correction may be identically performed in the encoder and the decoder by using the integer pixel range information and the sub-pixel range information.
[0473] In addition, the maximum pixel range of integer pixel units for the search area and the maximum sub-pixel range of sub-pixel units for the search area may be predetermined in the encoder and the decoder. Here, at least one of the information of the difference of the maximum pixel range and the information of the difference of the maximum sub-pixel range may be entropy encoded / decoded according to at least one of a video parameter set, a sequence parameter set, a picture parameter set, an adaptation parameter set, a parallel block group header, a CTU, and a block.
[0474] Here, in the encoder and the decoder, motion vector correction may be performed identically by using at least one of the information of the maximum pixel range and the difference value of the maximum pixel range. In addition, in the encoder and the decoder, motion vector correction may be performed identically by using at least one of the information of the difference value of the maximum sub-pixel range and the maximum sub-pixel range.
[0475] Furthermore, at least one piece of information of a difference between a pixel range of integer pixel units for the search area and a maximum pixel range of integer pixel units for the search area may be entropy encoded / decoded based on information of whether the search for the corrected motion vector is performed in integer pixel units.
[0476] Furthermore, at least one piece of information of a difference between a sub-pixel range of a search area in sub-pixel units and a maximum sub-pixel range of a search area in sub-pixel units may be entropy encoded / decoded based on information of whether the search for the corrected motion vector is performed in sub-pixel units.
[0477] In addition, the information of the sub-pixel unit may be entropy encoded / decoded based on the information of whether the search for the corrected motion vector is performed in sub-pixel units. Here, the sub-pixel unit may represent 1 / 2 pixel, 1 / 4 pixel, 1 / 8 pixel, 1 / 16 pixel, 1 / 32 pixel, etc.
[0478] Furthermore, motion vector correction can be performed identically in the encoder and the decoder by using information in sub-pixel units.
[0479] In addition, entropy encoding / decoding may be performed on information of a plurality of sub-pixel units. When information of a plurality of sub-pixel units is entropy encoded / decoded, motion vector correction may be performed for each sub-pixel unit corresponding to information of a plurality of coding units.
[0480] In order to reduce the memory access bandwidth, at least one of the search area of integer pixel units and the search area of sub-pixel units may be determined based on the size of the current block. Here, the size of the current block may represent at least one of a horizontal size, a vertical size, and a surface area. The following N, M, I, R, J, K, S, and T may be positive integers including 0.
[0481] When the size of the current block increases, at least one size of the search area of integer pixel units and the search area of sub-pixel units may increase.
[0482] When the current block has a size of N*M and N*M is equal to or less than 1, at least one of the search area of the integer pixel unit and the search area of the sub-pixel unit may be determined as −R to +R.
[0483] For example, when N*M is 32 (4*8 or 8*4) and I is 64, R may be determined to be 1.
[0484] For example, when N*M is 256 (16*16) and I is 512, R may be determined to be 2.
[0485] For example, when N*M is 16384 (128*128) and I is 16384, R may be determined to be 3.
[0486] Similarly, when the current block has a size of N*M and N*M is greater than 1, at least one of the search area of the integer pixel unit and the search area of the sub-pixel unit may be determined as −R to +R.
[0487] For example, when N*M is 64 (8*8) and I is 32, R may be determined to be 1.
[0488] For example, when N*M is 256 (16*16) and I is 64, R may be determined to be 2.
[0489] For example, when N*M is 16384 (128*128) and I is 256, R may be determined to be 4.
[0490] When the current block has a size of N*M and N is equal to or less than J, at least one of the search area of the integer pixel unit and the search area of the sub-pixel unit may be determined as -S to +S.
[0491] For example, when N is 8 and J is 8, S may be determined to be 1.
[0492] For example, when N is 16 and J is 32, S may be determined to be 2.
[0493] For example, when N is 64 and J is 64, S may be determined to be 3.
[0494] Similarly, when the current block has a size of N*M and N is greater than J, at least one of the search area of the integer pixel unit and the search area of the sub-pixel unit may be determined as -S to +S.
[0495] For example, when N is 16 and J is 8, S may be determined to be 1.
[0496] For example, when N is 32 and J is 16, S may be determined to be 2.
[0497] For example, when N is 128 and J is 32, S may be determined to be 4.
[0498] When the current block has a size of N×M and M is equal to or less than K, at least one of the search area of the integer pixel unit and the search area of the sub-pixel unit may be determined as −T to +T.
[0499] For example, when M is 8 and K is 16, T may be determined to be 1.
[0500] For example, when M is 16 and K is 64, T may be determined to be 2.
[0501] For example, when M is 128 and K is 128, T may be determined to be 3.
[0502] Similarly, when the current block has a size of N×M and M is greater than K, at least one of the search area of the integer pixel unit and the search area of the sub-pixel unit may be determined as −T to +T.
[0503] For example, when M is 8 and K is 4, T may be determined to be 1.
[0504] For example, when M is 32 and K is 8, T may be determined to be 2.
[0505] For example, when M is 128 and K is 64, T may be determined to be 4.
[0506] In addition, when the current block has a size of N×M, at least one of the search area of integer pixel units and the search area of sub-pixel units may be determined as -N×M / D to +N×M / D. Here, D may be a positive integer equal to or greater than 1 and may be a predefined value.
[0507] For example, when N*M is 32 (8*4) and D is 64, at least one of the search area of the integer pixel unit and the search area of the sub-pixel unit may be determined to be -1 / 2 to +1 / 2.
[0508] For example, when N*M is 64 (8*8) and D is 64, at least one of the search area of the integer pixel unit and the search area of the sub-pixel unit may be determined as -1 to +1.
[0509] For example, when N*M is 128 (16*8) and D is 64, at least one of the search area of the integer pixel unit and the search area of the sub-pixel unit may be determined as -2 to +2.
[0510] When the current block has a size of N×M and N is greater than M, at least one of the search area of the integer pixel unit and the search area of the sub-pixel unit may be determined as −N / D to +N / D.
[0511] For example, when N*M is 8*4 and D is 16, at least one of the search area of the integer pixel unit and the search area of the sub-pixel unit may be determined to be −1 / 2 to +1 / 2.
[0512] For example, when N*M is 16*8 and D is 16, at least one of the search area of the integer pixel unit and the search area of the sub-pixel unit may be determined as -1 to +1.
[0513] When the current block has a size of N×M and M is greater than N, at least one of the search area of the integer pixel unit and the search area of the sub-pixel unit may be determined as −M / D to +M / D.
[0514] For example, when N*M is 8*16 and D is 32, at least one of the search area of the integer pixel unit and the search area of the sub-pixel unit may be determined to be −1 / 2 to +1 / 2.
[0515] For example, when N*M is 16*32 and D is 32, at least one of the search area of the integer pixel unit and the search area of the sub-pixel unit may be determined as -1 to +1.
[0516] When the current block has a size of N×M and N and M are the same, at least one of the search area of the integer pixel unit and the search area of the sub-pixel unit may be determined as −M / D to +M / D.
[0517] For example, when N*M is 8*8 and D is 16, at least one of the search area of the integer pixel unit and the search area of the sub-pixel unit may be determined to be −1 / 2 to +1 / 2.
[0518] For example, when N*M is 16*16 and D is 16, at least one of the search area of the integer pixel unit and the search area of the sub-pixel unit may be determined as -1 to +1.
[0519] For example, when N*M is 32*32 and D is 16, at least one of the search area of the integer pixel unit and the search area of the sub-pixel unit may be determined as -2 to +2.
[0520] When bilateral template matching is used as described above, motion vectors can be improved in the decoder without additionally sending syntax elements. Furthermore, in case of bi-predictive merge mode or decoder motion vector derivation mode, bilateral template matching can be applied.
[0521] According to an embodiment, when the current block does not correspond to any one of the uni-prediction merge candidate mode, the local illumination compensation mode, the affine motion compensation mode, and the sub-CU merge mode, bilateral template matching may be performed.
[0522] Hereinafter, a motion correction mode related to motion information correction will be described. Here, the motion correction mode may refer to a prediction mode for deriving motion information for inter-picture prediction. For example, the motion correction mode may be a merge mode performed based on the motion information correction using bilateral template matching described above.
[0523] Using the motion correction mode, areas with zero or little motion in the picture can be efficiently encoded.
[0524] The motion correction mode is a motion information derivation scheme based on the above motion information correction, and the above motion information correction method can be applied to the motion correction mode.
[0525] On the other hand, whether the motion correction mode is available or whether the motion correction mode is applicable may be determined based on information signaled through a bitstream. The information may be signaled on a unit-by-unit basis, where a unit may be a video, a sequence, a picture, a slice, a tile, a coding tree unit (CTU), a coding unit (CU), a prediction unit (PU), a transform unit (TU), or a block. Whether the motion correction mode is available may indicate whether the motion correction mode is available for the above-mentioned corresponding unit. Whether the motion correction mode is applicable may indicate whether the motion correction mode is applicable to the corresponding unit.
[0526] The motion correction mode may include the following steps: (1) deriving motion information, and (2) generating corrected motion information by performing motion information correction on the derived motion information. Here, the motion information correction method may include the above-mentioned bilateral template matching. The corrected motion information obtained by performing the motion correction mode may be used for motion compensation of a current block to be encoded / decoded (also referred to as an encoding / decoding target block).
[0527] Furthermore, the motion information that may be corrected by the motion correction mode may include at least one of a motion vector, a reference picture, an inter-picture indicator, a reference picture index, a prediction list utilization flag, a weighting factor, and an offset.
[0528] 1) Derivation of motion information
[0529] In the motion correction mode, the motion information may be derived from at least one type of information among motion information of spatially adjacent neighboring blocks, motion information of temporally adjacent neighboring blocks, predefined motion information, and motion information that most frequently appears in a reference picture. Here, the derived motion information may be recorded as a motion correction candidate in a list of motion correction modes (motion correction mode list). The motion information derived from the motion information of spatially adjacent neighboring blocks is referred to as a spatial motion correction candidate, and the motion information derived from the motion information of temporally adjacent neighboring blocks is referred to as a temporal motion correction candidate.
[0530] Hereinafter, a method of deriving a motion vector as a piece of motion information will be described in detail.
[0531] In the motion correction mode, a motion vector may be derived from motion vectors of spatially adjacent neighboring blocks, motion vectors of temporally adjacent neighboring blocks, predefined motion vectors, and a motion vector that most frequently appears in a reference picture.
[0532] A. Spatially adjacent neighboring blocks
[0533] In the motion correction mode, the motion vector may be derived from a motion vector of at least one of the spatially adjacent neighboring blocks. Here, the spatially adjacent neighboring blocks may include Fig. 9 The neighboring blocks shown in .
[0534] In the motion correction mode, the motion vector may be derived from a plurality of motion vectors of neighboring blocks that satisfy at least one of the following conditions among spatially adjacent neighboring blocks:
[0535] - blocks with a coded block flag of 0 (i.e., blocks without residual signal),
[0536] - blocks corresponding to the merge skip mode,
[0537] - a block whose motion vector difference value among neighboring blocks spatially adjacent to the current block is zero (e.g., MVD=(0,0)),
[0538] - blocks corresponding to the merge mode,
[0539] - a block whose inter-picture prediction indicator is PRED_LX (X is a number among integers including zero), and
[0540] - The inter-picture prediction indicator is a block of any one of PRED_BI, PRED_TRI, and PRED_QUAD.
[0541] In the motion correction mode, a motion vector may be derived from a plurality of motion vectors of at least one of the spatial skip / merge candidates.
[0542] For example, when the reference picture of the spatial skip / merge candidate is different from the reference picture of the encoding / decoding target block (i.e., when the picture order count (POC) of the reference picture of the spatial skip / merge candidate is different from the picture order count (POC) of the reference picture of the encoding / decoding target block), the motion vector can be derived based on the POC value of the picture by scaling the motion vector derived from multiple motion vectors of spatially adjacent neighboring blocks.
[0543] B. Temporally adjacent neighboring blocks
[0544] In the motion correction mode, the motion vector may be derived from a plurality of motion vectors of at least one of the temporally adjacent neighboring blocks. Here, the temporally adjacent neighboring blocks may include a co-located block within a reference picture and a co-located block within a co-located picture.
[0545] Here, the co-located block in the reference picture may be at least one of the following blocks:
[0546] - a co-located block corresponding to the upper left position (0, 0) of the encoding / decoding target block in the reference picture (hereinafter, referred to as the co-located reference block),
[0547] - Co-located blocks in the reference picture ( Fig.10 C3 in
[0548] - Co-located blocks in the reference picture ( Fig.10 C1 in
[0549] - Co-located blocks in the reference picture ( Fig.10 H in , and
[0550] - Co-located blocks in the reference picture ( Fig.10 At least one of C4, C5, C6, C0, D0, D1).
[0551] Here, the co-located block in the co-located picture may be at least one of the following blocks:
[0552] - a co-located block corresponding to the upper left position (0,0) of the encoding / decoding target block in the co-located picture,
[0553] -Co-located blocks in the co-located screen ( Fig.10 C3 in
[0554] -Co-located blocks in the co-located screen ( Fig.10 C1 in
[0555] -Co-located blocks in the co-located screen ( Fig.10 H in , and
[0556] -Co-located blocks in the co-located screen ( Fig.10At least one of C4, C5, C6, C0, D0, D1).
[0557] In the motion correction mode, the motion vector may be derived from a plurality of motion vectors of neighboring blocks satisfying at least one of the following conditions among temporally adjacent neighboring blocks:
[0558] - blocks whose coded block flag is 0 (i.e., blocks without residual signals) arranged in the co-located block and in the blocks outside the co-located block,
[0559] - blocks corresponding to the merge skip mode arranged in the co-located block and the blocks outside the co-located block,
[0560] - a block whose motion vector difference value among blocks arranged inside and outside the co-located block is zero (e.g., MVD=(0,0)),
[0561] - blocks corresponding to the merge mode arranged in the co-located block and in the blocks outside the co-located block,
[0562] - a block whose inter-picture prediction indicator is PRED_LX (X is a number among integers including zero), and
[0563] - A block whose inter-picture prediction indicator is any one of PRED_BI, PRED_TRI, and PRED_QUAD is arranged in a block inside and outside the co-located block.
[0564] In the motion correction mode, a motion vector may be derived from a plurality of motion vectors of at least one of the temporal skip / merge candidates.
[0565] For example, when the reference picture or the co-located picture of the time skip / merge candidate is different from the reference picture of the encoding / decoding target block (i.e., when the picture order counts (POCs) of the reference pictures of the time skip / merge candidate and the encoding / decoding target block are different from each other), a motion vector can be derived by scaling a motion vector derived from multiple motion vectors of temporally adjacent neighboring blocks based on the POC value of the picture.
[0566] In addition, a reference picture index indicating a reference picture including a temporally adjacent neighboring block and a co-located picture may be derived from a reference picture index of a spatially adjacent neighboring block.
[0567] C. Predefined motion vector
[0568] In the motion correction mode, a zero motion vector having a value of (0,0) may be derived as the motion vector.
[0569] D. The most frequently occurring motion vector in the reference picture
[0570] In the motion correction mode, the most frequently occurring motion vectors in the reference picture can be derived.
[0571] In this case, the motion vectors are arranged in the order of frequency of occurrence of the motion vectors in the reference picture, and a maximum of L motion vectors can be derived according to the frequency order. Wherein L is a positive integer. In this case, L can be a fixed value preset in the encoder / decoder, or can be a variable value encoded by the encoder and sent with a signal.
[0572] On the other hand, in the motion correction mode, the motion vector can be derived in the order of items A, B, C and D. In this case, the derivation of the motion vector can be repeatedly performed until a total of M motion vectors are derived. In addition, the motion vector can be derived sequentially in the above order. When a newly derived motion vector already exists in the previously derived motion vector, the derivation of the motion vector is stopped, and thus at most M motion vectors can be derived and used in the motion correction mode. Here, M is a positive integer, and it can be a fixed value preset in the encoder / decoder or a variable value encoded by the encoder and sent with a signal. Here, when there are multiple spatially adjacent neighboring blocks, a motion vector of bidirectional prediction can be preferentially derived and inserted into the motion correction mode list, and then a motion vector of unidirectional prediction can be derived and inserted into the motion correction mode list.
[0573] The derivation of motion vectors in the above-described motion correction mode may be performed according to the process described below.
[0574] For example, the motion vector may be derived in the order of the motion vector of the spatially adjacent neighboring block, the motion vector of the temporally adjacent neighboring block, and the zero motion vector (0, 0). Fig. 9 B0, A0, B2 and Fig.10 The motion vector is derived in the order of C3, C0 and zero motion vector (0,0).
[0575] As another example, the motion vector may be derived in the order of a motion vector of a temporally adjacent neighboring block, a motion vector of a spatially adjacent neighboring block, and a zero motion vector (0, 0).
[0576] Here, the motion vectors of spatially adjacent neighboring blocks may be derived in the order in which spatial skip / merge candidates are derived, and the motion vectors of temporally adjacent neighboring blocks may be derived in the order in which temporal skip / merge candidates are derived.
[0577] In the motion correction mode, the derived motion vector can be used only when the value of the motion vector derived by the above method is not (0,0).
[0578] For example, when bidirectional prediction is used in the motion correction mode, the derived motion vector may be used only when both the values of the derived first motion vector and the second motion vector are not (0,0).
[0579] When N-directional prediction (N is an integer of 2 or greater) is used in the motion correction mode, the derived motion vectors may be used only when none of the values of the derived N motion vectors is (0,0).
[0580] In N-directional prediction (N is an integer of 2 or more), among the derived motion vectors, only some motion vectors having values other than (0, 0) may be used for the motion vectors in the motion correction mode.
[0581] In addition, when bidirectional prediction is used in the motion correction mode, when the derived first motion vector and the derived second motion vector are not equal to each other, one or both of the derived first motion vector and the derived second motion vector may be used in the motion correction mode.
[0582] On the other hand, when the inter-picture prediction indicator of the spatially adjacent neighboring block or the temporally adjacent neighboring block indicates at least one of bidirectional prediction, tridirectional prediction, and quadridirectional prediction, the derived motion vector may be used in the motion correction mode. That is, when the inter-picture prediction indicator of the spatially adjacent neighboring block or the temporally adjacent neighboring block does not indicate unidirectional prediction, the derived motion vector may be used in the motion correction mode.
[0583] When the inter-picture prediction indicator of the spatially adjacent neighboring block or the temporally adjacent neighboring block indicates unidirectional prediction, the inter-picture prediction indicator is changed to indicate bidirectional prediction, and the sign of the motion vector of the unidirectional prediction is changed oppositely. Thus, a motion vector for unidirectional prediction having a direction opposite to that of the previous (original) unidirectional prediction is derived and used as a motion vector in the motion correction mode.
[0584] For example, when the inter-picture prediction indicator of the spatially adjacent neighboring block or the temporally adjacent neighboring block indicates L0 unidirectional prediction and the motion vector of the spatially adjacent neighboring block or the temporally adjacent neighboring block is (-4, 6), the inter-picture prediction indicator is changed to indicate bidirectional prediction, and the motion vector (4, -6) is derived for L1 directional prediction. Therefore, the two motion vectors (-4, 6) and (4, -6) can be used as motion vectors in the motion correction mode. In this case, a reference picture located at a first distance from the current picture can be determined as a reference picture for L1 unidirectional prediction, wherein the first distance is the same as or proportional to a second distance that is the distance between the current picture and the reference picture for L0 unidirectional prediction.
[0585] In a motion correction mode using N-directional prediction (N is an integer of 2 or more), when the number of derived motion vectors is not N, one or more motion vectors may be further derived from one or more previously derived motion vectors to obtain a total of N motion vectors. Specifically, the motion vector(s) may be generated by scaling the previously derived motion vector(s) generated based on the picture sequence count of the current picture and / or the reference picture.
[0586] For example, when bidirectional prediction is used in motion correction mode and there is only a previously derived first motion vector, the first motion vector is scaled to generate a second motion vector based on the reference picture in reference picture list 1. Therefore, the previously derived first motion vector and the newly generated second motion vector may be used as motion vectors in motion correction mode.
[0587] Alternatively, when bidirectional prediction is used in motion correction mode and only the previously derived second motion vector exists, the second motion vector is scaled to generate the first motion vector based on the reference picture in reference picture list 0. Therefore, the previously derived second motion vector and the newly generated first motion vector may be used as motion vectors in motion correction mode.
[0588] Hereinafter, a method of deriving a reference picture which is also a piece of motion information will be described in detail.
[0589] In the motion correction mode, at least one of the following methods may be used to derive the reference picture.
[0590] For example, a reference picture having a reference picture index of 0 may be derived from reference pictures included in reference picture list 0 and reference pictures included in reference picture list 1 as a reference picture to be used in motion correction mode. Among the reference pictures included in reference picture list 0 and reference picture list 1, there may be a case where the reference pictures having reference picture index 0 in reference picture list 0 and reference picture list 1, respectively, may be identical to each other. In this case, among the reference pictures included in reference picture list 1, a reference picture different from the reference picture having reference picture index 0 included in reference picture list 0 may be derived as a reference picture to be used in motion correction mode. Among the reference pictures included in reference picture list 0 and reference picture list 1, there may be a case where the reference pictures having reference picture index 0 in reference picture list 0 and reference picture list 1, respectively, may be identical to each other. In this case, among the reference pictures included in reference picture list 0, a reference picture different from the reference picture having reference picture index 0 included in reference picture list 1 may be derived as a reference picture to be used in motion correction mode.
[0591] In order to derive a reference picture to be used in the motion correction mode as described above, a reference picture index 0 may be used. However, the present invention is not limited thereto, and a reference picture index having a non-zero value may be used to derive a reference picture to be used in the motion correction mode.
[0592] The method of deriving the reference image of the motion correction mode by using a non-zero reference image index may be as follows.
[0593] In one example, the median of the reference image indexes belonging to at least one of the A1 position, B1 position, B0 position, A0 position, B2 position, B3 position, and A2 position spatially adjacent to the encoding / decoding target block may be determined as the reference image index. Here, various statistical values (such as minimum value, maximum value, average value, weighted average value, mode, etc.) may be used instead of the median value. The reference image indicated by the determined reference image index may be derived as a reference image of the motion correction mode. Here, the reference image index may be determined by using the above method for each reference image list.
[0594] Furthermore, when a block exists at the A1 position and the corresponding block has an inter prediction mode, a reference image index belonging to the A1 position may be determined as a reference image index.
[0595] Furthermore, when a block exists at the B1 position and the corresponding block has an inter prediction mode, a reference image index belonging to the B1 position may be determined as a reference image index.
[0596] Furthermore, when a block exists at the B0 position and the corresponding block has an inter prediction mode, a reference image index belonging to the B0 position may be determined as a reference image index.
[0597] Furthermore, when a block exists at the A0 position and the corresponding block has an inter prediction mode, a reference image index belonging to the A0 position may be determined as a reference image index.
[0598] Furthermore, when a block exists at the B2 position and the corresponding block has an inter prediction mode, a reference image index belonging to the B2 position may be determined as a reference image index.
[0599] Furthermore, when a block exists at the B3 position and the corresponding block has an inter prediction mode, a reference image index belonging to the A0 position may be determined as a reference image index.
[0600] Furthermore, when a block exists at the A2 position and the corresponding block has an inter prediction mode, a reference image index belonging to the B2 position may be determined as a reference image index.
[0601] A corrected reference picture may be generated by applying the motion information correction method to the reference picture derived by using the above-described method, and the corrected reference picture may be used as a reference picture of the motion correction mode.
[0602] In the above embodiment, it is assumed that two reference picture lists will be used. However, the number of reference picture lists is not limited thereto. There may be N reference picture lists. In this case, N may be an integer of 2 or more. When there are N reference picture lists, the derivation of reference pictures may be performed in a manner similar to the above method (i.e., based on determination of consistency of reference pictures in the N reference picture lists). That is, N reference pictures different from each other may be derived.
[0603] In another embodiment, among the reference pictures included in the reference picture list i (i=0, ..., or N, where N is an integer of 0 or greater), a reference picture having a minimum picture order count (POC) difference with the encoding / decoding target picture can be derived as a reference picture of the motion correction mode.
[0604] Optionally, among the reference pictures included in the reference picture list i (i=0, ..., or N, where N is an integer of 1 or greater), a reference picture having a minimum picture order count (POC) difference with the encoding / decoding target picture and / or a minimum temporal layer identifier value can be derived as a reference picture of the motion correction mode.
[0605] Further optionally, the reference picture of the spatially adjacent neighboring block may be derived as the reference picture of the motion correction mode.
[0606] Further optionally, the reference picture of the temporally adjacent neighboring block may be derived as the reference picture of the motion correction mode.
[0607] Further optionally, one or more reference pictures selected from the skip / merge candidates may be derived as reference pictures of the motion correction mode.
[0608] Further optionally, motion information correction processing is performed on all or part of the reference pictures included in the reference picture list, and then the reference picture presenting the minimum distortion value after the correction processing is derived as the reference picture of the motion correction mode.
[0609] Hereinafter, a method of deriving an inter-picture prediction indicator as a piece of motion information will be described in detail.
[0610] In the motion correction mode, the inter-picture prediction indicator may be derived using at least one of the following methods.
[0611] For example, in the motion correction mode, the inter-picture prediction indicator may be fixed to one of uni-directional prediction, bi-directional prediction, tri-directional prediction, quad-directional prediction, and N-directional prediction.
[0612] As another example, the inter-picture prediction indicator of a spatially adjacent neighboring block may be derived as the inter-picture prediction indicator to be used in the motion correction mode.
[0613] As another example, the inter-picture prediction indicator of the temporally adjacent neighboring block may be derived as the inter-picture prediction indicator to be used in the motion correction mode.
[0614] As another example, one or more inter-picture prediction indicators in the skip / merge candidates may be derived as inter-picture prediction indicators to be used in the motion correction mode.
[0615] As yet another example, an inter-picture prediction indicator may be derived according to the number of available reference pictures in the reference pictures derived by the above method as the inter-picture prediction indicator to be used in the motion compensation mode. Specifically, when there is only one available reference picture, the inter-picture prediction indicator may be derived by using unidirectional prediction. Specifically, when there are two available reference pictures, the inter-picture prediction indicator may be derived by using bidirectional prediction.
[0616] A modified inter-picture prediction indicator is generated by applying the motion information modification method to the previously derived inter-picture prediction indicator, and the modified inter-picture prediction indicator may be used as the inter-picture prediction indicator in the motion correction mode of the current block.
[0617] 2) Correction of motion information
[0618] The encoder / decoder generates a modified reference picture by performing motion information correction on the reference picture derived by using the above method, and uses the modified motion information as motion information in the motion correction mode. Specifically, motion compensation can be performed by generating a prediction block using at least one of the modified motion vector, the modified reference picture, and the modified inter-picture prediction indicator. Similarly, the encoder / decoder generates a modified reference picture index, a modified prediction list utilization flag, a modified weighting factor, and a modified offset by applying the motion information correction method to at least one of the previously derived reference picture index, the previously derived prediction list utilization flag, the previously derived weighting factor, and the previously derived offset, and creates a prediction block by using at least one of the modified reference picture index, the modified prediction list utilization flag, the modified weighting factor, and the modified offset. Here, the motion information correction method may include the above-mentioned bilateral template matching. In this case, the weighted sum of the prediction block is calculated based on the picture sequence count between the encoding / decoding target picture and each reference picture to obtain a final prediction block, and thus motion compensation can be performed.
[0619] On the other hand, when the motion information modification process is applied to a previously derived motion vector, the previously derived motion vector may be used to determine an initial motion search position. A modified motion vector may be determined based on the initial motion search position.
[0620] The encoder / decoder may perform motion information correction on a maximum of M pieces of derived motion information. In this case, M may be a fixed value preset in the encoder / decoder, or may be a variable value encoded by the encoder and transmitted by a signal. For example, at least one piece of motion information among the first piece of motion information, the second piece of motion information, ..., and the Nth piece of motion information may be corrected. In this case, M is an integer equal to or greater than 4.
[0621] In addition, without applying motion information correction to all or part of the previously derived motion information, the previously derived motion information may be used as motion information of the motion correction mode. Specifically, motion compensation may be performed by generating a prediction block using at least one of a derived motion vector, a derived reference picture, and a derived inter-picture prediction indicator.
[0622] Furthermore, in the motion correction mode, motion information correction (in one example, bilateral template matching) may be selectively applied.
[0623] According to an embodiment, when the current block does not correspond to any one of the uni-prediction merge candidate mode, the local illumination compensation mode, the affine motion compensation mode, and the sub-CU merge mode, motion information correction may be performed.
[0624] In addition, when the current block corresponds to the bi-prediction merge mode, motion information correction may be performed.
[0625] In addition, when the difference (POC) between the POC of the first reference image associated with the first prediction direction (eg, the L0 prediction direction) and the POC of the encoding / decoding target image is ref0 –POC curr ) and a difference (POC) between the POC of the second reference image associated with the second prediction direction (eg, the L1 prediction direction) and the POC of the encoding / decoding target image ref1 –POC curr ) has a negative value, motion information correction can be performed.
[0626] In addition, when the difference (POC) between the POC of the first reference image associated with the first prediction direction (eg, the L0 prediction direction) and the POC of the encoding / decoding target image is ref0 –POC curr ) and a difference (POC) between the POC of the second reference image associated with the second prediction direction (eg, the L1 prediction direction) and the POC of the encoding / decoding target image ref1 –POC curr ) are the same, motion information correction can be performed.
[0627] Furthermore, whether to perform motion information correction may be determined based on the size (width, height, or surface area) of the current block.
[0628] In one example, when the width, height, or surface area of the current block is equal to or greater than N, motion information correction may be performed. Here, N may be 8.
[0629] In one example, when the surface area of the current block is equal to or greater than M, motion information correction may be performed. Here, M may be 64.
[0630] In addition, when the first prediction direction and the second prediction direction are different from each other, motion information correction may be performed.
[0631] The encoder / decoder may perform bilateral template matching on at least one piece of derived motion information obtained by the above method, generate a prediction block using the piece of motion information with minimum distortion, and perform motion compensation. At this time, the at least one piece of motion information may include at least one of L0 motion information and L1 motion information.
[0632] The encoder / decoder may perform bilateral template matching on one or more candidates present in the skip candidate list or the merge candidate list, generate a prediction block using one candidate having minimum distortion, and perform motion compensation.
[0633] The encoder / decoder may perform bilateral template matching on at least one piece of derived motion information obtained by the above method, generate M prediction blocks by using M pieces of motion information each having minimum distortion, calculate a weighted sum of the M prediction blocks, and use the weighted sum as a final prediction block of the encoding / decoding target block. Here, M may be a positive integer and may be equal to or greater than 2.
[0634] The encoder / decoder may perform bilateral template matching on one or more candidates included in the skip candidate list or the merge candidate list, generate M prediction blocks using M candidates each having minimum distortion, calculate a weighted sum of the M prediction blocks, and use the weighted sum as a final prediction block of the encoding / decoding target block. Here, M may be a positive integer and may be equal to or greater than 2.
[0635] The encoder / decoder may perform entropy encoding / decoding on a skip index or a merge index for a unidirectional prediction candidate present in a skip candidate list or a merge candidate list, and may generate a prediction block using at least one candidate with minimum distortion among M or more bidirectional prediction candidates, thereby performing motion compensation. In this case, a flag or an index is entropy encoded / decoded to indicate M or more bidirectional prediction candidates. That is, a skip index or a merge index is assigned to each unidirectional prediction candidate, and a skip index or a merge index is jointly assigned to M or more bidirectional prediction candidates. For M or more bidirectional prediction candidates, since at least one candidate representing the minimum distortion can be determined by using bilateral template matching, a skip index or a merge index may not be assigned to each bidirectional prediction candidate.
[0636] Whether to use the motion correction mode may be determined according to the motion correction mode utilization information. Here, the motion correction mode utilization information may be entropy encoded / decoded using at least one of the flag information and the index information.
[0637] The motion correction mode utilization information may be encoded / decoded based on the value of the skip flag. The encoding / decoding timing of the motion correction mode utilization information may be determined based on the encoding / decoding timing of the skip flag. For example, when the skip flag is 1 (i.e., when the skip mode is used), the motion correction mode utilization information may be entropy encoded / decoded. In this case, the motion correction mode utilization information may be entropy encoded / decoded after the skip flag is entropy encoded / decoded.
[0638] Instead, encoding / decoding may be performed on the skip flag based on the value of the motion correction mode utilization information. For example, when the motion correction mode utilization information is 1 (i.e., when the motion correction mode is used), the skip flag may be entropy decoded / decoded. In this case, the motion correction mode utilization information may be entropy encoded / decoded before the skip flag is entropy encoded / decoded.
[0639] The motion correction mode utilization information may be encoded / decoded based on the value of the merge flag. The encoding / decoding timing of the motion correction mode utilization information may be determined based on the encoding / decoding timing of the merge flag. For example, when the merge flag is 1 (i.e., when the merge mode is used), the motion correction mode utilization information may be entropy encoded / decoded. In this case, the motion correction mode utilization information may be entropy encoded / decoded after the merge flag is entropy encoded / decoded.
[0640] Instead, the merge flag may be encoded / decoded based on the value of the motion correction mode utilization information. For example, when the motion correction mode utilization information is 1 (i.e., when the motion correction mode is used), the merge flag may be entropy encoded / decoded. In this case, the motion correction mode utilization information may be entropy encoded / decoded before the merge flag is entropy encoded / decoded.
[0641] The motion correction mode utilization information may be encoded / decoded based on a specific motion compensation mode. For example, when the affine motion compensation mode is not used, the motion correction mode utilization information may be entropy encoded / decoded.
[0642] The decoder motion vector derivation mode flag may be encoded / decoded based on the value of the motion correction mode utilization information. That is, when the motion correction mode utilization information is 0 (i.e., when the motion correction mode is not used), the decoder side motion vector derivation mode flag may be entropy encoded / decoded.
[0643] Entropy encoding / decoding of the motion correction mode utilization information may be performed based on motion correction mode utilization information of an upper level entropy encoded / decoded in at least one of a video parameter set, a sequence parameter set, a picture parameter set, an adaptation parameter set, and a tile group header.
[0644] For example, when the motion correction mode utilization information of the upper level indicates the first value, the motion correction mode utilization information may be entropy encoded / decoded. In addition, the motion correction mode may be used within the corresponding level.
[0645] For example, when the motion correction mode utilization information of the upper level indicates the second value, the motion correction mode utilization information may not be entropy encoded / decoded. In addition, the motion correction mode may not be used in the corresponding level.
[0646] In addition, when the motion correction mode utilization information is not signaled, the motion correction mode utilization information may be derived based on the motion correction mode utilization information of one or more neighboring blocks of the encoding / decoding target block. The one or more neighboring blocks of the encoding / decoding target block may include one or more spatially adjacent neighboring blocks and / or temporally adjacent neighboring blocks. The one or more spatially adjacent blocks may include a block located to the left of the encoding / decoding target block and / or a block located above the encoding / decoding target block.
[0647] When at least one of blocks spatially adjacent to the encoding / decoding target block uses a skip mode, the motion correction mode utilization information may be entropy encoded / decoded.
[0648] When at least one of blocks spatially adjacent to the encoding / decoding target block uses the merge mode, the motion correction mode utilization information may be entropy encoded / decoded.
[0649] When at least one of blocks spatially adjacent to the encoding / decoding target block uses the inter-picture mode, the motion correction mode utilization information may be entropy encoded / decoded.
[0650] The bypass mode may be used to entropy encode / decode the motion correction mode utilization information.
[0651] When the motion correction mode and the skip mode are used together, the residual signal may not be entropy encoded / decoded.
[0652] When the motion correction mode and the merge mode are used together, the residual signal may be entropy encoded / decoded.
[0653] In the motion correction mode, only a portion of the residual signal may be entropy encoded / decoded for use. In this case, the portion of the residual signal may be a DC quantization level (DC transform coefficient).
[0654] When the motion correction mode is used, information other than the motion correction mode utilization information may not be entropy encoded / decoded. Here, the information other than the motion correction mode utilization information may be at least one piece of information on motion compensation.
[0655] In the above, the motion correction mode related to motion information correction has been described. Hereinafter, the motion correction candidates derived in the motion correction mode will be described.
[0656] The motion correction candidate may be motion information including at least one of the following information: a derived motion vector derived by a motion correction mode, a derived reference picture, and a derived inter-picture prediction indicator. The encoder / decoder may add the motion correction candidate as a skip candidate to a skip candidate list or add the motion correction candidate as a merge candidate to a merge candidate list in a skip mode or a merge mode.
[0657] An embodiment of adding motion correction candidates to the skip / merge candidate list will be described below.
[0658] When there is a skip / merge candidate identical to a motion correction candidate in the skip / merge candidate list, the motion correction candidate may not be added to the skip / merge candidate list. When there is a motion correction candidate identical to a skip / merge candidate in the skip / merge candidate list, the skip / merge candidate may not be added to the skip / merge candidate list.
[0659] Motion correction candidates may be derived earlier than spatial skip / merge candidates and may be added to the skip / merge candidate list.
[0660] The motion correction candidate may be derived earlier than the spatial skip / merge candidate derived from the specific position, and then the motion correction candidate may be added to the skip / merge candidate list. Here, the specific position may be Fig.10 At least one of positions A1, B1, B0, A0, B2, B3 and A2.
[0661] Also, the motion correction candidate may be derived earlier than at least one of a temporal skip / merge candidate, a combined merge candidate, and a merge candidate having a predetermined motion information value, and the motion correction candidate may be added to the skip / merge candidate list.
[0662] The encoder / decoder may determine motion information including at least one of a corrected motion vector obtained by motion information correction in a motion correction mode, a corrected reference picture, and a corrected inter-picture prediction indicator as a motion correction candidate. The encoder / decoder may add the motion correction candidate as a skip candidate to a skip candidate list in skip mode or add the motion correction candidate as a merge candidate to a merge candidate list in merge mode.
[0663] Motion correction mode can be used instead of skip mode. That is, the picture can be encoded / decoded by using motion correction mode instead of skip mode. Motion correction mode can be used instead of merge mode. That is, the picture can be encoded / decoded by using motion correction mode instead of merge mode.
[0664] At least one of the overlapped block motion compensation mode, the local illumination compensation mode, and the bidirectional optical flow mode may be applied to the final prediction block generated using the motion correction mode.
[0665] Furthermore, in the motion correction mode, the motion information correction method may be applied only to one piece of motion information or a part of the motion information without generating a list of motion information candidates.
[0666] Hereinafter, conditions for performing motion information correction in the motion correction mode (MRM) will be described.
[0667] Whether to perform motion information correction may be determined based on a picture order count (POC) of a reference picture of a motion vector.
[0668] For example, the motion information correction method can be performed when there are the following two motion vectors: a first motion vector indicating a reference picture whose picture sequence count is smaller than the picture sequence count of the encoding / decoding target picture; and a second motion vector indicating a reference picture whose picture sequence count is larger than the picture sequence count of the encoding / decoding target picture.
[0669] Alternatively, when there are two motion vectors indicating reference pictures having a picture order count smaller than a picture order count of an encoding / decoding target picture, the motion information correction method may be performed.
[0670] Further alternatively, when there are two motion vectors indicating reference pictures having a picture order count greater than a picture order count of an encoding / decoding target picture, the motion information correction method may be performed.
[0671] When at least one of the affine motion compensation mode, the decoder motion vector derivation mode, and the local illumination compensation mode is not used, motion information correction may be performed on the encoding / decoding target block.
[0672] When a difference between a picture sequence count of an encoding / decoding target picture and a picture sequence count of a first reference picture or a difference between a picture sequence count of an encoding / decoding target picture and a picture sequence count of a second reference picture is less than N (N is an integer greater than or equal to 0), motion information correction may be performed. In this case, the first reference picture may refer to a reference picture indicated by a first motion vector, and the second reference picture may refer to a reference picture indicated by a second motion vector.
[0673] Motion information correction may be performed based on the first motion vector and the second motion vector which are targets of the motion information correction.
[0674] For example, when the first motion vector is the same as the second motion vector and when the reference picture indicated by the first motion vector is the same as the reference picture indicated by the second motion vector, motion information correction may not be performed. That is, motion information correction may be performed only when the first motion information and the second motion information are different from each other and / or when the reference pictures indicated by the first motion information and the second motion information are different from each other. In addition, when the first motion vector is the same as the second motion vector, motion information correction may not be performed. That is, motion information correction may be performed only when the first motion vector and the second motion vector are different from each other.
[0675] Alternatively, motion information correction may be performed only when the reference picture indicated by the first motion vector is the same as the reference picture indicated by the second motion vector. Conversely, motion information correction may be performed only when the reference picture indicated by the first motion vector is different from the reference picture indicated by the second motion vector.
[0676] As another example, motion information correction may be performed on a motion vector whose motion vector value is not (0, 0). Here, when the inter-picture prediction indicator indicates bidirectionality and when the values of both the first motion vector and the second motion vector are not (0, 0), motion information correction may be performed. In addition, when the inter-picture prediction indicator indicates N-direction (N is an integer equal to or greater than 2) and when the values of all or a predetermined number of N motion vectors are not (0, 0), motion information correction may not be performed.
[0677] Motion information modification may be performed only when the inter-picture prediction indicator indicates a certain number of directions. For example, motion information modification may be performed only when the inter-picture prediction indicator of the skip / merge candidate indicates bidirectionality.
[0678] Motion information modification may be performed only when the reference picture index of the reference picture indicated by the first motion vector is 0 and the reference picture index of the reference picture indicated by the second motion vector is 0. Alternatively, motion information modification may be performed only when the reference picture index of the reference picture indicated by each motion vector has a specific value.
[0679] Motion information modification may be performed for at least one case of a spatial skip / merge candidate, a temporal skip / merge candidate, a combined skip / merge candidate, and a skip / merge candidate having a predetermined motion information value.
[0680] The motion information correction may be performed in units of subblocks in which the encoding / decoding target block is divided.
[0681] In addition, when the encoding / decoding target block has motion information or motion vectors in units of sub-blocks, motion information correction may be performed in units of sub-blocks to improve encoding efficiency. Here, the sub-blocks of the encoding / decoding target block may have different motion information or motion vectors from each other, or may have the same motion information or motion vectors. Here, when the sub-blocks of the encoding / decoding target block have different motion information or motion vectors from each other, motion information correction may be performed.
[0682] In addition, when the encoding / decoding target block has motion information or motion vector in units of sub-blocks, motion information correction may not be performed for the motion information or motion vector in units of sub-blocks in order to reduce computational complexity. Alternatively, in order to reduce computational complexity, motion information correction may be performed when all sub-blocks of the encoding / decoding target block have the same motion information or motion vector.
[0683] When the method of correcting motion information is performed in units of sub-blocks as described above, initial motion information of each sub-block may be determined as motion information of an encoding / decoding target block to which the sub-block belongs.
[0684] In addition, when the motion information correction method is performed in units of sub-blocks, the initial motion information of each sub-block can be determined by adding a predetermined offset value to the motion information of the encoding / decoding target block to which the sub-block belongs. The motion information correction method is performed for the sub-block unit, and then the initial motion information of each sub-block can be corrected by the corrected motion information of each sub-block.
[0685] In addition, when the motion information correction method is performed in units of sub-blocks, in order to reduce the memory access bandwidth, the sub-blocks may have units of M×N size. M and N may be positive integers, and M and N may have different values from each other. For example, each of M and N may be 8.
[0686] In addition, when the motion information correction method is performed in units of sub-blocks, the size of the sub-block may be determined based on the size of the current block. For example, when the size of the current block is greater than P×P, the size of the sub-block may be determined to be P×P. Here, P may be a positive integer and may be 16.
[0687] In addition, when the motion information correction method is performed in units of sub-blocks, the number of sub-blocks may be determined based on at least one encoding parameter of the current block.
[0688] For example, the number of sub-blocks may vary according to the size of the current block. When the current block has a size of 32×32, the number of sub-blocks may be 16, and when the current block has a size of 64×64, the number of sub-blocks may be 4.
[0689] For example, the number of sub-blocks may vary according to the shape of the current block. When the shape of the current block is a 16×32 vertically long non-square, the number of sub-blocks may be 8, and when the shape of the current block is a 32×32 square, the number of sub-blocks may be 16.
[0690] Furthermore, when the motion information correction method is performed in units of sub-blocks, the shape of the sub-block may be fixed to a square regardless of the shape of the current block.
[0691] Furthermore, when the motion information correction method is performed in units of sub-blocks, the shape of the sub-block may be determined to be the same as the block shape of the current block.
[0692] For example, when the shape of the current block is a vertically long non-square, the shape of the sub-block may be determined to be a vertically long non-square.
[0693] In addition, when the motion information correction method is performed in units of sub-blocks, information on whether to perform in units of sub-blocks may be determined based on information transmitted by a bitstream signal. Here, the information on whether to perform in units of sub-blocks may be transmitted by a signal in units of at least one of a video, a sequence, a picture, a slice, a tile group, a tile, a coding tree unit, a coding unit, a prediction unit, a transform unit, and a block. The information on whether to perform in units of sub-blocks may indicate whether the motion information correction method in units of sub-blocks is performed for the corresponding unit.
[0694] Also, the motion information correction method may be performed in sub-block units with respect to motion information of a sub-block unit determined by using at least one of the motion information prediction methods in sub-block units.
[0695] Furthermore, the motion information correction method may be performed in units of at least one of samples and blocks.
[0696] When the encoding / decoding target block is N-directional prediction such as tri-directional prediction, quad-directional prediction, etc., motion information correction may be performed by using N motion vector calculation templates such as three motion vectors, four motion vectors, etc. Here, N may be a positive integer equal to or greater than 3.
[0697] At least one of the motion information corrected by the motion information correction method may be used for motion compensation. Here, the corrected motion information may include a corrected motion vector.
[0698] At least one of the motion information corrected by the motion information correction method may be used to store a motion vector of a temporal motion vector. Here, the corrected motion information may include a corrected motion vector. Here, when the corrected motion information is stored as a motion vector of a temporal motion vector, a corrected motion vector stored in a current picture of a picture / strip / tile / tile group other than a current picture / strip / tile / tile group may be used as a temporal motion vector.
[0699] Furthermore, the motion information correction method may be operated with a specific rule in a first motion information correction mode that may be performed in an encoder / decoder without signaling additional information of the motion information correction.
[0700] In addition, the motion information correction method may be performed in the encoder, and then the motion information correction method may be operated in a second motion information correction mode, wherein the second motion information correction mode may entropy encode / decode information related to the motion information correction method (at least one of information on whether to use motion information correction, information indicating an initial motion to be a target of motion information correction, information on a search direction, and information on a search distance) and transmit it to the decoder. Here, the decoder may calculate the corrected motion information by using the information related to the motion information correction method without using additional motion information correction.
[0701] When bilateral template matching is used as motion information modification, new motion information may be generated by performing scaling on already derived motion information.
[0702] In one example, when a single piece of first motion information or a single first motion vector exists in a spatially adjacent neighboring block or a temporally adjacent neighboring block, second motion information or a second motion vector may be generated by performing scaling based on the corresponding first motion information or first motion vector, and thus motion information correction may be performed.
[0703] Similarly, when a single piece of second motion information or a single second motion vector exists in a spatially adjacent neighboring block or a temporally adjacent neighboring block, first motion information or a first motion vector may be generated by performing scaling based on the corresponding second motion information or second motion vector, and thus motion information correction may be performed.
[0704] In another example, when the reference image indicated by the initial motion vector and the reference image indicated by the revised motion vector candidate are different, the revised motion vector candidate may be calculated by performing scaling on the initial motion vector based on the POC value of each reference image. Here, the revised motion vector candidate may represent a motion vector indicating an area for searching for the revised motion vector, and bilateral template matching may be performed by comparing another revised motion vector candidate with a distortion value in a position indicated by the initial motion vector.
[0705] Furthermore, a corrected motion vector may be calculated by using motion information correction of at least one of a single luminance component motion vector and two chrominance component motion vectors.
[0706] Furthermore, after performing motion information correction on all or part of the reference images in the reference image list, a reference image with minimum distortion may be derived as a corrected reference image.
[0707] When motion information modification is performed in motion vector derivation mode (decoder-side motion vector derivation mode), the conditions for performing the above-mentioned motion information modification may be applied to the decoder.
[0708] In addition, based on Figures 16 to 21 The search area is used to perform motion information correction.
[0709] In detail, you can Figures 16 to 21 The initial motion vector derived in the merge candidate list or motion vector candidate list is corrected within the search area.
[0710] In one example, the initial motion vector may be modified within the search area based on at least one of the search direction information and the search distance information. Here, the search direction information and the search distance information may be information sent from the encoder to the decoder and may be sent in a flag or index form.
[0711] In one example, a motion vector to be used for correction may be derived by using at least one of information of a search direction and information of a search distance, and a corrected motion vector may be derived by adding the derived motion vector to an initial motion vector. Here, the motion vector to be used for correction may be added to the initial motion vector by scaling the motion vector according to a POC difference between a current picture and a reference picture.
[0712] Fig. 22 is a diagram illustrating a flowchart of an image decoding method according to an embodiment of the present invention.
[0713] Reference Fig. 22, at S2201, the decoder may derive initial motion information of the current block from at least one of motion information of spatially neighboring blocks, motion information of temporally neighboring blocks, and predefined motion information.
[0714] Subsequently, at S2202 , the decoder may generate modified motion information by performing motion information modification on the initial motion information.
[0715] The above-mentioned step of generating the corrected motion information (S2202) may include: generating a bilateral template based on the initial motion information; calculating a distortion value by comparing the bilateral template with a search area within a reference image indicated by the initial motion information; and generating the corrected motion information by correcting the initial motion information based on the distortion value.
[0716] In addition, in the step of generating the modified motion information (S2202), the modified motion information may be generated in units of sub-blocks of the current block.
[0717] Here, the search area may be an area having a square shape, wherein the area is defined by a pixel range of a predefined integer pixel unit in a horizontal direction and a vertical direction from a pixel indicated by the initial motion information.
[0718] In addition, when the current block has a bi-prediction merge mode, the above step of generating the modified motion information (S2202) may be performed.
[0719] In addition, whether to perform the above-mentioned step of generating the corrected motion information (S2202) may be determined based on the size of the current block.
[0720] In addition, whether to perform the above-mentioned step of generating the corrected motion information (S2202) may be determined according to the motion correction mode utilization information.
[0721] In addition, the distortion value used in S2202 may be calculated based on at least one of the sum of absolute differences (SAD), the sum of absolute transformed differences (SATD), the sum of squared errors (SSE), the mean of squared errors (MSE), and the mean-SAD removed (MR-SAD).
[0722] In addition, in the step of generating the modified motion information ( S2202 ), the initial motion information may be modified within the search area.
[0723] Here, the search area may be cross-shaped.
[0724] In addition, the above-mentioned step of generating the corrected motion information (S2202) may be performed when the current block has a merge mode.
[0725] Furthermore, in the step of generating the corrected motion information ( S2202 ), the initial motion vector may be corrected within the search area based on at least one of the information of the search direction and the information of the search distance.
[0726] Fig.23 is a diagram illustrating a flowchart of an image decoding method according to an embodiment of the present invention.
[0727] Reference Fig.23 , at S2301, the decoder may determine a co-located image of the current block.
[0728] In detail, in the step of determining the co-located image of the current block (S2301), the co-located image may be determined based on information indicating the co-located image. Here, the information indicating the co-located image may be information transmitted by a signal in units of at least one of a sequence, a picture, and a parallel block group.
[0729] Subsequently, at S2302 , the decoder may determine a co-located block in the co-located image by using motion information of at least one block spatially adjacent to the block.
[0730] Subsequently, at S2303 , the decoder may derive merging candidates in units of sub-blocks from sub-blocks obtained by dividing the co-located blocks.
[0731] Here, when the size of the current block is equal to or greater than a preset value, a merge candidate in units of sub-blocks may be derived. Here, the preset value may be 8×8.
[0732] In addition, the sub-block may have a square shape, wherein the square shape has a preset size. Here, the preset size may be 8×8.
[0733] Subsequently, at S2304 , the decoder may generate a merge candidate list by using merge candidates in units of sub-blocks.
[0734] The image decoding method according to the present invention has been described. Each step of the above-described image decoding method can be identically performed in the image encoding method. In addition, a bit stream can be generated by using the image encoding method.
[0735] The above embodiments may be performed in the same way in an encoder and a decoder.
[0736] At least one of the above embodiments or a combination thereof may be used to encode / decode an image.
[0737] The order in which the above embodiments are applied may be different between an encoder and a decoder, or the order in which the above embodiments are applied may be the same in an encoder and a decoder.
[0738] The above embodiments may be performed on each of the luminance signal and the chrominance signal, or may be performed identically on the luminance signal and the chrominance signal.
[0739] The block shape to which the above embodiments of the present invention are applied may have a square shape or a non-square shape.
[0740] The above embodiments of the present invention may be applied according to the size of at least one of a coding block, a prediction block, a transform block, a block, a current block, a coding unit, a prediction unit, a transform unit, a unit, and a current unit. Here, the size may be defined as a minimum size or a maximum size or both a minimum size and a maximum size to which the above embodiments are applied, or may be defined as a fixed size to which the above embodiments are applied. In addition, in the above embodiments, the first embodiment may be applied to a first size, and the second embodiment may be applied to a second size. In other words, the above embodiments may be applied in combination according to the size. In addition, the above embodiments may be applied when the size is equal to or greater than the minimum size and equal to or less than the maximum size. In other words, the above embodiments may be applied when the block size is included in a specific range.
[0741] For example, when the size of the current block is 8×8 or larger, the above embodiment may be applied. For example, when the size of the current block is 4×4 or larger, the above embodiment may be applied. For example, when the size of the current block is 16×16 or larger, the above embodiment may be applied. For example, when the size of the current block is equal to or larger than 16×16 and equal to or smaller than 64×64, the above embodiment may be applied.
[0742] The above embodiments of the present invention may be applied according to the time layer. In order to identify the time layer to which the above embodiments may be applied, an identifier identifying the specified time layer may be sent by a signal, and the above embodiments may be applied to the specified time layer identified by the corresponding identifier. Here, the identifier may be defined as the lowest layer or the highest layer or both the lowest layer and the highest layer to which the above embodiments may be applied, or may be defined as a specific layer indicating that the embodiment is applied. In addition, a fixed time layer to which the embodiment is applied may be defined.
[0743] For example, when the temporal layer of the current image is the lowest layer, the above embodiment may be applied. For example, when the temporal layer identifier of the current image is 1, the above embodiment may be applied. For example, when the temporal layer of the current image is the highest layer, the above embodiment may be applied.
[0744] A stripe type to which the above embodiments of the present invention are applied may be defined, and the above embodiments may be applied according to the corresponding stripe type.
[0745] According to the present invention as described above, when calculating the boundary strength in the deblocking filter, the motion information before being corrected can be used.
[0746] According to the present invention mentioned above, the motion vector may have at least one of a 16-pixel (16-pel) unit, an 8-pixel (8-pel) unit, a 4-pixel (4-pel) unit, an integer-pixel (integer-pel) unit, a 1 / 2-pixel (1 / 2-pel) unit, a 1 / 4-pixel (1 / 4-pel) unit, a 1 / 8-pixel (1 / 8-pel) unit, a 1 / 16-pixel (1 / 16-pel) unit, a 1 / 32-pixel (1 / 32-pel) unit, and a 1 / 64-pixel (1 / 64-pel) unit.
[0747] The embodiments of the present invention may be applied to an inter prediction and motion compensation method using motion vector prediction, as well as to an inter prediction and motion compensation method using a skip mode, a merge mode, and the like.
[0748] The embodiment of the present invention may be applied to an intra-block copy mode that uses a reconstructed area within a current image as a prediction block by using a block vector.
[0749] In the above embodiments, the method is described based on a flow chart with a series of steps or units, but the present invention is not limited to the order of these steps, but some steps can be performed simultaneously with other steps or in a different order. In addition, it should be understood by those skilled in the art that the steps in the flow chart are not mutually exclusive, and other steps can be added to the flow chart or some of the steps can be deleted from the flow chart without affecting the scope of the present invention.
[0750] The embodiments include various aspects of the examples. All possible combinations of the various aspects may not be described, but those skilled in the art will be able to recognize different combinations. Therefore, the present invention may include all substitutions, modifications and changes within the scope of the claims.
[0751] Embodiments of the present invention can be implemented in the form of program instructions, wherein the program instructions can be executed by various computer components and recorded in a computer-readable recording medium. The computer-readable recording medium can include program instructions, data files, data structures, etc. alone, or can include a combination of program instructions, data files, data structures, etc. The program instructions recorded in the computer-readable recording medium can be specially designed and constructed for the present invention, or are well known to those of ordinary skill in the field of computer software technology. Examples of computer-readable recording media include magnetic recording media (such as hard disks, floppy disks, and magnetic tapes), optical data storage media (such as CD-ROMs or DVD-ROMs), magneto-optical media (such as floppy disks), and hardware devices (such as read-only memories (ROMs), random access memories (RAMs), flash memories, etc.) that are specially constructed to store and implement program instructions. Examples of program instructions include not only machine language codes formatted by compilers, but also high-level language codes that can be implemented by computers using interpreters. The hardware device can be configured to be operated by one or more software modules or vice versa to implement processing according to the present invention.
[0752] Although the present invention has been described in terms of specific items (such as, detailed elements) and limited embodiments and drawings, they are only provided to help a more comprehensive understanding of the present invention, and the present invention is not limited to the above embodiments. It will be appreciated by those skilled in the art that various modifications and changes may be made to the above description.
[0753] Therefore, the spirit of the present invention should not be limited to the above-described embodiments, and the full scope of the appended claims and their equivalents should fall within the scope and spirit of the present invention.
[0754] Industrial Applicability
[0755] The present invention can be used when encoding / decoding an image.
Claims
1. An image decoding device for image decoding, the image decoding device comprising: Memory; as well as at least one processor connected to the memory and configured to: Obtaining image information including residual information of a current block and information related to motion vector accuracy, wherein the information related to motion vector accuracy indicates accuracy of the motion vector of the current block; Derivation of the motion vector of the current block based on the motion vector accuracy related information; deriving a prediction sample of the current block based on the motion vector; Derives residual samples of the current block based on the residual information; and Generate a reconstructed picture based on the prediction sample points and the residual sample points, wherein, based on the motion vector precision related information, the precision of the motion vector of the current block is derived as quarter pixel precision, one eighth pixel precision or integer pixel precision, and The information related to the motion vector accuracy is decoded by a fixed-length binarization method.
2. An image encoding device for image encoding, the image encoding device comprising: Memory; as well as at least one processor connected to the memory and configured to: Derivation of the motion vector of the current block; deriving a prediction sample of the current block based on the motion vector; deriving residual samples of the current block based on the prediction samples; and Encoding image information including residual information of the current block and information related to motion vector accuracy, The motion vector accuracy related information indicates the accuracy of the motion vector of the current block, and The motion vector accuracy related information is encoded by a fixed-length binarization method.
3. A device for transmitting image data, the device being configured to: Obtain a bit stream of encoded image information, where The encoded image information is generated based on the following operations: deriving a motion vector of a current block, deriving a prediction sample of the current block based on the motion vector, and deriving a residual sample of the current block based on the prediction sample; and encoding image information including residual information of the current block and information related to motion vector accuracy; and sending the image data, the image data comprising the bit stream, The motion vector accuracy related information indicates the accuracy of the motion vector of the current block, and The motion vector accuracy related information is encoded by a fixed-length binarization method.
