Method for processing image based on inter-prediction mode and device therefor

Abstract

To provide methods and devices for reducing hardware resource usage in performing an inter prediction based on a similarity of temporal motion information.SOLUTION: A method for decoding a video signal based on an inter prediction mode includes the steps of: if a merge mode is applied to a current block, generating a merge candidate list by using a spatial neighboring block and a temporal merge candidate of the current block; obtaining a merge index indicating a candidate to be used for an inter prediction of the current block in the merge candidate list; deriving a motion vector of each of subblocks included in the current block based on a motion vector of the candidate used for the inter prediction of the current block; and generating a prediction sample of the current block by using the motion vector of each of subblocks.SELECTED DRAWING: Figure 14

Description

[Technical field]

[0001] The present invention relates to a method for processing still or moving images, and more particularly to a method for encoding / decoding still or moving images based on an inter prediction mode and an apparatus supporting the same. [Background technology]

[0002] Compression coding refers to a series of signal processing techniques for transferring (sending) digitized information over communication lines or storing it in a form suitable for storage media. Media such as video, images, and audio can be subject to compression coding, and the technology that performs compression coding on video in particular is called video compression.

[0003] Next-generation video content will be characterized by high spatial resolution, high frame rate, and high dimensionality of scene representation. Processing such content will bring about a tremendous increase in memory storage, memory access rate, and processing power.

[0004] Therefore, there is a need to design coding tools to more efficiently process the next generation of video content. Summary of the Invention [Problem to be solved by the invention]

[0005] The object of the present invention is to propose a method and an apparatus for reducing the usage of hardware resources when performing inter-prediction (or inter-frame prediction) based on the similarity of temporal motion information.

[0006] The technical problems to be solved by the present invention are not limited to those mentioned above, and other technical problems not mentioned will be clearly understood by those having ordinary skill in the art to which the present invention pertains from the description below. [Means for solving the problem]

[0007] One aspect of the present invention is a method for decoding an image based on an inter prediction mode, comprising the steps of: generating a merge candidate list using spatial neighboring blocks and temporal merge candidates of a current block when a merge mode is applied to a current block; acquiring a merge index indicating a candidate used for inter prediction of the current block in the merge candidate list; deriving a motion vector of each of a plurality of sub-blocks included in the current block based on a motion vector of the candidate used for inter prediction of the current block; and generating a prediction sample of the current block using the motion vector of each of the sub-blocks, wherein the size of the sub-blocks may be set to a fixed specific size of 8x8 or more.

[0008] In an embodiment, generating a merge candidate list may include adding to the merge candidate list temporal neighboring blocks of the current block identified by the motion vectors of the spatial neighboring blocks in a collocated picture of the current block.

[0009] In an embodiment, the step of generating a merge candidate list may include a step of adding to the merge candidate list a temporal neighboring block of the current block identified by a motion vector of a first spatial neighboring block of the current block in a predefined order in a collocated picture of the current block.

[0010] In an embodiment, the step of generating a merge candidate list may include a step of checking whether the width and height of the current block are smaller than a predefined minimum size, and if the check results in the width and height of the current block being equal to or greater than the predefined minimum size, adding to the merge candidate list a temporal neighboring block of the current block identified by a motion vector of a spatial neighboring block in a collocated picture of the current block.

[0011] In an embodiment, the predefined minimum size may be set to the size of a sub-block.

[0012] Another aspect of the present invention is an apparatus for decoding an image based on an inter prediction mode, the apparatus including: a merge candidate list generation unit that generates a merge candidate list using spatial merge candidates and temporal merge candidates of a current block when a merge mode is applied to a current block; a merge index acquisition unit that acquires a merge index indicating a merge candidate used for inter prediction of the current block in the merge candidate list; a sub-block motion vector derivation unit that derives motion vectors of each of a plurality of sub-blocks included in the current block based on motion vectors of the merge candidates used for inter prediction of the current block; and a prediction block generation unit that generates a prediction block of the current block using the motion vectors of each of the sub-blocks, wherein a size of the sub-blocks may be set to a fixed specific size of 8x8 or more.

[0013] In an embodiment, the merging candidate list generator may add to the merging candidate list temporal neighboring blocks of the current block identified by the motion vector of the spatial neighboring blocks in a collocated picture of the current block.

[0014] In an embodiment, the merge candidate list generator may add to the merge candidate list a temporal neighboring block of the current block identified by a motion vector of a first spatial neighboring block of the current block in a predefined order in a collocated picture of the current block.

[0015] In an embodiment, the merge candidate list generator checks whether the width and height of the current block are smaller than a predefined minimum size, and if the check results in the width and height of the current block being greater than or equal to the predefined minimum size, the merge candidate list generator may add a temporal neighboring block of the current block identified by a motion vector of a spatial neighboring block in a collocated picture of the current block to the merge candidate list.

[0016] In an embodiment, the predefined minimum size may be set to the size of a sub-block. Effect of the Invention

[0017] According to an embodiment of the present invention, the hardware gate count can be effectively reduced when generating a prediction block using motion information of an already decoded picture.

[0018] In addition, according to an embodiment of the present invention, by using motion information of an already decoded picture to perform inter prediction, it is possible to reduce the hardware gate count while minimizing the associated loss of compression performance.

[0019] The effects obtained by the present invention are not limited to those mentioned above, and other effects not mentioned should be clearly understood by those having ordinary skill in the art to which the present invention pertains from the following description. [Brief description of the drawings]

[0020] [Figure 1] 1 is a schematic block diagram of an encoding device for encoding a video / image signal as an embodiment to which the present invention is applied. [Diagram 2] 1 is a schematic block diagram of a decoding device for decoding a video / image signal as an embodiment to which the present invention is applied. [Diagram 3] FIG. 2 is a diagram showing an example of a multi-type tree structure as an embodiment to which the present invention can be applied. [Figure 4] FIG. 1 illustrates a signaling mechanism for partitioning information of a quadtree with nested multi-type tree structure as an embodiment to which the present invention can be applied. [Diagram 5] FIG. 10 illustrates a method for dividing a CTU into multiple CUs based on a quadtree and nested multi-type tree structure as an embodiment to which the present invention can be applied. [Figure 6] FIG. 2 is a diagram illustrating a method for restricting ternary-tree division as an embodiment to which the present invention can be applied. [Figure 7] 1 is a diagram illustrating an example of a redundant partitioning pattern that can occur in binary tree partitioning and ternary tree partitioning as an embodiment to which the present invention can be applied. FIG. [Figure 8]1 is a diagram illustrating an inter-prediction based video / image encoding method according to an embodiment of the present invention and an inter-prediction unit in an encoding device according to an embodiment of the present invention. [Figure 9] 1 is a diagram illustrating an inter-prediction based video / image encoding method according to an embodiment of the present invention and an inter-prediction unit in an encoding device according to an embodiment of the present invention. [Figure 10] 1 is a diagram illustrating a method for decoding an inter-prediction based video / image according to an embodiment of the present invention and an inter-prediction unit in a decoding device according to an embodiment of the present invention; [Figure 11] 1 is a diagram illustrating a method for decoding an inter-prediction based video / image according to an embodiment of the present invention and an inter-prediction unit in a decoding device according to an embodiment of the present invention; [Figure 12] FIG. 1 is a diagram illustrating peripheral blocks used in a merge mode or a skip mode as an embodiment to which the present invention is applied. [Figure 13] 1 is a flowchart illustrating a method for constructing a merge candidate list according to an embodiment of the present invention; [Figure 14] 1 is a flowchart illustrating a method for constructing a merge candidate list according to an embodiment of the present invention; [Figure 15] FIG. 1 is a diagram for explaining a method for deriving ATMVP (Advanced Temporal Motion Vector Prediction) candidates as an embodiment to which the present invention is applied. [Figure 16] FIG. 1 is a diagram for explaining a method for deriving ATMVP (Advanced Temporal Motion Vector Prediction) candidates as an embodiment to which the present invention is applied. [Figure 17] FIG. 1 is a diagram illustrating a method for deriving Advanced Temporal Motion Vector Prediction (ATMVP) candidates as an embodiment to which the present invention is applied. [Figure 18]1 is a diagram illustrating a method for compressing temporal motion vector data according to an embodiment of the present invention and the positions of spatial candidates used therein; [Figure 19] 1 is a diagram illustrating a method for compressing temporal motion vector data according to an embodiment of the present invention and the positions of spatial candidates used therein; [Figure 20] 1 is a flowchart illustrating a method for generating an inter predicted block based on a candidate list according to an embodiment of the present invention. [Figure 21] 1 is a flowchart illustrating a method for generating a candidate list according to an embodiment of the present invention. [Figure 22] FIG. 1 is a diagram showing an example of neighboring blocks used in inter prediction according to an embodiment to which the present invention is applied. [Diagram 23] FIG. 1 is a diagram illustrating a method for deriving an extended Advanced Temporal Motion Vector Prediction (ATMVP) candidate according to an embodiment to which the present invention is applied. [Figure 24] FIG. 2 illustrates a process of fetching (patching) motion vectors of temporally neighboring blocks in a reference picture according to an embodiment of the present invention; [Diagram 25] FIG. 2 illustrates a process of fetching motion vectors of temporally neighboring blocks in a reference picture according to an embodiment of the present invention; [Figure 26] FIG. 2 is a diagram illustrating a fetching process of motion vectors of temporal neighboring blocks on a sub-block basis according to an embodiment to which the present invention is applied. [Figure 27]4 is a flowchart illustrating a method for generating a candidate list according to an embodiment of the present invention. [Figure 28] 1 is a flowchart illustrating a method for generating an inter-prediction block according to an embodiment to which the present invention is applied. [Figure 29] 1 is a diagram illustrating an inter prediction device according to an embodiment to which the present invention is applied. [Diagram 30] 1 is a diagram showing a video coding system to which the present invention is applied; [Diagram 31] FIG. 1 is a structural diagram of a content streaming system as an embodiment to which the present invention is applied. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0021] The accompanying drawings, which are included as part of the detailed description to facilitate understanding of the present invention, provide embodiments of the present invention and, together with the detailed description, explain the technical features of the present invention.

[0022] Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. The detailed description disclosed below in conjunction with the accompanying drawings is intended to describe exemplary embodiments of the present invention, and is not intended to show the only embodiments in which the present invention can be implemented. The following detailed description includes specific details to provide a complete understanding of the present invention. However, those skilled in the art will recognize that the present invention can be implemented without such specific details.

[0023] In some cases, in order to avoid obscuring the concept of the present invention, well-known structures and devices may be omitted or shown in the form of a block diagram focusing on the core functions of each structure and device.

[0024] Furthermore, the terms used in the present invention are generally used as widely as possible, but in certain cases, the applicant may use terms arbitrarily selected by the applicant. In such cases, the meanings will be clearly described in the detailed description of the relevant part, and it is made clear that the terms should not be interpreted simply based on the names of the terms used in the present invention, but should be interpreted while also understanding the meanings of the corresponding terms.

[0025] The specific terms used in the following description are provided to aid in understanding the present invention, and the use of such specific terms may be changed to other forms without departing from the technical spirit of the present invention. For example, in the case of a signal, data, sample, picture, frame, block, etc., they may be appropriately substituted and interpreted in each coding process.

[0026] Hereinafter, in this specification, a "processing unit" refers to a unit in which an encoding / decoding process such as prediction, transformation, and / or quantization is performed. Hereinafter, for convenience of explanation, a processing unit may also be referred to as a "processing block" or a "block."

[0027] The processing unit may be interpreted as including a unit of a luminance (luma) component and a unit of a chrominance (chroma) component. For example, the processing unit may correspond to a coding tree unit (CTU), a coding unit (CU), a prediction unit (PU), or a transform unit (TU).

[0028] Also, the processing unit may be interpreted as a unit of a luma component or a unit of a chroma component. For example, the processing unit may correspond to a coding tree block (CTB), a coding block (CB), a prediction block (PU), or a transform block (TB) of the luma component. Alternatively, the processing unit may correspond to a coding tree block (CTB), a coding block (CB), a prediction block (PU), or a transform block (TB) of the chroma component. Also, without being limited thereto, the processing unit may be interpreted to include a unit of a luma component and a unit of a chroma component.

[0029] Furthermore, the processing units are not necessarily limited to square blocks, but may also be arranged in the shape of polygons having three or more vertices.

[0030] In the following description, pixels or picture elements are collectively referred to as samples. Using a sample can mean using a pixel value or picture element value.

[0031] FIG. 1 is a schematic block diagram of an encoding device for encoding a video / image signal as an embodiment to which the present invention is applied.

[0032] Referring to FIG. 1, the encoding device 100 may include an image division unit 110, a subtraction unit 115, a transformation unit 120, a quantization unit 130, an inverse quantization unit 140, an inverse transformation unit 150, an addition unit 155, a filtering unit 160, a memory 170, an inter prediction unit 180, an intra prediction unit 185, and an entropy encoding unit 190. The inter prediction unit 180 and the intra prediction unit 185 may be collectively referred to as a prediction unit. That is, the prediction unit may include the inter prediction unit 180 and the intra prediction unit 185. The transformation unit 120, the quantization unit 130, the inverse quantization unit 140, and the inverse transformation unit 150 may be included in a residual processing unit. The residual processing unit may further include a subtraction unit 115. In one embodiment, the image division unit 110, subtraction unit 115, transformation unit 120, quantization unit 130, inverse quantization unit 140, inverse transformation unit 150, addition unit 155, filtering unit 160, inter prediction unit 180, intra prediction unit 185, and entropy encoding unit 190 may be configured as one hardware component (e.g., an encoder or processor). Also, the memory 170 may include a decoded picture buffer (DPB) and may be configured as a digital storage medium.

[0033] The image division unit 110 may divide an input image (or a picture or a frame) input to the encoding device 100 into one or more processing units. As an example, the processing unit may be called a coding unit (CU). In this case, the coding unit may be recursively divided from a coding tree unit (CTU) or a largest coding unit (LCU) based on a quad-tree binary-tree (QTBT) structure. For example, one coding unit may be divided into a plurality of coding units of a deeper sub-depth based on a quad-tree structure and / or a binary tree structure. In this case, for example, the quad-tree structure may be applied first, and the binary tree structure may be applied later. Alternatively, the binary tree structure may be applied first. A coding procedure according to the present invention may be performed based on a final coding unit that is not further divided. In this case, the coding unit may be immediately used as the final coding unit based on the coding efficiency according to the characteristics of the image, or the coding unit may be recursively divided into coding units of lower depths as necessary, and the coding unit of the optimal size may be used as the final coding unit. Here, the coding procedure may include procedures such as prediction, transformation, and restoration, which will be described later. As another example, the processing unit may further include a prediction unit (PU) or a transform unit (TU). In this case, the prediction unit and the transform unit may be divided or partitioned from the final coding unit described above. The prediction unit may be a unit of sample prediction, and the transform unit may be a unit for deriving transform coefficients and / or a unit for deriving a residual signal from the transform coefficients.

[0034] The unit may be mixed with terms such as block or area in some cases. In the general case, an MxN block may refer to a set of samples or transform coefficients consisting of M columns and N rows. A sample may generally refer to a pixel or a pixel value, or may refer to only a pixel / pixel value of a luma component, or may refer to only a pixel / pixel value of a chroma component. A sample may be used as a term corresponding to a pixel or pel of one picture (or image).

[0035] The encoding device 100 may subtract a prediction signal (predicted block, prediction sample array) output from the inter prediction unit 180 or the intra prediction unit 185 from an input video signal (original block, original sample array) to generate a residual signal (residual signal, residual block, remaining sample array), and the generated residual signal is transferred to the conversion unit 120. In this case, as shown in the figure, a unit that subtracts a prediction signal (prediction block, prediction sample array) from an input video signal (original block, original sample array) in the encoding device 100 may be called a subtraction unit 115. The prediction unit may predict a block to be processed (hereinafter, referred to as a current block) and generate a predicted block including a prediction sample of the current block. The prediction unit may determine whether intra prediction or inter prediction is applied in units of the current block or CU. The prediction unit may generate various information related to prediction, such as prediction mode information, and transmit the information to the entropy encoding unit 190, as described later in the description of each prediction mode. The prediction information can be encoded by the entropy encoding unit 190 and output in the form of a bitstream.

[0036] The intra prediction unit 185 may predict the current block by referring to samples in the current picture. The referenced samples may be located in the neighborhood of the current block or may be located away from the current block depending on the prediction mode. In intra prediction, the prediction mode may include a plurality of non-directional modes and a plurality of directional modes. The non-directional mode may include, for example, a DC mode and a planar mode. The directional mode may include, for example, 33 directional prediction modes or 65 directional prediction modes depending on the fineness of the prediction direction. However, this is an example, and more or less directional prediction modes may be used depending on the setting. The intra prediction unit 185 may also determine the prediction mode to be applied to the current block using the prediction modes applied to the neighboring blocks.

[0037] The inter prediction unit 180 may derive a predicted block of the current block based on a reference block (an array of reference samples) identified by a motion vector on a reference picture. In this case, in order to reduce the amount of motion information transferred in the inter prediction mode, the motion information may be predicted in units of blocks, sub-blocks, or samples based on the correlation of the motion information between the neighboring blocks and the current block. The motion information may include a motion vector and a reference picture index. The motion information may further include information on an inter prediction direction (such as L0 prediction, L1 prediction, or Bi prediction). In the case of inter prediction, the neighboring blocks may include a spatial neighboring block present in the current picture and a temporal neighboring block present in the reference picture. The reference picture including the reference block and the reference picture including the temporal neighboring block may be the same or different. The temporal surrounding blocks may be called collocated reference blocks, collocated CUs (colCUs), etc., and the reference picture including the temporal surrounding blocks may be called collocated pictures (colPic). For example, the inter prediction unit 180 may construct a motion information candidate list based on the surrounding blocks and generate information indicating which candidate is used to derive the motion vector and / or reference picture index of the current block. Inter prediction may be performed based on various prediction modes, and for example, in the case of skip mode and merge mode, the inter prediction unit 180 may use the motion information of the surrounding blocks as the motion information of the current block. In the case of skip mode, unlike the merge mode, a residual signal may not be transferred.In the case of a Motion Vector Prediction (MVP) mode, the motion vector of the current block can be indicated by signaling the motion vector difference using the motion vector of a neighboring block as a motion vector predictor.

[0038] The prediction signal generated via the inter prediction unit 180 or the intra prediction unit 185 can be used to generate a reconstructed signal or can be used to generate a residual signal.

[0039] The transform unit 120 may generate transform coefficients by applying a transform technique to the residual signal. For example, the transform technique may include at least one of a discrete cosine transform (DCT), a discrete sine transform (DST), a Karhunen-Loeve transform (KLT), a graph-based transform (GBT), or a conditionally non-linear transform (CNT). Here, the GBT refers to a transform obtained from a graph when the relationship information between pixels is represented by a graph. The CNT refers to a transform obtained based on a predicted signal generated using all previously reconstructed pixels. In addition, the transform process may be applied to pixel blocks having the same square size, or may be applied to non-square variable-sized blocks.

[0040] The quantization unit 130 quantizes the transform coefficients and transmits them to the entropy encoding unit 190, which may encode the quantized signal (information on the quantized transform coefficients) and output it as a bitstream. The information on the quantized transform coefficients may be called residual information. The quantization unit 130 may rearrange the quantized transform coefficients in a block form into a one-dimensional vector form based on a coefficient scan order, and may generate information on the quantized transform coefficients based on the quantized transform coefficients in the one-dimensional vector form. The entropy encoding unit 190 may perform various encoding methods, such as exponential Golomb, Context-Adaptive Variable Length Coding (CAVLC), and Context-Adaptive Binary Arithmetic Coding (CABAC). The entropy encoding unit 190 may encode information required for video / image restoration (e.g., values ​​of syntax elements) together with or separately from the quantized transform coefficients. The encoded information (e.g., encoded video / image information) may be transferred or stored in the form of a bitstream in Network Abstraction Layer (NAL) units. The bitstream may be transferred via a network or stored in a digital storage medium. Here, the network may include a broadcast network and / or a network, and the digital storage medium may include various storage media such as a USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. A transmission (sending) unit (not shown) that transmits (sends) the signal output from the entropy encoding unit 190 and / or a storage unit (not shown) that stores the signal may be configured as an internal / external element of the encoding device 100, or the transmission (sending) unit may be a component of the entropy encoding unit 190.

[0041] The quantized transform coefficients output from the quantizer 130 may be used to generate a prediction signal. For example, a residual signal may be restored by applying inverse quantization and inverse transform to the quantized transform coefficients via the inverse quantizer 140 and the inverse transformer 150 in the loop. The adder 155 may generate a reconstructed signal (reconstructed picture, reconstructed block, reconstructed sample array) by adding the reconstructed residual signal to the prediction signal output from the inter prediction unit 180 or the intra prediction unit 185. When there is no residual of the block to be processed, such as when the skip mode is applied, the predicted block may be used as the reconstructed block. The adder 155 may be referred to as a reconstruction unit or a reconstructed block generator. The generated reconstructed signal may be used for intra prediction of the next block to be processed in the current picture, and may also be used for inter prediction of the next picture after filtering as described below.

[0042] The filtering unit 160 may apply a filter to the reconstructed signal to improve subjective / objective image quality. For example, the filtering unit 160 may apply various filtering methods to the reconstructed picture to generate a modified reconstructed picture, and may store the modified reconstructed picture in the memory 170, specifically, in the DPB of the memory 170. The various filtering methods may include, for example, deblocking (deblocking) filtering, sample adaptive offset, an adaptive loop filter, a bilateral filter, etc. The filtering unit 160 may generate various information related to filtering and transmit it to the entropy encoding unit 190, as will be described later in the description of each filtering method. The information related to filtering may be encoded by the entropy encoding unit 190 and output in the form of a bitstream.

[0043] The modified reconstructed picture transferred to the memory 170 can be used as a reference picture by the inter prediction unit 180. Through this, when inter prediction is applied, the encoding apparatus can avoid a prediction mismatch between the encoding apparatus 100 and the decoding apparatus, and can also improve encoding efficiency.

[0044] The memory 170 DPB may store the modified reconstructed picture for use as a reference picture in the inter prediction unit 180. The memory 170 may store motion information of a block from which motion information in the current picture is derived (or encoded) and / or motion information of a block in an already reconstructed picture. The stored motion information may be transmitted to the inter prediction unit 180 to be utilized as motion information of a spatially neighboring block or motion information of a temporally neighboring block. The memory 170 may store reconstructed samples of reconstructed blocks in the current picture and may transmit them to the intra prediction unit 185.

[0045] FIG. 2 is a schematic block diagram of a decoding device for decoding a video / image signal as an embodiment to which the present invention is applied.

[0046] Referring to FIG. 2, the decoding device 200 may include an entropy decoding unit 210, an inverse quantization unit 220, an inverse transform unit 230, an adder 235, a filtering unit 240, a memory 250, an inter prediction unit 260, and an intra prediction unit 265. The inter prediction unit 260 and the intra prediction unit 265 may be collectively referred to as a prediction unit. That is, the prediction unit may include the inter prediction unit 260 and the intra prediction unit 265. The inverse quantization unit 220 and the inverse transform unit 230 may be collectively referred to as a residual processing unit. That is, the residual processing unit may include the inverse quantization unit 220 and the inverse transform unit 230. The entropy decoding unit 210, the inverse quantization unit 220, the inverse transform unit 230, the adder 235, the filtering unit 240, the inter prediction unit 260, and the intra prediction unit 265 may be configured by one hardware component (e.g., a decoder or a processor) according to an embodiment. The memory 170 may also include a Decoded Picture Buffer (DPB) and may be configured with a digital storage medium.

[0047] When a bitstream including video / image information is input, the decoding device 200 can restore an image corresponding to the process in which the video / image information is processed by the encoding device of Fig. 1. For example, the decoding device 200 can perform decoding using a processing unit applied in the encoding device. Thus, the decoding processing unit can be, for example, a coding unit, and the coding unit can be divided from a coding tree unit or a maximal coding unit according to a quadtree structure and / or a binary tree structure. Then, the restored image signal decoded and output via the decoding device 200 can be reproduced by a reproduction device.

[0048] The decoding apparatus 200 may receive a signal output from the encoding apparatus of Fig. 1 in the form of a bitstream, and may decode the received signal via the entropy decoding unit 210. For example, the entropy decoding unit 210 may analyze the bitstream to derive information (e.g., video / image information) required for image restoration (or picture restoration). For example, the entropy decoding unit 210 may decode information in the bitstream based on a coding method such as exponential-Golomb coding, CAVLC, or CABAC, and output values ​​of syntax elements required for image restoration and quantized values ​​of transform coefficients related to residuals. In more detail, the CABAC entropy decoding method receives bins corresponding to each syntax element in a bitstream, determines a context model using information of the syntax element to be decoded and decode information of the neighboring and to-be-decoded blocks, or information of symbols / bins decoded in a previous stage, predicts the occurrence probability of the bin based on the determined context model, and performs arithmetic decoding of the bin to generate a symbol corresponding to the value of each syntax element. In this case, after determining the context model, the CABAC entropy decoding method may update the context model using information of the decoded symbol / bin for the next symbol / empty context model. Information related to prediction among the information decoded by the entropy decoding unit 210 is provided to a prediction unit (inter prediction unit 260 and intra prediction unit 265), and the entropy decoded residual value, i.e., the quantized transform coefficient and related parameter information, may be input from the entropy decoding unit 210 to the inverse quantization unit 220. In addition, information regarding filtering of the information decoded by the entropy decoding unit 210 may be provided to the filtering unit 240. Meanwhile, a receiving unit (not shown) for receiving a signal output from the encoding device may be further configured as an internal / external element of the decoding device 200, or the receiving unit may be a component of the entropy decoding unit 210.

[0049] The inverse quantization unit 220 may inverse quantize the quantized transform coefficients to output transform coefficients. The inverse quantization unit 220 may rearrange the quantized transform coefficients in the form of a two-dimensional block. In this case, the rearrangement may be performed based on a coefficient scan order performed in the encoding device. The inverse quantization unit 220 may perform inverse quantization of the quantized transform coefficients using a quantization parameter (e.g., quantization step size information) to obtain transform coefficients.

[0050] In the inverse transform unit 230, the transform coefficients are inverse transformed to obtain a residual signal (residual block, residual sample array).

[0051] The prediction unit may perform prediction of the current block and generate a predicted block including a prediction sample of the current block. The prediction unit may determine whether intra prediction or inter prediction is applied to the current block based on the information on the prediction output from the entropy decoding unit 210, and may determine a specific intra / inter prediction mode.

[0052] The intra prediction unit 265 can predict the current block by referring to samples in the current picture. The referenced samples can be located in the neighborhood of the current block or can be located far away depending on the prediction mode. In intra prediction, the prediction mode can include a plurality of non-directional modes and a plurality of directional modes. The intra prediction unit 265 can also determine the prediction mode applied to the current block using the prediction modes applied to the neighboring blocks.

[0053] The inter prediction unit 260 may derive a predicted block of the current block based on a reference block (an array of reference samples) identified by a motion vector on a reference picture. In this case, the motion information may be predicted in units of blocks, sub-blocks, or samples based on correlation of motion information between neighboring blocks and the current block in order to reduce the amount of motion information transferred in the inter prediction mode. The motion information may include a motion vector and a reference picture index. The motion information may further include information on an inter prediction direction (such as L0 prediction, L1 prediction, or Bi prediction). In the case of inter prediction, the neighboring blocks may include spatial neighboring blocks present in the current picture and temporal neighboring blocks present in the reference picture. For example, the inter prediction unit 260 may configure a motion information candidate list based on the neighboring blocks, and derive a motion vector and / or a reference picture index of the current block based on the received candidate selection information. Inter prediction may be performed based on various prediction modes, and the information on the prediction may include information indicating a mode of inter prediction of the current block.

[0054] The adder 235 may generate a reconstructed signal (reconstructed picture, reconstructed block, reconstructed sample array) by adding the acquired residual signal to a prediction signal (predicted block, predicted sample array) output from the inter prediction unit 260 or the intra prediction unit 265. When there is no residual for the current block, such as when the skip mode is applied, the predicted block may be used as the reconstructed block.

[0055] The adder 235 may be referred to as a reconstruction unit or a reconstruction block generator. The generated reconstruction signal may be used for intra prediction of the next block to be processed in the current picture, and may also be used for inter prediction of the next picture after filtering as described below.

[0056] The filtering unit 240 may apply filtering to the reconstructed signal to improve subjective / objective image quality. For example, the filtering unit 240 may apply various filtering methods to the reconstructed picture to generate a modified reconstructed picture, and may transfer the modified reconstructed picture to the memory 250, specifically, to the DPB of the memory 250. The various filtering methods may include, for example, deblock filtering, sample adaptive offset, an adaptive loop filter, a bilateral filter, etc.

[0057] The (modified) reconstructed picture stored in the DPB of the memory 250 can be used as a reference picture by the inter prediction unit 260. The memory 250 can store motion information of a block from which motion information in the current picture is derived (or decoded) and / or motion information of a block in an already reconstructed picture. The stored motion information can be transmitted to the inter prediction unit 260 to be utilized as motion information of a spatially neighboring block or motion information of a temporally neighboring block. The memory 250 can store reconstructed samples of reconstructed blocks in the current picture and transmit them to the intra prediction unit 265.

[0058] In this specification, the embodiments described for the filtering unit 160, inter prediction unit 180 and intra prediction unit 185 of the encoding device 100 can also be applied identically or correspondingly to the filtering unit 240, inter prediction unit 260 and intra prediction unit 265 of the decoding device 200, respectively.

[0059] Block Partitioning

[0060] The video / image (video) coding method according to the present document may be performed based on various detailed techniques, each of which is briefly described as follows: It is obvious to those skilled in the art that the techniques described below may be related to related procedures such as prediction, residual processing ((inverse) transformation, (inverse) quantization, etc.), coding of syntax elements, filtering, partitioning / division, etc. in the video / image encoding / decoding procedures described above and / or below.

[0061] The block partitioning procedure according to this document is performed in the image partitioning unit 110 of the encoding device described above, and partitioning-related information can be processed (encoded) by the entropy encoding unit 190 and transmitted to the decoding device in the form of a bitstream. The entropy decoding unit 210 of the decoding device can derive a block partitioning structure of the current picture based on the partitioning-related information acquired from the bitstream, and perform a series of procedures for image decoding based on the derived block partitioning structure (e.g., prediction, residual processing, block reconstruction, in-loop filtering, etc.).

[0062] Partitioning of picture into CTUs

[0063] A picture may be divided into a sequence of coding tree units (CTUs). A CTU may correspond to a coding tree block (CTB). Alternatively, a CTU may include a coding tree block of luma samples and two coding tree blocks of corresponding chroma samples. That is, for a picture containing three sample arrays, a CTU may include an NxN block of luma samples and two corresponding blocks of chroma samples.

[0064] The maximum allowed size of a CTU for coding and prediction, etc. may differ from the maximum allowed size of a CTU for transforms. For example, the maximum allowed size of a luma block in a CTU may be 128x128.

[0065] Partitioning of the CTUs using a tree structure

[0066] A CTU may be divided into CUs based on a quaternary tree (QT) structure. A quaternary tree structure may be called a quaternary tree structure because it reflects various local characteristics. Meanwhile, in this document, a CTU may be divided based on division of a multi-type tree structure including not only a quaternary tree but also a binary tree (BT) and a ternary tree (TT). Hereinafter, a QTBT structure may include a quaternary tree and a binary tree-based division structure, and a QTBTTT may include a quaternary tree, a binary tree and a ternary tree-based division structure. Alternatively, a QTBT structure may include a quaternary tree, a binary tree and a ternary tree-based division structure. In a coding tree structure, a CU may have a square or rectangular shape. A CTU may first be divided into a quaternary tree structure. Thereafter, a leaf node of the quaternary tree structure may be further divided by a multi-type tree structure.

[0067] FIG. 3 is a diagram showing an example of a multi-type tree structure as an embodiment to which the present invention can be applied.

[0068] In one embodiment of the present invention, the multi-type tree structure may include four split types as shown in FIG. 3. The four split types may include Vertical Binary Splitting (SPLIT_BT_VER), Horizontal Binary Splitting (SPLIT_BT_HOR), Vertical Ternary Splitting (SPLIT_TT_VER), and Horizontal Ternary Splitting (SPLIT_TT_HOR). The leaf nodes of the multi-type tree structure may be called CUs. Such CUs may be used for prediction and transformation procedures. In this document, generally, CUs, PUs, and TUs may have the same block size. However, if the maximum supported transform length is smaller than the width or height of a color component of the CU, the CUs and TUs may have different block sizes.

[0069] FIG. 4 is a diagram illustrating a signaling mechanism of partition division information of a quadtree with nested multi-type tree structure as an embodiment to which the present invention can be applied.

[0070] Here, the CTU is treated as the root of a quadtree and is first partitioned in a quadtree structure. Leaf nodes of each quadtree can then be further partitioned in a multi-type tree structure. In the multi-type tree structure, a first flag (e.g., mtt_split_cu_flag) is signaled to indicate whether the node is to be additionally partitioned. If the node is to be additionally partitioned, a second flag (e.g., mtt_split_cu_vertical_flag) can be signaled to indicate the splitting direction. Then, a third flag (e.g., mtt_split_cu_binary_flag) can be signaled to indicate whether the split type is binary or ternary. For example, based on the mtt_split_cu_vertical_flag and the mtt_split_cu_binary_flag, the multi-type tree splitting mode (MttSplitMode) of a CU can be derived as shown in Table 1 below.

[0071] [Table 1]

[0072] FIG. 5 is a diagram illustrating a method for dividing a CTU into multiple CUs based on a quadtree and nested multi-type tree structure as an embodiment to which the present invention can be applied.

[0073] Here, the bold block edges indicate quadtree partitioning, and the remaining edges indicate multi-type tree partitioning. Quadtree partitioning with multi-type trees can provide a content-adapted coding tree structure. A CU can correspond to a coding block (CB). Alternatively, a CU can include a coding block of luma samples and two coding blocks of corresponding chroma samples. The size of a CU can be as large as a CTU or as small as 4x4 in luma sample units. For example, in a 4:2:0 color format, the maximum chroma CB size can be 64x64 and the minimum chroma CB size can be 2x2.

[0074] In this document, for example, the maximum allowed luma TB size may be 64x64 and the maximum allowed chroma TB size may be 32x32. If the width or height of a CB split along the above tree structure is greater than the maximum transform width or height, the CB may be automatically (or implicitly) split until it meets the horizontal and vertical TB size restrictions.

[0075] Meanwhile, for a quadtree coding tree scheme with multiple types of trees, the following parameters can be defined and identified in the SPS syntax elements:

[0076] -CTU size: size of quaternary tree root node

[0077] -MinQTSize: The minimum allowed size of a quaternary tree leaf node.

[0078] -MaxBtSize: The maximum allowed size of a binary tree root node.

[0079] -MaxTtSize: The maximum allowed size of a ternary tree root node.

[0080] -MaxMttDepth: The maximum allowed depth of a multi-type tree split at a quadtree leaf.

[0081] -MinBtSize: The minimum allowed size of a binary tree leaf node.

[0082] -MinTtSize: The maximum allowed size of a ternary tree leaf node.

[0083] As an example of a quadtree coding tree structure with multiple types of trees, the CTU size may be set to 64x64 blocks of 128x128 luma samples and two corresponding chroma samples (in 4:2:0 chroma format). In this case, MinOTSize may be set to 16x16, MaxBtSize may be set to 128x128, MaxTtSzie may be set to 64x64, MinBtSize and MinTtSize (for both width and height) may be 4x4, and MaxMttDepth may be set to 4. Quadtree partitioning may be applied to the CTU to generate quadtree leaf nodes. The quadtree leaf nodes may be referred to as leaf QT nodes. The quadtree leaf nodes may have a size from 16x16 (i.e., the MinOTSize) to 128x128 (i.e., the CTU size). If the leaf QT node is 128x128, it may not be further split into a binary / ternary tree because in this case the split would exceed MaxBtsize and MaxTtszie (i.e. 64x64). Otherwise, the leaf QT node may be further split into a multitype tree. Thus, the leaf QT node is the root node of the multitype tree, and the leaf QT node may have a multitype tree depth (mttDepth) value of 0. When the multitype tree depth reaches MaxMttdepth (e.g. 4), no further splits may be considered. When the width of the multitype tree node is equal to MinBtSize and less than or equal to 2xMinTtSize, no further horizontal splits may be considered. When the height of the multitype tree node is equal to MinBtSize and less than or equal to 2xMinTtSize, no further vertical splits may be considered.

[0084] FIG. 6 is a diagram illustrating a method for restricting ternary-tree division as an embodiment to which the present invention can be applied.

[0085] Referring to Figure 6, in order to enable the design of a 64x64 luma block and a 32x32 chroma pipeline in a hardware decoder, the TT partitioning may be restricted in certain cases. For example, if the width or height of a luma coding block is greater than a preset specific value (e.g., 32, 64), the TT partitioning may be restricted as shown in Figure 6.

[0086] In this document, the coding tree scheme can support luma and chroma blocks to have separate block tree structures. For P and B slices, the luma and chroma CTBs in one CTU can be restricted to have the same coding tree structure. However, for I slices, the luma and chroma blocks can have separate block tree structures from each other. When the separate block tree mode is applied, the luma CTB can be divided into CUs based on a specific coding tree structure, and the chroma CTB can be divided into chroma CUs based on another coding tree structure. This can mean that a CU in an I slice can be composed of a coding block of a luma component or a coding block of two chroma components, and a CU in a P or B slice can be composed of blocks of three color components.

[0087] Although the quadtree coding tree structure with a multi-type tree has been described in the above "Partitioning of the CTUs using a tree structure", the structure in which the CU is divided is not limited thereto. For example, the BT structure and the TT structure can be interpreted as a concept included in the structure of a multiple partitioning tree (MPT), and the CU can be interpreted as being divided via the QT structure and the MPT structure. In an example in which the CU is divided via the QT structure and the MPT structure, a syntax element (e.g., MPT_split_type) including information on how many blocks the leaf node of the QT structure is divided into and a syntax element (e.g., MPT_split_mode) including information on whether the leaf node of the QT structure is divided vertically or horizontally are signaled, so that the division structure can be determined.

[0088] In yet another example, the CU may be divided in a manner other than the QT structure, the BT structure, or the TT structure. That is, unlike the manner in which the CU of the lower depth is divided into 1 / 4 size of the CU of the higher depth based on the QT structure, the CU of the lower depth is divided into 1 / 2 size of the CU of the higher depth based on the BT structure, and the CU of the lower depth is divided into 1 / 4 or 1 / 2 size of the CU of the higher depth based on the TT structure, the CU of the lower depth may be divided into 1 / 5, 1 / 3, 3 / 8, 3 / 5, 2 / 3, or 5 / 8 size of the CU of the parent depth, depending on the case, and the manner in which the CU is divided is not limited thereto.

[0089] If a portion of a tree node block exceeds the bottom or right picture boundary, the tree node block may be constrained such that all samples of all coded CUs are located within the picture boundary. In this case, for example, the following splitting rules may be applied:

[0090] If a portion of a tree node block extends beyond both the left and right picture boundaries,

[0091] - If the block is a QT node and the size of the block is larger than the minimum QT size, the block is forcibly split in QT split mode.

[0092] - Otherwise, the block is forcibly split in Horizontal Binary Splitting (SPLIT_BT_HOR) mode.

[0093] Otherwise, if a portion of the tree node block extends beyond the lower picture boundary,

[0094] - If the block is a QT node, and the size of the block is larger than the minimum QT size, and the size of the block is larger than the maximum BT size, the block is forcibly split in QT split mode.

[0095] - Otherwise, if the block is a QT node and the size of the block is greater than the minimum QT size and the size of the block is less than or equal to the maximum BT size, the block is forcibly split in QT split mode or Horizontal Binary Splitting (SPLIT_BT_HOR) mode.

[0096] - Otherwise (if the block is a BTT node or the size of the block is smaller than or equal to the minimum QT size), the block is force split in Horizontal Binary Splitting (SPLIT_BT_HOR) mode.

[0097] Otherwise, if a portion of the tree node block extends beyond all of the right picture border,

[0098] - If the block is a QT node, and the size of the block is larger than the minimum QT size, and the size of the block is larger than the maximum BT size, the block is forcibly split in QT split mode.

[0099] - Otherwise, if the block is a QT node and the size of the block is greater than the minimum QT size and the size of the block is less than or equal to the maximum BT size, the block is forcibly split in QT splitting mode or Vertical Binary Splitting (SPLIT_BT_VER) mode.

[0100] - Otherwise (the block is a BTT node or the size of the block is smaller than or equal to the minimum QT size), the block is force split in Vertical Binary Splitting (SPLIT_BT_VER) mode.

[0101] Meanwhile, the quadtree coding block structure with multiple types of trees described above can provide a very flexible block partitioning structure. Because of the partitioning types supported by multiple types of trees, other partitioning patterns may potentially result in the same coding block structure. By limiting the occurrence of such redundant partitioning patterns, the amount of data in the partitioning information can be reduced. This will be described with reference to the following figure.

[0102] FIG. 7 is a diagram illustrating redundant division patterns that can occur in binary tree division and ternary tree division as an embodiment to which the present invention can be applied.

[0103] As shown in FIG. 7, two levels of consecutive binary splits in one direction have the same coding block structure as the bisection of the central partition after trisection. In such a case, the binary tree split of the central partition of the trisection in the given direction may be restricted. Such a restriction may be applied to the CU of all pictures. When such a specific split is restricted, the signaling of the corresponding syntax element may be modified to reflect such a restricted case, thereby reducing the number of bits signaled for partitioning. For example, as in the example shown in FIG. 7, when the binary tree split of the central partition of the CU is restricted, the mtt_split_cu_binary_flag syntax element indicating whether the split is bisection or trisection is not signaled, and its value may be inferred as 0 by the decoder.

[0104] Prediction

[0105] To reconstruct the current processing unit for which decoding is performed, decoded portions of the current picture or other pictures in which the current processing unit is included can be used.

[0106] A picture (slice) that uses only the current picture for reconstruction, i.e., performs only intra-screen prediction, can be called an intra-picture or I-picture (slice); a picture (slice) that uses up to one motion vector and reference index to predict each unit can be called a predictive picture or P-picture (slice); and a picture (slice) that uses up to two motion vectors and reference indexes can be called a bi-directional (pair) predictive picture or B-picture (slice).

[0107] Intra prediction refers to a prediction method that derives a current processing block from data elements (e.g., sample values, etc.) of the same decoded picture (or slice), i.e., a method of predicting pixel values ​​of a current processing block by referring to a reconstructed region in the current picture.

[0108] Below we look more closely at inter prediction.

[0109] Inter prediction (or inter prediction)

[0110] Inter prediction refers to a prediction method for deriving a current processing block based on data elements (e.g., sample values ​​or motion vectors) of pictures other than the current picture, i.e., a method for predicting pixel values ​​of a current processing block by referring to a reconstructed area in another reconstructed picture other than the current picture.

[0111] Inter-prediction (or inter-picture prediction) is a technique that removes redundancy between pictures, and is mostly done through motion estimation and motion compensation.

[0112] The present invention describes detailed techniques of the inter prediction method previously described in Figures 1 and 2, and in the case of a decoder, it can be represented by a method for decoding an inter prediction based video / image in Figure 10 and an inter prediction unit in a decoding device in Figure 11, which will be described later. In addition, in the case of an encoder, it can be represented by a method for encoding an inter prediction based video / image in Figure 8 and an inter prediction unit in an encoding device in Figure 9, which will be described later. Furthermore, data encoded according to Figures 8 and 9 can be stored in the form of a bitstream.

[0113] The prediction unit of the encoding device / decoding device can perform inter prediction on a block-by-block basis to derive a prediction sample. Inter prediction can refer to a prediction derived in a manner that depends on data elements (e.g., sample values, motion information, etc.) of a picture other than the current picture. When inter prediction is applied to the current block, a predicted block (prediction sample array) of the current block can be derived based on a reference block (reference sample array) identified by a motion vector on a reference picture indicated by an index of the reference picture.

[0114] In this case, in order to reduce the amount of motion information transferred in the inter prediction mode, the motion information of the current block may be predicted in units of blocks, sub-blocks, or samples based on the correlation of the motion information between the neighboring blocks and the current block. The motion information may include a motion vector and a reference picture index. The motion information may further include information on an inter prediction type (such as L0 prediction, L1 prediction, or Bi prediction).

[0115] When inter prediction is applied, the neighboring blocks may include spatial neighboring blocks existing in the current picture and temporal neighboring blocks existing in the reference picture. The reference picture including the reference block and the reference picture including the temporal neighboring block may be the same or different. The temporal neighboring block may be called a collocated reference block, a collocated CU (colCU), etc., and the reference picture including the temporal neighboring block may be called a collocated picture (colPic). For example, a motion information candidate list may be constructed based on the neighboring blocks of the current block, and a flag or index information indicating which candidate is selected (used) to derive the motion vector and / or the index of the reference picture of the current block may be signaled.

[0116] Inter prediction may be performed based on various prediction modes, for example, in the case of skip mode and merge mode, the motion information of the current block may be the same as that of the selected neighboring block. In the case of skip mode, unlike the merge mode, a residual signal may not be transmitted. In the case of a Motion Vector Prediction (MVP) mode, a motion vector difference may be signaled using the motion vector of the selected neighboring block as a motion vector predictor. In this case, the motion vector of the current block may be derived using the sum of the motion vector predictor and the motion vector difference.

[0117] 8 and 9 are diagrams illustrating an inter-prediction based video / image encoding method according to an embodiment of the present invention and an inter-prediction unit in an encoding device according to an embodiment of the present invention.

[0118] 8 and 9, S801 may be performed by the inter prediction unit 180 of the encoding apparatus, and S802 may be performed by the residual processing unit of the encoding apparatus. Specifically, S802 may be performed by the subtraction unit 115 of the encoding apparatus. In S803, prediction information may be derived by the inter prediction unit 180 and encoded by the entropy encoding unit 190. In S803, residual information may be derived by the residual processing unit and encoded by the entropy encoding unit 190. The residual information is information on the residual sample. The residual information may include information on quantized transform coefficients for the residual sample.

[0119] As described above, the residual samples may be derived as transform coefficients through the transform unit 120 of the encoding device, and the transform coefficients may be derived as quantized transform coefficients through the quantization unit 130. Information about the quantized transform coefficients may be encoded in the entropy encoding unit 190 through a residual coding procedure.

[0120] The encoding device performs inter prediction of a current block (S801). The encoding device can derive an inter prediction mode and motion information of the current block and generate a prediction sample of the current block. Here, the steps of determining an inter prediction mode, deriving motion information, and generating a prediction sample can be performed simultaneously, or any one of the steps can be performed prior to the other steps. For example, the inter prediction unit 180 of the encoding device can include a prediction mode determination unit 181, a motion information derivation unit 182, and a prediction sample derivation unit 183, and the prediction mode determination unit 181 can determine a prediction mode of the current block, the motion information derivation unit 182 can derive motion information of the current block, and the prediction sample derivation unit 183 can derive a motion sample of the current block.

[0121] For example, the inter prediction unit 180 of the encoding device may search for a block similar to the current block within a certain region (search area) of a reference picture through motion estimation, and derive a reference block whose difference from the current block is minimum or equal to or less than a certain criterion. Based on this, an index of a reference picture indicating a reference picture in which the reference block is located may be derived, and a motion vector may be derived based on a difference in position between the reference block and the current block. The encoding device may determine a mode to be applied to the current block among various prediction modes. The encoding device may compare RD costs for the various prediction modes, and determine an optimal prediction mode for the current block.

[0122] For example, when a skip mode or a merge mode is applied to the current block, the encoding apparatus may construct a merge candidate list (to be described later) and derive a reference block whose difference from the current block is minimum or equal to a certain criterion among reference blocks indicated by merge candidates included in the merge candidate list. In this case, a merge candidate associated with the derived reference block may be selected, and merge index information indicating the selected merge candidate may be generated and signaled to a decoding apparatus. Motion information of the current block may be derived using motion information of the selected merge candidate.

[0123] As another example, when the (A)MVP mode is applied to the current block, the encoding apparatus may construct an (A)MVP candidate list described later, and may use a motion vector of a selected MVP candidate among the MVP (motion vector predictor) candidates included in the (A)MVP candidate list as the MVP of the current block. In this case, for example, a motion vector indicating a reference block derived by the above-mentioned motion estimation may be used as the motion vector of the current block, and an MVP candidate having a motion vector with the smallest difference from the motion vector of the current block among the MVP candidates may become the selected MVP candidate. A motion vector difference (MVD), which is a difference obtained by subtracting the MVP from the motion vector of the current block, may be derived. In this case, information regarding the MVD may be signaled to the decoding apparatus. In addition, when the (A)MVP mode is applied, the value of the reference picture index may be configured as reference picture index information and may be separately signaled to the decoding apparatus.

[0124] The encoding apparatus may derive residual samples based on the predicted samples (S802). The encoding apparatus may derive the residual samples through a comparison between original samples of the current block and the predicted samples.

[0125] The encoding apparatus encodes video information including prediction information and residual information (S803). The encoding apparatus may output the encoded video information in the form of a bitstream. The prediction information is information related to the prediction procedure and may include prediction mode information (e.g., skip flag, merge flag or mode index, etc.) and information on motion information. The information on the motion information may include candidate selection information (e.g., merge index, mvp flag or mvp index), which is information for deriving a motion vector. In addition, the information on the motion information may include the above-mentioned information on MVD and / or index information of a reference picture.

[0126] The information on the motion information may include information indicating whether L0 prediction, L1 prediction, or bi-directional (bi) prediction is applied. The residual information is information on the residual sample. The residual information may include information on quantized transform coefficients for the residual sample.

[0127] The output bitstream can be stored on a (digital) storage medium and transmitted to the decoding device, or can be transmitted to the decoding device via a network.

[0128] Meanwhile, as described above, the encoding apparatus can generate a reconstructed picture (including reconstructed samples and reconstructed blocks) based on the reference samples and the residual samples. This is because the encoding apparatus derives the same prediction result as that performed in the decoding apparatus, thereby improving coding efficiency. Therefore, the encoding apparatus can store the reconstructed picture (or reconstructed samples, reconstructed blocks) in a memory and use it as a reference picture for inter prediction. As described above, an in-loop filtering procedure, etc. can be further applied to the reconstructed picture.

[0129] 10 and 11 are diagrams illustrating a method for decoding inter-prediction based video / picture according to an embodiment of the present invention, and an inter-prediction unit in a decoding device according to an embodiment of the present invention.

[0130] 10 and 11, the decoding device may perform operations corresponding to those performed by the encoding device. The decoding device may perform prediction on the current block based on the received prediction information and derive a prediction sample.

[0131] S1001 to S1003 may be performed by the inter prediction unit 260 of the decoding device, and the residual information of S1004 may be acquired from a bitstream by the entropy decoding unit 210 of the decoding device. The residual processing unit of the decoding device may derive a residual sample of the current block based on the residual information. Specifically, the inverse quantization unit 220 of the residual processing unit may perform inverse quantization to derive a transform coefficient based on the quantized transform coefficient derived based on the residual information, and the inverse transform unit 230 of the residual processing unit may perform inverse transform of the transform coefficient to derive a residual sample of the current block. S1005 may be performed by the addition unit 235 or a reconstruction unit of the decoding device.

[0132] Specifically, the decoding device may determine a prediction mode of the current block based on received prediction information (S1001). The decoding device may determine which inter prediction mode is applied to the current block based on prediction mode information in the prediction information.

[0133] For example, it may be determined whether the merge mode is applied to the current block or whether the (A)MVP mode is determined based on the merge flag. Alternatively, it may be possible to select one of various inter prediction mode candidates based on the mode index. The inter prediction mode candidates may include a skip mode, a merge mode, and / or an (A)MVP mode, or may include various inter prediction modes described below.

[0134] The decoding apparatus derives motion information of the current block based on the determined inter prediction mode (S1002). For example, when a skip mode or a merge mode is applied to the current block, the decoding apparatus may form a merge candidate list (described later) and select one merge candidate from among the merge candidates included in the merge candidate list. The selection may be performed based on the above-mentioned selection information (merge index). Motion information of the selected merge candidate may be used to derive motion information of the current block. The motion information of the selected merge candidate may be used as motion information of the current block.

[0135] As another example, when the (A)MVP mode is applied to the current block, the decoding apparatus may construct an (A)MVP candidate list described later, and may use a motion vector of a selected MVP candidate among the MVP (motion vector predictor) candidates included in the (A)MVP candidate list as the MVP of the current block. The selection may be performed based on the above-mentioned selection information (mvp flag or mvp index). In this case, the MVD of the current block may be derived based on information on the MVD, and the motion vector of the current block may be derived based on the mvp of the current block and the MVD. Also, the reference picture index of the current block may be derived based on index information of the reference picture. In the reference picture list for the current block, a picture pointed to by the index of the reference picture may be derived as a reference picture referenced for inter prediction of the current block.

[0136] On the other hand, as described below, the motion information of the current block may be derived without constructing a candidate list, in which case the motion information of the current block may be derived according to a procedure started in a prediction mode, as described below. In this case, the construction of the candidate list as described above may be omitted.

[0137] The decoding device may generate a prediction sample of the current block based on the motion information of the current block (S1003). In this case, the reference picture may be derived based on the reference picture index of the current block, and the prediction sample of the current block may be derived using a sample of the reference block to which the motion vector of the current block points on the reference picture. In this case, as described below, a filtering procedure may be performed on all or a part of the prediction samples of the current block, depending on the case.

[0138] For example, the inter prediction unit 260 of the decoding device may include a prediction mode determination unit 261, a motion information derivation unit 262, and a prediction sample derivation unit 263, and may determine a prediction mode of the current block based on prediction mode information received by the prediction mode determination unit 261, derive motion information (such as a motion vector and / or a reference picture index) of the current block based on information regarding the motion information received from the motion information derivation unit 262, and derive a prediction sample of the current block in the prediction sample derivation unit 263.

[0139] The decoding apparatus generates a residual sample of the current block based on the received residual information (S1004). The decoding apparatus generates a reconstructed sample of the current block based on the prediction sample and the residual sample, and can generate a reconstructed picture based on the reconstructed sample (S1005). Thereafter, an in-loop filter procedure, etc. is further applied to the reconstructed picture, as described above.

[0140] As described above, the inter prediction procedure may include a step of determining an inter prediction mode, a step of deriving motion information according to the determined prediction mode, and a step of performing prediction (generating a prediction sample) based on the derived motion information.

[0141] Determination of inter prediction mode

[0142] Various inter prediction modes are used for prediction of a current block in a picture. For example, various modes such as merge mode, skip mode, MVP mode, and affine mode are used. In the Decoder side Motion Vector Refinement (DMVR) mode, the Adaptive Motion Vector Resolution (AMVR) mode can be further used as an auxiliary mode. The affine mode is also called the affine motion prediction mode. The MVP mode is also called the Advanced Motion Vector Prediction (AMVP) mode.

[0143] Prediction mode information indicating an inter prediction mode of a current block may be signaled from an encoding device to a decoding device. The prediction mode information may be included in a bitstream and received by the decoding device. The prediction mode information may include index information indicating any one of a number of candidate modes. Alternatively, the inter prediction mode may be indicated through hierarchical signaling of flag information. In this case, the prediction mode information may include a number of flags.

[0144] For example, a skip flag may be signaled to indicate whether or not a skip mode can be applied, and if the skip mode cannot be applied, a merge flag may be signaled to indicate whether or not a merge mode can be applied, and if the merge mode cannot be applied, an MVP mode may be applied, or a flag for additional division may be further signaled. The affine mode may be signaled as an independent mode, or may be signaled as a mode subordinate to the merge mode or MVP mode. For example, in the affine mode, it may be configured as one candidate in the merge candidate list or MVP candidate list, as described below.

[0145] Derivation of motion information according to inter prediction mode

[0146] Inter prediction can be performed using the motion information of the current block. The encoding device can derive optimal motion information of the current block through a motion estimation procedure. For example, the encoding device can search for a similar reference block with high correlation in a fractional pixel unit within a determined search range in the reference picture using an original block in the original picture of the current block, thereby deriving the motion information. The similarity of the block can be derived based on a phase-based sample value difference. For example, the similarity of the block can be calculated based on the SAD between the current block (or the template of the current block) and the reference block (or the template of the reference block). In this case, the motion information can be derived based on the reference block with the smallest SAD in the search space. The derived motion information can be signaled to the decoding device based on various methods based on the inter prediction mode.

[0147] Merge and Skip Modes

[0148] FIG. 12 is a diagram for explaining peripheral blocks used in the merge mode or the skip mode as an embodiment to which the present invention is applied.

[0149] When a merge mode is applied, the motion information of the current prediction block is not directly transmitted, but is derived using the motion information of the surrounding prediction blocks. Therefore, the motion information of the current prediction block can be indicated by transmitting flag information indicating that the merge mode is used and a merge index indicating which surrounding prediction block is used.

[0150] To perform the merge mode, the encoder may search for merge candidate blocks used to derive motion information of the current prediction block. For example, up to five merge candidate blocks may be used, but the present invention is not limited thereto. The maximum number of merge candidate blocks may be transmitted in a slice header (or a tile group header), but the present invention is not limited thereto. After finding the merge candidate blocks, the encoder may generate a merge candidate list, and may select a merge candidate block having the smallest cost among them as a final merge candidate block.

[0151] The present invention provides various embodiments of the merging candidate blocks that make up the merging candidate list.

[0152] The merge candidate list may use, for example, five merge candidate blocks. For example, four spatial merge candidates and one temporal merge candidate may be used. As a specific example, in the case of spatial merge candidates, the blocks shown in FIG. 12 may be used as spatial merge candidates.

[0153] FIG. 13 is a flowchart illustrating a method for constructing a merge candidate list according to an embodiment of the present invention.

[0154] Referring to FIG. 13, a coding apparatus (encoder / decoder) searches spatial neighboring blocks of a current block and inserts derived spatial merge candidates into a merge candidate list (S1301). For example, the spatial neighboring blocks may include a lower left corner neighboring block, a left peripheral block, a right upper corner neighboring block, an upper peripheral block, and an upper left corner neighboring block of the current block. However, this is for example only, and additional neighboring blocks such as a right peripheral block, a lower peripheral block, and a lower right peripheral block may be further used as the spatial neighboring blocks in addition to the above-mentioned spatial neighboring blocks. The coding apparatus may search the spatial neighboring blocks based on a priority order to detect usable blocks, and derive motion information of the detected blocks as the spatial merge candidates. For example, the encoder and decoder may search the five blocks shown in FIG. 12 in the order of A1, B1, B0, A0, and B2, and sequentially index usable candidates to configure a merge candidate list.

[0155] The coding apparatus searches for temporal neighboring blocks of the current block and inserts derived temporal merging candidates into the merging candidate list (S1302). The temporal neighboring blocks may be located on a reference picture that is a different picture from the current picture in which the current block is located. The reference picture in which the temporal neighboring blocks are located may be called a collocated picture or a col picture. The temporal neighboring blocks may be searched for in the order of a lower right corner neighboring block and a lower right center block of a co-located block of the current block on the col picture.

[0156] Meanwhile, when motion data compression is applied, specific motion information can be stored as representative motion information for each fixed storage unit in the col picture. In this case, it is not necessary to store motion information of all blocks in the fixed storage unit, and thus a motion data compression effect can be obtained. In this case, the fixed storage unit may be predetermined, for example, in units of 16x16 samples or 8x8 samples, or size information of the fixed (scheduled) storage unit may be signaled from the encoder to the decoder. When the motion data compression is applied, the motion information of the temporally neighboring blocks can be replaced with the representative motion information of the fixed storage unit in which the temporally neighboring blocks are located.

[0157] That is, in this case, from the perspective of implementation, the temporal merge candidate may be derived based on motion information of a prediction block that is arithmetically shifted right by a certain value based on the coordinates (upper left sample position) of the temporal neighboring block, rather than the prediction block located at the coordinates of the temporal neighboring block, and then arithmetically shifted left. For example, if the certain storage unit is a 2nx2n sample unit, and the coordinates of the temporal neighboring block are (xTnb, yTnb), the motion information of a prediction block located at a modified position ((xTnb >> n) << n), (yTnb >> n) << n)) may be used for the temporal merge candidate.

[0158] Specifically, for example, when the certain storage unit is a 16x16 sample unit, and the coordinates of the temporal neighboring block are (xTnb, yTnb), the motion information of the prediction block located at the modified position ((xTnb >> 4) << 4), (yTnb >> 4) << 4) may be used for the temporal merge candidate. Alternatively, for example, when the certain storage unit is an 8x8 sample unit, and the coordinates of the temporal neighboring block are (xTnb, yTnb), the motion information of the prediction block located at the modified position ((xTnb >> 3) << 3), (yTnb >> 3) << 3) may be used for the temporal merge candidate.

[0159] The coding apparatus may check whether the number of current merging candidates is smaller than the number of maximum merging candidates (S1303). The number of maximum merging candidates may be predefined or signaled from the encoder to the decoder. For example, the encoder may generate information regarding the number of maximum merging candidates, encode the information, and transmit the information to the decoder in the form of a bitstream. When the number of maximum merging candidates is full, the process of adding subsequent candidates may not proceed.

[0160] If the check results in the number of the current merge candidates being less than the maximum number of merge candidates, the code device inserts an additional merge candidate into the merge candidate list (S1304). The additional merge candidate may include, for example, an ATMVP, a combined bi-predictive merge candidate (if the slice type of the current slice is type B), and / or a zero vector merge candidate.

[0161] If the checking result indicates that the number of current merge candidates is not less than the maximum number of merge candidates, the code device may terminate construction of the merge candidate list. In this case, the encoder may select an optimal merge candidate from among the merge candidates constituting the merge candidate list based on a rate-distortion (RD) cost, and may signal selection information (e.g., a merge index) indicating the selected merge candidate to a decoder. The decoder may select the optimal merge candidate based on the merge candidate list and the selection information.

[0162] As described above, the motion information of the selected merging candidate may be used as the motion information of the current block, and a prediction sample of the current block may be derived based on the motion information of the current block. An encoder may derive a residual sample of the current block based on the prediction sample, and may signal residual information regarding the residual sample to a decoder. As described above, a decoder may generate a reconstructed sample based on the residual sample derived based on the residual information and the prediction sample, and generate a reconstructed picture based on the reconstructed sample.

[0163] When a skip mode is applied, the motion information of the current block can be derived in the same manner as when the previous merge mode is applied, except that when the skip mode is applied, the residual signal of the block is omitted, and thus the prediction samples can be directly used as reconstructed samples.

[0164] MVP Mode

[0165] FIG. 14 is a flowchart illustrating a method for constructing a merge candidate list according to an embodiment of the present invention.

[0166] When the MVP (Motion Vector Prediction) mode is applied, a candidate list of motion vector predictors (mvp) can be generated using the motion vectors of the restored spatial neighboring blocks (which may be the neighboring blocks described in FIG. 12 above) and / or the motion vectors corresponding to the temporal neighboring blocks (or Col blocks). That is, the motion vectors of the restored spatial neighboring blocks and / or the motion vectors corresponding to the temporal neighboring blocks can be used as motion vector predictor candidates.

[0167] The prediction information may include selection information (e.g., MVP flag or MVP index) indicating a selected optimal motion vector predictor candidate among the motion vector predictor candidates included in the list. In this case, the prediction unit may select a motion vector predictor of the current block among the motion vector predictor candidates included in the motion vector candidate list using the selection information. The prediction unit of the encoding device may obtain a motion vector difference (MVD) between the motion vector of the current block and the motion vector predictor, encode the motion vector difference, and output the motion vector predictor in the form of a bitstream. That is, the MVD may be obtained by subtracting the motion vector predictor from the motion vector of the current block. In this case, the prediction unit of the decoding device may obtain a motion vector difference included in the prediction information, and derive the motion vector of the current block through addition of the motion vector difference and the motion vector predictor. The prediction unit of the decoding device may obtain or derive an index of a reference picture indicating a reference picture from the prediction information. For example, the motion vector predictor candidate list may be configured as shown in FIG. 14.

[0168] ATMVP(Advanced Temporal Motion Vector Prediction)

[0169] 15 and 16 are diagrams for explaining a method of deriving ATMVP (Advanced Temporal Motion Vector Prediction) candidates as an embodiment to which the present invention is applied.

[0170] Referring to Fig. 15, ATMVP is a method for deriving motion information of sub-blocks of a coding unit based on motion information of collocated blocks of temporally adjacent pictures. Through this, it is possible to improve the performance of temporal motion vector prediction (TMVP) and reduce general or worst case complexity. In the present invention, ATMVP is also referred to as subblock-based temporal merging candidate (SbTMVP).

[0171] In one embodiment of the present invention, the ATMVP may be derived by the following process.

[0172] First, the encoder / decoder can add motion vectors from spatially neighboring coding units if the neighboring coding units are available and the motion vector of the coding unit used is different from the motion vector in the current candidate list. As an example, referring to FIG. 16, the above process can be performed according to the order of A1, B1, B0, A0, B2. As another example, for complexity improvement, the above process can also derive ATMVP using only the motion vector of a block at a fixed position (e.g., the position of A1).

[0173] The encoder / decoder can use the first motion vector candidate among the No space candidates used to determine the position to derive the collocated picture and the motion information of each subblock. Here, No indicates the number of space candidates used. If No is 0, the collocated picture and the collocated position of motion 0 can be used to derive the motion information of each subblock.

[0174] When multiple reference pictures are used, the collocated pictures of different coding units in ATMVP may not be the same. For different coding units in the current picture, having different collocated pictures for the derivation of ATMVP means that motion information fields of multiple reference pictures should be derived, which is undesirable for increasing memory bandwidth.

[0175] Therefore, the present invention provides a further simplified design of using the same collocated picture when deriving ATMVP. For example, the method of using the same collocated picture may be defined in the header of a slice (or a group of tiles), but the present invention is not limited thereto. As an example, at the block level, if the reference picture of a neighboring block A is different from the collocated picture, the motion vector of the neighboring block A can be scaled based on the scaling method of the temporal motion vector. Then, the scaled motion vector of the neighboring block A can be used in ATMVP.

[0176] FIG. 17 is a diagram illustrating a method for deriving an ATMVP (Advanced Temporal Motion Vector Prediction) candidate as an embodiment to which the present invention is applied.

[0177] Referring to Figure 17, in one embodiment of the present invention, TMVP using the motion vector of the block at the right-bottom of the current block or the temporally adjacent block (or colPB) located at the center of the current block does not reflect motion within the screen, so the encoder / decoder can use the motion vector of the colPB at the position pointed to by the motion vector of the surrounding block as the MVP.

[0178] For example, the encoder / decoder can find the motion vector of the spatial neighbor block used first while checking with the construction procedure of the merge candidates shown in Figure 17. Then, the position pointed to by the motion vector in the reference picture can be derived as col-PB (i.e., ATMVP candidate).

[0179] In addition, the motion vector is used as the motion vector of a corresponding block in each sub-block unit. In this case, if a motion vector does not exist for a specific sub-block, the motion vector of a center block located at the center of the corresponding block is used as the motion vector for the unavailable sub-block and can be stored as a representative motion vector.

[0180] Temporal motion vector data storage reduction

[0181] In one embodiment of the present invention, for temporal motion vector data compression, a method for reducing temporal motion vector storage based on spatial candidate motion vector data is proposed.

[0182] 18 and 19 are diagrams illustrating a method for compressing temporal motion vector data and the locations of spatial candidates used therein, according to one embodiment of the present invention.

[0183] Referring to Figure 18, in an embodiment of the present invention, when a spatial candidate is predicted by inter prediction, the motion vector of the spatial candidate can be set as a default (base) motion vector for compression. For example, up to five spatial candidates can be used as reference temporal motion information for deriving a default temporal motion vector. In one embodiment, the above five spatial candidates can be set as shown in Figure 19.

[0184] Also, the temporal motion vector data may be compressed based on the motion vectors of the spatial candidates. The order of searching for the spatial candidates may be the same as that shown in Fig. 18. The spatial candidates may be checked in the order of center block (C), upper left block (TL), upper right block (TR), lower left block (BL), and lower right block (BR). This is only one embodiment, and the present invention is not limited thereto, and other combinable orders may be applied.

[0185] First, the encoder / decoder can determine that the center block (C) is inter-predicted. If the center block (C) is inter-predicted, the encoder / decoder can set the motion vector of the center block (C) to a default (default: base value) for motion vector prediction.

[0186] If the center block (C) is not inter-predicted, the encoder / decoder may determine that the top left block (TL) is inter-predicted. If the top left block (TL) is inter-predicted, the encoder / decoder may set the motion vector of the top left block (TL) as a default for motion vector prediction.

[0187] If the top left block (TL) is not inter predicted, the encoder / decoder may determine that the top right block (TR) is inter predicted. If the top right block (TR) is inter predicted, the encoder / decoder may set the motion vector of the top right block (TR) to a default for motion vector prediction.

[0188] If the top right block (TR) is not inter predicted, the encoder / decoder may determine that the bottom left block (BL) is inter predicted. If the bottom left block (BL) is inter predicted, the encoder / decoder may set the motion vector of the bottom left block (BL) to a default for motion vector prediction.

[0189] If the lower left block (BL) is not inter predicted, the encoder / decoder may determine that the lower right block (BR) is inter predicted. If the lower right block (BR) is inter predicted, the encoder / decoder may set the motion vector of the lower right block (BR) to a default (default: base value) for motion vector prediction.

[0190] If the bottom right block (BR) was not inter predicted, the encoder / decoder may set the intra mode to a default value (default).

[0191] Through the above process, the encoder / decoder can compress a default motion vector into motion information.

[0192] Embodiments implementing ATMVP based on adaptive sub-block size

[0193] In one embodiment of the present invention, we propose a method to perform ATMVP based on adaptive sub-block size, for example, the sub-block size used for ATMVP derivation can be adaptively applied at slice level.

[0194] Meanwhile, when ATMVP motion information is derived in units of 4x4 blocks, a problem may occur in that motion derivation and motion compensation are performed in units of 4x4 sub-blocks within one ATMVP coding unit.

[0195] To solve this, the encoder can signal to the decoder at the sequence level one default sub-block size to be used for ATMVP motion derivation.

[0196] As another example, if a default sub-block size is used in the current slice, a flag can be signaled at the picture or slice level, and if the flag is false, the ATMVP sub-block size can be further signaled from the slice header.

[0197] 1. An embodiment of limiting an area for derivation of a collocated block

[0198] In the present invention, the area of ​​the collocated block for ATMVP may include the current CTU and one column of NxN blocks in the collocated picture. For example, the NxN blocks may be 4x4 blocks, but the present invention is not limited thereto.

[0199] If the ATMVP collocated block identified by the motion vector of the merge candidate is located outside the constrained region, it may be moved to be located within the constrained region, for example, to be located at the nearest boundary within the constrained region.

[0200] 2. Embodiment for Derivation of Subblock-based Temporal Merging Candidates

[0201] In one embodiment of the present invention, the encoder / decoder can add motion information of a collocated block (or collocated subblock) in a collocated picture, which is identified based on motion information of spatially adjacent blocks, to a subblock merging candidate list as a subblock-based temporal merging candidate.

[0202] In the present invention, the motion information of spatially adjacent blocks can be called a temporal motion vector. In an embodiment, the encoder / decoder can derive sub-block-based temporal merging candidates when the width and height of the current coding block are equal to or greater than a predetermined specific size. For example, the predetermined specific size can be 8.

[0203] As an embodiment, the encoder / decoder may set the motion information of the first spatial candidate among the spatial candidates used as the temporal motion vector. As an example, the encoder / decoder may search for available spatial candidates in the order of A1, B1, B0, and A0. In this case, the encoder / decoder may set the spatial candidate whose reference picture is the same as the collocated picture from among the available spatial candidates as the temporal motion vector. As another example, the encoder / decoder may check whether a spatial candidate at a fixed position is available, and if available, may set the motion vector of the candidate in that area as the temporal motion vector. For example, the above-mentioned one spatial candidate at a fixed position may be set as the block at the position of A1.

[0204] In addition, the encoder / decoder can specify the position of the collocated block in the collocated picture using the temporal motion vector. As an example, the following Equation 1 can be used.

[0205] <Formula 1>

number

[0206] Here, (xColCtrCb, yColCtrCb) indicates the top-left sample position of the collocated coding block that contains the sample on the lower right side of the center position, and tempMv indicates the above-mentioned temporal motion vector.

[0207] Also, the encoder / decoder may determine the position for deriving the motion information of each subblock in the current coding block in subblock units. In one embodiment, the position of the collocated subblock in the collocated picture may be derived using the following Equation 2.

[0208] <Formula 2>

number

[0209] Here, (xSb, ySb) denotes the position of the current sub-block.

[0210] In one embodiment, the encoder / decoder can use motion information of the collocated block identified using the temporal motion vector if the current collocated subblock is not available.

[0211] FIG. 20 is a flowchart illustrating a method for generating an inter predicted block based on a candidate list according to an embodiment of the present invention.

[0212] Referring to FIG. 20, an inter prediction process performed by the inter prediction unit (180, 260) described above in FIG. 1 and FIG. 2 is illustrated. In FIG. 20, it is assumed that a merge mode and an inter mode (i.e., (A)MVP mode) are used as a method of deriving predicted motion information (PMI) through inter prediction and performing prediction. However, the present invention is not limited thereto, and various other modes can be applied for inter prediction.

[0213] As mentioned above, merge mode and inter mode can be classified according to whether or not at least one of a motion vector difference (MVD), a reference list (or reference picture list), and / or a reference picture index is transmitted via a bitstream in addition to a candidate index to derive a final PMI used to generate a predictive block.

[0214] Specifically, the encoder / decoder checks whether a merge mode is applied to the current block (S2001).

[0215] When a merge mode is applied to the current block, the encoder / decoder generates a merge candidate (or a merge candidate list) and selects a PMI used for inter prediction of the current block (S2002, 2003). When a merge mode is applied, the encoder can transmit the selected PMI information to the decoder as a merge index. The merge index can indicate a specific merge candidate from among the merge candidates. The encoder / decoder can use the motion information of the specific merge candidate as a motion vector for inter prediction of the current block.

[0216] If the merge mode is not applied to the current block (i.e., if the inter mode is applied), the encoder / decoder generates an inter candidate (or an inter candidate list) and selects an initial PMI (or a motion vector predictor) used for inter prediction of the current block (S2004, S2005). If the inter mode is applied, the encoder can transfer the selected PMI information to the decoder as a motion vector predictor flag (or index). The motion vector predictor flag can indicate a specific inter candidate from among the inter candidates. The inter candidate can then be called a motion vector predictor candidate. The encoder / decoder derives a motion vector used for prediction by adding a motion vector differential to the initial PMI (S2006).

[0217] The encoder / decoder uses the derived motion vector to generate a prediction block for the current block (S2007).

[0218] That is, when the merge mode is applied, the encoder / decoder does not use additional syntax (or syntax elements) other than the signaling of the candidate index (i.e., merge index) to derive the final PMI, and generates a prediction block of the current block using the motion information of the selected candidate via the candidate index. At this time, the candidate generation process is performed in step (or block) S2002.

[0219] When the inter mode is applied, the candidate generation process is performed in the step S2004 in the same manner as in the merge mode. When the inter mode is applied, the encoder / decoder receives additional motion information based on the motion information of the selected candidate, derives a final PMI, and uses the final PMI to generate a prediction block for the current block.

[0220] In the following, we explain how to generate candidates (or candidate lists) using motion information of neighboring blocks.

[0221] FIG. 21 is a flowchart showing a method for generating a candidate list according to an embodiment of the present invention.

[0222] For convenience of explanation, the decoder will be mainly described with reference to Fig. 21, but the method of generating a candidate list according to the present embodiment can be performed in the encoder as well. In one embodiment, the methods described with reference to Figs. 12 to 17 can be applied, and the duplicated explanations will be omitted.

[0223] The candidate configuration procedure shown in Fig. 21 is merely an example, and the present invention is not limited thereto, and the order may be changed, some steps may be omitted, or some steps may be added. In addition, the candidates added in Fig. 21 are merely an example, and various other candidates may be added, and some of the candidates shown in Fig. 21 may not be used.

[0224] The decoder adds a spatial neighboring block (or a spatial candidate) to a candidate list (S2101). For example, the decoder can add the neighboring candidate block of FIG. 16 described above to the candidate list.

[0225] The decoder adds temporal neighboring blocks (or temporal candidates) to a candidate list (S2102).

[0226] The decoder adds the combined motion information (or the combined motion vector) to a candidate list and adds the zero motion vector to the candidate list (S2103, S2104).

[0227] FIG. 22 shows an example of neighboring blocks used in inter prediction according to an embodiment to which the present invention is applied.

[0228] Referring to Figure 22, the encoder / decoder can use the motion information of the spatial neighboring blocks based on spatial similarity and the motion information of the temporal neighboring blocks based on temporal similarity to construct a candidate list. In the present invention, the motion information of the spatial neighboring blocks can be called spatial neighboring information, and the motion information of the temporal neighboring blocks can be called temporal neighboring information or corresponding temporal information.

[0229] In an embodiment, the encoder / decoder may construct a candidate list using spatial candidates (or spatial neighboring information) as shown in Fig. 22. And the encoder / decoder may construct a candidate list using temporal candidates (or corresponding temporal information) as shown in Fig. 22.

[0230] As described above, an adaptive temporal motion vector predictor (ATMVP) can be used to correct the temporal similarity information considering the spatial similarity. ATMVP is a method for deriving motion information of a sub-block based on motion information of a collocated block of a temporally adjacent picture. This can improve the performance and reduce the complexity of temporal motion vector prediction (TMVP). In the present invention, ATMVP can be referred to as a subblock-based temporal merging candidate, SbTMVP, etc.

[0231] In other words, TMVP, which uses the motion vector of a collocated block located to the right-bottom or center of the current block, does not reflect motion within the picture, so the motion vector of the collocated block pointed to by the motion vector of the surrounding block is used in the motion vector predictor.

[0232] Referring again to FIG. 17 described above, in one embodiment, the encoder / decoder may find the motion vector of the first available spatial neighbor block while checking the pre-defined merge candidates in a configuration order. Thereafter, the encoder / decoder may determine the position indicated by the corresponding motion vector (i.e., temporal motion vector) in a reference picture as a collocated block. This may increase the accuracy of the corresponding block (or temporal motion vector). In one embodiment, the reference picture may be defined as a collocated picture.

[0233] The encoder / decoder may derive a motion vector of a corresponding block identified by a temporal motion vector in a sub-block unit in the current coding block. In one embodiment, if a motion vector corresponding to a particular sub-block does not exist (or is not available), the encoder / decoder may set (or store) a motion vector of a block located in the center of the collocated block as a representative motion vector and use it as a motion vector for the particular sub-block.

[0234] In one embodiment, to use the ATMVP mode, the encoder / decoder can apply an increased number of merge candidates, and no additional syntax (or syntax elements) may be used for this purpose. The syntax indicating the maximum number of merge candidates may be transferred from the encoder to the decoder via a higher level syntax. The higher level syntax may be, for example, a sequence parameter set (SPS), a picture parameter set (PPS), or a slice header. As an example, the maximum number of merge candidates may be six.

[0235] In addition, in one embodiment, an ATMVP-ext candidate indicating an extended ATMVP candidate may be added to the candidate list. Like the above-mentioned ATMVP, ATMVP-ext is a method for improving the existing TMVP, and can derive a motion vector for each subblock by combining spatially adjacent blocks and temporally adjacent blocks.

[0236] FIG. 23 is a diagram illustrating a method for deriving an extended Advanced Temporal Motion Vector Prediction (ATMVP) candidate according to an embodiment to which the present invention is applied.

[0237] 23, it is assumed that the size of a current block 2301 is 16x16. Also, in deriving a motion vector in sub-block units in FIG. 23, it is assumed that the size of a sub-block is 4x4, but the present invention is not limited thereto, and the size of the sub-block may be defined as a fixed size of 8x8 or more, which will be described later.

[0238] The encoder / decoder uses the motion vectors of the spatially adjacent blocks at the left horizontal and upper vertical positions, and the motion vectors of the temporally adjacent blocks at the right horizontal and lower vertical positions, to derive the motion vectors of each sub-block within the current block 2301.

[0239] For example, the encoder / decoder may derive motion vectors of spatially adjacent blocks at positions L-0 and A-0 in order to derive a motion vector for a subblock at position 0 in the current block 2301, derive motion vectors of temporally adjacent blocks of a subblock at positions 1 and 4 in the current block 2301, and calculate an average value of the four motion vectors. The encoder / decoder may determine the average value of the four motion vectors as the motion vector of the subblock at position 0 in the current block 2301. In one embodiment, when only a part of the four blocks is available, the encoder / decoder may calculate an average value of the motion vectors of the available blocks. In one embodiment, the reference index may be preset (e.g., reference index 0 or 1, co-located picture).

[0240] In one embodiment, similar to the ATMVP mode, to use the extended ATMVP mode, the encoder / decoder can apply an increased number of merge candidates, and therefore no additional syntax (or syntax elements) may be used.

[0241] 24 and 25 are diagrams illustrating the fetching process of motion vectors of temporal neighboring blocks in a reference picture according to an embodiment of the present invention.

[0242] Referring to Figure 24, the encoder / decoder can fetch (or derive, acquire) motion information stored at a corresponding position within a reference picture (or co-located picture) in a memory (170 in Figure 1 above, 250 in Figure 2) (specifically, the DPB of the memory) in order to derive a motion vector (hereinafter sometimes referred to as a temporal motion vector) of a temporally adjacent block of the current block.

[0243] In the present invention, for convenience of explanation, the memory will be mainly described as being DRAM, but this is not limited thereto, and the memory may include various storage media such as DRAM, USB, SD, CD, DVD, Blu-ray (registered trademark), HDD, SSD, etc.

[0244] In an embodiment, the corresponding position in the reference picture (or co-located picture) may be a right-bottom block of the current block or a center position of the current block (i.e., TMVP), or may be a position identified by a motion vector of a spatially adjacent block of the current block (i.e., ATMVP).

[0245] 25, assume that a temporal motion vector is fetched in units of a current block from a DRAM 2501. In this case, as shown in FIG. 25, the encoder / decoder fetches one motion vector stored in a block 2502 corresponding to the current block in the DRAM 2501 as a temporal motion vector.

[0246] On the other hand, when the ATMVP according to the embodiment of the present invention (i.e., sub-block based temporal merge candidate) or the extended ATMVP is applied, motion information is fetched in sub-block units, so that it is not possible to fetch one motion information unit of the current block as shown in Fig. 25. In other words, when the ATMVP or the extended ATMVP is applied, a larger amount of data must be fetched from the DRAM 2501. This will be described with reference to the following diagram.

[0247] FIG. 26 is a diagram illustrating a fetching process of motion vectors of temporal neighboring blocks in sub-block units according to an embodiment to which the present invention is applied.

[0248] 26, it is assumed that the current block is divided into four sub-blocks and motion compensation is performed. As described above, for convenience of explanation, the present invention will be described mainly on the case where the memory is the DRAM 2601, but the present invention is not limited thereto, and the memory may include various storage media such as the DRAM 2601, USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc.

[0249] As shown in Fig. 26, when fetching motion information in units of four sub-blocks, the encoder / decoder fetches four pieces of motion information corresponding to each sub-block in a corresponding position block 2602 of a current block in a reference picture (or a co-located picture) stored in a DRAM 2601. In other words, when fetching motion information in units of four sub-blocks, the encoder / decoder must fetch about four times as much data from the DRAM 2601 as compared to the case described above in Fig. 25.

[0250] 25 and 26, it can be seen that the size of the sub-block has a decisive effect on the amount of data fetched from the DRAM 2601. Therefore, the size of the sub-block can have a significant effect on the decoder and encoder pipeline configuration and throughput depending on the hardware and fetch performance.

[0251] In a sub-block-based motion compensation method such as affine motion prediction, a motion vector is derived on a sub-block basis based on the motion vector of the derived control point, but in the case of ATMVP, i.e., a sub-block-based temporal merge candidate (or predictor), a motion vector stored in memory on a sub-block basis within a corresponding block of a reference picture (or co-located picture) is brought in. Therefore, even though the sub-block-based motion compensation is similar, the hardware load (burden) according to the size of the sub-block is significantly different between affine motion prediction and ATMVP.

[0252] If the current block is divided into too many sub-blocks, problems may arise in that multiple fetches must be performed depending on the size of the memory bus over which the fetches are performed.

[0253] Therefore, in order to solve this problem, the present invention proposes a method of setting (or determining, defining) a fixed minimum sub-block size and performing a temporal motion information fetching process in the set sub-block units.

[0254] In the case of conventional ATMVP or ATVMP-ext, the current block is divided into sub-block units of 4x4 size, and fetching and motion compensation are performed. In this case, as described above, depending on the size of the current block, the memory required for fetching may become excessively large. The load on the hardware may become very large, and the compression performance may decrease.

[0255] Therefore, in the embodiment of the present invention, a method of dividing into sub-blocks of a fixed size of 8x8 or sub-blocks of a fixed size of 16x16 is proposed. The method proposed in this embodiment has been proven through experimental results to have little loss of compression performance in comparison with an increase in hardware complexity, and the experimental results are the same as those in Tables 2 to 5 below.

[0256] First, Table 2 below shows the compression performance when ATMVP is executed in units of 4x4 sub-blocks.

[0257] [Table 2]

[0258] Table 3 below shows the compression performance when ATMVP is executed in units of 8x8 sub-blocks.

[0259] [Table 3]

[0260] Table 4 below shows the compression performance when ATMVP is executed in units of 16x16 sub-blocks.

[0261] [Table 4]

[0262] Table 5 below shows the compression performance when ATMVP is executed in units of 32x32 sub-blocks.

[0263] [Table 5]

[0264] Referring to the experimental results in Tables 2 to 5 above, it can be seen that although there is a trade-off between compression efficiency and decoding speed depending on the size of the sub-block, the loss in compression performance relative to the improvement in hardware complexity is very small.

[0265] In an embodiment of the present invention, the encoder / decoder can perform sub-block-based motion compensation (i.e., ATMVP, ATMVP-ext) in units larger than 4x4, i.e., 8x8, 16x16 or 32x32 units, and can fetch sub-block-based temporal motion information in sub-block units of that size.

[0266] According to an embodiment of the present invention, sub-block based motion compensation can be achieved with high hardware complexity versus relatively low compression loss.

[0267] In the following, in order to increase (improve) the hardware complexity, we propose a method of limiting the application of the sub-block-based temporal motion information derivation method proposed in the present invention (i.e., ATMVP, ATMVP-ext) depending on the size of the coding block.

[0268] In an embodiment of the present invention, the encoder / decoder may not perform a sub-block-based temporal motion information derivation method (i.e., ATMVP, ATMVP-ext) if the width or height of the current block (or current coding block) is smaller than a predefined size.

[0269] 24 to 26, the hardware complexity is improved by reducing the amount of data fetched for motion information per unit of the same area, whereas the present embodiment can further improve the worst case complexity. Conventional video compression techniques divide pictures or blocks into various shapes to perform prediction, transformation, and other coding.

[0270] Accordingly, the size of a coding block (coding unit or prediction unit) may be divided into blocks of smaller sizes such as 4x4, 4x8, 8x4, etc. In this case, the size of the current block may be smaller than the size of the smallest sub-block for fetching temporal motion information (or temporal motion vector). In this case, in terms of hardware, fetching may be performed with the size of the smallest coding block, not the size of the smallest sub-block.

[0271] Therefore, in the present invention, in order to improve such a problem, if the size of the currently processed block (or the width and / or height of the current block) is smaller than a predefined minimum size, the block-based temporal motion vector candidate may not be used. That is, in an embodiment of the present invention, the encoder / decoder may add a sub-block-based temporal motion vector candidate to a candidate list only if the size of the currently processed block (or the width and / or height of the current block) is equal to or larger than a predefined minimum size. As an embodiment, the following Equation 3 may be used as a condition for using a sub-block-based temporal motion vector candidate.

[0272] <Formula 3>

number

[0273] FIG. 27 is a flowchart showing a method for generating a candidate list according to an embodiment to which the present invention is applied.

[0274] For convenience of explanation, the description will be focused on a decoder with reference to Figure 27, but the candidate list generating method of the present embodiment can be performed in the encoder as well. In one embodiment, the methods described in Figures 12 to 17 and Figures 21 to 26 above can be applied, and the duplicated description thereof will be omitted.

[0275] The candidate configuration procedure shown in Fig. 27 is merely an example, and the present invention is not limited thereto, and the order may be changed, some steps may be omitted, or some steps may be added. Also, the candidates added in Fig. 27 are merely an example, and in addition, various other candidates may be added, and some of the candidates shown in Fig. 27 may not be used.

[0276] The decoder adds a spatial neighboring block (or a spatial candidate) to a candidate list (S2701). For example, the decoder can add the neighboring candidate block of FIG. 16 described above to the candidate list.

[0277] The decoder checks whether the width and height of the current block are greater than or equal to a predefined minimum size (S2702).

[0278] The decoder adds a temporal neighboring block (or a temporal candidate) to the candidate list if the width and height of the current block are equal to or greater than a predefined minimum size (S2703). If the width or height of the current block is smaller than a predefined minimum size, the decoder may not add the temporal neighboring block (or a temporal candidate) to the candidate list.

[0279] The decoder adds the combined motion information (or the combined motion vector) to a candidate list and adds the zero motion vector to the candidate list (S2704, S2705).

[0280] Although the embodiments of the present invention described above have been described separately for convenience of explanation, the present invention is not limited thereto. In other words, the embodiments described above may be implemented independently, or one or more of the embodiments may be implemented in combination.

[0281] FIG. 28 is a flowchart illustrating a method for generating an inter-predicted block according to an embodiment to which the present invention is applied.

[0282] Referring to Figure 28, for convenience of explanation, the description will focus on a decoder, but the present invention is not limited thereto, and the method of generating an inter-prediction block based on inter-prediction can be applied in substantially the same way in an encoder.

[0283] When a merge mode is applied to a current block, the decoder generates a merge candidate list using spatial neighboring blocks and temporal merge candidates of the current block (S2801).

[0284] The decoder obtains a merge index indicating a candidate to be used for inter prediction of the current block in the merge candidate list (S2802).

[0285] The decoder derives motion vectors for each of a plurality of sub-blocks included in the current block, based on candidate motion vectors used for inter prediction of the current block (S2803).

[0286] The decoder generates a prediction sample for the current block using the motion vectors of each of the sub-blocks (S2804).

[0287] In an embodiment, the size of the sub-blocks may be set to a fixed specific size of 8x8 or more.

[0288] In an embodiment, the step of generating the merge candidate list may include a step of adding a temporal neighboring block of the current block identified by a motion vector of the spatial neighboring block in a collocated picture of the current block to the merge candidate list.

[0289] In an embodiment, the step of generating the merge candidate list may include a step of adding to the merge candidate list a temporal neighboring block of the current block identified by a motion vector of a first spatial neighboring block of the current block in a predefined order within a collocated picture of the current block.

[0290] In an embodiment, the step of generating the merge candidate list may include a step of checking whether the width and height of the current block are smaller than a predefined minimum size, and if the check results in the width and height of the current block being equal to or greater than the predefined minimum size, adding a temporal neighboring block of the current block identified by the motion vector of the spatial neighboring block in a collocated picture of the current block to the merge candidate list.

[0291] In an embodiment, the predefined minimum size may be set to the size of the sub-block.

[0292] FIG. 29 is a diagram illustrating an inter prediction device according to an embodiment to which the present invention is applied.

[0293] Although the inter prediction unit is illustrated as a single block in FIG. 29 for ease of explanation, the inter prediction unit may be realized in a configuration included in the encoder and / or decoder.

[0294] 29, prior to the inter prediction unit, the functions, processes and / or methods proposed in Figures 8 to 32 are implemented. Specifically, the inter prediction unit may include an inter prediction unit merge candidate list generation unit 2901, a merge index acquisition unit 2902, a sub-block motion vector derivation unit 2903 and a prediction block generation unit 2904.

[0295] When a merge mode is applied to a current block, the merge candidate list generating unit 2901 generates a merge candidate list using spatial neighboring blocks and temporal merge candidates of the current block.

[0296] A merge index acquisition unit 2902 acquires a merge index indicating a candidate to be used for inter prediction of the current block in the merge candidate list.

[0297] The sub-block motion vector derivation unit 2903 derives motion vectors for each of the sub-blocks included in the current block, based on candidate motion vectors used for inter prediction of the current block.

[0298] The prediction block generator 2904 generates a prediction sample of the current block using the motion vectors of each of the sub-blocks.

[0299] In an embodiment, the size of the sub-blocks may be set to a fixed specific size of 8x8 or more.

[0300] In an embodiment, the merging candidate list generator 2901 may add a temporal neighboring block of the current block identified by a motion vector of the spatial neighboring block in a collocated picture of the current block to the merging candidate list.

[0301] In an embodiment, the merge candidate list generation unit 2901 may add to the merge candidate list a temporal neighboring block of the current block identified by a motion vector of a first spatial neighboring block of the current block in a predefined order among the spatial neighboring blocks of the current block in a collocated picture of the current block.

[0302] In an embodiment, the merge candidate list generation unit 2901 checks whether the width and height of the current block are smaller than a predefined minimum size, and if the check results in the width and height of the current block being equal to or greater than the predefined minimum size, the merge candidate list generation unit 2901 may add a temporal neighboring block of the current block identified by the motion vector of the spatial neighboring block in a collocated picture of the current block to the merge candidate list.

[0303] As an example, the predefined minimum size may be set to the size of the sub-block.

[0304] FIG. 30 shows a video coding system to which the present invention is applied.

[0305] A video coding system may include a source device and a receiving device. The source device may transmit encoded video / image information or data to the receiving device in a file or streaming format via a digital storage medium or a network.

[0306] The source device may include a video source, an encoding apparatus, and a transmitter. The receiving device may include a receiver, a decoding apparatus, and a renderer. The encoding device may be called a video / video encoding device, and the decoding device may be called a video / video decoding device. The transmitter may be included in the encoding device. The receiver may be included in the decoding device. The renderer may include a display unit, which may be configured as another device or an external component.

[0307] A video source may acquire video / video through a video / video capture, synthesis, or generation process, etc. A video source may include a video / video capture device and / or a video / video generation device. A video / video capture device may include, for example, one or more cameras, a video / video archive containing previously captured video / video, etc. A video / video generation device may include, for example, a computer, a tablet, a smartphone, etc., and may (electronically) generate video / video. For example, a virtual video / video may be generated through a computer, etc., in which case the video / video capture process may take the place of the process by which the associated data is generated.

[0308] An encoding device can encode an input video / image. The encoding device can perform a series of steps such as prediction, transformation, quantization, etc. for compression and coding efficiency. The encoded data (encoded video / image information) can be output in the form of a bitstream.

[0309] The transmitting unit may transmit the encoded video / image information or data output in the form of a bitstream to a receiving unit of a receiving device via a digital storage medium or a network in a file or streaming format. The digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. The transmitting unit may include an element for generating a media file via a predetermined file format and may include an element for transmitting via a broadcasting / communication network. The receiving unit may extract the bitstream and transmit it to a decoding device.

[0310] The decoding device can decode the video / image by performing a series of steps such as inverse quantization, inverse transformation, prediction, etc. that correspond to the operations of the encoding device.

[0311] The renderer can render the decoded video / image. The rendered video / image can be displayed via a display unit.

[0312] FIG. 31 shows a structural diagram of a content streaming system as an embodiment to which the present invention is applied.

[0313] Referring to FIG. 31, a content streaming system to which the present invention is applied may generally include an encoding server, a streaming server, a Web server, a media storage, a user device, and a multimedia input device.

[0314] The encoding server compresses content input from a multimedia input device such as a smartphone, camera, or video camera into digital data to generate a bitstream and transmits the bitstream to the streaming server. As another example, if a multimedia input device such as a smartphone, camera, or video camera directly generates a bitstream, the encoding server may be omitted.

[0315] The bitstream can be generated by an encoding method or a bitstream generating method to which the present invention is applied, and the streaming server can temporarily store the bitstream in the process of transmitting or receiving the bitstream.

[0316] The streaming server transfers multimedia data to a user device based on a user's request via a web server, and the web server acts as an intermediary to inform the user of available services. When a user requests a desired service from the web server, the web server transfers it to the streaming server, and the streaming server transfers the multimedia data to the user. In this case, the content streaming system may include a separate control server, and in this case, the control server plays a role of controlling commands / responses between each device in the content streaming system.

[0317] The streaming server may receive the content from a media storage and / or an encoding server. For example, if the content is to be received from the encoding server, the content may be received in real time. In this case, the streaming server may store the bitstream for a certain period of time in order to provide a smooth streaming service.

[0318] Examples of the user devices include mobile phones, smartphones, laptop computers, digital broadcasting terminals, Personal Digital Assistants (PDAs), Portable Multimedia Players (PMPs), navigation systems, slate PCs, tablet PCs, Ultrabooks (ULTRABOOK (registered trademark)), wearable devices (e.g., smartwatches, smart glass, Head Mounted Displays (HMDs)), digital TVs, desktop computers, digital signage, etc.

[0319] Each server in the content streaming system can be operated as a distributed server, in which case data received by each server can be processed in a distributed manner.

[0320] As described above, the embodiments described in the present invention may be implemented and executed on a processor, microprocessor, controller, or chip. For example, the functional units shown in each figure may be implemented and executed on a computer, processor, microprocessor, controller, or chip.

[0321] In addition, the decoder and encoder to which the present invention is applied can be included in a multimedia broadcasting transmitting / receiving device, a mobile communication terminal, a home cinema video device, a digital cinema video device, a surveillance camera, a video interactive device, a real-time communication device such as video communication, a mobile streaming device, a storage medium, a video camera, a video on demand (VoD) service providing device, an OTT video (Over The Top video) device, an Internet streaming service providing device, a three-dimensional (3D) video device, a video phone, a video device, and a medical video device, and can be used to process a video signal or a data signal. For example, an OTT video (Over The Top video) device can include a game console, a Blu-ray player, an Internet-connected TV, a home theater system, a smartphone, a tablet PC, a DVR (Digital Video Recorder), and the like.

[0322] In addition, the processing method to which the present invention is applied can be produced in the form of a program executed by a computer and can be stored in a computer-readable recording medium. Multimedia data having a data structure according to the present invention can also be stored in a computer-readable recording medium. The computer-readable storage medium includes any kind of storage device and distributed storage device in which computer-readable data is stored. The computer-readable storage medium can include, for example, a Blu-ray Disc (BD), a Universal Serial Bus (USB), a ROM, a PROM, an EPROM, an EEPROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, and an optical data storage device. In addition, the computer-readable recording medium includes a medium realized in the form of a carrier wave (e.g., transmission through the Internet). In addition, the bit stream generated by the encoding method can be stored in a computer-readable recording medium or transferred via a wired or wireless communication network.

[0323] Furthermore, the embodiments of the present invention may be realized as a computer program product by a program code, which may be executed by a computer according to the embodiments of the present invention. The program code may be stored on a carrier readable by a computer.

[0324] The above-described embodiments are combinations of the components and features of the present invention in a predetermined form. Each component or feature should be considered as optional unless otherwise explicitly stated. Each component or feature may be implemented in a form not combined with other components or features. It is also possible to combine some components and / or features to configure an embodiment of the present invention. The order of operations described in the embodiments of the present invention may be changed. Some configurations or features of any embodiment may be included in other embodiments, or may be replaced (alternative) with configurations or features corresponding to other embodiments. It is obvious that claims that do not have an explicit reference relationship in the claims may be combined to configure an embodiment, or may be included as a new claim by amendment after filing.

[0325] Embodiments of the present invention may be implemented by various means, such as hardware, firmware, software, or a combination thereof. In a hardware implementation, an embodiment of the present invention may be implemented by one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), Field Programmable Gate Arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, etc.

[0326] In the case of implementation by firmware or software, an embodiment of the present invention may be implemented in the form of a module, procedure, function, etc. that performs the functions or operations described above. The software code may be stored in a memory and driven by a processor. The memory may be located inside or outside the processor and may transmit and receive data to and from the processor by various means already known in the art.

[0327] It is obvious to those skilled in the art that the present invention can be embodied in other specific forms without departing from the essential features of the present invention. Therefore, the above detailed description should not be interpreted as limiting in all respects, but should be regarded as illustrative. The scope of the present invention should be determined by a reasonable interpretation of the appended claims, and all modifications within the equivalent scope of the present invention are included in the scope of the present invention. [Industrial Applicability]

[0328] The above-described preferred embodiments of the present invention have been disclosed for illustrative purposes, and those skilled in the art may improve, modify, substitute or add various other embodiments within the technical spirit and scope of the present invention disclosed in the appended claims.

Claims

1. 1. A method for decoding a video signal, comprising the steps of: generating a merging candidate list based on a merging mode applied to a current block, the merging candidate list including at least one of a spatial neighboring block and a temporal neighboring block of the current block, the merging candidate list including candidates up to a maximum number of merging candidates; obtaining a merge index indicating a candidate in the merge candidate list to be used for inter prediction of the current block; deriving motion vectors of each of the sub-blocks included in the current block based on the motion vectors of the candidates indicated by the merge index; generating a prediction block of the current block based on the motion vector of each sub-block; performing an inverse transform on the transformed coefficients of the current block to generate a residual block of the current block; generating a reconstructed block of the current block based on the predicted block and the residual block; The generating of the merge candidate list includes adding the temporal neighboring block located in a co-located picture of the current block and identified by the motion vector of the spatial neighboring block to the merge candidate list based on a width and a height of the current block being equal to or greater than a predefined minimum size, respectively; based on the width or the height of the current block being smaller than the predefined minimum size, the temporal neighboring block is not included in the merging candidate list; based on the merge index indicating the temporal neighboring block as the candidate, the motion vector of each of the sub-blocks included in the current block is derived based on the motion vector of a sub-block included in the temporal neighboring block; A method wherein zero candidates are added to the merge candidate list.

2. the temporal neighboring block is identified by the motion vector of a first spatial neighboring block in the merging candidate list; The method of claim 1 , wherein the initial spatial neighboring block is determined based on a predefined order among two or more spatial neighboring blocks of the current block.

3. The method of claim 1 , wherein the predefined minimum magnitude is set to eight.

4. 1. A method for encoding a video signal, comprising the steps of: generating a merging candidate list based on a merging mode applied to a current block, the merging candidate list including at least one of a spatial neighboring block and a temporal neighboring block of the current block, the merging candidate list including candidates up to a maximum number of merging candidates; deriving a motion vector for each sub-block included in the current block based on the motion vectors of candidates included in the merge candidate list; generating a prediction block of the current block based on the motion vector of each sub-block; generating a residual block of the current block based on the predicted block; generating a merge index that points to the candidate in the merge candidate list; and performing a transform on the residual block; The generating of the merge candidate list includes adding the temporal neighboring block located in a co-located picture of the current block and identified by the motion vector of the spatial neighboring block to the merge candidate list based on a width and a height of the current block being equal to or greater than a predefined minimum size, respectively; based on the width or the height of the current block being smaller than the predefined minimum size, the temporal neighboring block is not included in the merging candidate list; the motion vectors of the respective sub-blocks of the current block are derived based on motion vectors of sub-blocks split from the temporal neighboring blocks on the basis that the motion vectors of the temporal neighboring blocks in the merge candidate list are used to derive the motion vectors of the respective sub-blocks of the current block; A method wherein zero candidates are added to the merge candidate list.

5. the temporal neighboring block is identified by the motion vector of a first spatial neighboring block in the merging candidate list; The method of claim 4 , wherein the first spatial neighboring block is determined based on a predefined order among two or more spatial neighboring blocks of the current block.

6. The method of claim 5 , wherein the predefined minimum magnitude is set to eight.

7. 1. A transmission method for data including a video related bitstream, comprising: generating the bitstream for the video; transmitting the data including the bitstream; The bitstream comprises: generating a merging candidate list based on a merging mode applied to a current block, the merging candidate list including at least one of a spatial neighboring block and a temporal neighboring block of the current block, the merging candidate list including candidates up to a maximum number of merging candidates; deriving a motion vector for each sub-block included in the current block based on the motion vectors of candidates included in the merge candidate list; generating a prediction block of the current block based on the motion vector of each sub-block; generating a residual block of the current block based on the predicted block; generating a merge index that points to the candidate in the merge candidate list; applying a transform to the residual block; The generating of the merge candidate list includes adding the temporal neighboring block located in a co-located picture of the current block and identified by the motion vector of the spatial neighboring block to the merge candidate list based on a width and a height of the current block being equal to or greater than a predefined minimum size, respectively; based on the width or the height of the current block being smaller than the predefined minimum size, the temporal neighboring block is not included in the merging candidate list; the motion vectors of the respective sub-blocks of the current block are derived based on motion vectors of sub-blocks split from the temporal neighboring blocks on the basis that the motion vectors of the temporal neighboring blocks in the merge candidate list are used to derive the motion vectors of the respective sub-blocks of the current block; A method wherein zero candidates are added to the merge candidate list.

Images