Image processing method and apparatus based on interpredictive mode

Abstract

To provide methods and devices for reducing hardware resource usage in performing an inter prediction based on a similarity of temporal motion information.SOLUTION: A method for decoding a video signal based on an inter prediction mode includes the steps of: if a merge mode is applied to a current block, generating a merge candidate list by using a spatial neighboring block and a temporal merge candidate of the current block; obtaining a merge index indicating a candidate to be used for an inter prediction of the current block in the merge candidate list; deriving a motion vector of each of subblocks included in the current block based on a motion vector of the candidate used for the inter prediction of the current block; and generating a prediction sample of the current block by using the motion vector of each of subblocks.SELECTED DRAWING: Figure 14

Description

Technical Field

[0001] The present invention relates to a method for processing still images or moving images. More specifically, the present invention relates to a method for encoding / decoding still images or moving images based on an inter prediction mode and an apparatus for supporting (assisting) the same.

Background Art

[0002] Compression encoding refers to a series of signal processing techniques for transferring (transmitting) digitized information via a communication line or storing it in a form suitable for a storage medium. Media such as video, images, and audio can be the targets of compression encoding. In particular, a technique for performing compression encoding on video is called video compression.

[0003] Next-generation video content will have characteristics such as high spatial resolution, high frame rate, and high dimensionality of scene representation. In order to process such content, it will bring an extremely large increase in terms of memory storage, memory access rate, and processing power.

[0004] Therefore, it is necessary to design coding tools for more efficiently processing next-generation video content.

Summary of the Invention

Problems to be Solved by the Invention

[0005] An object of the present invention is to propose a method and an apparatus for reducing the use of hardware resources in performing inter prediction (or prediction between frames) based on the similarity of temporal motion information.

[0006] The technical problems that this invention aims to solve are not limited to those mentioned above, and any other technical problems not mentioned should be clearly understood by a person with ordinary skill in the art to which this invention belongs from the description below. [Means for solving the problem]

[0007] One aspect of the present invention is a method for decoding video based on an interprediction mode, wherein, when a merge mode is applied to the current block, the method comprises: generating a merge candidate list using the spatial neighboring block and temporal neighboring block of the current block; obtaining a merge index that indicates a candidate to be used for interprediction of the current block within the merge candidate list; deriving the motion vectors of each of the multiple subblocks contained in the current block based on the motion vectors of the candidates used for interprediction of the current block; and generating a prediction sample of the current block using the motion vectors of each of the subblocks, wherein the size of the subblock can be set to a fixed specific size of 8x8 or larger.

[0008] In one embodiment, the step of generating a merge candidate list may include adding to the merge candidate list the time-adjacent blocks of the current block, which are identified by the motion vectors of the spatially adjacent blocks within the current block's collocated picture.

[0009] In one embodiment, the step of generating a merge candidate list may include adding to the merge candidate list the time-adjacent blocks of the current block, which are identified by the motion vector of the first spatially adjacent block in a predefined order among the spatially adjacent blocks of the current block, within the collocated picture of the current block.

[0010] As an embodiment, the step of generating a merge candidate list may include the steps of: checking whether the width and height of the current block are smaller than a predefined minimum size; and, if the check shows that the width and height of the current block are greater than or equal to the predefined minimum size, adding the time-adjacent blocks of the current block, identified by the motion vectors of the spatially adjacent blocks within the collocated picture of the current block, to the merge candidate list.

[0011] In one embodiment, the predefined minimum size can be set to the size of the subblock.

[0012] Another embodiment of the present invention is a device for decoding video based on an inter-prediction mode, comprising: a merge candidate list generation unit that generates a merge candidate list using spatial merge candidates and temporal merge candidates of the current block when a merge mode is applied to the current block; a merge index acquisition unit that obtains a merge index that indicates a merge candidate to be used for inter-prediction of the current block within the merge candidate list; a sub-block motion vector derivation unit that derives the motion vectors of each of a plurality of sub-blocks in the current block based on the motion vectors of the merge candidates used for inter-prediction of the current block; and a prediction block generation unit that generates a prediction block of the current block using the motion vectors of each of the sub-blocks, wherein the size of the sub-block can be set to a fixed specific size of 8x8 or larger.

[0013] In one embodiment, the merge candidate list generation unit can add the time-adjacent blocks of the current block, identified by the motion vectors of spatially adjacent blocks within the current block's collocated picture, to the merge candidate list.

[0014] In one embodiment, the merge candidate list generation unit can add to the merge candidate list the time-adjacent blocks of the current block, which are identified by the motion vector of the first spatially adjacent block in a predefined order among the spatially adjacent blocks of the current block, within the collocated picture of the current block.

[0015] In one embodiment, the merge candidate list generation unit checks whether the width and height of the current block are smaller than a predefined minimum size. If the check finds that the width and height of the current block are larger than or equal to the predefined minimum size, it can add the time-adjacent blocks of the current block, identified by the motion vectors of the spatially adjacent blocks within the current block's collocated picture, to the merge candidate list.

[0016] In one embodiment, the predefined minimum size can be set to the size of the subblock. [Effects of the Invention]

[0017] According to embodiments of the present invention, the hardware gate count can be effectively reduced when generating prediction blocks using motion information of already decoded pictures.

[0018] Furthermore, according to the embodiment of the present invention, when performing interpretation using motion information of already decoded pictures, it is possible to reduce the hardware gate count while simultaneously minimizing the corresponding loss of compression performance.

[0019] The effects obtained by the present invention are not limited to those mentioned above, and any other effects not mentioned should be clearly understood by those with ordinary skill in the art to which the present invention pertains from the following description. [Brief explanation of the drawing]

[0020] [Figure 1] This is a schematic block diagram of an encoding device used to encode video / image signals, which is an embodiment to which the present invention is applied. [Figure 2] This is a schematic block diagram of a decoding device that performs video / image signal decoding, which is an embodiment to which the present invention is applied. [Figure 3] This figure shows an example of a multi-type tree structure to which the present invention can be applied. [Figure 4] This figure illustrates an example of an embodiment to which the present invention can be applied: a signaling mechanism for partition information in a quadtree with nested multi-type trees. [Figure 5] This figure illustrates an embodiment to which the present invention can be applied, in which a method for dividing a CTU into multiple CUs based on a quadtree and nested multi-type tree structure is shown. [Figure 6] This figure illustrates an embodiment to which the present invention can be applied: a method for restricting ternary-tree partitioning. [Figure 7] This figure illustrates redundant partitioning patterns that can occur in binary tree partitioning and ternary tree partitioning, as embodiments to which the present invention can be applied. [Figure 8]A diagram illustrating an inter-prediction-based video / visual encoding method according to an embodiment of the present invention and an inter-prediction unit in an encoding apparatus according to an embodiment of the present invention. [Figure 9] A diagram illustrating an inter-prediction-based video / visual encoding method according to an embodiment of the present invention and an inter-prediction unit in an encoding apparatus according to an embodiment of the present invention. [Figure 10] A diagram illustrating a method for decoding an inter-prediction-based video / visual according to an embodiment of the present invention and an inter-prediction unit in a decoding apparatus according to an embodiment of the present invention. [Figure 11] A diagram illustrating a method for decoding an inter-prediction-based video / visual according to an embodiment of the present invention and an inter-prediction unit in a decoding apparatus according to an embodiment of the present invention. [Figure 12] As an embodiment to which the present invention is applied, a diagram for explaining peripheral blocks used in merge mode or skip mode. [Figure 13] A flowchart illustrating a method for constructing a merge candidate list according to an embodiment to which the present invention is applied. [Figure 14] A flowchart illustrating a method for constructing a merge candidate list according to an embodiment to which the present invention is applied. [Figure 15] As an embodiment to which the present invention is applied, a diagram for explaining a method for deriving ATMVP (Advanced Temporal Motion Vector Prediction) candidates. [Figure 16] As an embodiment to which the present invention is applied, a diagram for explaining a method for deriving ATMVP (Advanced Temporal Motion Vector Prediction) candidates. [Figure 17] As an embodiment to which the present invention is applied, a diagram illustrating a method for deriving ATMVP (Advanced Temporal Motion Vector Prediction) candidates. [Figure 18]This figure illustrates a method for compressing temporal motion vector data and the positions of candidate spatial locations used therein, according to one embodiment to which the present invention is applied. [Figure 19] This figure illustrates a method for compressing temporal motion vector data and the positions of candidate spatial locations used therein, according to one embodiment to which the present invention is applied. [Figure 20] This flowchart shows a method for generating an inter-predicted block based on a candidate list according to an embodiment to which the present invention is applied. [Figure 21] This is a flowchart showing a method for generating a candidate list according to an embodiment to which the present invention is applied. [Figure 22] This figure shows an example of a peripheral block used in inter prediction according to an embodiment to which the present invention is applied. [Figure 23] This figure illustrates a method for deriving an extended ATMVP (Advanced Temporal Motion Vector Prediction) candidate according to an embodiment to which the present invention is applied. [Figure 24] This figure illustrates a fetching process for fetching (patching) motion vectors of time-adjacent blocks within a reference picture according to an embodiment to which the present invention is applied. [Figure 25] This figure illustrates a fetching process for fetching motion vectors of time-adjacent blocks within a reference picture according to an embodiment to which the present invention is applied. [Figure 26] This figure shows a fetching process for fetching motion vectors of time-adjacent blocks in sub-block units according to an embodiment to which the present invention is applied. [Figure 27]This is a flowchart showing a method for generating a candidate list according to an embodiment to which the present invention is applied. [Figure 28] This flowchart illustrates a method for generating an interpretation block according to an embodiment to which the present invention is applied. [Figure 29] This figure illustrates an inter-prediction device according to an embodiment to which the present invention is applied. [Figure 30] This figure shows a video coding system to which the present invention is applied. [Figure 31] This is a structural diagram of a content streaming system, which is an embodiment to which the present invention is applied. [Modes for carrying out the invention]

[0021] The accompanying drawings, included as part of the detailed description to aid in understanding the present invention, provide embodiments of the invention and, together with the detailed description, illustrate the technical features of the invention.

[0022] Preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings. The detailed description disclosed below, together with the accompanying drawings, is intended to illustrate exemplary embodiments of the present invention and is not intended to show only the only embodiments in which the present invention can be carried out. The following detailed description includes specific details to provide a complete understanding of the present invention. However, those skilled in the art will know that the present invention can be carried out without such specific details.

[0023] In some cases, known structures and devices may be omitted or shown in the form of block diagrams focusing on the core function of each structure and device, in order to avoid ambiguity of the concept of the present invention.

[0024] Furthermore, while the terminology used in this invention has been selected as widely used and common terms whenever possible, in certain cases, the applicant has chosen terminology at their discretion for explanation. In such cases, the meaning will be clearly described in the detailed explanation of the relevant section, and it should be made clear that the terminology used in the description of this invention should not be interpreted simply by its name alone, but rather by understanding the meaning of the corresponding term.

[0025] Certain terms used in the following description are provided to aid in understanding the present invention, and the use of such specific terms may be modified in other ways without departing from the technical spirit of the invention. For example, terms such as signals, data, samples, pictures, frames, and blocks may be appropriately substituted and interpreted during each coding process.

[0026] Hereafter, in this specification, “processing unit” means a unit in which an encoding / decoding process, such as prediction, transformation, and / or quantization, is performed. Hereafter, for convenience of explanation, a processing unit may also be referred to as a “processing block” or “block.”

[0027] The term "processing unit" can be interpreted as including units for both the luminance (luma) component and the chroma component. For example, a processing unit can correspond to a coding tree unit (CTU), a coding unit (CU), a prediction unit (PU), or a transform unit (TU).

[0028] Furthermore, a processing unit can be interpreted as a unit of the luminance (luma) component or a unit of the chroma component. For example, a processing unit can correspond to a coding tree block (CTB), coding block (CB), prediction block (PU), or transform block (TB) of the luminance (luma) component. Alternatively, it can correspond to a coding tree block (CTB), coding block (CB), prediction block (PU), or transform block (TB) of the chroma component. However, it is not limited to these interpretations, and a processing unit can also be interpreted in a sense that includes units of both the luminance (luma) and chroma components.

[0029] Furthermore, the processing unit is not necessarily limited to square blocks, but can also be composed of polygons with three or more vertices.

[0030] In this specification, pixels or picture elements will be collectively referred to as samples. The use of samples can be interpreted as the use of pixel values ​​or picture element values.

[0031] Figure 1 shows a schematic block diagram of an encoding device in which video / image signals are encoded, as an embodiment to which the present invention is applied.

[0032] Referring to Figure 1, the encoding device 100 may be configured to include a video splitting unit 110, a subtraction unit 115, a conversion unit 120, a quantization unit 130, an inverse quantization unit 140, an inverse conversion unit 150, an addition unit 155, a filtering unit 160, a memory 170, an inter-prediction unit 180, an intra-prediction unit 185, and an entropy encoding unit 190. The inter-prediction unit 180 and the intra-prediction unit 185 can be collectively referred to as the prediction unit. In other words, the prediction unit can include the inter-prediction unit 180 and the intra-prediction unit 185. The conversion unit 120, quantization unit 130, inverse quantization unit 140, and inverse conversion unit 150 can be included in the residual processing unit. The residual processing unit may further include a subtraction unit 115. In one embodiment, the aforementioned video splitting unit 110, subtraction unit 115, conversion unit 120, quantization unit 130, inverse quantization unit 140, inverse conversion unit 150, addition unit 155, filtering unit 160, inter-prediction unit 180, intra-prediction unit 185, and entropy encoding unit 190 can be configured by a single hardware component (e.g., an encoder or processor). Furthermore, the memory 170 may include a DPB (Decoded Picture Buffer) and may be configured by a digital storage medium.

[0033] The video splitting unit 110 can split the input video (or picture, frame) input to the encoding device 100 into one or more processing units. For example, the processing units may be called coding units (CUs). In this case, coding units can be recursively split from a coding tree unit (CTU) or a largeest coding unit (LCU) based on a quad-tree binary-tree (QTBT) structure. For example, a single coding unit can be split into multiple sub-depth coding units based on a quad-tree structure and / or a binary-tree structure. In this case, for example, the quad-tree structure may be applied first and the binary-tree structure later. Alternatively, the binary-tree structure may be applied first. Based on the final coding unit that cannot be further split, the coding procedure according to the present invention can be executed. In this case, based on the encoding efficiency according to the characteristics of the video, the coding unit can be immediately used as the final coding unit, or, if necessary, the coding unit can be recursively divided into further lower-depth coding units, and the optimally sized coding unit can be used as the final coding unit. Here, the coding procedure may include procedures such as prediction, transformation, and reconstruction, which will be described later. As another example, the above processing unit may further include a prediction unit (PU) or a transformation unit (TU). In this case, the prediction unit and the transformation unit can be divided or partitioned from the final coding unit described above, respectively. The prediction unit may be a unit of sample prediction, and the transformation unit may be a unit for deriving transformation coefficients and / or a unit for deriving a residual signal from transformation coefficients.

[0034] The term "unit" can sometimes be confused with terms such as "block" or "area." Generally, an MxN block can represent a set of samples or transform coefficients consisting of M columns and N rows. A sample can generally represent a pixel or a pixel value, sometimes only representing the luminance (luma) component pixel / pixel value, or sometimes only the chroma component pixel / pixel value. A sample can be used as the term corresponding to a single picture (or image) pixel or pel.

[0035] The encoding device 100 can generate a residual signal (remaining blocks, remaining sample array) by subtracting the predicted signal (predicted blocks, predicted sample array) output from the inter-prediction unit 180 or intra-prediction unit 185 from the input video signal (original blocks, original sample array), and the generated residual signal is transferred to the conversion unit 120. In this case, as shown in the figure, the unit that subtracts the predicted signal (predicted blocks, predicted sample array) from the input video signal (original blocks, original sample array) within the encoding device 100 can be called the subtraction unit 115. The prediction unit can predict the blocks to be processed (hereinafter referred to as the current blocks) and generate predicted blocks that include the predicted samples of the current blocks. The prediction unit can determine whether intra-prediction or inter-prediction is applied to the current blocks or on a CU basis. As will be described later in the explanation of each prediction mode, the prediction unit can generate various prediction-related information such as prediction mode information and transmit it to the entropy encoding unit 190. Prediction information can be encoded by the entropy encoding unit 190 and output in bitstream format.

[0036] The intra-prediction unit 185 can predict the current block by referring to a sample in the current picture. The referenced sample may be located in the vicinity (neighbor) of the current block or at a distance, depending on the prediction mode. In intra-prediction, the prediction mode may include multiple non-directional modes and multiple directional modes. Non-directional modes may include, for example, DC mode and planar mode. Directional modes may include, for example, 33 directional prediction modes or 65 directional prediction modes, depending on the degree of fineness of the prediction direction. However, this is an example, and more or fewer directional prediction modes may be used depending on the settings. The intra-prediction unit 185 may also determine the prediction mode to be applied to the current block using the prediction modes applied to the surrounding blocks.

[0037] The interprediction unit 180 can derive a predicted block for the current block based on a reference block (an array of reference samples) identified by motion vectors on the reference picture. In this case, in order to reduce the amount of motion information transferred in interprediction mode, motion information can be predicted on a block, subblock, or sample basis based on the correlation of motion information between the surrounding blocks and the current block. The motion information may include motion vectors and reference picture indices. The motion information may further include information on the interprediction direction (L0 prediction, L1 prediction, Bi prediction, etc.). In the case of interprediction, surrounding blocks may include spatial neighboring blocks that exist in the current picture and temporal neighboring blocks that exist in the reference picture. The reference picture containing the above-mentioned reference block and the reference picture containing the above-mentioned temporal neighboring block may be the same or different. The temporal peripheral blocks described above can be referred to as collocated reference blocks (colCU), collocated CUs, etc., and the reference picture containing the temporal peripheral blocks can be referred to as a collocated picture (colPic). For example, the interpretation unit 180 can construct a motion information candidate list based on the peripheral blocks and generate information indicating which candidate is used to derive the motion vector and / or the index of the reference picture of the current block. Interpretation can be performed based on various prediction modes; for example, in skip mode and merge mode, the interpretation unit 180 can use the motion information of the peripheral blocks as the motion information of the current block. In skip mode, unlike merge mode, the residual signal may not be transferred.In Motion Vector Prediction (MVP) mode, the motion vector of the current block can be instructed by signaling the motion vector difference using the motion vectors of surrounding blocks as motion vector predictors.

[0038] The prediction signal generated via the above-mentioned interpretation unit 180 or intrapretation unit 185 can be used to generate a reconstruction signal or a residual signal.

[0039] The transformation unit 120 can generate transformation coefficients by applying a transformation technique to the residual signal. For example, the transformation technique may include at least one of the following: DCT (Discrete Cosine Transform), DST (Discrete Sine Transform), KLT (Karhunen-Loeve Transform), GBT (Graph-Based Transform), or CNT (Conditionally Non-linear Transform). Here, GBT refers to a transformation obtained from a graph representing the relationship information between pixels. CNT refers to a transformation obtained by generating a prediction signal using all previously reconstructed pixels. The transformation process may also be applied to pixel blocks of the same size and square shape, or to blocks of variable size that are not square.

[0040] The quantization unit 130 quantizes the conversion coefficients and transfers them to the entropy encoding unit 190, which can encode the quantized signal (information about the quantized conversion coefficients) and output it as a bitstream. The information about the quantized conversion coefficients can be called residual information. The quantization unit 130 can rearrange the quantized conversion coefficients in block form into a one-dimensional vector form based on the coefficient scan order, and can also generate information about the quantized conversion coefficients based on the quantized conversion coefficients in the form of a one-dimensional vector. The entropy encoding unit 190 can perform various encoding methods, such as exponential Golomb, CAVLC (Context-Adaptive Variable Length Coding), and CABAC (Context-Adaptive Binary Arithmetic Coding). In addition to the quantized conversion coefficients, the entropy encoding unit 190 can also encode information necessary for video / image restoration (such as the values ​​of syntax elements) together with or separately from the quantized conversion coefficients. Encoded information (e.g., encoded video / image information) can be transferred or stored in the form of a bitstream in units of NAL (Network Abstraction Layer) units. The bitstream can be transferred over a network or stored on a digital storage medium. Here, the network can include broadcast networks and / or other networks, and the digital storage medium can include various storage media such as USB, SD, CD, DVD, Blu-ray®, HDD, SSD, etc. A transmission (transmit) unit (not shown) that transmits (transmits) the signal output from the entropy encoding unit 190 and / or a storage unit (not shown) that stores it can be configured as an internal / external element of the encoding device 100, or the transfer (transmit) unit can be a component of the entropy encoding unit 190.

[0041] The quantized conversion coefficients output from the quantization unit 130 can be used to generate a prediction signal. For example, the residual signal can be reconstructed by applying inverse quantization and inverse transformation to the quantized conversion coefficients via the inverse quantization unit 140 and inverse transformation unit 150 in the loop. The adder 155 can generate a reconstructed signal (reconstructed picture, reconstructed block, reconstructed sample array) by adding the reconstructed residual signal to the prediction signal output from the inter-prediction unit 180 or intra-prediction unit 185. If there are no residuals in the block to be processed, such as when skip mode is applied, the predicted block can be used as the reconstructed block. The adder 155 can be called the reconstruction unit or reconstructed block generation unit. The generated reconstructed signal can be used for intra-prediction of the next block to be processed in the current picture, and can also be used for inter-prediction of the next picture after filtering, as described later.

[0042] The filtering unit 160 can improve subjective / objective image quality by applying filters to the restored signal. For example, the filtering unit 160 can apply various filtering methods to the restored picture to generate a modified restored picture, and the modified restored picture can be stored in the memory 170, specifically in the DPB of the memory 170. The various filtering methods can include, for example, deblocking filtering, sample adaptive offset, adaptive loop filter, and bilateral filter. As will be described later in the description of each filtering method, the filtering unit 160 can generate various filtering-related information and transmit it to the entropy encoding unit 190. The filtering-related information can be encoded by the entropy encoding unit 190 and output in bitstream format.

[0043] The corrected restored picture transferred to memory 170 can be used as a reference picture by the interpretation unit 180. Through this, the encoding device can avoid prediction mismatches between the encoding device 100 and the decoding device when interpretation is applied, and can also improve encoding efficiency.

[0044] Memory 170DPB can store the corrected restored picture for use as a reference picture in the inter-prediction unit 180. Memory 170 can store motion information of blocks from which motion information in the current picture has been derived (or encoded) and / or motion information of blocks in the already restored picture. The stored motion information can be transmitted to the inter-prediction unit 180 for use as motion information of spatially surrounding blocks or motion information of temporally surrounding blocks. Memory 170 can store restored samples of restored blocks in the current picture and transmit them to the intra-prediction unit 185.

[0045] Figure 2 shows a schematic block diagram of a decoding device that performs video / image signal decoding, as an embodiment to which the present invention is applied.

[0046] Referring to Figure 2, the decoding device 200 can be configured to include an entropy decoding unit 210, an inverse quantization unit 220, an inverse transformation unit 230, an addition unit 235, a filtering unit 240, a memory 250, an inter-prediction unit 260, and an intra-prediction unit 265. The inter-prediction unit 260 and the intra-prediction unit 265 can be collectively called the prediction unit. In other words, the prediction unit can include the inter-prediction unit 260 and the intra-prediction unit 265. The inverse quantization unit 220 and the inverse transformation unit 230 can be collectively called the residual processing unit. In other words, the residual processing unit can include the inverse quantization unit 220 and the inverse transformation unit 230. The aforementioned entropy decoding unit 210, inverse quantization unit 220, inverse transformation unit 230, addition unit 235, filtering unit 240, inter-prediction unit 260, and intra-prediction unit 265 can be configured by a single hardware component (e.g., a decoder or a processor) depending on the embodiment. Furthermore, memory 170 may include a DPB (Decoded Picture Buffer) and may be composed of a digital storage medium.

[0047] When a bitstream containing video / image information is input, the decoding device 200 can reconstruct the image in accordance with the process by which the video / image information was processed by the encoding device in Figure 1. For example, the decoding device 200 can decode using the processing units applied in the encoding device. Thus, the decoding processing unit may be, for example, a coding unit, which can be divided along a quadtree structure and / or binary tree structure from a coding tree unit or a maximum coding unit. The reconstructed video signal decoded and output via the decoding device 200 can then be reproduced by a playback device.

[0048] The decoding device 200 can receive the signal output from the encoding device shown in Figure 1 in bitstream form, and the received signal can be decoded via the entropy decoding unit 210. For example, the entropy decoding unit 210 can analyze the bitstream and derive information necessary for image restoration (or picture restoration) (e.g., video / image information). For example, the entropy decoding unit 210 can decode the information in the bitstream based on a coding method such as exponential Golomb coding, CAVLC, or CABAC, and output the values ​​of syntactic elements necessary for image restoration and the quantized values ​​of conversion coefficients related to residuals. More specifically, the CABAC entropi decoding method receives a bin corresponding to each syntactic element in a bitstream, determines a context model using information about the syntactic element to be decoded, as well as decoded information about the surrounding and decoded blocks, or symbol / bin information decoded in a previous step, predicts the probability of bin occurrence based on the determined context model, and performs arithmetic decoding of the bins to generate symbols corresponding to the values ​​of each syntactic element. At this time, after determining the context model, the CABAC entropi decoding method can update the context model using the decoded symbol / bin information for the next symbol / empty context model. Information regarding predictions among the information decoded by the entropi decoding unit 210 is provided to the prediction unit (inter-prediction unit 260 and intra-prediction unit 265), and residual values ​​from the entropi decoding unit 210, i.e., quantized conversion coefficients and related parameter information, can be input to the inverse quantization unit 220. Furthermore, information related to filtering among the information decoded by the entropy decode unit 210 can be provided to the filtering unit 240. On the other hand, a receiving unit (not shown) that receives the signal output from the encoding device may be further configured as an internal / external element of the decoding device 200, or the receiving unit may be a component of the entropy decode unit 210.

[0049] The inverse quantization unit 220 can inversely quantize the quantized transformation coefficients and output the transformation coefficients. The inverse quantization unit 220 can rearrange the quantized transformation coefficients in the form of two-dimensional blocks. In this case, the rearrangement can be performed based on the coefficient scan order performed by the encoding device. The inverse quantization unit 220 can perform inverse quantization of the quantized transformation coefficients using quantization parameters (e.g., quantization stage size information) to obtain the transformation coefficients.

[0050] In the inverse conversion unit 230, the conversion coefficients are inversely converted to obtain the residual signal (residual block, residual sample array).

[0051] The prediction unit can make predictions about the current block and generate a predicted block that includes prediction samples of the current block. Based on the prediction information output from the entropy decode unit 210, the prediction unit can determine whether intra-prediction or inter-prediction is applied to the current block, and can determine a specific intra / inter-prediction mode.

[0052] The intra-prediction unit 265 can predict the current block by referring to a sample within the current picture. The referenced sample may be located in the vicinity (neighbor) of the current block or at a distance, depending on the prediction mode. In intra-prediction, the prediction mode may include multiple non-directional modes and multiple directional modes. The intra-prediction unit 265 can also determine the prediction mode to be applied to the current block using the prediction modes applied to the surrounding blocks.

[0053] The interprediction unit 260 can derive a predicted block for the current block based on a reference block (an array of reference samples) identified by motion vectors on a reference picture. In this case, to reduce the amount of motion information transferred in interprediction mode, motion information can be predicted on a block, subblock, or sample basis based on the correlation of motion information between surrounding blocks and the current block. The motion information may include motion vectors and reference picture indices. The motion information may further include information on the interprediction direction (L0 prediction, L1 prediction, Bi prediction, etc.). In interprediction, surrounding blocks may include spatial neighboring blocks within the current picture and temporal neighboring blocks in the reference picture. For example, the interprediction unit 260 can construct a motion information candidate list based on surrounding blocks and derive the motion vector and / or reference picture index of the current block based on the received candidate selection information. Interprediction can be performed based on various prediction modes, and the prediction information may include information indicating the interprediction mode for the current block.

[0054] The summing unit 235 can generate a reconstructed signal (reconstructed picture, reconstructed block, reconstructed sample array) by adding the acquired residual signal to the predicted signal (predicted block, predicted sample array) output from the inter-prediction unit 260 or intra-prediction unit 265. If there is no residual in the block to be processed, such as when skip mode is applied, the predicted block can be used as the reconstructed block.

[0055] The addition unit 235 can be called the restoration unit or restoration block generation unit. The generated restoration signal can be used for intra-prediction of the next block to be processed in the current picture, and can also be used for intra-prediction of the next picture after filtering, as described later.

[0056] The filtering unit 240 can improve subjective / objective image quality by applying filtering to the restored signal. For example, the filtering unit 240 can apply various filtering methods to the restored picture to generate a modified restored picture, and can transfer the modified restored picture to the memory 250, specifically to the DPB of the memory 250. The various filtering methods can include, for example, deblocking filtering, sample adaptive offset, adaptive loop filter, and bilateral filter.

[0057] The restored picture stored (modified) in the DPB of memory 250 can be used as a reference picture by the inter-prediction unit 260. Memory 250 can store motion information of blocks from which motion information in the current picture has been derived (or decoded) and / or motion information of blocks in already restored pictures. The stored motion information can be transmitted to the inter-prediction unit 260 for use as motion information of spatially surrounding blocks or motion information of temporally surrounding blocks. Memory 250 can store restored samples of restored blocks in the current picture and transmit them to the intra-prediction unit 265.

[0058] In this specification, the embodiments described for the filtering unit 160, inter-prediction unit 180, and intra-prediction unit 185 of the encoding device 100 can be applied identically or in a corresponding manner to the filtering unit 240, inter-prediction unit 260, and intra-prediction unit 265 of the decoding device 200, respectively.

[0059] Block Partitioning

[0060] The video coding methods described herein can be carried out based on various detailed techniques, which are outlined below. It will be obvious to those skilled in the art that the techniques described below can be related to the aforementioned and / or later video encoding / decoding procedures, such as prediction, residual processing (inverse transformation, inverse quantization, etc.), coding of syntactic elements, filtering, and partitioning.

[0061] The block partitioning procedure described herein is performed in the video splitting unit 110 of the encoding device described above, and the partitioning-related information is processed (encoded) by the entropy encoding unit 190 and transmitted to the decoding device in the form of a bitstream. The entropy decoding unit 210 of the decoding device derives the block partitioning structure of the current picture based on the partitioning-related information obtained from the bitstream, and can then perform a series of procedures for video decoding (e.g., prediction, residual processing, block reconstruction, in-loop filtering, etc.) based on this.

[0062] Partitioning of picture into CTUs

[0063] A picture can be divided into a sequence of coding tree units (CTUs). A CTU can correspond to a coding tree block (CTB). Alternatively, a CTU can include a coding tree block for a luma sample and two corresponding coding tree blocks for chroma samples. That is, for a picture containing three sample arrays, a CTU can include an NxN block for the luma sample and two corresponding blocks for the chroma sample.

[0064] The maximum allowable size of a CTU for coding and prediction may differ from the maximum allowable size of a CTU for transformation. For example, the maximum allowable size of a luma block within a CTU may be 128x128.

[0065] Partitioning of the CTUs using a tree structure

[0066] A Coding Tree Unit (CTU) can be partitioned into Cubic Units (CUs) based on a Quad-Tree (QT) structure. A Quad-Tree structure can also be called a Quaternary Tree structure because it reflects various local characteristics. However, in this document, a CTU can be partitioned based on a multi-type tree structure that includes not only Quad-Tree structures but also Binary-Tree (BT) and Ternary-Tree (TT) structures. Hereinafter, a QTBT structure can include partition structures based on Quad-Tree and Binary-Tree, and a QTBTTT can include partition structures based on Quad-Tree, Binary-Tree, and Ternary-Tree. Alternatively, a QTBT structure can also include partition structures based on Quad-Tree, Binary-Tree, and Ternary-Tree. In a coding tree structure, a CU can have a square or rectangular shape. A CTU can first be partitioned into a Quad-Tree structure. Subsequently, the leaf nodes of the Quad-Tree structure can be further partitioned by a multi-type tree structure.

[0067] Figure 3 shows an example of a multi-type tree structure as an embodiment to which the present invention can be applied.

[0068] In one embodiment of the present invention, the multi-type tree structure may include four splitting types as shown in Figure 3. These four splitting types may include vertical binary splitting (SPLIT_BT_VER), horizontal binary splitting (SPLIT_BT_HOR), vertical ternary splitting (SPLIT_TT_VER), and horizontal ternary splitting (SPLIT_TT_HOR). The leaf nodes of the multi-type tree structure may be called CUs. Such CUs may be used for prediction and transformation procedures. In general, CUs, PUs, and TUs may have the same block size. However, CUs and TUs may have different block sizes if the maximum supported transform length is smaller than the width or height of the color component of the CU.

[0069] Figure 4 illustrates a signaling mechanism for partition information in a quadtree with nested multi-type tree structure, as an embodiment to which the present invention can be applied.

[0070] Here, the CTU is treated as the root of the quadtree and is partitioned for the first time in the quadtree structure. Each leaf node of the quadtree can subsequently be further partitioned in a multi-type tree structure. In a multi-type tree structure, a first flag (e.g., mtt_split_cu_flag) is signaled to indicate whether the node will be further partitioned. If the node is further partitioned, a second flag (e.g., mtt_split_cu_verticla_flag) may be signaled to indicate the splitting direction. Subsequently, a third flag (e.g., mtt_split_cu_binary_flag) may be signaled to indicate whether the splitting type is bipartite or ternary. For example, based on the above mtt_split_cu_vertical_flag and mtt_split_cu_binary_flag, the multi-type tree splitting mode (MttSplitMode) of the CU can be derived as shown in Table 1 below.

[0071] [Table 1]

[0072] Figure 5 illustrates an embodiment to which the present invention can be applied, illustrating a method for dividing a CTU into multiple CUs based on a quadtree and nested multi-type tree structure.

[0073] Here, bold block edges indicate quadtree partitioning, while the remaining edges indicate multi-type tree partitioning. Quadtree partitions with multi-type trees can provide a content-adapted coding tree structure. A CU can correspond to a coding block (CB). Alternatively, a CU can contain a coding block for a luma sample and two coding blocks for a corresponding chroma sample. The size of a CU can be as large as a CTU, or as small as 4x4 per luma sample. For example, in a 4:2:0 color format (or chroma format), the maximum chroma CB size can be 64x64, and the minimum chroma CB size can be 2x2.

[0074] In this document, for example, the maximum allowable luma TB size may be 64x64, and the maximum allowable chroma TB size may be 32x32. If the width or height of a CB divided along the above tree structure is greater than the maximum transformation width or height, the CB may be automatically (or implicitly) divided until the horizontal and vertical TB size limits are met.

[0075] On the other hand, for a quadtree coding tree scheme with multi-type trees, the following parameters can be defined and identified in SPS syntax elements.

[0076] -CTU size: Size of the root node of a quaternary tree.

[0077] -MinQTSize: Minimum allowable size of a quaternary tree leaf node

[0078] - MaxBtSize: Maximum allowable size of the binary tree root node.

[0079] - MaxTtSize: Maximum allowable size of the root node of a ternary tree.

[0080] -MaxMttDepth: The maximum allowable depth of layers in a multi-type tree divided by quadtree leaves.

[0081] -MinBtSize: Minimum allowable size of a binary tree leaf node

[0082] -MinTtSize: Maximum allowable size of a ternary tree leaf node

[0083] As an example of a quadtree coding tree structure with a multi-type tree, the CTU size can be set to a 64x64 block of 128x128 chroma samples and two corresponding chroma samples (in a 4:2:0 chroma format). In this case, MinOTSize can be set to 16x16, MaxBtSize to 128x128, MaxTtSzie to 64x64, MinBtSize and MinTtSize (for both width and height) to 4x4, and MaxMttDepth to 4. Quadtree partitioning can be applied to the CTU to generate leaf nodes of the quadtree. Leaf nodes of a quadtree can be called leaf QT nodes. Leaf nodes of a quadtree can have sizes ranging from 16x16 (i.e., the MinOTSize) to 128x128 (i.e., the CTU size). If a leaf QT node is 128x128, it may not be further divided into a binary / ternary tree. This is because, in this case, even if it were divided, it would exceed MaxBtsize and MaxTtzie (i.e., 64x64). Otherwise, the leaf QT node can be further divided into a multitype tree. Thus, the leaf QT node is the root node of the multitype tree, and the leaf QT node can have a multitype tree depth (mttDepth) of 0. If the multitype tree depth reaches MaxMttdepth (e.g., 4), further divisions may not be considered. If the width of a multitype tree node is the same as MinBtSize and less than or equal to 2xMinTtSize, further horizontal divisions may not be considered. If the height of a multitype tree node is the same as MinBtSize and less than or equal to 2xMinTtSize, further vertical divisions may not be considered.

[0084] Figure 6 illustrates an embodiment to which the present invention can be applied, illustrating a method for restricting ternary-tree partitioning.

[0085] Referring to Figure 6, TT division can be restricted in certain cases to enable the design of 64x64 chroma blocks and 32x32 chroma pipelines in hardware decoders. For example, if the width or height of a chroma coding block is greater than a predetermined value (e.g., 32, 64), TT division can be restricted as shown in Figure 6.

[0086] In this document, the coding tree scheme can support the idea that luma and chroma blocks have separate block tree structures. For P and B slices, within a single CTU, luma and chroma CTBs can be restricted to having the same coding tree structure. However, for I slices, luma and chroma blocks can have separate block tree structures from each other. When the separate block tree mode is applied, a luma CTB can be divided into CUs based on a particular coding tree structure, and a chroma CTB can be divided into chroma CUs based on other coding tree structures. This means that a CU within an I slice may consist of coding blocks for luma components or coding blocks for two chroma components, while a CU in a P or B slice may consist of blocks for three color components.

[0087] In the previously mentioned “Partitioning of the CTUs using a tree structure,” a quadtree coding tree structure with a multitype tree was described, but the structures through which CUs are partitioned are not limited to this. For example, BT and TT structures can be interpreted as concepts included in the structure of a Multiple Partitioning Tree (MPT), and CUs can be interpreted as being partitioned via QT and MPT structures. In one example of CUs being partitioned via QT and MPT structures, the partition structure can be determined by signaling syntactic elements containing information about how many blocks the leaf nodes of the QT structure are partitioned into (e.g., MPT_split_type) and syntactic elements containing information about whether the leaf nodes of the QT structure are partitioned vertically or horizontally (e.g., MPT_split_mode).

[0088] In other examples, the CU can be divided in ways other than the QT, BT, or TT structures. That is, unlike the QT structure in which the lower-depth CU is divided to 1 / 4 the size of the upper-depth CU, or the BT structure in which the lower-depth CU is divided to 1 / 2 the size of the upper-depth CU, or the TT structure in which the lower-depth CU is divided to 1 / 4 or 1 / 2 the size of the upper-depth CU, the lower-depth CU can, depending on the case, be divided to 1 / 5, 1 / 3, 3 / 8, 3 / 5, 2 / 3, or 5 / 8 the size of the parent-depth CU, and the way in which the CU is divided is not limited to this.

[0089] If a portion of a tree node block exceeds the bottom or right picture boundary, that tree node block can be restricted so that all samples of all coded CUs are located within the aforementioned picture boundary. In this case, for example, the following partitioning rule may be applied:

[0090] - If a portion of a tree node block extends beyond both the left and right picture boundaries,

[0091] - If the above block is a QT node and its size is greater than the minimum QT size, the above block will be forcibly partitioned in QT partitioning mode.

[0092] - Otherwise, the above block will be forced to split in Horizontal Binary Splitting (SPLIT_BT_HOR) mode.

[0093] - Instead, if a portion of the tree node block crosses the lower picture boundary,

[0094] - If the above block is a QT node, and the size of the above block is greater than the minimum QT size and the size of the above block is greater than the maximum BT size, then the above block will be forcibly partitioned in QT partitioning mode.

[0095] - If the above block is a QT node, and the size of the above block is greater than the minimum QT size and the size of the above block is less than or equal to the maximum BT size, then the above block is forced to split in QT splitting mode or Horizontal Binary Splitting (SPLIT_BT_HOR) mode.

[0096] - Otherwise (if the above block is a BTT node or the size of the above block is less than or equal to the minimum QT size), the above block is forced to split in Horizontal Binary Splitting (SPLIT_BT_HOR) mode.

[0097] - Instead, if a portion of the tree node block extends beyond the entire right-hand picture boundary,

[0098] - If the above block is a QT node, and the size of the above block is greater than the minimum QT size and the size of the above block is greater than the maximum BT size, then the above block will be forcibly partitioned in QT partitioning mode.

[0099] - If the above block is a QT node, and the size of the block is greater than the minimum QT size, and the size of the above block is less than or equal to the maximum BT size, then the above block is forced to split in QT splitting mode or Vertical Binary Splitting (SPLIT_BT_VER) mode.

[0100] - Otherwise (if the above block is a BTT node or if the size of the above block is less than or equal to the minimum QT size), the above block is forced to split in Vertical Binary Splitting (SPLIT_BT_VER) mode.

[0101] On the other hand, the quadtree coding block structure with the aforementioned multi-type trees can provide a highly flexible block partitioning structure. Because multi-type trees are supported by the partitioning type, other partitioning patterns may, in some cases, result in potentially the same coding block structure. By limiting the occurrence of such redundant partitioning patterns, the amount of data in the partitioning information can be reduced. This is illustrated in the diagram below.

[0102] Figure 7 illustrates redundant partitioning patterns that can occur in binary tree partitioning and ternary tree partitioning as embodiments to which the present invention can be applied.

[0103] As shown in Figure 7, two levels of consecutive binary splits in one direction have the same coding block structure as a binary split of the central partition after a triplicate. In such cases, the binary tree split of the central partition of a ternary tree split (in the given direction) can be restricted. Such restrictions can be applied to the CU of all pictures. When such a particular split is restricted, the signaling of the corresponding syntactic element can be modified to reflect this restriction, thereby reducing the number of bits signaled for partitioning. For example, if the binary tree split of the central partition of a CU is restricted, as in the example shown in Figure 7, the mtt_split_cu_binary_flag syntactic element, which indicates whether the split is a triplicate or a tertiary split, is not signaled, and its value may be inferred as 0 by the decoder.

[0104] prediction

[0105] To restore the current processing unit in which decoding is performed, the decoded portion of the current picture or another picture containing the current processing unit can be used.

[0106] A picture (slice) that uses only the current picture for reconstruction, i.e., performs only in-screen prediction, can be called an intra-picture or I-picture (slice); a picture (slice) that uses up to one motion vector and reference index to predict each unit can be called a predictive picture or P-picture (slice); and a picture (slice) that uses up to two motion vectors and reference indexes can be called a bi-predictive picture or B-picture (slice).

[0107] Intra-prediction refers to a prediction method that derives the current processing block from data elements (e.g., sample values) of the same decoded picture (or slice). In other words, it refers to a method of predicting the pixel values ​​of the current processing block by referring to the restored region within the current picture.

[0108] Below, we will examine the interpretation forecast in more detail.

[0109] Interpretation (or screen-to-screen prediction)

[0110] Interpretation refers to a prediction method that derives the current processing block based on data elements (e.g., sample values ​​or motion vectors) from pictures other than the current picture. In other words, it refers to a method of predicting the pixel values ​​of the current processing block by referring to restored regions in other restored pictures other than the current picture.

[0111] Inter-picture prediction (or inter-picture prediction) is a technique that removes redundancy (overlap) between pictures, and is mostly done through motion estimation and motion compensation.

[0112] This invention describes in detail the techniques of the interpretation method previously explained in Figures 1 and 2. In the case of a decoder, this can be shown in Figure 10, which describes the interpretation-based video / image decoding method, and in Figure 11, which describes the interpretation unit within the decoding device. In addition, in the case of an encoder, this can be shown in Figure 8, which describes the interpretation-based video / image encoding method, and in Figure 9, which describes the interpretation unit within the encoding device. Furthermore, the data encoded by Figures 8 and 9 can be stored in the form of a bitstream.

[0113] The prediction unit of the encoding / decoding device can perform interpretation on a block-by-block basis to derive predicted samples. Interpretation can indicate predictions derived in a way that depends on data elements of pictures other than the current picture (e.g., sample values ​​or motion information). When interpretation is applied to the current block, the predicted block (predicted sample array) of the current block can be derived based on the reference block (reference sample array) identified by the motion vector on the reference picture indicated by the index of the reference picture.

[0114] In this case, in order to reduce the amount of motion information transferred in inter-prediction mode, the motion information of the current block can be predicted at the block, sub-block, or sample level based on the correlation of motion information between the surrounding block and the current block. The motion information may include motion vectors and reference picture indices. The motion information may further include information on the inter-prediction type (L0 prediction, L1 prediction, Bi prediction, etc.).

[0115] When interpretation is applied, surrounding blocks can include spatial neighboring blocks present in the current picture and temporal neighboring blocks present in the reference picture. The reference picture containing the above reference block and the reference picture containing the above temporal neighboring block may be the same or different. The above temporal neighboring block may be called a collocated reference block or collocated CU (colCU), and the reference picture containing the above temporal neighboring block may be called a collocated picture (colPic). For example, a list of motion information candidates may be constructed based on the surrounding blocks of the current block, and flags or index information may be signaled to indicate which candidates are selected (used) to derive the motion vector of the current block and / or the index of the reference picture.

[0116] Interpretation can be performed based on various prediction modes. For example, in skip mode and merge mode, the motion information of the current block may be the same as the motion information of the selected surrounding blocks. In skip mode, unlike merge mode, the residual signal may not be transmitted. In Motion Vector Prediction (MVP) mode, the motion vector of the selected surrounding block can be used as a motion vector predictor, and the motion vector difference can be signaled. In this case, the motion vector of the current block can be derived using the sum of the motion vector predictor and the motion vector difference.

[0117] Figures 8 and 9 illustrate an interpretation-based video / image encoding method and an interpretation unit within an encoding device according to an embodiment of the present invention.

[0118] Referring to Figures 8 and 9, S801 can be performed by the inter-prediction unit 180 of the encoding device, and S802 can be performed by the residual processing unit of the encoding device. Specifically, S802 can be performed by the subtraction unit 115 of the encoding device. In S803, the prediction information can be derived by the inter-prediction unit 180 and encoded by the entropy encoding unit 190. In S803, the residual information can be derived by the residual processing unit and encoded by the entropy encoding unit 190. The residual information is information about the residual sample. The residual information may include information about the quantized transformation coefficients for the residual sample.

[0119] As described above, the residual sample can be derived as a conversion coefficient via the conversion unit 120 of the encoding device, and the conversion coefficient can be derived as a quantized conversion coefficient via the quantization unit 130. Information regarding the quantized conversion coefficient can be encoded in the entropy encoding unit 190 via the residual coding procedure.

[0120] The encoding device performs interpretation of the current block (S801). The encoding device can derive the interpretation mode and motion information of the current block and generate prediction samples of the current block. Here, the procedures for determining the interpretation mode, deriving motion information, and generating prediction samples may be performed simultaneously, or one of the procedures may be performed before the others. For example, the interpretation unit 180 of the encoding device may include a prediction mode determination unit 181, a motion information derivation unit 182, and a prediction sample derivation unit 183. The prediction mode determination unit 181 determines the prediction mode of the current block, the motion information derivation unit 182 derives motion information of the current block, and the prediction sample derivation unit 183 derives motion samples of the current block.

[0121] For example, the interpretation unit 180 of the encoding device can search for blocks similar to the current block within a certain area (search area) of the reference picture via motion estimation, and derive reference blocks whose difference from the current block is minimal or below a certain standard. Based on this, it can derive the index of the reference picture that points to the reference picture where the reference block is located, and derive a motion vector based on the difference in position between the reference block and the current block. The encoding device can determine which of the various prediction modes is to be applied to the current block. The encoding device can compare the RD costs for the various prediction modes and determine the optimal prediction mode for the current block.

[0122] For example, if skip mode or merge mode is applied to the current block, the encoding device can configure a merge candidate list, as described later, and derive a reference block from among the reference blocks pointed to by the merge candidates included in the merge candidate list whose difference from the current block is the minimum or below a certain standard. In this case, the merge candidate associated with the derived reference block is selected, merge index information pointing to the selected merge candidate is generated, and this can be signaled to the decoding device. The movement information of the current block may be derived using the movement information of the selected merge candidate.

[0123] As another example, when the (A)MVP mode is applied to the current block, the encoding device can configure the (A)MVP candidate list described later, and use the motion vector of the selected mvp candidate from among the mvp (motion vector predictor) candidates included in the (A)MVP candidate list as the mvp of the current block. In this case, for example, the motion vector pointing to the reference block derived by the motion estimation described above can be used as the motion vector of the current block, and the mvp candidate having the motion vector with the smallest difference from the motion vector of the current block can become the selected mvp candidate. The MVD (Motion Vector Difference), which is the difference obtained by subtracting the mvp from the motion vector of the current block, can be derived. In this case, information regarding the MVD can be signaled to the decoding device. Also, when the (A)MVP mode is applied, the value of the reference picture index is composed of reference picture index information and can be separately signaled to the decoding device.

[0124] The encoding device can derive the residual sample based on the predicted sample (S802). The encoding device can derive the residual sample by comparing the original sample of the current block with the predicted sample.

[0125] The encoding device encodes video information including prediction information and residual information (S803). The encoding device can output the encoded video information in the form of a bitstream. The prediction information is information related to the prediction procedure and may include prediction mode information (e.g., skip flag, merge flag or mode index) and motion information. The motion information may include candidate selection information (e.g., merge index, mvp flag or mvp index) which is information for deriving a motion vector. The motion information may also include the aforementioned MVD information and / or reference picture index information.

[0126] Furthermore, the motion information may include information indicating whether L0 prediction, L1 prediction, or bidirectional (bi) prediction is applied. The residual information is information about the residual sample. The residual information may include information about the quantized transformation coefficients for the residual sample.

[0127] The output bitstream can be stored in a (digital) storage medium and transmitted to a decoding device, or it can be transmitted to a decoding device via a network.

[0128] On the other hand, as mentioned above, the encoding device can generate a reconstructed picture (including reconstructed samples and reconstructed blocks) based on the above-mentioned reference samples and residual samples. This is because the encoding device can derive the same prediction results as the decoding device, thereby improving coding efficiency. Therefore, the encoding device can store the reconstructed picture (or reconstructed samples, reconstructed blocks) in memory and use it as a reference picture for interpretation. As mentioned above, in-loop filtering procedures and the like can be further applied to the above-mentioned reconstructed picture.

[0129] Figures 10 and 11 illustrate an interprediction-based video / image decoding method according to an embodiment of the present invention, and an interprediction unit within a decoding device according to an embodiment of the present invention.

[0130] Referring to Figures 10 and 11, the decoding device can perform operations corresponding to those performed by the encoding device. Based on the received prediction information, the decoding device can make predictions for the current block and derive prediction samples.

[0131] S1001 to S1003 can be performed by the interpretation unit 260 of the decoding device, and the residual information in S1004 can be obtained from the bitstream by the entropy decoding unit 210 of the decoding device. The residual processing unit of the decoding device can derive the residual sample of the current block based on the residual information. Specifically, the inverse quantization unit 220 of the residual processing unit performs inverse quantization to derive the conversion coefficients based on the quantized conversion coefficients derived from the residual information, and the inverse transformation unit 230 of the residual processing unit performs inverse transformation of the conversion coefficients to derive the residual sample of the current block. S1005 can be performed by the addition unit 235 or the restoration unit of the decoding device.

[0132] Specifically, the decoding device can determine the prediction mode for the current block based on the received prediction information (S1001). Based on the prediction mode information in the prediction information, the decoding device can determine which inter-prediction mode is applied to the current block.

[0133] For example, based on the merge flag above, it can be determined whether the merge mode above is applied to the current block, or whether the (A)MVP mode is determined. Alternatively, based on the mode index above, one of the various inter-prediction mode candidates can be selected. The inter-prediction mode candidates above may include skip mode, merge mode and / or (A)MVP mode, or may include the various inter-prediction modes described below.

[0134] The decoding device derives motion information for the current block based on the determined inter prediction mode (S1002). For example, if a skip mode or merge mode is applied to the current block, the decoding device can configure a merge candidate list, as described later, and select one merge candidate from among the merge candidates included in the merge candidate list. This selection can be performed based on the selection information (merge index) described above. Motion information for the current block can be derived using the motion information of the selected merge candidate. Motion information for the selected merge candidate can be used as motion information for the current block.

[0135] As another example, when the (A)MVP mode is applied to the current block, the decoding device can configure the (A)MVP candidate list described later, and use the motion vector of the selected mvp candidate from the mvp (motion vector predictor) candidates included in the (A)MVP candidate list as the mvp of the current block. The above selection can be performed based on the selection information (mvp flag or mvp index) described above. In this case, the MVD of the current block can be derived based on the information about the MVD, and the motion vector of the current block can be derived based on the mvp of the current block and the MVD. In addition, the reference picture index of the current block can be derived based on the index information of the reference picture. Within the reference picture list for the current block, the picture pointed to by the index of the reference picture can be derived as the reference picture referenced for interpretation of the current block.

[0136] On the other hand, as will be described later, the movement information of the current block can be derived without constructing a candidate list. In this case, the movement information of the current block can be derived by following the procedure initiated in the prediction mode described later. In this case, the construction of the candidate list as described above can be omitted.

[0137] The decoding device can generate predicted samples of the current block based on the motion information of the current block (S1003). In this case, the reference picture can be derived based on the reference picture index of the current block, and the predicted samples of the current block can be derived using the sample of the reference block pointed to on the reference picture by the motion vector of the current block. In this case, as will be described later, a filtering procedure can be performed on all or some of the predicted samples of the current block, depending on the circumstances.

[0138] For example, the interpretation unit 260 of the decoding device may include a prediction mode determination unit 261, a motion information derivation unit 262, and a prediction sample derivation unit 263. The prediction mode determination unit 261 determines the prediction mode of the current block based on the prediction mode information received, derives motion information of the current block (such as motion vectors and / or reference picture indices) based on the motion information information received from the motion information derivation unit 262, and derives prediction samples of the current block in the prediction sample derivation unit 263.

[0139] The decoding device generates residual samples of the current block based on the received residual information (S1004). Based on the predicted samples and the residual samples, the decoding device can generate restored samples of the current block and generate a restored picture based on these (S1005). As previously mentioned, in-loop filtering procedures and the like may be further applied to the restored picture thereafter.

[0140] As mentioned above, the interpretation procedure may include the steps of determining the interpretation mode, deriving motion information corresponding to the determined prediction mode, and performing predictions (generating prediction samples) based on the derived motion information.

[0141] Interpretation mode determination

[0142] Various interpretation modes are used to predict the current block within a picture. For example, various modes such as merge mode, skip mode, MVP mode, and affine mode are used. In DMVR (Decoder side Motion Vector Refinement) mode, AMVR (Adaptive Motion Vector Resolution) mode and others can be used as auxiliary modes. Affine mode is sometimes called affine motion prediction mode. MVP mode is sometimes called AMVP (Advanced Motion Vector Prediction) mode.

[0143] Prediction mode information, indicating the inter-prediction mode of the current block, can be signaled from the encoding device to the decoding device. This prediction mode information can be included in the bitstream and received by the decoding device. This prediction mode information may include index information indicating one of a number of candidate modes. Alternatively, the inter-prediction mode can be indicated via hierarchical signaling of flag information. In this case, the prediction mode information may include multiple flags.

[0144] For example, a skip flag can be signaled to indicate whether or not skip mode can be applied, a merge flag can be signaled if skip mode cannot be applied to indicate whether or not merge mode can be applied, and if merge mode cannot be applied, MVP mode can be applied. Additional flags can also be signaled for further distinctions. Affine mode can be signaled in an independent mode, or it can be signaled in a mode that is subordinate to merge mode or MVP mode, etc. For example, affine mode can be configured as one of the candidates in a merge candidate list or MVP candidate list, as will be described later.

[0145] Derivation of motion information according to interpretation mode

[0146] Interpretation can be performed using the motion information of the current block. The encoding device can derive optimal motion information for the current block through a motion estimation procedure. For example, the encoding device can use the original block in the original picture of the current block to search for highly correlated similar reference blocks in fractional pixel units within a defined search range in the reference picture, thereby deriving motion information. Block similarity can be derived based on the difference in phase-based sample values. For example, block similarity can be calculated based on the SAD between the current block (or the template of the current block) and the reference block (or the template of the reference block). In this case, motion information can be derived based on the reference block with the smallest SAD in the search space. The derived motion information can be signaled to the decoding device in various ways based on the interpretation mode.

[0147] Merge mode and skip mode

[0148] Figure 12 illustrates a peripheral block used in merge mode or skip mode as an embodiment to which the present invention is applied.

[0149] When merge mode is applied, the movement information of the current prediction block is not directly transferred; instead, the movement information of the surrounding prediction blocks is used to derive the movement information of the current prediction block. Therefore, the movement information of the current prediction block can be indicated by transferring a flag that indicates that merge mode has been used and a merge index that indicates which surrounding prediction block was used.

[0150] The encoder can search for merge candidate blocks to use in order to perform a merge mode, in order to derive motion information for the current predicted block. For example, up to five merge candidate blocks may be used, but the present invention is not limited thereto. The maximum number of merge candidate blocks may be transmitted in the slice header (or tile group header), but the present invention is not limited thereto. After finding the merge candidate blocks, the encoder can generate a merge candidate list and select the merge candidate block with the lowest cost among them as the final merge candidate block.

[0151] The present invention provides various embodiments of merge candidate blocks that constitute the merge candidate list described above.

[0152] The above list of merge candidates can, for example, use five merge candidate blocks. For example, it can use four spatial merge candidates and one temporal merge candidate. As a specific example, in the case of spatial merge candidates, the blocks shown in Figure 12 can be used as spatial merge candidates.

[0153] Figure 13 is a flowchart illustrating a method for configuring a merge candidate list according to one embodiment to which the present invention is applied.

[0154] Referring to Figure 13, the coding device (encoder / decoder) searches for spatially surrounding blocks of the current block and inserts the derived spatial merge candidates into the merge candidate list (S1301). For example, the spatially surrounding blocks may include the block around the lower left corner, the left side, the upper right corner, the top side, and the upper left corner of the current block. However, as an example, additional surrounding blocks such as the right side, bottom side, and lower right side can also be used as spatially surrounding blocks. The coding device searches for the spatially surrounding blocks based on priority to detect usable blocks and derives the movement information of the detected blocks as spatial merge candidates. For example, the encoder and decoder can search the five blocks shown in Figure 12 in the order A1, B1, B0, A0, B2, and sequentially index the usable candidates to form a merge candidate list.

[0155] The coding device searches for temporally surrounding blocks of the current block and inserts the derived temporal merge candidate into the merge candidate list (S1302). The temporally surrounding block can be located on a reference picture that is a different picture from the current picture on which the current block is located. The reference picture on which the temporally surrounding block is located can be called a collocated picture or col picture. The temporally surrounding block can be searched for in the order of the lower right corner block and the lower right center block of the co-located block at the same position as the current block on the col picture.

[0156] On the other hand, when motion data compression is applied, specific motion information can be stored as representative motion information for each certain storage unit in the col picture. In this case, it is not necessary to store the motion information of all blocks within the certain storage unit, thereby achieving the motion data compression effect. In this case, the certain storage unit may be predetermined, for example, in units of 16x16 samples or 8x8 samples, or the size information of the certain (scheduled) storage unit may be signaled from the encoder to the decoder. When motion data compression is applied, the motion information of the temporally surrounding blocks can be replaced with the representative motion information of the certain storage unit in which the temporally surrounding blocks are located.

[0157] In other words, in this case, from an implementation standpoint, the temporal merge candidate can be derived not from a predicted block located at the coordinates of the temporal peripheral block, but from motion information of a predicted block covering a position obtained by arithmetically shifting to the right by a certain value and then arithmetically shifting to the left based on the coordinates of the temporal peripheral block (top-left sample position). For example, if the constant storage unit is 2nx2n sample units, and the coordinates of the temporal peripheral block are (xTnb, yTnb), then motion information of a predicted block located at the modified position ((xTnb >> n)<< n), (yTnb >> n)<< n) can be used for the temporal merge candidate.

[0158] Specifically, for example, if the above-mentioned storage unit is 16x16 sample units, and the coordinates of the temporally surrounding block are (xTnb, yTnb), then the motion information of the predicted block located at the corrected position ((xTnb >> 4)<< 4), (yTnb >> 4)<< 4)) can be used for the above-mentioned temporal merge candidate. Alternatively, for example, if the above-mentioned storage unit is 8x8 sample units, and the coordinates of the temporally surrounding block are (xTnb, yTnb), then the motion information of the predicted block located at the corrected position ((xTnb >> 3)<< 3), (yTnb>> 3)<< 3)) can be used for the above-mentioned temporal merge candidate.

[0159] The coding device can check whether the current number of merge candidates is less than the maximum number of merge candidates (S1303). The maximum number of merge candidates can be predefined or signaled from the encoder to the decoder. For example, the encoder can generate and encode information about the maximum number of merge candidates and transmit it to the decoder in the form of a bitstream. If the maximum number of merge candidates is fully met, the process of adding further candidates may not proceed.

[0160] If, as a result of the above check, the number of current merge candidates is less than the number of maximum merge candidates, the coding device inserts additional merge candidates into the merge candidate list (S1304). These additional merge candidates may include, for example, ATMVP, combined bi-predictive merge candidates (if the slice type of the current slice is type B), and / or zero vector merge candidates.

[0161] If, as a result of the above check, the number of current merge candidates is not less than the maximum number of merge candidates, the coding device can terminate the construction of the merge candidate list. In this case, the encoder can select the best merge candidate from among the merge candidates constituting the merge candidate list based on the RD (Rate-Distortion) cost, and can signal selection information (e.g., merge index) pointing to the selected merge candidate to the decoder. The decoder can select the best merge candidate based on the merge candidate list and the selection information.

[0162] As previously mentioned, the motion information of the selected merge candidate can be used as the motion information of the current block, and based on the motion information of the current block, predicted samples of the current block can be derived. The encoder can derive residual samples of the current block based on the predicted samples and signal residual information regarding the residual samples to the decoder. As previously mentioned, the decoder can generate reconstructed samples based on the residual samples derived based on the residual information and the predicted samples, and generate a reconstructed picture based on these.

[0163] When skip mode is applied, the motion information of the current block can be derived in the same way as when the previous merge mode is applied. However, when skip mode is applied, the residual signal of that block is omitted, and therefore, the predicted sample can be used directly as the reconstructed sample.

[0164] MVP Mode

[0165] Figure 14 is a flowchart illustrating a method for configuring a merge candidate list according to one embodiment to which the present invention is applied.

[0166] When MVP (Motion Vector Prediction) mode is applied, a candidate list of motion vector predictors (MVP) can be generated using the motion vectors of the reconstructed spatially surrounding blocks (which may be, for example, the surrounding blocks described in Figure 12 above) and / or the motion vectors corresponding to the temporally surrounding blocks (or Col blocks). In other words, the motion vectors of the reconstructed spatially surrounding blocks and / or the motion vectors corresponding to the temporally surrounding blocks can be used as motion vector predictor candidates.

[0167] The information regarding the above prediction may include selection information (e.g., an MVP flag or MVP index) that indicates the selected optimal motion vector predictor candidate from among the motion vector predictor candidates included in the above list. In this case, the prediction unit can use the above selection information to select the motion vector predictor for the current block from among the motion vector predictor candidates included in the motion vector candidate list. The prediction unit of the encoding device can calculate the difference in motion vectors (MVD) between the motion vector of the current block and the motion vector predictor, encode this, and output it in the form of a bitstream. That is, the MVD can be obtained by subtracting the motion vector predictor from the motion vector of the current block. In this case, the prediction unit of the decoding device can obtain the difference in motion vectors included in the information regarding the above prediction, and derive the motion vector of the current block by adding the difference in motion vectors and the motion vector predictor. The prediction unit of the decoding device can obtain or derive the index of a reference picture that indicates a reference picture from the information regarding the above prediction. For example, the motion vector predictor candidate list can be configured as shown in Figure 14.

[0168] ATMVP(Advanced Temporal Motion Vector Prediction)

[0169] Figures 15 and 16 illustrate a method for deriving an ATMVP (Advanced Temporal Motion Vector Prediction) candidate as one embodiment to which the present invention is applied.

[0170] Referring to Figure 15, ATMVP is a method for deriving motion information of subblocks of a coding unit based on motion information of collocated blocks of temporally adjacent pictures. Through this, the performance of Temporal Motion Vector Prediction (TMVP) can be improved, and the complexity of general or worst-case cases can be reduced. In this invention, ATMVP is sometimes referred to as subblock-based temporal merging candidate, or SbTMVP.

[0171] In one embodiment of the present invention, ATMVP can be derived by the following process:

[0172] First, the encoder / decoder can add motion vectors from spatially adjacent coding units if adjacent coding units are available and the motion vectors of the coding units used differ from the motion vectors in the current candidate list. As an example, referring to Figure 16, the process described above can be executed in the order A1, B1, B0, A0, B2. As another example, for the sake of reducing complexity, the process described above can also derive the ATMVP using only the motion vectors of blocks at fixed positions (e.g., position A1).

[0173] The above encoder / decoder can use the first motion vector candidate out of the No space candidates used to determine the position from which to derive the motion information of the collocated picture and each subblock. Here, No indicates the number of space candidates used. If No is 0, the collocated picture and the collocated position of motion 0 can be used to derive the motion information of each subblock.

[0174] When multiple reference pictures are used, the collocated pictures of different coding units in ATMVP may not be the same. Having different collocated pictures for different coding units within the current picture for ATMVP derivation means that information fields for the movement of multiple reference pictures must be derived, which is undesirable in terms of increasing memory bandwidth.

[0175] Therefore, the present invention provides a further simplified design that uses the same collocated picture when deriving the ATMVP. For example, the method of using the same collocated picture may be defined in the header of the slice (or group of tiles), but the present invention is not limited thereto. As an example, if, at the block level, the reference picture of adjacent block A is different from the collocated picture, the motion vector of adjacent block A can be scaled based on a scaling method for the temporal motion vector. The scaled motion vector of adjacent block A can then be used in the ATMVP.

[0176] Figure 17 illustrates a method for deriving an ATMVP (Advanced Temporal Motion Vector Prediction) candidate as one embodiment to which the present invention is applied.

[0177] Referring to Figure 17, in one embodiment of the present invention, the TMVP, which uses the motion vector of the time-adjacent block (or colPB) at the right-bottom end of the current block or at the center of the current block, does not reflect the movement within the screen. Therefore, the encoder / decoder can use the motion vector of colPB at the position pointed to by the motion vector of the surrounding block as the MVP.

[0178] For example, the encoder / decoder can find the motion vector of the first spatial neighbor block to be used, while checking it along with the configuration procedure of the merge candidate shown in Figure 17. Then, in the reference picture, the position pointed to by the above motion vector can be derived as col-PB (i.e., ATMVP candidate).

[0179] Furthermore, the above motion vectors are used as the motion vectors for the corresponding blocks in each subblock unit. In this case, if a motion vector does not exist for a particular subblock, the motion vector of the center block located at the center of the corresponding block can be used as the motion vector for the unavailable subblock and stored as a representative motion vector.

[0180] Temporal motion vector data storage reduction

[0181] In one embodiment of the present invention, a method is proposed to reduce the storage of temporal motion vectors based on candidate spatial motion vector data in order to compress temporal motion vector data.

[0182] Figures 18 and 19 illustrate a method for compressing temporal motion vector data and the positions of spatial candidates used therein, according to one embodiment to which the present invention is applied.

[0183] Referring to Figure 18, in an embodiment of the present invention, once spatial candidates are predicted by interpretation, the motion vectors of the spatial candidates can be set as default motion vectors for compression. For example, up to five spatial candidates can be used as reference time motion information to derive default time motion vectors. In one embodiment, the above five spatial candidates can be set as shown in Figure 19.

[0184] Furthermore, time-motion vector data can be compressed based on the motion vectors of spatial candidates. The order in which spatial candidates are searched may be the same as that shown in Figure 18. Spatial candidates can be examined in the order of center block (C), upper left block (TL), upper right block (TR), lower left block (BL), and lower right block (BR). This is only one embodiment, and the present invention is not limited thereto, and other possible combinations of orders can be applied.

[0185] First, the encoder / decoder can confirm that the center block (C) has been predicted. If the center block (C) has been predicted, the encoder / decoder can set the motion vector of the center block (C) to the default value for motion vector prediction.

[0186] If the center block (C) is not intercepted, the encoder / decoder can confirm that the upper left block (TL) is intercepted. If the upper left block (TL) is intercepted, the encoder / decoder can set the motion vector of the upper left block (TL) to the default for motion vector prediction.

[0187] If the upper left block (TL) above is not intercepted, the encoder / decoder can confirm that the upper right block (TR) is intercepted. If the upper right block (TR) above is intercepted, the encoder / decoder can set the motion vector of the upper right block (TR) above to the default for motion vector prediction.

[0188] If the upper right block (TR) is not intercepted, the encoder / decoder can confirm that the lower left block (BL) is intercepted. If the lower left block (BL) is intercepted, the encoder / decoder can set the motion vector of the lower left block (BL) to the default for motion vector prediction.

[0189] If the lower left block (BL) above is not intercepted, the encoder / decoder can confirm that the lower right block (BR) is intercepted. If the lower right block (BR) above is intercepted, the encoder / decoder can set the motion vector of the lower right block (BR) above to the default value for motion vector prediction.

[0190] If the upper right block (BR) is not predicted, the encoder / decoder can set the intra mode to the default value.

[0191] Through the process described above, the encoder / decoder can compress the default motion vector into motion information.

[0192] Embodiments that perform ATMVP based on adaptive subblock size

[0193] In one embodiment of the present invention, we propose a method for performing ATMVP based on the size of an adaptive subblock. For example, the size of the subblock used for ATMVP derivation can be adaptively applied at the slice level.

[0194] On the other hand, when ATMVP motion information is derived in units of 4x4 blocks, a problem can arise in that motion derivation and motion compensation are performed within a single ATMVP coding unit for each 4x4 subblock.

[0195] To address this, the encoder can signal a default sub-block size used for ATMVP motion derivation to the decoder at the sequence level.

[0196] As another example, if the default sub-block size is used in the current slice, a flag can be signaled at the picture or slice level. If the above flag is false, the ATMVP sub-block size can be further signaled from the slice header.

[0197] Embodiments that limit the region for derivation of collocated blocks

[0198] In the present invention, the region of the Collocated block for ATMVP may include the current CTU and one column of NxN blocks in the collocated picture. For example, the above NxN block may be a 4x4 block, but the present invention is not limited thereto.

[0199] If an ATMVP Collocated block identified by the motion vector of a merge candidate is located outside the restricted region, it can be moved to be located within the restricted region. For example, it can be moved to be located at the nearest boundary within the restricted region.

[0200] Embodiment for deriving a subblock-based temporal merging candidate

[0201] In one embodiment of the present invention, an encoder / decoder can add motion information of a collocated block (or collocated subblock) within a collocated picture, identified based on motion information of spatially adjacent blocks, to a subblock merging candidate list as a subblock-based temporal merging candidate.

[0202] In this invention, motion information of spatially adjacent blocks can be called a temporal motion vector. In one embodiment, an encoder / decoder can derive subblock-based time merge candidates if the width and height of the current coding block are greater than or equal to a predetermined specific size. For example, the predetermined specific size may be 8.

[0203] As an embodiment, the encoder / decoder can set the motion information of the first spatial candidate among the spatial candidates used as a time-motion vector. For example, the encoder / decoder can search for available spatial candidates in the order A1, B1, B0, A0. In this case, the encoder / decoder can set the spatial candidate that is the same as the collocated picture among the available spatial candidates as a time-motion vector. As another example, the encoder / decoder can check whether a spatial candidate at a fixed position is available, and if so, it can set the motion vector of the candidate in that region as a time-motion vector. For example, the spatial candidate at the fixed position can be set as the block at position A1.

[0204] Furthermore, the encoder / decoder can use the above time-motion vector to determine the position of the collocated block within the collocated picture. As an example, the following equation 1 can be used.

[0205] <Formula 1>

number

[0206] Here, (xColCtrCb, yColCtrCb) indicates the top-left sample position of the collocated coding block containing the sample on the lower right side of the central position, and tempMv indicates the time motion vector mentioned above.

[0207] Furthermore, the encoder / decoder can determine the position from which to derive motion information for each subblock within the current coding block, on a subblock-by-subblock basis. In one embodiment, the position of a collocated subblock within a collocated picture can be derived using the following equation 2.

[0208] <Formula 2>

number

[0209] Here, (xSb, ySb) indicates the current subblock position.

[0210] In one embodiment, if the current collocated subblock is not available, the encoder / decoder may use motion information of a collocated block identified using a time motion vector.

[0211] Figure 20 is a flowchart showing a method for generating an inter-predicted block based on a candidate list according to an embodiment to which the present invention is applied.

[0212] Referring to Figure 20, an example of the interprediction process performed by the interprediction units (180, 260) described in Figures 1 and 2 is shown. In Figure 20, it is assumed that the merge mode and the intermode (i.e., (A)MVP mode) are used as methods for deriving predicted motion information (PMI) and performing predictions via interprediction. However, the present invention is not limited to this, and various other modes can be applied for interprediction.

[0213] As mentioned above, merge mode and intermode can be distinguished depending on whether, in order to derive the final PMI used for generating the prediction block, at least one of the following is transmitted via the bitstream: motion vector difference (MVD), reference list (or reference picture list), and / or reference picture index, in addition to the candidate index.

[0214] Specifically, the encoder / decoder checks whether merge mode is applied to the current block (S2001).

[0215] When merge mode is applied to the current block, the encoder / decoder generates merge candidates (or a list of merge candidates) and selects the PMI used for inter-prediction of the current block (S2002, 2003). When merge mode is applied, the encoder can transfer the selected PMI information to the decoder as a merge index. The merge index can indicate a specific merge candidate from among the merge candidates. The encoder / decoder can use the motion information of the specific merge candidate(s) as a motion vector for inter-prediction of the current block.

[0216] If merge mode is not applied to the current block (i.e., inter-mode is applied), the encoder / decoder generates an inter-candidate (or inter-candidate list) and selects the first PMI (or motion vector predictor) to be used for inter-prediction of the current block (S2004, S2005). If inter-mode is applied, the encoder can transfer the above selected PMI information to the decoder as a motion vector predictor flag (or index). The above motion vector predictor flag can indicate a specific inter-candidate from among the inter-candidates. The above inter-candidate can then be called a motion vector predictor candidate. The encoder / decoder derives the motion vector used for prediction by adding the motion vector difference to the above initial PMI (S2006).

[0217] The encoder / decoder uses the derived motion vector to generate a predicted block for the current block (S2007).

[0218] In other words, when merge mode is applied, the encoder / decoder generates a predicted block for the current block using the motion information of the selected candidate via the candidate index, without using any additional syntax (or syntactic elements) other than signaling the candidate index (i.e., the merge index) to derive the final PMI. In this case, the candidate generation process takes place at stage (or block) S2002.

[0219] Even when intermode is applied, the candidate generation process is performed at stage S2004 in the same way as in merge mode. When intermode is applied, the encoder / decoder receives additional motion information based on the motion information of the selected candidate, derives the final PMI, and uses this to generate the predicted block for the current block.

[0220] The following describes how to generate candidates (or candidate lists) using the movement information of adjacent blocks.

[0221] Figure 21 is a flowchart showing a method for generating a candidate list according to an embodiment to which the present invention is applied.

[0222] Referring to Figure 21, although the explanation will focus on the decoder for convenience, the method for generating the candidate list in this embodiment can be performed in the same way for the encoder. In one embodiment, the method described in Figures 12 to 17 above can be applied, and a redundant explanation of this will be omitted.

[0223] The candidate configuration procedure shown in Figure 21 is just one example, and the present invention is not limited thereto. The order can be changed, and some steps can be omitted or added. Also, the candidates added in Figure 21 are just one example; many other candidates can be added, and some of the candidates shown in Figure 21 may not be used.

[0224] The decoder adds a spatial neighboring block (or spatial candidate) to the candidate list (S2101). For example, the decoder can add the surrounding candidate block shown in Figure 16, described earlier, to the candidate list.

[0225] The decoder adds the temporal neighboring block (or temporal candidate) to the candidate list (S2102).

[0226] The decoder adds the combined motion information (or combined motion vector) to the candidate list and adds the zero motion vector to the candidate list (S2103, S2104).

[0227] Figure 22 shows an example of a peripheral block used in inter prediction according to an embodiment to which the present invention is applied.

[0228] Referring to Figure 22, the encoder / decoder can construct a candidate list using motion information of spatially adjacent blocks based on spatial similarity and motion information of temporally adjacent blocks based on temporal similarity. In this invention, motion information of spatially adjacent blocks can be called spatial adjacent information, and motion information of temporally adjacent blocks can be called temporal adjacent information or corresponding temporal information.

[0229] As an embodiment, the encoder / decoder can construct a candidate list using spatial candidates (or spatial adjacency information) as shown in Figure 22. Furthermore, the encoder / decoder can construct a candidate list using temporal candidates (or corresponding temporal information) as shown in Figure 22.

[0230] As mentioned above, an Adaptive Temporal Motion Vector Prediction (ATMVP) can be used to correct temporal similarity information that takes spatial similarity into account. ATMVP is a method for deriving subblock motion information based on the motion information of collocated blocks of temporally adjacent pictures. This improves the performance of Temporal Motion Vector Prediction (TMVP) and reduces complexity. In this invention, ATMVP may be called a subblock-based temporal merging candidate, SbTMVP, etc.

[0231] In other words, TMVP, which uses the motion vector of the collated block at the right-bottom or center of the current block, does not reflect the movement within the picture. Therefore, the motion vector of the collated block at the position pointed to by the motion vector of the surrounding block is used in the motion vector predictor.

[0232] Referring again to Figure 17 described earlier, in one embodiment, the encoder / decoder can find the motion vector of the first available spatial neighbor block by checking the configuration order of a pre-set merge candidate. Thereafter, the encoder / decoder can determine the collated block in the reference picture where the corresponding motion vector (i.e., the temporal motion vector) is located. This improves the accuracy of the corresponding block (or temporal motion vector). In one embodiment, the above reference picture can be defined as a collocated picture.

[0233] The encoder / decoder can derive the motion vector of a corresponding block identified by a time motion vector on a subblock basis within the current coding block. In one embodiment, if a motion vector corresponding to a particular subblock does not exist (or is unavailable), the encoder / decoder can set (or store) the motion vector of a block located in the center of the collated block as a representative motion vector and use it as the motion vector for the particular subblock.

[0234] In one embodiment, to use ATMVP mode, the encoder / decoder can apply an increased number of merge candidates, and no additional syntax (or syntactic elements) may be used for this purpose. Syntax indicating the maximum number of merge candidates can be transferred from the encoder to the decoder via higher-level syntax. Such higher-level syntax could be, for example, a Sequence Parameter Set (SPS), a Picture Parameter Set (PPS), or a slice header. As an example, the maximum number of merge candidates could be six.

[0235] In one embodiment, ATMVP-ext candidates, which represent extended ATMVP candidates, may also be added to the candidate list. ATMVP-ext, like ATMVP described above, is a method for improving existing TMVP, and can derive motion vectors at the subblock level by combining spatially adjacent blocks and temporally adjacent blocks.

[0236] Figure 23 illustrates a method for deriving an extended ATMVP (Advanced Temporal Motion Vector Prediction) candidate according to an embodiment to which the present invention is applied.

[0237] Referring to Figure 23, we assume that the current block 2301 has a size of 16x16. Also, in Figure 23, when deriving motion vectors in units of subblocks, we assume that the size of the subblock is 4x4, but the present invention is not limited to this, and the size of the subblock can be defined as a fixed size of 8x8 or more, as will be described later.

[0238] The encoder / decoder uses the motion vectors of spatially adjacent blocks in the left horizontal and upper vertical directions, and the motion vectors of temporally adjacent blocks in the right horizontal and lower vertical directions, to derive the motion vectors of each subblock within the current block 2301.

[0239] For example, the encoder / decoder can derive the motion vectors of the spatially adjacent blocks at positions L-0 and A-0, and the motion vectors of the temporally adjacent blocks at positions 1 and 4 within the current block 2301, in order to derive the motion vector for the subblock at position 0 within the current block 2301, and calculate the average value of the four motion vectors. The encoder / decoder can then determine the average value of the above four motion vectors as the motion vector for the subblock at position 0 within the current block 2301. In one embodiment, if only a portion of the four blocks are available, the encoder / decoder can calculate the average value of the motion vectors of the available blocks. Also, in one embodiment, the reference index can be preset (e.g., reference index 0 or 1, collated picture).

[0240] In one embodiment, to use an extended ATMVP mode, similar to the ATMVP mode, the encoder / decoder can apply an increased number of merge candidates, and no additional syntax (or syntactic elements) may be used for this purpose.

[0241] Figures 24 and 25 illustrate a fetching process for time-adjacent blocks within a reference picture according to an embodiment to which the present invention is applied.

[0242] Referring to Figure 24, the encoder / decoder can fetch (or derive, acquire) motion information stored at the corresponding position in the reference picture (or collated picture) of the memory (170 in Figure 1, 250 in Figure 2) (specifically, the DPB of the memory) in order to derive the motion vector of the time-adjacent block of the current block (hereinafter sometimes referred to as the time-motion vector).

[0243] In this invention, for the sake of convenience, the description will mainly focus on the case where the memory is DRAM, but it is not limited to this, and the memory can include various storage media such as DRAM, USB, SD, CD, DVD, Blu-ray (registered trademark), HDD, SSD, etc.

[0244] In one embodiment, the corresponding position within the reference picture (or collated picture) may be the right-bottom block of the current block or the center of the current block (i.e., TMVP), or it may be a position determined by the motion vector of the spatially adjacent block of the current block (i.e., ATMVP).

[0245] Referring to Figure 25, let's assume that the time motion vector is fetched from DRAM 2501 on a current block basis. In this case, as shown in Figure 25, the encoder / decoder fetches a single motion vector stored in the corresponding block 2502 of the current block within DRAM 2501 as the time motion vector.

[0246] On the other hand, when an embodiment of the present invention, such as ATMVP (i.e., subblock-based time merge candidate) or an extended ATMVP, is applied, motion information is fetched in subblock units, and therefore, as shown in Figure 25, it is not possible to fetch motion information for a single current block unit. In other words, when ATMVP or an extended ATMVP is applied, a much larger amount of data must be fetched from the DRAM 2501. This will be explained with reference to the figure below.

[0247] Figure 26 shows a fetching process for fetching motion vectors of time-adjacent blocks in subblock units according to an embodiment to which the present invention is applied.

[0248] Referring to Figure 26, we assume that the current block is divided into four sub-blocks and motion compensation is performed. Also, as mentioned above, in this invention, for the sake of explanation, we will mainly describe the case where the memory is DRAM2601, but we are not limited to this, and the memory can include various storage media such as DRAM2601, USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc.

[0249] As shown in Figure 26, when fetching motion information in units of four subblocks, the encoder / decoder fetches four pieces of motion information corresponding to each subblock within the corresponding block 2602 of the current block, using the reference picture (or collated picture) stored in DRAM 2601. In other words, when fetching motion information in units of four subblocks, the encoder / decoder must fetch approximately four times more data from DRAM 2601 compared to the case described in Figure 25.

[0250] Comparing Figures 25 and 26, it can be seen that the subblock size has a decisive impact on the amount of data fetched from the DRAM2601. Therefore, the subblock size can significantly affect the decoder and encoder pipeline configuration and throughput, depending on the hardware and fetch performance.

[0251] Subblock-based motion compensation methods such as affine motion prediction derive motion vectors on a subblock basis based on the derived motion vectors of control points. However, ATMVP, which is a subblock-based time merge candidate (or predictor), retrieves motion vectors stored in memory on a subblock basis within the corresponding block of the reference picture (or collated picture). Therefore, even though they are similar subblock-based motion compensation methods, the hardware load (burden) depending on the size of the subblock differs significantly between affine motion prediction and ATMVP.

[0252] If the current block is excessively divided into multiple subblocks, a problem may arise where multiple fetch operations must be performed depending on the size of the memory bus used for fetching.

[0253] Therefore, in order to solve these problems, the present invention proposes a method for setting (or determining, defining) a fixed minimum subblock size and performing a time motion information fetching process in units of the set subblock.

[0254] In conventional ATMVP or ATVMP-ext, the current block is divided into 4x4 subblock units for fetching and motion compensation. In this case, as mentioned above, the amount of memory required for fetching can become excessively large depending on the size of the current block. This can lead to a very high hardware load and a decrease in compression performance.

[0255] Therefore, in the embodiments of the present invention, a method of dividing into fixed 8x8-sized sub-blocks or fixed 16x16-sized sub-block units is proposed. The method proposed in this embodiment has been proven through experiments to have less loss of compression performance (compared) with respect to the improvement of hardware complexity, and the above experimental results are the same as Tables 2 to 5 below.

[0256] First, Table 2 below shows the compression performance obtained by executing ATMVP in 4x4 sub-block units.

[0257] [Table 2]

[0258] The following Table 3 shows the compression performance obtained by executing ATMVP in 8x8 sub-block units.

[0259] [Table 3]

[0260] The following Table 4 shows the compression performance obtained by executing ATMVP in 16x16 sub-block units.

[0261] [Table 4]

[0262] The following Table 5 shows the compression performance obtained by executing ATMVP in 32x32 sub-block units.

[0263] [Table 5]

[0264] Referring to the experimental results in Tables 2 to 5 above, it can be confirmed that depending on the size of the sub-block, there is a trade-off between the compression efficiency and the decoding speed, but the loss of compression performance (in contrast) for the improvement of hardware complexity is very small.

[0265] In an embodiment of the present invention, the encoder / decoder can perform sub-block-based motion compensation (i.e., ATMVP, ATMVP-ext) in units larger than 4x4, that is, 8x8, 16x16 or 32x32 units, and can fetch sub-block-based temporal motion information in sub-block units of such size.

[0266] According to an embodiment of the present invention, in performing sub-block-based motion compensation, a relatively high improvement effect on hardware complexity can be obtained (in contrast) with relatively little compression loss.

[0267] In the following, in order to improve (optimize) the hardware complexity, a method of restricting the application of the sub-block-based temporal motion information derivation method (i.e., ATMVP, ATMVP-ext) proposed in the present invention according to the size of the coding block is proposed.

[0268] In an embodiment of the present invention, when the width or height of the current block (or current coding block) is smaller than a predefined size, the encoder / decoder may not execute the sub-block-based temporal motion information derivation method (i.e., ATMVP, ATMVP-ext).

[0269] The method described above with reference to FIGS. 24 to 26 improves the hardware complexity by reducing the data fetch amount of motion information in units of the same area. On the contrary, in this embodiment, the worst-case complexity can be further improved. Conventional video compression technologies perform prediction, transformation, and other coding by dividing pictures or blocks into various forms.

[0270] This allows the size of a coding block (coding unit or prediction unit) to be divided into smaller blocks, such as 4x4, 4x8, or 8x4. In such cases, the size of the current block may be smaller than the size of the smallest subblock from which to fetch time motion information (or time motion vectors). In this case, the hardware can perform the fetch using the smallest coding block size, which is not the size of the smallest subblock.

[0271] Therefore, in order to improve this problem, in the present invention, if the size of the current processing block (or the width and / or height of the current block) is smaller than a predetermined minimum size, a block-unit time motion vector candidate may not be used. In other words, in embodiments of the present invention, the encoder / decoder can add a subblock-based time motion vector candidate to the candidate list only if the size of the current processing block (or the width and / or height of the current block) is greater than or equal to a predetermined minimum size. In one embodiment, the following equation 3 can be used as a condition for using a subblock-unit time motion vector candidate.

[0272] <Formula 3>

number

[0273] Figure 27 is a flowchart showing a method for generating a candidate list according to an embodiment to which the present invention is applied.

[0274] Referring to Figure 27, although the explanation will focus on the decoder for convenience, the method for generating the candidate list in this embodiment can be performed in the same way for the encoder. In one embodiment, the method described in Figures 12 to 17 and 21 to 26 can be applied, and a redundant explanation of this will be omitted.

[0275] The candidate configuration procedure shown in Figure 27 is just one example, and the present invention is not limited thereto. The order may be changed, and some steps may be omitted or added. In addition, the candidates added in Figure 27 are just one example; various other candidates may be added, and some of the candidates shown in Figure 27 may not be used.

[0276] The decoder adds a spatial neighboring block (or spatial candidate) to the candidate list (S2701). For example, the decoder can add the surrounding candidate block shown in Figure 16, described earlier, to the candidate list.

[0277] The decoder checks whether the width and height of the current block are greater than or equal to the predefined minimum size (S2702).

[0278] The decoder adds a temporal neighboring block (or temporal candidate) to the candidate list if the width and height of the current block are greater than or equal to the predefined minimum size (S2703). If the width or height of the current block is less than the predefined minimum size, the decoder may not add a temporal neighboring block (or temporal candidate) to the candidate list.

[0279] The decoder adds the combined motion information (or combined motion vector) to the candidate list and adds the zero motion vector to the candidate list (S2704, S2705).

[0280] The embodiments of the present invention described above have been described separately for the sake of clarity, but the present invention is not limited thereto. In other words, the embodiments described above may be carried out independently, or one or more embodiments may be carried out in combination.

[0281] FIG. 28 is a flowchart illustrating a method for generating an inter-prediction block according to an embodiment to which the present invention is applied.

[0282] Referring to FIG. 28, for the sake of convenience of explanation, the decoder will be mainly described, but the present invention is not limited thereto. Based on inter-prediction, the method for generating an inter-prediction block can be substantially applied in the same way even in an encoder.

[0283] When the merge mode is applied to the current block, the decoder generates a merge candidate list (S2801) using the spatial neighboring blocks and temporal neighboring blocks of the current block.

[0284] The decoder obtains a merge index (S2802) in the merge candidate list that indicates a candidate used for inter-prediction of the current block.

[0285] The decoder derives the motion vector of each of the plurality of sub-blocks included in the current block based on the motion vector of the candidate used for inter-prediction of the current block (S2803).

[0286] The decoder generates a predicted sample of the current block using the motion vector of each of the sub-blocks (S2804).

[0287] As an embodiment, the size of the sub-block can be set to a specific fixed size of 8x8 or more.

[0288] In one embodiment, the step of generating the merge candidate list may include adding to the merge candidate list the time-adjacent blocks of the current block, which are identified by the motion vectors of the spatially adjacent blocks within the collocated picture of the current block.

[0289] In one embodiment, the step of generating the merge candidate list may include adding to the merge candidate list the time-adjacent blocks of the current block, which are identified in the current block's collocated picture by the motion vector of the first spatially adjacent block in a predefined order among the current block's spatially adjacent blocks.

[0290] As an embodiment, the step of generating the merge candidate list may include the steps of: checking whether the width and height of the current block are smaller than a predefined minimum size; and, if the result of the check is that the width and height of the current block are greater than or equal to the predefined minimum size, adding the time-adjacent blocks of the current block, identified by the motion vector of the spatially adjacent blocks within the collocated picture of the current block, to the merge candidate list.

[0291] In one embodiment, the predefined minimum size can be set to the size of the subblock.

[0292] Figure 29 is a diagram illustrating an inter-prediction device according to an embodiment to which the present invention is applied.

[0293] In Figure 29, for the sake of explanation, the interpretation unit is shown as a single block, but the interpretation unit can be implemented in a configuration that is included in the encoder and / or decoder.

[0294] Referring to Figure 29, the functions, processes, and / or methods proposed in Figures 8 to 32 are implemented prior to the inter-prediction unit. Specifically, the inter-prediction unit can be configured to include a merge candidate list generation unit 2901, a merge index acquisition unit 2902, a sub-block motion vector derivation unit 2903, and a prediction block generation unit 2904.

[0295] The merge candidate list generation unit 2901 generates a merge candidate list using the spatial neighboring blocks and temporal neighboring blocks of the current block when a merge mode is applied to the current block.

[0296] The merge index acquisition unit 2902 acquires a merge index that indicates the candidate to be used for inter prediction of the current block from the merge candidate list mentioned above.

[0297] The subblock motion vector derivation unit 2903 derives the motion vectors of each of the multiple subblocks included in the current block based on the candidate motion vectors used for interpretation of the current block.

[0298] The prediction block generation unit 2904 generates prediction samples for the current block using the motion vectors of each of the subblocks.

[0299] In one embodiment, the size of the sub-block can be set to a fixed, specific size of 8x8 or larger.

[0300] In one embodiment, the merge candidate list generation unit 2901 can add the time-adjacent blocks of the current block, which are identified by the motion vector of the spatially adjacent blocks within the collocated picture of the current block, to the merge candidate list.

[0301] In one embodiment, the merge candidate list generation unit 2901 can add to the merge candidate list a time-adjacent block of the current block, which is identified by the motion vector of the first spatially adjacent block in a predefined order among the spatially adjacent blocks of the current block, within the collocated picture of the current block.

[0302] In one embodiment, the merge candidate list generation unit 2901 checks whether the width and height of the current block are smaller than a predetermined minimum size. If, as a result of the check, the width and height of the current block are greater than or equal to the predetermined minimum size, the time-adjacent block of the current block, identified by the motion vector of the spatially adjacent block within the collocated picture of the current block, can be added to the merge candidate list.

[0303] As an example, the predefined minimum size can be set to the size of the subblock.

[0304] Figure 30 shows a video coding system to which the present invention is applied.

[0305] A video coding system may include a source device and a receiving device. The source device can transmit encoded video / image information or data to the receiving device via a digital storage medium or network in file or streaming format.

[0306] The source device may include a video source, an encoding apparatus, and a transmitter. The receiving device may include a receiver, a decoding apparatus, and a renderer. The encoding apparatus may be called a video / image encoding apparatus, and the decoding apparatus may be called a video / image decoding apparatus. The transmitter may be included in the encoding apparatus. The receiver may be included in the decoding apparatus. The renderer may include a display unit, which may consist of a separate device or external component.

[0307] A video source can acquire video / images through processes such as video / image capture, synthesis, or generation. A video source may include video / image capture devices and / or video / image generation devices. Video / image capture devices may include, for example, one or more cameras, or a video / image archive containing previously captured video / images. Video / image generation devices may include, for example, computers, tablets, and smartphones, and can generate video / images (electronically). For example, virtual video / images can be generated through a computer, in which case the video / image capture process can be replaced as the process by which the associated data is generated.

[0308] An encoding device can encode input video / image data. An encoding device can perform a series of steps such as prediction, transformation, and quantization for compression and coding efficiency. The encoded data (encoded video / image information) can be output in the form of a bitstream.

[0309] The transfer unit can transmit encoded video / image information or data output in the form of a bitstream to the receiving unit of a receiving device via a digital storage medium or network in file or streaming format. The digital storage medium can include various storage media such as USB, SD, CD, DVD, Blu-ray®, HDD, SSD, etc. The transfer unit may include elements for generating media files via a predetermined file format and elements for transmission via a broadcast / communication network. The receiving unit can extract the bitstream and transmit it to a decoding device.

[0310] A decoding device can decode video / images by performing a series of steps, such as inverse quantization, inverse transformation, and prediction, corresponding to the operation of an encoding device.

[0311] The renderer can render the decoded video / image. The rendered video / image can be displayed via the display unit.

[0312] Figure 31 shows a diagram of a content streaming system structure as an embodiment to which the present invention is applied.

[0313] Referring to Figure 31, a content streaming system to which the present invention applies may generally include an encoding server, a streaming server, a web server, media storage, user equipment, and multimedia input devices.

[0314] The above-mentioned encoding server is responsible for compressing content input from multimedia input devices such as smartphones, cameras, and video cameras into digital data to generate a bitstream, and then transferring this bitstream to the above-mentioned streaming server. In other cases, if the multimedia input device such as a smartphone, camera, or video camera directly generates the bitstream, the above-mentioned encoding server can be omitted.

[0315] The bitstream described above can be generated by an encoding method or a bitstream generation method to which the present invention is applied, and the streaming server can temporarily store the bitstream in the process of transferring or receiving the bitstream.

[0316] The streaming server transfers multimedia data to the user's device based on the user's request via a web server, and the web server acts as an intermediary to inform the user about the available services. When the user requests a desired service from the web server, the web server transmits this to the streaming server, and the streaming server transfers the multimedia data to the user. In this case, the content streaming system may include a separate control server, in which case the control server controls the commands / responses between the devices within the content streaming system.

[0317] The above streaming server can receive content from media storage and / or encoding servers. For example, if it receives content from the above encoding server, it can receive the content in real time. In this case, in order to provide a smooth streaming service, the above streaming server can store the above bitstream for a certain period of time.

[0318] Examples of user devices mentioned above include mobile phones, smartphones, laptop computers, digital broadcasting terminals, PDAs (Personal Digital Assistants), PMPs (Portable Multimedia Players), navigation systems, slate PCs, tablet PCs, ultrabooks (ULTRABOOK®), wearable devices (such as smartwatches, smart glasses, and HMDs (Head Mounted Displays)), digital TVs, desktop computers, and digital signage.

[0319] Each server in the above content streaming system can be operated as a distributed server, in which case the data received by each server can be processed in a distributed manner.

[0320] As described above, the embodiments described in the present invention can be implemented and executed on a processor, microprocessor, controller, or chip. For example, the functional units shown in each figure can be implemented and executed on a computer, processor, microprocessor, controller, or chip.

[0321] Furthermore, decoders and encoders to which the present invention applies can include multimedia broadcasting transceivers, mobile communication terminals, home cinema video equipment, digital cinema video equipment, surveillance cameras, video interaction devices, real-time communication devices such as video communication, mobile streaming devices, storage media, video cameras, video-on-demand (VoD) service providers, OTT video (Over The Top video) devices, internet streaming service providers, 3D video devices, video phones, video equipment, and medical video equipment, and can be used to process video signals or data signals. For example, OTT video (Over The Top video) devices can include game consoles, Blu-ray players, internet-connected TVs, home theater systems, smartphones, tablet PCs, DVRs (Digital Video Recorders), etc.

[0322] Furthermore, the processing method to which the present invention is applied can be produced in the form of a program executed on a computer and stored on a computer-readable recording medium. Multimedia data having the data structure according to the present invention can also be stored on a computer-readable recording medium. The computer-readable storage medium includes all kinds of storage devices and distributed storage devices on which computer-readable data is stored. The computer-readable storage medium can include, for example, Blu-ray discs (BDs), Universal Serial Bus (USB), ROMs, PROMs, EPROMs, EEPROMs, RAMs, CD-ROMs, magnetic tapes, floppy disks, and optical data storage devices. The computer-readable recording medium also includes media implemented in the form of a carrier wave (e.g., transmission over the Internet). Furthermore, the bitstream generated by the encoding method can be stored on a computer-readable recording medium or transmitted over a wireless network.

[0323] Furthermore, embodiments of the present invention can be realized as a computer program product using program code, and the program code can be executed on a computer according to embodiments of the present invention. The program code can be stored on a computer-readable carrier.

[0324] The embodiments described above are combinations of the components and features of the present invention in a predetermined form. Each component or feature should be considered optional unless otherwise explicitly mentioned. Each component or feature can be implemented in a form that is not combined with other components or features. It is also possible to combine some components and / or features to constitute embodiments of the present invention. The order of operations described in the embodiments of the present invention can be changed. Some components or features of any embodiment can be included in other embodiments, or can be substituted for components or features corresponding to other embodiments. It is obvious that claims that do not have an explicit reference relationship in the claims can be combined to constitute embodiments, or can be included as new claims by amendments after filing.

[0325] Embodiments of the present invention can be realized by various means, such as hardware, firmware, software, or a combination thereof. In the case of hardware implementation, one embodiment of the present invention can be realized by one or more ASICs (Application Specific Integrated Circuits), DSPs (Digital Signal Processors), DSPDs (Digital Signal Processing Devices), PLDs (Programmable Logic Devices), FPGAs (Field Programmable Gate Arrays), processors, controllers, microcontrollers, microprocessors, etc.

[0326] In the case of implementation using firmware or software, one embodiment of the present invention can be implemented in the form of a module, procedure, function, etc., that performs the functions or operations described above. The software code can be stored in memory and driven by a processor. The memory can be located inside or outside the processor and can send and receive data to and from the processor by various already known means.

[0327] It will be obvious to those skilled in the art that the present invention can be embodied in other specific forms without departing from the essential features of the invention. Therefore, the above-mentioned detailed description should not be constrained in any way and should be considered illustrative. The scope of the invention should be determined by a reasonable interpretation of the appended claims, and all modifications within the equivalent scope of the invention are included within the scope of the invention. [Industrial applicability]

[0328] The preferred embodiments of the present invention described above are disclosed for illustrative purposes only, and those skilled in the art can improve, modify, substitute, or add various other embodiments within the technical concept and technical scope of the present invention disclosed in the claims appended below.

Claims

1. A method for decoding a video signal, The process involves obtaining video information from a bitstream, including prediction-related information and residual information. A step of generating a merge candidate list that includes at least one spatially adjacent block and a temporally adjacent block of the current block, as well as candidates corresponding to affine motion predictions. A step of obtaining a merge index that indicates a candidate in the merge candidate list to be used for inter prediction of the current block, A step of deriving the motion vectors of each subblock included in the current block based on the candidate motion vectors indicated by the merge index, wherein if the merge index indicates a candidate corresponding to an affine motion prediction, the step of deriving the motion vectors of the subblocks based on the motion vectors of the affine control points, A step of generating a predicted block for the current block based on the motion vector of each subblock and the prediction-related information, The step includes generating a restored block of the current block based on the residual information and the predicted block, The step of generating the merge candidate list includes adding to the merge candidate list the time-adjacent blocks located within the collated picture of the current block and identified by the motion vector of the spatially adjacent blocks, based on the fact that the width and height of the current block are each greater than or equal to a predetermined minimum size. Based on the fact that the width or height of the current block is smaller than the predefined minimum size, the time-adjacent block is not included in the merge candidate list. Based on the merge index indicating the time-adjacent block as a candidate, the motion vectors of each subblock included in the current block are derived based on the motion vectors of the subblocks included in the time-adjacent block. A method by which zero candidates are added to the merge candidate list.

2. The aforementioned time-adjacent block is identified by the motion vector of the first spatially adjacent block in the merge candidate list, The method according to claim 1, wherein the first spatially adjacent block is determined based on a predetermined order between two or more spatially adjacent blocks of the current block.

3. The method according to claim 1, wherein the predefined minimum size is set to 8.

4. A method for encoding a video signal, A step of generating a merge candidate list that includes at least one spatially adjacent block and a temporally adjacent block of the current block, as well as candidates corresponding to affine motion predictions. A step of deriving the motion vectors of each subblock included in the current block based on the motion vectors of the candidates included in the merge candidate list, wherein if a candidate corresponding to the affine motion prediction is selected from the merge candidate list, the motion vectors of the subblocks are derived based on the motion vectors of the affine control points, A step of generating a predicted block for the current block based on the motion vector of each subblock, The stage of generating prediction-related information, The steps include: deriving the residual block of the current block based on the predicted block of the current block; A step of generating residual information based on the residual block of the current block, A step of generating a restored block of the current block based on the residual block and the predicted block, The process includes the step of encoding video information including the prediction-related information and the residual information, The step of generating the merge candidate list includes adding to the merge candidate list the time-adjacent blocks located within the collated picture of the current block and identified by the motion vector of the spatially adjacent blocks, based on the fact that the width and height of the current block are each greater than or equal to a predetermined minimum size. Based on the fact that the width or height of the current block is smaller than the predefined minimum size, the time-adjacent block is not included in the merge candidate list. Based on the fact that the motion vectors of the time-adjacent blocks in the merge candidate list are used to derive the motion vectors of each of the subblocks of the current block, the motion vectors of each of the subblocks of the current block are derived based on the motion vectors of the subblocks separated from the time-adjacent blocks. A method by which zero candidates are added to the merge candidate list.

5. The aforementioned time-adjacent block is identified by the motion vector of the first spatially adjacent block in the merge candidate list, The method according to claim 4, wherein the first spatially adjacent block is determined based on a predetermined order between two or more spatially adjacent blocks of the current block.

6. The method according to claim 4, wherein the predefined minimum size is set to 8.

7. A method for transmitting data including a bitstream relating to video, A step of generating the bitstream relating to the aforementioned video, The step of transmitting the data including the bitstream, The aforementioned bitstream is A step of generating a merge candidate list that includes at least one spatially adjacent block and a temporally adjacent block of the current block, as well as candidates corresponding to affine motion predictions. A step of deriving the motion vectors of each subblock included in the current block based on the motion vectors of the candidates included in the merge candidate list, wherein if a candidate corresponding to the affine motion prediction is selected from the merge candidate list, the motion vectors of the subblocks are derived based on the motion vectors of the affine control points, A step of generating a predicted block for the current block based on the motion vector of each subblock, The stage of generating prediction-related information, The steps include: deriving the residual block of the current block based on the predicted block of the current block; A step of generating residual information based on the residual block of the current block, A step of generating a restored block of the current block based on the residual block and the predicted block, The video information including the prediction-related information and the residual information are encoded, and the result is generated by performing these steps. The step of generating the merge candidate list includes adding to the merge candidate list the time-adjacent blocks located within the collated picture of the current block and identified by the motion vector of the spatially adjacent blocks, based on the fact that the width and height of the current block are each greater than or equal to a predetermined minimum size. Based on the fact that the width or height of the current block is smaller than the predefined minimum size, the time-adjacent block is not included in the merge candidate list. Based on the fact that the motion vectors of the time-adjacent blocks in the merge candidate list are used to derive the motion vectors of each of the subblocks of the current block, the motion vectors of each of the subblocks of the current block are derived based on the motion vectors of the subblocks separated from the time-adjacent blocks. A method by which zero candidates are added to the merge candidate list.

Images