Method and apparatus for processing video signal for inter prediction

Abstract

The present specification relates to a method and apparatus for processing a video signal for inter prediction. A decoding method according to an embodiment of the present specification comprises: a step for acquiring, from first coding information on a first level unit, a first flag related to whether or not second MVD information is encoded in first motion vector difference (MVD) information for predicting a first direction and the second MVD information for predicting a second direction; a step for acquiring, on the basis of the first flag, a second flag related to whether to apply a symmetric MVD (SMVD) to the current block from second coding information on a second level unit lower than the first level unit; a step for determining a first MVD with respect to the current block on the basis of the first MVD information; a step for determining a second MVD on the basis of the second flag; a step for determining a first motion vector and a second motion vector on the basis of the first MVD and the second MVD; and a step for generating a prediction sample of the current block on the basis of the first motion vector and the second motion vector.

Description

[0001] This application is a divisional application of a patent application with application number 202080018752.4 (PCT / KR2020 / 003120), which was submitted to the China Patent Office on September 3, 2021, with an international application date of March 5, 2020, and the invention name is “Method and device for processing video signals for inter-frame prediction”. Technical Field

[0002] The present invention relates to a video / image compression coding system, and in particular to a method and device for performing inter-frame prediction in a video encoding / decoding process. Background Art

[0003] Compression coding refers to a series of signal processing techniques for sending digitized information through communication lines or a technology for storing information in a form suitable for storage media. Media including pictures, images, audio, etc. can be the target of compression coding, and specifically, the technology for performing compression coding on pictures is called video image compression.

[0004] The next generation of video content is expected to have the characteristics of high spatial resolution, high frame rate and high dimensionality of scene representation. In order to process such content, it will lead to a sharp increase in memory storage, memory access rate and processing power.

[0005] Inter prediction is a method of performing prediction on a current picture with reference to a reconstructed sample of another picture. In order to improve the efficiency of inter prediction, various motion vector derivation methods are being discussed together with new inter prediction techniques. Summary of the invention

[0006] Technical issues

[0007] Embodiments of the present disclosure provide a method and apparatus for improving signaling efficiency of information indicating whether a symmetric motion vector difference (SMVD) is applied during an encoding / decoding process for information for inter prediction.

[0008] The technical objectives to be achieved by the embodiments of the present disclosure are not limited to the above-mentioned technical objectives, and from the following description, a person skilled in the art in the art to which the embodiments of the present disclosure belong will obviously understand other technical objectives not mentioned above.

[0009] Technical Solution

[0010] The embodiment of the present disclosure provides an encoding method and a decoding method of a video signal for inter-frame prediction. The decoding method according to the embodiment of the present disclosure includes: obtaining a first flag related to whether the second MVD information is encoded in the first motion vector difference (MVD) information for first direction prediction and the second MVD information for second direction prediction from the first coding information for the first level unit; based on the first flag, obtaining a second flag from the second coding information for the second level unit lower than the first level unit, the second flag being related to whether a symmetric MVD (SMVD) is applied to the current block; determining a first MVD for the current block based on the first MVD information; determining a second MVD based on the second flag; determining a first motion vector and a second motion vector based on the first MVD and the second MVD; and generating a prediction sample of the current block based on the first motion vector and the second motion vector.

[0011] In an embodiment, the first level unit may correspond to one of: a picture, a tile group or a slice, and the second level unit may correspond to a coding unit.

[0012] In an embodiment, when the first flag is 0, decoding of the second MVD information may be performed, and when the first flag is 1, decoding of the second MVD information may be omitted.

[0013] In an embodiment, obtaining the second flag may include: when the first flag is 0 and an additional condition is satisfied, decoding the second flag, and when the first flag is 1, inferring that the second flag is 0 without decoding the second flag.

[0014] In an embodiment, determining the second MVD may include: determining the second MVD from the second MVD information when the second flag is 0, and determining the second MVD from the first MVD based on the SMVD when the second flag is 1.

[0015] In an embodiment, when the second flag is 1, the second MVD may have the same magnitude as the first MVD and an opposite sign as the first MVD.

[0016] In an embodiment, determining the first motion vector and the second motion vector may include: obtaining first motion vector predictor (MVP) information for the first direction prediction and second MVP information for the second direction prediction, determining a first candidate motion vector corresponding to the first MVP information in a first MVP candidate list for the first direction prediction, and determining a second candidate motion vector corresponding to the second MVP information in a second MVP candidate list for the second direction prediction, determining the first motion vector by adding the first MVD to the first candidate motion vector, and determining the second motion vector by adding the second MVD to the second candidate motion vector.

[0017] In an embodiment, generating the predicted sample of the current block may include: determining a first reference picture for first direction prediction and a second reference picture for second direction prediction, and generating the predicted sample of the current block based on a first reference sample indicated by the first motion vector in the first reference picture and a second reference sample indicated by the second motion vector in the second reference picture.

[0018] In an embodiment, the first reference picture may correspond to a reference picture that is preceding and closest to the current picture in display order in a first reference picture list for first directional prediction, and the second reference picture may correspond to a reference picture that is following and closest to the current picture in display order in a second reference picture list for second directional prediction.

[0019] According to an embodiment of the present disclosure, an encoding method includes: encoding first coding information for a first level unit; encoding second coding information for a second level unit lower than the first level unit, the first coding information including a first flag related to whether second MVD information is encoded in first motion vector difference (MVD) information for first direction prediction and second MVD information for second direction prediction, and the second coding information including a second flag related to whether symmetric MVD (SMVD) is applied to a current block corresponding to the second level unit; and encoding the second flag based on the first flag.

[0020] In an embodiment, if the first flag is 0, encoding of the second MVD information may be performed, and if the first flag is 1, encoding of the second MVD information may be omitted.

[0021] In an embodiment, when encoding the second coding information, when the first flag is 0, the second flag may be encoded based on a search process of a first motion vector for the first direction prediction and a second motion vector for the second direction prediction.

[0022] According to an embodiment of the present disclosure, a decoding device includes: a memory storing the video signal; and a processor connected to the memory and processing the video signal. The processor is configured to: obtain a first flag in a first level unit, the first flag being related to whether second MVD information is encoded in first motion vector difference (MVD) information for first direction prediction and second MVD information for second direction prediction; based on the first flag, obtain a second flag, the second flag being related to whether symmetric MVD (SMVD) is applied to a current block corresponding to a second level unit lower than the first level unit; determine a first MVD for the current block based on the first MVD information; determine a first motion vector and a second motion vector based on the first MVD and the second MVD; and generate a prediction sample of the current block based on the first motion vector and the second motion vector.

[0023] According to an embodiment of the present disclosure, an encoding device includes: a memory storing the video signal; and a processor connected to the memory and processing the video signal. The processor is configured to: encode first coding information for a first level unit, and encode second coding information for a second level unit lower than the first level unit. The first coding information includes a first flag related to whether the second MVD information is encoded in the first motion vector difference (MVD) information for first direction prediction and the second MVD information for second direction prediction, and the second coding information includes a second flag related to whether a symmetric MVD (SMVD) is applied to a current block corresponding to the second level unit, and the second flag is encoded based on the first flag.

[0024] In addition, an embodiment of the present disclosure provides a non-transitory computer-readable medium storing one or more instructions. The one or more instructions control a video signal processing device to: obtain a first flag in a first level unit, the first flag being related to whether second MVD information is encoded in first motion vector difference (MVD) information for first direction prediction and second MVD information for second direction prediction; obtain a second flag based on the first flag, the second flag being related to whether symmetric MVD (SMVD) is applied to a current block corresponding to a second level unit lower than the first level unit; determine a first MVD for the current block based on the first MVD information; determine a second MVD based on the second flag; determine a first motion vector and a second motion vector based on the first MVD and the second MVD; and generate a prediction sample of the current block based on the first motion vector and the second motion vector.

[0025] In addition, the one or more instructions control the video signal processing device to encode first coding information for a first level unit and encode second coding information for a second level unit lower than the first level unit. The first coding information includes a first flag as to whether second MVD information is encoded in first motion vector difference (MVD) information for first direction prediction and second MVD information for second direction prediction, and the second coding information includes a second flag as to whether symmetric MVD (SMVD) is applied to a current block corresponding to the second level unit, and the second flag is encoded based on the first flag.

[0026] Beneficial Effects

[0027] According to an embodiment of the present disclosure, information indicating whether to use a symmetric motion vector difference (SMVD) to which symmetric bidirectional prediction is applied is prevented from being signaled unnecessarily even when one of the bidirectional prediction information is not encoded, thereby reducing the data amount and coding complexity / time of information required for inter-frame prediction.

[0028] Effects that can be obtained by the embodiments of the present disclosure are not limited to the above effects, and from the following description, a person skilled in the art to which the embodiments of the present disclosure belong can clearly understand other technical effects not described above. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] The accompanying drawings are included as a part of the detailed description to help understanding of the present disclosure, provide embodiments of the present disclosure and describe technical features of the present disclosure together with the detailed description.

[0030] Figure 1 An example of a video coding system according to an embodiment of the present disclosure is shown.

[0031] Figure 2 1 is a schematic block diagram of an encoding apparatus for encoding a video / image signal to which an embodiment of the present disclosure is applied.

[0032] Figure 3 1 is a schematic block diagram of a decoding device for decoding a video / image signal to which the present disclosure is applied.

[0033] Figure 4 An example of a content streaming system according to an embodiment of the present disclosure is shown.

[0034] Figure 5 An example of an apparatus for processing a video signal according to an embodiment of the present disclosure is shown.

[0035] Figure 6 An example of a partition structure of a picture according to an embodiment of the present disclosure is shown.

[0036] Figures 7a to 7d An example of a block partition structure according to an embodiment of the present disclosure is shown.

[0037] Figure 8 An example of a case where ternary tree (TT) and binary tree (BT) partitions are restricted according to an embodiment of the present disclosure is shown.

[0038] Fig. 9 An example of a flowchart for encoding pictures constituting a video signal according to an embodiment of the present disclosure is shown.

[0039] Fig.10 An example of a flowchart for decoding pictures constituting a video signal according to an embodiment of the present disclosure is shown.

[0040] Fig.11 An example of a hierarchical structure of an encoded image according to an embodiment of the present disclosure is shown.

[0041] Fig.12 An example of a flowchart for performing inter-frame prediction during an encoding process of a video signal according to an embodiment of the present disclosure is shown.

[0042] Fig.13 An example of an inter-frame predictor in an encoding device according to an embodiment of the present disclosure is shown.

[0043] Fig.14 An example of a flowchart for performing inter-frame prediction during a decoding process of a video signal according to an embodiment of the present invention is shown.

[0044] Fig.15 An example of an inter-frame predictor in a decoding device according to an embodiment of the present disclosure is shown.

[0045] Fig.16 An example of spatially neighboring blocks used as spatial merging candidates according to an embodiment of the present disclosure is shown.

[0046] Fig.17 An example of a flowchart for constructing a merge candidate list according to an embodiment of the present disclosure is shown.

[0047] Fig.18 An example of a flowchart for constructing a motion vector predictor (MVP) candidate list according to an embodiment of the present disclosure is shown.

[0048] Fig.19 An example of a case where a symmetric motion vector difference (MVD) mode is applied according to an embodiment of the present disclosure is shown.

[0049] Fig. 20 An example of an affine motion model according to an embodiment of the present disclosure is shown.

[0050] Fig.21a and21b An example of a motion vector of each control point according to an embodiment of the present disclosure is shown.

[0051] Fig. 22 An example of a motion vector of each sub-block according to an embodiment of the present disclosure is shown.

[0052] Fig.23 An example of a flowchart for constructing an affine merge candidate list according to an embodiment of the present disclosure is shown.

[0053] Fig.24 An example of a block for deriving an inherited affine motion predictor according to an embodiment of the present disclosure is shown.

[0054] Fig.25 An example of control point motion vectors for deriving an inherited affine motion predictor according to an embodiment of the present disclosure is shown.

[0055] Fig.26 An example of a block for deriving a constructed affine merge candidate according to an embodiment of the present invention is shown.

[0056] Fig. 27 An example of a flowchart for constructing an affine MVP candidate list according to an embodiment of the present disclosure is shown.

[0057] Fig.28 An example of a flowchart for deriving a motion vector according to an embodiment of the present disclosure is shown.

[0058] Fig.29 An example of a flowchart for estimating motion according to an embodiment of the present disclosure is shown.

[0059] Fig.30 An example of a flow chart for encoding a video signal for inter-frame prediction according to an embodiment of the present disclosure is shown.

[0060] Fig.31 An example of a decoding flowchart of a video signal for inter-frame prediction according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0061] Hereinafter, preferred embodiments of the present disclosure will be described with reference to the accompanying drawings. The specification described below with reference to the accompanying drawings is for describing exemplary embodiments of the present disclosure, and is not intended to describe the only embodiments that can implement the present disclosure. The following description includes specific details in order to provide a perfect understanding of the present disclosure. However, it will be understood by those skilled in the art that the present disclosure can be specifically implemented without these specific details. In some cases, in order to prevent the technical concepts of the present disclosure from being unclear, well-known structures or devices may be omitted, or may be depicted as block diagrams centered on the core functions of the structures or devices.

[0062] In some cases, in order to prevent the technical concepts of the present disclosure from being unclear, well-known structures or devices may be omitted, or may be depicted as a block diagram centered on the core functions of the structure or device.

[0063] In addition, although general terms currently widely used are selected as terms as much as possible in the present disclosure, terms arbitrarily selected by the applicant are used in specific cases. Since the meaning of the terms will be clearly described in the corresponding part of the specification in this case, it is understood that the present disclosure is not simply interpreted by the terms used only in the description of the present disclosure, but the meaning of the terms should be clarified.

[0064] The specific terms used in the following description may be provided to help understand the present disclosure. In addition, within the scope of the technical concept of the present disclosure, these specific terms may be modified into other forms. For example, signals, data, samples, pictures, frames, blocks, etc. may be appropriately replaced and interpreted in each compilation process.

[0065] Hereinafter, in the present disclosure, a "processing unit" refers to a unit that performs encoding / decoding processing such as prediction, transformation, and / or quantization. A processing unit may also be interpreted as including a meaning of a unit for a luma component and a unit for a chroma component. For example, a processing unit may correspond to a coding tree unit (CTU), a coding unit (CU), a prediction unit (PU), or a transform unit (TU).

[0066] In addition, the processing unit may also be interpreted as a unit for a luma component or a unit for a chroma component. For example, the processing unit may correspond to a coding tree block (CTB), a coding block (CB), a prediction block (PB), or a transform block (TB) for a luma component. Alternatively, the processing unit may correspond to a coding tree block (CTB), a coding block (CB), a prediction block (PB), or a transform block (TB) for a chroma component. In addition, the processing unit is not limited thereto, and the processing unit may be interpreted as including the meaning of a unit for a luma component and a unit for a chroma component.

[0067] Furthermore, the processing unit is not necessarily limited to a square block, and may be constructed in the form of a polygon having three or more vertices.

[0068] Furthermore, hereinafter, in the present disclosure, a pixel is generally referred to as a sample. Furthermore, using a sample may mean using a pixel value.

[0069] Figure 1 An example of a video coding system according to an embodiment of the present disclosure is shown. The video coding system may include a source device 10 and a receiving device 20. The source device 10 may send encoded video / image information or data to the receiving device 20 in a file or streaming format through a storage medium or a network.

[0070] The source device 10 may include a video source 11, an encoding device 12, and a transmitter 13. The receiving device 20 may include a receiver 21, a decoding device 22, and a renderer 23. The source device may be referred to as a video / image encoding device, and the receiving device may be referred to as a video / image decoding device. The transmitter 13 may be included in the encoding device 12. The receiver 21 may be included in the decoding device 22. The renderer may include a display, and the display may be configured as a separate device or an external component.

[0071] The video source 11 may acquire video / image data by a capture, synthesis or generation process of a video / image. The video source 11 may include a video / image capture device and / or a video / image generation device. The video / image capture device may include, for example, one or more cameras, a previously captured video / image archive, etc. The video / image generation device may include, for example, a computer, a tablet computer, and a smart phone, and may generate video / image data electronically. For example, virtual video / image data may be generated by a computer, etc., and in this case, the video / image capture process may be replaced by a process that generates relevant data.

[0072] The encoding device 12 can encode the input video / image. The encoding device 12 can perform a series of procedures, such as prediction, transformation, and quantization, for compression and coding efficiency. The encoded data (encoded video / video information) can be output in the form of a bit stream.

[0073] The transmitter 13 can send the encoded video / video information or data output in the form of a bit stream to the receiver 21 of the receiving device 20 through a digital storage medium or a network in a file or streaming format. The digital storage medium may include various storage media such as a universal serial bus USB, a secure digital SD card, a compact disc CD, a digital video disc DVD, a blue-ray, a hard disk drive HDD, and a solid-state drive SSD. The transmitter 13 may include an element for generating a media file in a predetermined file format, and may include an element for transmitting through a broadcast / communication network. The receiver 21 may extract the bit stream and send it to a decoding device 22.

[0074] The decoding device 22 can decode video / image data by performing a series of processes such as dequantization, inverse transformation, and prediction corresponding to the operations of the encoding device 12.

[0075] The renderer 23 may render the decoded video / image. The rendered video / image may be displayed through a display.

[0076] Figure 2 is an embodiment to which the present disclosure is applied, and is a schematic block diagram of an encoding apparatus for encoding a video / image signal. Figure 2The encoding device 100 may correspond to Figure 1 The encoding device 12.

[0077] Reference Figure 2 , the encoding device 100 may be configured to include an image segmenter 110, a subtractor 115, a transformer 120, a quantizer 130, a dequantizer 140, an inverse transformer 150, an adder 155, a filter 160, a memory 170, an inter-frame predictor 180, an intra-frame predictor 185, and an entropy encoder 190. The inter-frame predictor 180 and the intra-frame predictor 185 may be collectively referred to as a predictor. In other words, the predictor may include the inter-frame predictor 180 and the intra-frame predictor 185. The transformer 120, the quantizer 130, the dequantizer 140, and the inverse transformer 150 may be included in a residual processor. The residual processor may also include a subtractor 115. In one embodiment, the image segmenter 110, the subtractor 115, the transformer 120, the quantizer 130, the dequantizer 140, the inverse transformer 150, the adder 155, the filter 160, the inter-frame predictor 180, the intra-frame predictor 185, and the entropy encoder 190 may be configured as one hardware component (e.g., an encoder or a processor). In addition, the memory 170 may include a decoded picture buffer (DPB) and may be configured with a digital storage medium.

[0078] The image divider 110 may divide the input image (or picture or frame) input to the encoding device 100 into one or more processing units. For example, the processing unit may be referred to as a coding unit (CU). In this case, the coding unit may be recursively partitioned from a coding tree unit (CTU) or a maximum coding unit (LCU) based on a quadtree binary tree (QTBT) structure. For example, one coding unit may be divided into a plurality of coding units with a deeper depth based on a quadtree structure and / or a binary tree structure. In this case, for example, a quadtree structure may be applied first, and then a binary tree structure may be applied. Alternatively, a binary tree structure may be applied first. The coding process according to the present disclosure may be performed based on a final coding unit that is no longer partitioned. In this case, the maximum coding unit may be directly used as a final coding unit based on coding efficiency according to image characteristics, or the coding unit may be recursively partitioned into coding units with a deeper depth if necessary. Therefore, a coding unit with an optimal size may be used as a final coding unit. In this case, the coding process may include a process such as prediction, transformation, or reconstruction described later. For another example, the processing unit may also include a prediction unit (PU) or a transformation unit (TU). In this case, each of the prediction unit and the transform unit may be divided or partitioned from each final coding unit. The prediction unit may be a unit for sample prediction, and the transform unit may be a unit from which a transform coefficient is derived and / or a unit from which a residual signal is derived.

[0079] The unit used in the present disclosure may be used interchangeably with a block or region, depending on the situation. In the present disclosure, an M×N block may indicate a set of samples or a set of transform coefficients configured with M columns and N rows. In general, a sample may indicate a pixel or a value of a pixel, and may indicate only a pixel / pixel value of a luminance component or only a pixel / pixel value of a chrominance component. In a sample, a picture (or image) may be used as a term corresponding to a pixel or a pel.

[0080] The encoding device 100 can generate a residual signal (residual block or residual sample array) by subtracting the prediction signal (prediction block or prediction sample array) output by the inter predictor 180 or the intra predictor 185 from the input image signal (original block or original sample array). The generated residual signal is sent to the transformer 120. In this case, as shown, the unit in which the prediction signal (prediction block or prediction sample array) is subtracted from the input image signal (original block or original sample array) in the encoding device 100 can be referred to as a subtractor 115. The predictor can perform prediction on the processing target block (hereinafter referred to as the current block) and can generate a prediction block including prediction samples for the current block. The predictor can determine whether to apply intra prediction or inter prediction in the current block or CU unit. The predictor can generate information about prediction, such as prediction mode information to be described later in the description of each prediction mode, and can send the information to the entropy encoder 190. The information about prediction can be encoded in the entropy encoder 190 and can be output in the form of a bitstream.

[0081] The intra-frame predictor 185 can predict the current block with reference to samples within the current picture. Depending on the prediction mode, the referenced sample can be positioned adjacent to the current block, or can be spaced apart from the current block. In intra-frame prediction, the prediction mode may include multiple non-angle modes and multiple angle modes. The non-angle mode may include, for example, a DC mode and a plane mode. The angle mode may include, for example, 33 angle prediction modes or 65 angle prediction modes, depending on the degree of refinement of the prediction direction. In this case, for example, more or less than 33 angle prediction modes or 65 angle prediction modes may be used according to the configuration. The intra-frame predictor 185 may use the prediction mode applied to the neighboring blocks to determine the prediction mode applied to the current block.

[0082] The inter-frame predictor 180 may derive a prediction block of the current block based on a reference block (reference sample array) specified by a motion vector on a reference picture. In this case, in order to reduce the amount of motion information sent in the inter-frame prediction mode, the motion information may be predicted in units of blocks, sub-blocks, or samples based on the correlation of the motion information between the adjacent blocks and the current block. The motion information may include a motion vector and a reference picture index. The motion information may also include inter-frame prediction direction (L0 prediction, L1 prediction, Bi prediction) information. In the case of inter-frame prediction, the adjacent blocks may include spatial adjacent blocks within the current picture and temporal adjacent blocks within the reference picture. The reference picture including the reference block and the reference picture including the temporal adjacent block may be the same or different. The temporal adjacent block may be referred to as a name called a co-located reference block or a co-located CU (colCU). The reference picture including the temporal adjacent block may be referred to as a co-located picture (colPic). For example, the inter-frame predictor 180 may construct a motion information candidate list based on the motion information of the adjacent blocks, and may generate information indicating which candidate is used to derive the motion vector and / or reference picture index of the current block. Inter-frame prediction may be performed based on various prediction modes. For example, in the case of skip mode and merge mode, the inter-frame predictor 180 may use the motion information of the neighboring block as the motion information of the current block. In the case of skip mode, unlike the merge mode, the residual signal may not be transmitted. In the case of motion vector prediction (MVP) mode, the motion vector of the neighboring block may be used as a motion vector predictor. The motion vector of the current block may be indicated by signaling a motion vector difference MVD.

[0083] The prediction unit may generate a prediction signal (prediction sample) based on various prediction methods described later. For example, the prediction unit may apply intra prediction or inter prediction to predict a block, and may apply intra prediction and inter prediction together (simultaneously). This may be referred to as combined inter and intra prediction (CIIP). In addition, the prediction unit may perform intra block copying (IBC) to predict the block. IBC may be used, for example, for content (e.g., game) video / video coding, such as screen content coding (SCC). In addition, IBC may also be referred to as current picture reference (CPR). IBC essentially performs prediction within the current picture, but may be performed similarly to inter prediction because a reference block is derived within the current picture. That is, IBC may use at least one of the inter prediction techniques described in the present disclosure.

[0084] The prediction signal generated by the prediction unit (including the inter-frame predictor 180 and / or the intra-frame predictor 185) can be used to generate a reconstruction signal or a residual signal. The transformer 120 can generate a transform coefficient by applying a transform scheme to the residual signal. For example, the transform scheme may include at least one of a discrete cosine transform (DCT), a discrete sine transform (DST), a Karhunen–Loève transform (KLT), a graph-based transform (GBT), or a conditional nonlinear transform (CNT). In this case, GBT means a transform obtained from a graph if the relationship information between pixels is represented as a graph. CNT means a transform obtained based on a prediction signal generated using all previously reconstructed pixels. In addition, the transform process can be applied to pixel blocks in a square form with the same size, or can be applied to blocks in a non-square form or blocks in a non-square form with a variable size.

[0085] The quantizer 130 may quantize the transform coefficients and send them to the entropy encoder 190. The entropy encoder 190 may encode the quantized signal (information about the quantized transform coefficients) and output the quantized signal in the form of a bitstream. Information about the quantized transform coefficients may be referred to as residual information. The quantizer 130 may rearrange the quantized transform coefficients in the form of a block in the form of a one-dimensional vector based on the coefficient scanning sequence, and may generate information about the quantized transform coefficients based on the characteristics of the quantized transform coefficients in the form of a one-dimensional vector. The entropy encoder 190 may perform various encoding methods such as exponential Golomb, context adaptive variable length coding (CAVLC), and context adaptive binary arithmetic coding (CABAC). In addition to the quantized transform coefficients, the entropy encoder 190 may also encode information required for video / image reconstruction (e.g., the value of a syntax element) together or separately. The encoding information (e.g., encoded video / image information) may be transmitted in the form of a bitstream or stored in a network abstraction layer NAL unit. The video / image information may also include information about various parameter sets, such as an adaptive parameter set (APS), a picture parameter set (PPS), a sequence parameter set (SPS), or a video parameter set (VPS). The signaled / transmitted information and / or syntax elements described later in this disclosure may be encoded by the above-mentioned encoding process and included in the bitstream. The bitstream may be sent over a network or may be stored in a digital storage medium. In this case, the network may include a broadcast network and / or a communication network. The digital storage medium may include various storage media, such as USB, SD, CD, DVD, Blu-ray, HDD, and SSD. A transmitter (not shown) that transmits a signal output by the entropy encoder 190 and / or a memory (not shown) for storing the signal may be configured as an internal / external element of the encoding device 100, or the transmitter may be an element of the entropy encoder 190.

[0086] The quantized transform coefficients output by the quantizer 130 can be used to generate a reconstructed signal. For example, the residual signal can be reconstructed by applying dequantization and inverse transformation to the quantized transform coefficients via the dequantizer 140 and the inverse transformer 150 within a loop. The adder 155 can add the reconstructed residual signal to the prediction signal output by the inter-frame predictor 180 or the intra-frame predictor 185, so that a reconstructed signal (reconstructed image, reconstructed block or reconstructed sample array) can be generated. If there is no residual signal for the processing target block, the prediction block can be used as a reconstructed block, such as when the skip mode has been applied. The adder 155 can be referred to as a reconstructor or a reconstructed block generator. The generated reconstructed signal can be used for intra-frame prediction of the next processing target block in the current picture, and can be used for inter-frame prediction of the next picture through filtering as will be described later.

[0087] The filter 160 can improve the subjective / objective picture quality by applying filtering to the reconstructed signal. For example, the filter 160 can generate a modified reconstructed picture by applying various filtering methods to the reconstructed picture. The modified reconstructed picture can be stored in the DPB 175 of the memory 170. For example, various filtering methods may include deblocking filtering, sample adaptive offset SAO, adaptive loop filter ALF and bilateral filter. The filter 160 can generate information for filtering, as will be described later in the description of each filtering method, and can send them to the entropy encoder 190. The filter information can be encoded by entropy coding in the entropy encoder 190 and output in the form of a bitstream.

[0088] The modified reconstructed picture sent to the DPB 175 may be used as a reference picture in the inter-frame predictor 180. If inter-frame prediction is applied, the encoding device can avoid prediction mismatch in the encoding device 100 and the decoding device by using the modified reconstructed picture, and improve encoding efficiency. The DPB 175 may store the modified reconstructed picture so as to use the modified reconstructed picture as a reference picture in the inter-frame predictor 180. The stored motion information may be sent to the inter-frame prediction unit 180 to be used as motion information of a spatial neighboring block or motion information of a temporal neighboring block. The memory 170 may store the reconstructed samples of the reconstructed blocks in the current picture, and send information about the reconstructed samples to the intra-frame prediction unit 185.

[0089] Figure 3 1 is a schematic block diagram of a decoding device for decoding a video / image signal to which the present disclosure is applied. Figure 3 The decoding device 200 may correspond to Figure 1 The decoding device 22.

[0090] Reference Figure 3, the decoding device 200 may be configured to include an entropy decoder 210, a dequantizer 220, an inverse transformer 230, an adder 235, a filter 240, a memory 250, an inter-frame predictor 260, and an intra-frame predictor 265. The inter-frame predictor 260 and the intra-frame predictor 265 may be collectively referred to as a predictor. That is, the predictor may include the inter-frame predictor 180 and the intra-frame predictor 185. The dequantizer 220 and the inverse transformer 230 may be collectively referred to as a residual processor. That is, the residual processor may include the dequantizer 220 and the inverse transformer 230. According to an embodiment, the entropy decoder 210, the dequantizer 220, the inverse transformer 230, the adder 235, the filter 240, the inter-frame predictor 260, and the intra-frame predictor 265 may be configured as one hardware component (e.g., a decoder or a processor). Furthermore, in an embodiment, the memory 250 may include a decoded picture buffer DPB 255 and may be configured with a hardware component (eg, a memory or a digital storage medium).

[0091] When a bit stream including video / image information is input, the decoding apparatus 200 can Figure 2 The image is reconstructed by processing the video / image information in the encoding device of the decoding apparatus, for example, the decoding apparatus 200 can perform decoding using the processing unit applied in the encoding apparatus. Thus, for example, the processing unit for decoding can be a coding unit. Depending on the quadtree structure and / or the binary tree structure, the coding unit can be split from the coding tree unit or the maximum coding unit. In addition, the reconstructed image signal decoded and output by the decoding apparatus 200 can be played back by a playback device.

[0092] The decoding device 200 may receive the Figure 1The received signal may be decoded by the entropy decoder 210. For example, the entropy decoder 210 may derive information (e.g., video / image information) for image reconstruction (or picture reconstruction) by parsing the bitstream. The video / image information may also include information about various parameter sets, such as an adaptive parameter set (APS), a picture parameter set (PPS), a sequence parameter set (SPS), or a video parameter set (VPS). The decoding device may decode the picture based on the information about the parameter set. The signaled / received information and / or syntax elements described later in this disclosure may be decoded by a decoding process and obtained from the bitstream. For example, the entropy decoder 210 may obtain information within the bitstream based on a coding method such as exponential Golomb coding, CAVLC, or CABAC, and may output values ​​of syntax elements for image reconstruction or quantized values ​​of transform coefficients about the residual. More specifically, in the CABAC entropy decoding method, a bin corresponding to each syntax element may be received from a bitstream, the context model may be determined using the decoded target syntax element information and the decoded information of the adjacent and decoded target blocks or the information of the symbol / bin decoded in the previous step, the probability of the bin occurrence may be predicted based on the determined context model, and the symbol corresponding to the value of each syntax element may be generated by performing arithmetic decoding on the bin. In this case, in the CABAC entropy decoding method, after the context model is determined, the context model may be updated using the information of the symbol / bin decoded by the context model for the next symbol / bin. Information about prediction in the information decoded in the entropy decoder 210 may be provided to the predictor (inter-frame predictor 260 and intra-frame predictor 265). Parameter information related to the residual value (ie, quantized transform coefficient) on which entropy decoding has been performed in the entropy decoder 210 may be input to the dequantizer 220. In addition, information about filtering in the information decoded in the entropy decoder 210 may be provided to the filter 240. Meanwhile, a receiver (not shown) receiving a signal output by the encoding device may also be configured as an internal / external element of the decoding device 200, or the receiver may be an element of the entropy decoder 210. Meanwhile, the decoding device 200 according to the present specification may be referred to as a video / image / picture decoding device. The decoding device 200 may be divided into an information decoder (video / image / picture information decoder) and a sample decoder (video / image / picture sample decoder). The information decoder may include an entropy decoding unit 210, and the sample decoder may include at least one of a dequantizer 220, an inverse transform unit 230, an adder 235, a filter 240, a memory 250, an inter-frame prediction unit, and an intra-frame prediction unit 265.

[0093] The dequantizer 220 may dequantize the quantized transform coefficient and output the transform coefficient. The dequantizer 220 may rearrange the quantized transform coefficient in a two-dimensional block form. In this case, the rearrangement may be performed based on a coefficient scanning sequence performed in the encoding device. The dequantizer 220 may perform dequantization on the quantized transform coefficient using a quantization parameter (e.g., quantization step size information), and may obtain the transform coefficient.

[0094] The inverse transformer 230 may output a residual signal (a residual block or a residual sample array) by inversely transforming the transform coefficients.

[0095] The predictor may perform prediction on the current block and may generate a prediction block including a prediction sample of the current block. The predictor may determine whether to apply intra prediction to the current block or inter prediction to the current block based on the information about prediction output by the entropy decoder 210, and may determine a detailed intra / inter prediction mode.

[0096] The predictor can generate a prediction signal (prediction sample) based on various prediction methods to be described below. For example, the predictor can apply intra prediction or inter prediction for the prediction of a block, and apply intra prediction and inter prediction at the same time. This can be referred to as combined inter and intra prediction (CIIP). In addition, the predictor can perform intra block copying (IBC) to predict the block. IBC can be used for content image / video encoding, such as games, such as screen content coding (SCC). In addition, IBC can also be referred to as current picture reference (CPR). IBC basically performs prediction in the current picture, but can be performed similarly to inter prediction in terms of IBC deriving a reference block in the current picture. That is, IBC can use at least one of the inter prediction techniques described in the present disclosure.

[0097] The intra-frame predictor 265 may reference samples within the current picture to predict the current block. Depending on the prediction mode, the referenced samples may be located adjacent to the current block or may be spaced apart from the current block. In intra-frame prediction, the prediction mode may include multiple non-angle modes and multiple angle modes. The intra-frame predictor 265 may use the prediction mode applied to the neighboring blocks to determine the prediction mode applied to the current block.

[0098] The inter-frame predictor 260 may derive a prediction block of the current block based on a reference block (reference sample array) specified by a motion vector on a reference picture. In this case, in order to reduce the amount of motion information sent in the inter-frame prediction mode, the motion information may be predicted in blocks, sub-blocks, or sample units based on the correlation of motion information between adjacent blocks and the current block. The motion information may include a motion vector and a reference picture index. The motion information may also include inter-frame prediction direction (L0 prediction, L1 prediction, Bi prediction) information. In the case of inter-frame prediction, the adjacent blocks may include spatial adjacent blocks within the current picture and temporal adjacent blocks within the reference picture. For example, the inter-frame predictor 260 may configure a motion information candidate list based on adjacent blocks, and may derive a motion vector and / or a reference picture index of the current block based on the received candidate selection information. Inter-frame prediction may be performed based on various prediction modes. Information about the prediction may include information indicating an inter-frame prediction mode of the current block.

[0099] The adder 235 can generate a reconstructed signal (reconstructed image, reconstructed block or reconstructed sample array) by adding the obtained residual signal to the prediction signal (prediction block or prediction sample array) output by the prediction unit (including the inter-frame predictor 260 and / or the intra-frame predictor 265). As in the case where the skip mode has been applied, if there is no residual for the processing target block, the prediction block can be used as the reconstructed block.

[0100] The adder 235 may be referred to as a reconstructor or a reconstructed block generator. The generated reconstructed signal may be used for intra prediction of the next processing target block within the current picture, and may be used for inter prediction of the next picture through filtering as will be described later.

[0101] The filter 240 may improve the subjective / objective picture quality by applying filtering to the reconstructed signal. For example, the filter 240 may generate a modified reconstructed picture by applying various filtering methods to the reconstructed picture, and may send the modified reconstructed picture to the DPB 255 of the memory 250. For example, the various filtering methods may include deblocking filtering, sample adaptive offset SAO, adaptive loop filter ALF, and bilateral filter.

[0102] The reconstructed picture sent (modified) in the DPB 255 of the memory 250 can be used as a reference picture in the inter-frame predictor 260. The memory 250 can store the motion information of the block whose motion information in the current picture is derived (or decoded) and / or the motion information of the block in the reconstructed picture. The stored motion information can be sent to the inter-frame prediction unit 260 to be used as the motion information of the spatial neighboring block or the motion information of the temporal neighboring block. The memory 250 can store the reconstructed samples of the reconstructed blocks in the current picture, and can send the reconstructed samples to the intra-frame prediction unit 265.

[0103] In the present disclosure, the embodiments described in the filter 160, the inter-frame predictor 180, and the intra-frame predictor 185 of the encoding device 100 may be applied to the filter 240, the inter-frame predictor 260, and the intra-frame predictor 265 of the decoding device 200, respectively, in the same or in a corresponding manner.

[0104] Figure 4 An example of a content streaming system according to an embodiment of the present disclosure is shown. The content streaming system to which the present disclosure is applied may mainly include an encoding server 410 , a streaming server 420 , a web server 430 , a media storage 440 , a user device 450 , and a multimedia input device 460 .

[0105] The encoding server 410 may compress the content input from the multimedia input device 460 (such as a smartphone, a camera, a camcorder, etc.) into digital data to generate a bitstream and transmit it to the streaming server 420. As another example, when the multimedia input device 460 (such as a smartphone, a camera, and a camcorder) directly generates a bitstream, the encoding server 410 may be omitted.

[0106] The bitstream may be generated by applying the encoding method or the bitstream generating method of the present disclosure, and the streaming server 420 may temporarily store the bitstream in the process of transmitting or receiving the bitstream.

[0107] The streaming server 420 transmits multimedia data to the user device 450 through the web server 430 based on the user request, and the web server 430 serves as an intermediary to inform the user what services exist. When the user requests a desired service through the web server 430, the web server 430 transfers information about the desired service to the streaming server 420, and the streaming server 420 transmits the multimedia data to the user. At this time, the content streaming system may include a separate control server, in which case the control server is used to control commands / responses between devices in the content streaming system.

[0108] The streaming server 420 may receive content from the media storage 440 and / or the encoding server 410. For example, the streaming server 420 may receive content in real time from the encoding server 410. In this case, in order to provide a smooth streaming service, the streaming server 420 may store the bitstream for a predetermined time.

[0109] For example, user equipment 450 may include mobile phones, smart phones, laptop computers, terminals for digital broadcasting, personal digital assistants PDAs, portable multimedia players PMPs, navigation terminals, tablet computers, tablet PCs, ultrabooks, wearable devices (e.g., smart watches, smart glasses, head-mounted displays HMDs, digital TVs, desktop computers, and digital signs).

[0110] Each server in the content streaming system may operate as a distributed server, and in this case, data received from each server may be processed in a distributed manner.

[0111] Figure 5 An example of an apparatus for processing a video signal according to an embodiment of the present disclosure is shown. The video signal processing apparatus may correspond to Figure 1 The encoding device 100 or Figure 2 A decoding device 200.

[0112] The video signal processing device 500 for processing a video signal may include a memory 520 for storing a video signal and a processor 510 for processing the video signal while being combined with the memory 520. The processor 510 according to an embodiment of the present disclosure may be configured with at least one processing circuit for processing a video signal, and may process the video signal by executing an instruction for encoding or decoding a video signal. That is, the processor 510 may encode the original video data or decode the encoded video signal by executing the encoding or decoding method described below. The processor 510 may include a processor corresponding to Figure 2 or Figure 3 The memory 520 may correspond to one or more processors of each module. Figure 2 Memory 170 or Figure 3 of memory 250.

[0113] Partitioning Structure

[0114] The video / image coding method according to the present disclosure may be performed based on the partition structure to be described below. Specifically, processes such as prediction, residual processing (e.g., (inverse) transform, (de)quantization), syntax element coding, and filtering may be performed on the coding tree unit (CTU) and CU (and / or TU or PU) derived based on the partition structure. The block partitioning process according to the present disclosure is performed by the image segmentation unit 110 of the encoding device 100 described above, and the partition-related information may be processed (encoded) by the entropy encoding unit 190 and transmitted to the decoding device 200 in the form of a bitstream. The entropy decoding unit 210 of the decoding device 200 may derive the block partition structure of the current block based on the partition-related information obtained from the bitstream, and perform a series of processes for image decoding (e.g., prediction, residual processing, block / picture reconstruction, in-loop filtering, etc.) based on the derived block partition structure.

[0115] When compiling a video / image according to an embodiment of the present disclosure, the image processing unit may have a hierarchical structure. A picture may be divided into one or more tiles or tile groups. A tile group may include one or more tiles. A tile may include one or more CTUs. A CTU may be partitioned into one or more CTUs. A tile is a rectangular area of ​​a CTU within a specific tile column and a specific tile row in a picture. A tile group may include an integer number of tiles according to a raster scan of tiles within a picture. A tile group header may convey information / parameters applicable to a corresponding tile group. When the encoding device 100 / decoding device 200 has a multi-core processor, the encoding / decoding process for tiles or tile groups may be processed in parallel. Here, the tile group may have a type of tile group, including an intra (I) tile group, a predictive (P) tile group, and a bidirectional predictive (B) tile group. For the prediction of blocks in an intra I tile group, inter-frame prediction is not used, but only intra-frame prediction may be used. Of course, even for I-picture block groups, original sample values ​​coded without prediction can be signaled. For blocks in P-picture block groups, intra prediction or inter prediction can be used, and when inter prediction is used, only unidirectional prediction can be used. Meanwhile, for blocks in B-picture block groups, intra prediction or inter prediction can be used, and when inter prediction is used, not only unidirectional prediction but also bidirectional prediction can be used.

[0116] Figure 6 An example of a partition structure of a picture according to an embodiment of the present disclosure is shown. Figure 6 In , a picture with 216 (18 by 12) luma CTUs is partitioned into 12 tiles and 3 tile groups.

[0117] The encoder may determine the tile / tile group and the maximum and minimum coding unit sizes according to characteristics of the video image (eg, resolution) or by considering the efficiency of coding or parallel processing.

[0118] The decoder can obtain information indicating whether the tile / tile group of the current picture and the CTU in the tile are partitioned into multiple coding units. When the decoder does not continuously obtain (decode) information but obtains (decodes) information only under specific conditions, coding efficiency can be improved.

[0119] The tile group header (tile group header syntax) may include information / parameters that may be commonly applied to tile groups. APS (ASP syntax) or PPS (PPS syntax) may include information / parameters that may be commonly applied to one or more pictures. SPS (SPS syntax) may include information / parameters that may be commonly applied to one or more sequences. VPS (VPS syntax) may include information / parameters that may be commonly applied to the entire video. The higher-level syntax in the present disclosure may include at least one of the following: APS syntax, PPS syntax, SPS syntax, and VPS syntax.

[0120] Furthermore, for example, information about the partitioning and construction of tiles / tile groups may be constructed by the encoder through a higher level syntax and then sent to the decoder in the form of a bitstream.

[0121] Figures 7a to 7d An example of a block partition structure according to an embodiment of the present disclosure is shown. Figure 7a , 7b 7c and 7d show examples of block partition structures by quadtree (QT), binary tree (BT), ternary tree (TT) and asymmetric tree (AT), respectively.

[0122] In a video coding system, a block can be partitioned based on the QT partition scheme. In addition, a sub-block partitioned by the QT partition scheme can be further recursively partitioned according to the QT partition scheme. Leaf blocks that are no longer partitioned by the QT partition scheme can be partitioned by at least one of BT, TT, or AT. BT can have two types of partitions, such as horizontal BT (2N×N, 2N×N) and vertical BT (N×2N, N×2N). TT can have two types of partitions, such as horizontal TT (2N×1 / 2N, 2N×N, 2N×1 / 2N) and vertical TT (1 / 2N×2N, N×2N, 1 / 2N×2N). AT can have four types of partitions, such as horizontal upward AT (2N×1 / 2N, 2N×3 / 2N), horizontal downward AT (2N×3 / 2N, 2N×1 / 2N), vertical left AT (1 / 2N×2N, 3 / 2N×2N), vertical right AT (3 / 2N×2N, 1 / 2N×2N). BT, TT, and AT can be further recursively partitioned by using BT, TT, and AT, respectively.

[0123] Figure 7aAn example of QT partitioning is shown. QT can partition block A into four sub-blocks A0, A1, A2 and A3. QT can partition sub-block A1 again into four sub-blocks B0, B1, B2 and B3.

[0124] Figure 7b An example of BT partitioning is shown. Block B3, which is no longer partitioned by QT, can be partitioned by vertical BT (C0, C1) or horizontal BT (D0, D1). Similar to block C0, each sub-block can be further recursively partitioned in the form of horizontal BT (E0, E1) or vertical BT (F0, F1).

[0125] Figure 7c An example of TT partitioning is shown. Block B3, which is no longer partitioned by QT, can be partitioned into vertical TT (C0, C1, C2) or horizontal TT (D0, D1, D2). Similar to block C1, each sub-block can be further recursively partitioned, similar to the form of horizontal TT (E0, E1, E2) or vertical TT (F0, F1, F2).

[0126] Figure 7d An example of AT partitioning is shown. Block B3, which is no longer partitioned by QT, can be partitioned into vertical AT (C0, C1) or horizontal AT (D0, D1). Similar to block C1, each sub-block can be further recursively partitioned, similar to the form of horizontal AT (E0, E1) or vertical TT (F0, F1).

[0127] At the same time, BT, TT and AT partitions can be applied to a block at the same time. For example, a sub-block partitioned by BT can be partitioned by TT or AT. In addition, a sub-block partitioned by TT can be partitioned by BT or AT. A sub-block partitioned by AT can be partitioned by BT or TT. For example, after horizontal BT partitioning, each sub-block can be partitioned by vertical BT. In addition, after vertical BT partitioning, each sub-block can be partitioned by horizontal BT. In this case, the partition order is different, but the final partition shape is the same.

[0128] In addition, when a block is partitioned, the order in which the block is searched may be defined differently. Generally, the search may be performed from left to right and from top to bottom, and the search of the block may represent the order in which it is determined whether to further partition each partitioned sub-block, the coding order of each sub-block when each sub-block is no longer partitioned, or the search order when referring to information of another adjacent block in the sub-block.

[0129] In addition, virtual pipeline data units (VPDUs) can be defined for pipeline processing in a picture. VPDUs can be defined as non-overlapping units in one picture. In a hardware decoder, consecutive VPDUs can be processed simultaneously by multiple pipeline stages. In most pipeline stages, the VPDU size is roughly proportional to the buffer size. Therefore, it is important to keep the VPDU size small when considering the buffer size in hardware. In most hardware decoders, the VPDU size can be configured to be equal to the maximum TB size. For example, the VPDU size can be 64×64 (64×64 luma samples) in size. However, this is an example, and the VPDU size can be changed (increased or decreased) by considering the above-mentioned TT and / or BT partitions.

[0130] Figure 8 An example of a case where TT and BT partitions are limited according to an embodiment of the present disclosure is shown. In order to keep the VPDU size to the size of 64×64 luma samples, Figure 8 At least one of the following restrictions applies.

[0131] - For CUs with width or height or both width and height equal to 128, TT partitioning is not allowed.

[0132] - For 128xN CUs with N <= 64 (i.e., width equal to 128 and height less than 128), horizontal BT is not allowed.

[0133] - For Nx128 CUs with N <= 64 (i.e., height equal to 128 and width less than 128), vertical BT is not allowed.

[0134] Image / Video Compilation Process

[0135] In image / video coding, pictures that construct an image / video can be encoded / decoded according to a series of decoding orders. The picture order corresponding to the output order of the decoded pictures can be configured differently from the coding order, and not only forward prediction but also reverse prediction can be performed.

[0136] Fig. 9 An example of a flowchart for encoding a picture constituting a video signal according to an embodiment of the present disclosure is shown. Fig. 9 In step S910, Figure 2The step S910 may include the inter / intra prediction process described in the present disclosure, the step S920 may include the residual processing process described in the present disclosure, and the step S930 may include the information encoding process described in the present disclosure.

[0137] Reference Fig. 9 , the picture encoding process may illustratively include encoding information (eg, prediction information, residual information, and partition information) and performing the encoding as follows Figure 2 The process of outputting the encoding information in the form of a bitstream described in , and the process of generating a reconstructed picture of the current picture and the process of applying in-loop filtering to the reconstructed picture (optional). In step S910, the encoding device 100 may derive (modified) residual samples from the transform coefficients quantized by the dequantizer 140 and the inverse transformer 150, and may generate a reconstructed picture based on the prediction samples corresponding to the output and the (modified) residual samples. The generated reconstructed picture may be the same as the reconstructed picture generated by the decoding device 200. The modified reconstructed picture may be generated by an in-loop filtering process for reconstructing the picture, and the modified reconstructed picture may be stored in the memory 170 (DPB 175), and as in the case of the decoding device 200, the modified reconstructed picture may be used as a reference picture in the inter-frame prediction process during encoding of the picture. As described above, in some cases, part or all of the in-loop filtering process may be omitted. When an in-loop filtering process is performed, (in-loop) filtering related information (parameters) may be encoded by the entropy encoder 190 and output in the form of a bit stream, and the decoding device 200 may perform the in-loop filtering process in the same manner as the encoding device 100 based on the filtering related information.

[0138] Through the in-loop filtering process, noise generated during video / motion picture coding, such as block artifacts and ringing artifacts, can be reduced, and subjective / objective visual quality can be improved. In addition, both the encoding device 100 and the decoding device 200 perform the in-loop filtering process, and as a result, the encoding device 100 and the decoding device 200 can derive the same prediction process, increase the reliability of picture coding, and reduce the amount of data sent for picture coding.

[0139] Fig.10 FIG. 1 shows an example of a flowchart for decoding a picture constituting a video signal according to an embodiment of the present disclosure. Step S1010 may be performed by Figure 3The entropy decoding unit 210 in the decoding device 200 of the present invention may perform step S1020, the predictors 260 and 265 may perform step S1030, the residual processing units 220 and 230 may perform step S1040, the adding unit 235 may perform step S1050, and the filter 240 may perform step S1010. Step S1020 may include the information decoding process described in the present disclosure, step S1020 may include the inter / intra prediction process described in the present disclosure, step S1030 may include the residual processing process described in the present disclosure, step S1040 may include the block / picture construction process described in the present disclosure, and step S1050 may include the in-loop filtering process described in the present disclosure.

[0140] Reference Fig.10 The picture decoding process can schematically include a process of obtaining image / video information from a bit stream (through decoding) (S1010), a picture reconstruction process (S1020 to S1040), and Figure 2 The in-loop filtering process (S1050) of the reconstructed picture shown. The picture reconstruction process can be performed based on the prediction samples and residual samples obtained by the process of the inter-frame prediction S1020 and the residual processing S1030 (dequantization and inverse transformation of the quantized transform coefficients) described in the present disclosure. The modified reconstructed picture can be generated by the in-loop filtering process of the reconstructed picture generated by the picture reconstruction process, and the modified reconstructed picture can be output as a decoded picture. In addition, the modified reconstructed picture can be stored in the DPB 255 of the decoding device 200 and used as a reference picture in the inter-frame prediction process during the subsequent decoding of the picture. In some cases, the in-loop filtering process can be omitted, and in this case the reconstructed picture can be output as a decoded picture. In addition, the modified reconstructed picture can be stored in the DPB 255 of the decoding device 200 and used as a reference picture in the inter-frame prediction process during the subsequent decoding of the picture. The in-loop filtering process S1050 may include a deblocking filtering process, a sample adaptive offset (SAO) process, an adaptive loop filtering (ALF) process, and / or a bilateral filtering process, and some or all of them may be omitted. In addition, one or some of the deblocking filtering process, the SAO process, the ALF process, and the bilateral filtering process may be applied in sequence, or all processes may be applied in sequence. For example, after the deblocking filtering process is applied to the reconstructed image, the SAO process may be performed. In addition, for example, after the deblocking filtering process is applied to the reconstructed image, the ALF process may be performed. This may be similarly performed even in the encoding device 100.

[0141] As described above, the picture reconstruction process can be performed even in the encoding device 100 other than the decoding device 200. A reconstructed block can be generated based on intra prediction / inter prediction in units of each block, and a reconstructed picture including the reconstructed block can be generated. When the current picture / slice / tile group is an I picture / slice / tile group, the blocks included in the current picture / slice / tile group can be reconstructed based only on intra prediction. In this case, inter prediction can be applied to some blocks in the current picture / slice / tile group, and intra prediction can be applied to some remaining blocks. The color components of the picture may include a luminance component and a chrominance component, and when not explicitly limited in the present disclosure, the method and embodiments proposed in the present disclosure may be applied to the luminance component and the chrominance component.

[0142] Example of compilation layers and structures

[0143] Fig.11 An example of a hierarchical structure of an encoded image according to an embodiment of the present disclosure is shown.

[0144] The coded image can be divided into: a video coding layer (VCL), which performs a decoding process of the image and handles the decoding process; a lower layer system, which transmits and stores coding information; and a network abstraction layer (NAL), which exists between the VCL and the lower layer system and is used to perform a network adaptation function.

[0145] VCL data including image data (tile group data) compressed in the VCL or an additional required supplemental enhancement information (SEI) message may be generated during decoding of a parameter set including information such as a picture parameter set (PPS), a sequence parameter set (SPS) or a video parameter set (VPS) or an image.

[0146] In NAL, header information (NAL unit data) is added to the raw byte sequence payload (RSRP) generated in the VCL to generate a NAL unit. In this case, RSRP can be referred to as tile group data, parameter sets, and SEI messages generated in the VCL. The NAL unit header may include NAL unit type information specified according to the RSRP data included in the corresponding NAL unit.

[0147] like Fig.11 As shown, the NAL unit can be divided into a VCL NAL unit and a non-VCL NAL unit according to the RSRP generated in the VCL. The VCL NAL unit may refer to a NAL unit including information about the image (tile group data), and the non-VCL NAL unit may refer to a NAL unit including information required for decoding the image (parameter set or SEI message).

[0148] VCL NA units and non-VCL NAL units can be sent over the network while adding header information according to the data specification. For example, the NAL unit can be converted into a predetermined standardized data form (such as H.266 / VVC file format, real-time transport protocol (RTP) and transport stream (TS)) and sent over various networks.

[0149] As described above, for a NAL unit, a NAL unit type may be specified according to an RBSP data structure included in a corresponding NAL unit, and information about the NAL unit type may be stored in a NAL unit header and signaled.

[0150] For example, NAL units can generally be classified into VCL NAL unit types and non-VCL NAL unit types according to whether the NAL unit includes information about the image (tile group data). VCL NAL unit types can be classified according to the attributes and types of pictures included in the VCL NAL unit, and non-VCL NAL units can be classified according to the types of parameter sets.

[0151] The following are examples of NAL unit types specified according to the type of parameter sets included in a non-VCL NAL unit type.

[0152] - Adaptation Parameter Set (APS) NAL unit: The type of NAL unit that includes APS

[0153] - Video Parameter Set (VPS) NAL unit: the type of NAL unit that contains the VPS

[0154] - Sequence Parameter Set (SPS) NAL unit: the type of NAL unit that includes the SPS

[0155] -Picture parameter set (PPS) NAL unit: the type of NAL unit that includes the PPS

[0156] The above-mentioned NAL unit type may have syntax information of the NAL unit type, and the syntax information may be stored in the NAL unit header and signaled. For example, the syntax information may be nal_unit_type, and the NAL unit type may be specified by the value of nal_unit_type.

[0157] The tile group header (tile group header syntax) may include information / parameters that may be generally applicable to tile groups. APS (ASP syntax) or PPS (PPS syntax) may include information / parameters that may be generally applicable to one or more pictures. SPS (SPS syntax) may include information / parameters that may be generally applicable to one or more sequences. VPS (VPS syntax) may include information / parameters that may be generally applicable to the entire video. The higher-level syntax in the present disclosure may include at least one of the following: APS syntax, PPS syntax, SPS syntax, and VPS syntax.

[0158] In the present disclosure, the image / video information encoded from the encoding device 100 to the decoding device 200 and notified by a signal in the form of a bit stream may include information related to intra-picture partitions, intra-frame / inter-frame prediction information, residual information and in-loop filtering information, and may include information included in the APS, information included in the PPS, information included in the SPS and / or information included in the VPS.

[0159] Inter prediction

[0160] Hereinafter, the inter-frame prediction technology according to the embodiment of the present disclosure will be described. The inter-frame prediction described below can be performed by Figure 2 The inter-frame predictor 180 of the encoding device 100 or Figure 3 The inter-frame predictor 260 of the decoding device 200 performs, and according to an embodiment of the present invention, the encoded data can be stored in the form of a bit stream.

[0161] The prediction unit of the encoding device 100 / decoding device 200 may derive prediction samples by performing inter-frame prediction in units of blocks. Inter-frame prediction may be a prediction derived in a manner that depends on data elements (e.g., sample values ​​or motion information) of a picture other than the current picture. When inter-frame prediction is applied to the current block, the prediction block (prediction sample array) of the current block may be derived based on a reference block (reference sample array) specified by a motion vector on a reference picture indicated by a reference picture index. In this case, in order to reduce the amount of motion information sent in the inter-frame prediction mode, the motion information of the current block may be predicted in units of blocks, sub-blocks, or samples based on the correlation of motion information between adjacent blocks and the current block. The motion information may include a motion vector and a reference picture index. The motion information may also include inter-frame prediction type (L0 prediction, L1 prediction, Bi prediction, etc.) information. In the case of applying inter-frame prediction, the adjacent blocks may include spatial adjacent blocks present in the current picture and temporal adjacent blocks present in the reference picture. The reference picture including the reference block and the reference picture including the temporal adjacent block may be the same as or different from each other. Temporally adjacent blocks may be referred to by names such as co-located reference blocks, co-located CUs (colCUs), and the like, and reference pictures including temporally adjacent blocks may be referred to as co-located pictures (colPic). For example, a motion information candidate list may be configured based on adjacent blocks of a current block, and a flag or index information indicating which candidate is selected (used) may be signaled to derive a motion vector and / or a reference picture index of the current block. Inter-frame prediction may be performed based on various prediction modes, and for example, in the case of skip mode and merge mode, the motion information of the current block may be the same as the motion information of the selected adjacent block. In the case of skip mode, unlike merge mode, a residual signal may not be sent. In the case of a motion vector prediction (MVP) mode, the motion vector of the selected adjacent block may be used as a motion vector predictor, and the motion vector difference may be signaled. In this case, the motion vector of the current block may be derived by using the sum of the motion vector predictor and the motion vector difference.

[0162] Fig.12 is an example of a flowchart of inter-frame prediction in an encoding process of a video signal according to an embodiment of the present specification, and Fig.13 An example of an inter-frame prediction unit in the encoding device according to an embodiment of the present specification is shown.

[0163] The encoding device 100 performs inter prediction on the current block (S1210). The encoding device 100 may derive the inter prediction mode and motion information of the current block, and may generate a prediction sample of the current block. In this case, the inter prediction mode determination, motion information derivation, and prediction sample generation processes may be performed simultaneously, and any one process may be performed before another process. For example, the inter predictor 180 of the encoding device 100 may include a prediction mode determination unit 181, a motion information derivation unit 182, and a prediction sample derivation unit 183. The prediction mode determination unit 181 may determine the prediction mode of the current block. The motion information derivation unit 182 may derive motion information of the current block. The prediction sample derivation unit 183 may derive a prediction sample of the current block. For example, the inter predictor 180 of the encoding device 100 may search a given area (search area) of a reference image through motion estimation to find a block similar to the current block, and may derive a reference block having a minimum difference or a difference of a given reference or less relative to the current block. The inter-frame predictor 180 may derive a reference picture index indicating where the reference block is located based on the reference block, and may derive a motion vector based on the position difference between the reference block and the current block. The encoding device 100 may determine a mode applied to the current block among various prediction modes. The encoding device may compare the rate-distortion RD costs of various prediction modes, and may determine the best prediction mode for the current block.

[0164] For example, if the skip mode or the merge mode is applied to the current block, the encoding device 100 may configure a merge candidate list described later, and may derive a reference block having a minimum difference or a difference of a given reference or less with respect to the current block among the reference blocks indicated by the merge candidates included in the merge candidate list. In this case, a merge candidate associated with the derived reference block may be selected. Merge index information indicating the selected merge candidate may be generated and signaled to the decoding device 200. The motion information of the current block may be derived using the motion information of the selected merge candidate.

[0165] For another example, if the (A) MVP mode is applied to the current block, the encoding device may configure the (A) MVP candidate list described later, and may use the motion vector of the MVP candidate selected from the motion vector predictor (MVP) candidates included in the (A) MVP candidate list as the MVP of the current block. In this case, for example, a motion vector indicating a reference block derived by motion estimation may be used as the motion vector of the current block. The MVP candidate including a motion vector having the smallest difference value relative to the motion vector of the current block among the MVP candidates may become the selected MVP candidate. A motion vector difference (MVD), that is, a difference obtained by subtracting the MVP from the motion vector of the current block may be derived. In this case, information about the MVD may be signaled to the decoding device 200. In addition, if the (A) MVP mode is applied, the value of the reference picture index may be configured as reference picture index information, and may be signaled separately to the decoding device.

[0166] The encoding device 100 may induce residual samples based on the prediction samples (S1220). The encoding device 100 may induce residual samples through comparison between original samples and prediction samples of the current block.

[0167] The encoding device 100 encodes the image information including prediction information and residual information (S1230). The encoding device may output the encoded image information in the form of a bitstream. The prediction information may include information about prediction mode information (e.g., a skip flag, a merge flag, or a mode index) and motion information as information related to the prediction process. The information related to the motion information may include candidate selection information (e.g., a merge index, an mvp flag, or an mvp index), that is, information for deriving a motion vector. In addition, the information related to the motion information may include information about the MVD and / or reference picture index information. In addition, the information related to the motion information may include information indicating whether L0 prediction, L1 prediction, or bi-prediction is applied. The residual information is information about the residual sample. The residual information may include information about the quantized transform coefficients of the residual sample. The prediction mode information and the motion information may be collectively referred to as inter-frame prediction information.

[0168] The output bitstream may be stored in a (digital) storage medium and sent to a decoding device, or may be sent to a decoding device over a network.

[0169] At the same time, as described above, the encoding device can generate a reconstructed picture (including a reconstructed sample and a reconstructed block) based on the reference sample and the residual sample. This is to derive the same prediction result as the prediction result performed in the decoding device 200 in the encoding device 100. Therefore, the coding efficiency can be improved. Therefore, the encoding device 100 can store the reconstructed picture (or the reconstructed sample and the reconstructed block) in the memory, and can use the reconstructed picture as a reference picture for inter-frame prediction. As described above, the in-loop filtering process can also be applied to the reconstructed picture.

[0170] Fig.14 is an example of a flowchart for performing inter-frame prediction in a decoding process of a video signal according to an embodiment of the present specification, Fig.15 An example of an inter-frame prediction unit in a decoding device according to an embodiment of the present specification is shown.

[0171] The decoding device 200 may perform an operation corresponding to the operation performed in the encoding device 100. The decoding device 200 may perform prediction on the current block based on the received prediction information, and may derive a prediction sample.

[0172] Specifically, the decoding apparatus 200 may determine a prediction mode for a current block based on the received prediction information (S1410). The decoding apparatus 200 may determine which inter prediction mode to apply to the current block based on prediction mode information within the prediction information.

[0173] For example, the decoding device 200 may determine whether to apply the merge mode or the (A) MVP mode to the current block based on the merge flag. Alternatively, the decoding device 200 may select one of various inter prediction mode candidates based on the mode index. The inter prediction mode candidate may include a skip mode, a merge mode, and / or a (A) MVP mode, or may include various inter prediction modes described later.

[0174] The decoding device 200 derives motion information of the current block based on the determined inter prediction mode (S1420). For example, if the skip mode or the merge mode is applied to the current block, the decoding device 200 may configure a merge candidate list described later and select one of the merge candidates included in the merge candidate list. The selection of the merge candidate may be performed based on the merge index. The motion information of the current block may be derived from the motion information of the selected merge candidate. The motion information of the selected merge candidate may be used as the motion information of the current block.

[0175] For another example, if the (A) MVP mode is applied to the current block, the decoding device 200 may configure a (A) MVP candidate list described later, and may use a motion vector of a motion vector predictor (MVP) candidate selected from the MVP candidates included in the (A) MVP candidate list as the MVP of the current block. The selection may be performed based on selection information (MVP flag or MVP index). In this case, the decoding device 200 may derive the MVD of the current block based on information about the MVD. The decoding device may derive the motion vector of the current block based on the MVP and MVD of the current block. In addition, the decoding device may derive the reference picture index of the current block based on the reference picture index information. The picture indicated by the reference picture index within the reference picture list regarding the current block may be derived as a reference picture referenced for inter-frame prediction of the current block.

[0176] At the same time, as will be described later, the motion information of the current block can be derived without a candidate list configuration. In this case, the motion information of the current block can be derived according to the process disclosed in the prediction mode described later. In this case, the candidate list configuration such as described above can be omitted.

[0177] The decoding device 200 may generate a prediction sample of the current block based on the motion information of the current block (S1430). In this case, the decoding device 200 may derive a reference image based on a reference picture index of the current block, and may derive a prediction sample of the current block indicated on the reference image through a motion vector of the current block. In this case, as will be described later, a prediction sample filtering process may be further performed on some or all of the prediction samples of the current block, depending on circumstances.

[0178] For example, the inter-frame predictor 260 of the decoding device 200 may include a prediction mode determination unit 261, a motion information derivation unit 262, and a prediction sample derivation unit 263. The decoding device 200 may determine the prediction mode of the current block based on the prediction mode information received from the prediction mode determination unit 261, and may derive motion information (motion vector and / or reference picture index) of the current block based on the information related to the motion information received from the motion information derivation unit 262. The prediction sample derivation unit 263 may derive the prediction sample of the current block.

[0179] The decoding device 200 generates a residual sample of the current block based on the received residual information (S1440). The decoding device 200 may generate a reconstructed sample of the current block based on the predicted sample and the residual sample, and may generate a reconstructed picture based on the reconstructed sample (S1450). Thereafter, as described above, an in-loop filtering process may be further applied to the reconstructed picture.

[0180] As described above, the inter prediction process may include an inter prediction mode determining step, a motion information deriving step according to the determined prediction mode, and a prediction performing (prediction sample generating) step based on the derived motion information.

[0181] Determination of inter-frame prediction mode

[0182] Various inter prediction modes can be used to predict the current block within the picture. For example, various modes such as merge mode, skip mode, MVP mode, and affine mode can be used. Decoder-side motion vector refinement (DMVR) mode, adaptive motion vector resolution (AMVR) mode, etc. can also be used as additional modes. Affine mode can be called affine motion prediction mode. MVP mode can be called advanced motion vector prediction (AMVP) mode.

[0183] Prediction mode information indicating the inter prediction mode of the current block may be signaled from the encoding device to the decoding device. The prediction mode information may be included in a bitstream and received by the decoding device. The prediction mode information may include index information indicating one of a plurality of candidate modes. Alternatively, the inter prediction mode may be indicated by hierarchical signaling of flag information. In this case, the prediction mode information may include one or more flags. For example, the encoding device 100 may indicate whether the skip mode is applied by signaling a skip flag, indicate whether the merge mode is applied by signaling a merge flag when the skip mode is not applied, and indicate the application of the MVP mode or further signal a flag for additional identification when the merge mode is not applied. The affine mode may be signaled as an independent mode or may be signaled as a mode subordinate to the merge mode or the MVP mode. For example, the affine mode may be configured as one of the merge candidate list or the MVP candidate list, as will be described later.

[0184] Derivation of motion information

[0185] The encoding device 100 or the decoding device 200 may perform inter-frame prediction using the motion information of the current block. The encoding device 100 may derive the best motion information of the current block according to the motion estimation process. For example, the encoding device 100 may use the original block in the original picture for the current block to search for a similar reference block with high correlation in units of fractional pixels within a determined search range within the reference picture. Therefore, the encoding device may derive motion information. The similarity of the blocks may be derived based on the difference between the phase-based sample values. For example, the similarity of the blocks may be calculated based on the SAD (sum of absolute differences) between the current block (or the template of the current block) and the reference block (or the template of the reference block). In this case, the motion information may be derived based on the reference block with the minimum SAD within the search area. The derived motion information may be signaled to the decoding device using several methods based on the inter-frame prediction mode.

[0186] Merge mode and skip mode

[0187] If the merge mode is applied, the motion information of the current prediction block is not directly sent, and the motion information of the adjacent prediction block is used to derive the motion information of the current prediction block. Therefore, the encoding device 100 can indicate the motion information of the current prediction block by sending flag information for notifying that the merge mode has been used and a merge index for notifying which adjacent prediction block has been used.

[0188] The encoding device 100 should search for a merge candidate block for deriving motion information of the current prediction block in order to perform a merge mode. For example, up to 5 merge candidate blocks may be used, but the present disclosure is not limited thereto. In addition, the maximum number of merge candidate blocks may be sent in a slice header, and the present disclosure is not limited thereto. After searching for the merge candidate blocks, the encoding device 100 may generate a merge candidate list, and may select a merge candidate block with the minimum cost among the merge candidate blocks as the final merge candidate block.

[0189] For example, the merge candidate list may use 5 merge candidate blocks. For example, 4 spatial merge candidates and 1 temporal merge candidate may be used.

[0190] Fig.16 An example of spatially neighboring blocks used as spatial merging candidates according to an embodiment of the present specification is shown.

[0191] Reference Fig.16 , for the prediction of the current block, at least one of the following can be used: left neighboring block A1, lower left neighboring block A0, upper right neighboring block B0, upper neighboring block B1 and upper left neighboring block B2. Fig.12 The merge candidate list of the current block is configured by the process shown in .

[0192] Fig.17is a flowchart illustrating a method of configuring a merge candidate list according to an embodiment to which the present disclosure is applied.

[0193] The coding device (coding device 100 or decoding device 200) searches for spatial neighboring blocks of the current block, and inserts the derived spatial merge candidates into the merge candidate list (S1710). For example, the spatial neighboring blocks may include a lower left neighboring block, a left neighboring block, an upper right neighboring block, an upper neighboring block, and an upper left neighboring block of the current block. In this case, this is an example, and in addition to these spatial neighboring blocks, additional neighboring blocks such as right neighboring blocks, lower neighboring blocks, and lower right neighboring blocks may be further used as spatial neighboring blocks. The coding device may detect available blocks by searching for spatial neighboring blocks based on priority, and may derive motion information of the detected blocks as spatial merge candidates. For example, the coding device 100 or the decoding device 200 may search in the order of A1, B1, B0, A0, and B2. Fig.11 The 5 blocks shown in , and the merge candidate list can be configured by indexing the available candidates in sequence.

[0194] The compilation device searches for temporally adjacent blocks of the current block and inserts the derived temporal merge candidates into the merge candidate list (S1720). The temporally adjacent blocks may be located on a reference picture (i.e., a picture different from the current picture on which the current block is located). The reference picture on which the temporally adjacent blocks are located may be referred to as a co-located picture or a col picture. The temporally adjacent blocks may be searched in the order of the bottom-right adjacent block and the bottom-right center block of the co-located block for the current block on the col picture. Meanwhile, if motion data compression is applied, specific motion information may be stored in the col picture as the representative motion information for each given storage unit. In this case, it is not necessary to store the motion information of all blocks within the given storage unit, so a motion data compression effect can be obtained. In this case, for example, the given storage unit may be predetermined as a 16×16 sample unit or an 8×8 sample unit, or the size information of the given storage unit may be signaled from the encoding device 100 to the decoding device 200. If motion data compression is applied, the motion information of the temporally adjacent blocks may be replaced with the representative motion information of the given storage unit in which the temporally adjacent blocks are located. That is, in this case, in terms of the implementation manner, after performing an arithmetic right shift by a given value based on the coordinates of the temporally adjacent block (top-left sample position) rather than the coordinates of the prediction block in which the temporally adjacent block is located, the temporal merge candidates may be derived based on the motion information of the prediction block covering the arithmetic left shift position. For example, if the given storage unit is a 2n×2n sample unit, assuming the coordinates of the temporally adjacent block are (xTnb, yTnb), the motion information of the prediction block located at ((xTnb>>n)<<n), (yTnb>>n)<<n)) (i.e., the modified position) may be used for the temporal merge candidates. Specifically, for example, if the given storage unit is a 16×16 sample unit, assuming the coordinates of the temporally adjacent block are (xTnb, yTnb), the motion information of the prediction block located at ((xTnb>>4)<<4), (yTnb>>4)<<4)) (i.e., the modified position) may be used for the temporal merge candidates. Alternatively, for example, if the given storage unit is an 8×8 sample unit, assuming the coordinates of the temporally adjacent block are (xTnb, yTnb), the motion information of the prediction block located at ((xTnb>>3)<<3), (yTnb>>3)<<3)) (i.e., the modified position) may be used for the temporal merge candidates.

[0195] The coding device may check whether the current number of merge candidates is less than the maximum number of merge candidates (S1730). The maximum number of merge candidates may be predefined, or may be signaled from the encoding device 100 to the decoding device 200. For example, the encoding device 100 may generate information about the maximum number of merge candidates, may encode the information, and may transmit the information to the decoding device 200 in the form of a bitstream. If the maximum number of merge candidates is filled, the candidate addition process may not be performed.

[0196] If, as a result of the check, the current number of merge candidates is less than the maximum number of merge candidates, the coding apparatus inserts the added merge candidate into the merge candidate list (S1740). For example, the added merge candidate may include ATMVP (Adaptive Temporal Motion Vector Prediction), a combined bi-predictive merge candidate (if the slice type of the current slice is B type), and / or a zero vector merge candidate.

[0197] MVP Pattern

[0198] Fig.18 is an example of a flowchart for constructing a motion vector predictor (MVP) candidate list according to an embodiment of the present specification.

[0199] The MVP mode may be referred to as advanced MVP or adaptive MVP (AMVP). If the motion vector prediction (MVP) mode is applied, the reconstructed spatial neighboring blocks (e.g., Fig.16The motion vector predictor (MVP) candidate list is generated by using the motion vector of the neighboring block described in (i) and / or the motion vector corresponding to the temporal neighboring block (or Col block). That is, the motion vector of the reconstructed spatial neighboring block and / or the motion vector of the temporal neighboring block can be used as the motion vector predictor candidate. The information about the prediction may include selection information (e.g., MVP flag or MVP index) indicating the best motion vector predictor candidate selected from the motion vector predictor candidates included in the list. In this case, the predictor can select the motion vector predictor of the current block from the motion vector predictor candidates included in the motion vector candidate list using the selection information. The predictor of the encoding device 100 can calculate the motion vector difference (MVD) between the motion vector of the current block and the motion vector predictor, can encode the MVD, and can output the encoded MVD in the form of a bitstream. That is, the MVD can be calculated as a value obtained by subtracting the motion vector predictor from the motion vector of the current block. In this case, the predictor of the decoding device can obtain the motion vector difference included in the information about the prediction, and can derive the motion vector of the current block by adding the motion vector difference and the motion vector predictor. The predictor of the decoding device can obtain or derive the reference picture index indicating the reference picture from the information about the prediction. For example, the motion vector predictor candidate list can be as follows Fig.18 Configure as shown.

[0200] Reference Fig.18 , the coding device searches for a spatial candidate block for motion vector prediction and inserts it into a prediction candidate list (S1810). For example, the coding device may search for neighboring blocks according to a predetermined search order and add information of neighboring blocks that meet the conditions of the spatial candidate block to the prediction candidate list (MVP candidate list).

[0201] After constructing the spatial candidate block list, the coding device compares the number of spatial candidates included in the prediction candidate list with a preset reference number (e.g., 2) (S1820). If the number of spatial candidates included in the prediction candidate list is greater than or equal to the reference number (e.g., 2), the coding device may end the construction of the prediction candidate list.

[0202] However, if the number of spatial candidate blocks included in the prediction candidate list is less than the reference number (e.g., 2), the coding apparatus searches for a temporal candidate block and inserts it into the prediction candidate list (S1830), and when the temporal candidate block is not available, adds a zero motion vector to the prediction candidate list (S1840).

[0203] The prediction block of the current block can be derived based on the motion information derived according to the prediction mode. The prediction block may include prediction samples (prediction sample array) of the current block. When the motion vector of the current block indicates a fractional sample unit, an interpolation process may be performed, and through the interpolation process, the prediction sample of the current block may be derived based on the reference sample of the fractional sample unit in the reference picture. When affine inter-frame prediction is applied to the current block, the prediction sample may be generated based on the motion vector of the sample / sub-block unit. When bidirectional prediction is applied, the final prediction sample may be derived by the weighted (according to the phase) sum of the prediction sample derived based on the first direction prediction (e.g., L0 prediction) and the prediction sample derived based on the second direction prediction. Reconstructed samples and reconstructed pictures may be generated based on the derived prediction samples, and as described above, processes such as in-loop filtering may be subsequently performed.

[0204] Meanwhile, when the MVP mode is applied, the reference picture index may be explicitly signaled. In this case, the reference picture index refdIxL0 for L0 prediction and the reference picture index refdIxL1 for L1 prediction may be separately signaled. For example, when the MVP mode is applied and bidirectional prediction is applied, both information about refdIxL0 and information about refdIxL1 may be signaled.

[0205] When the MVP mode is applied, as described above, information about the MVD derived by the encoding device 100 may be signaled to the decoding device 200. The information about the MVD may include, for example, an MVD absolute value and information about the x and y components of a symbol. In this case, whether the MVD absolute value is greater than 0 (abs_mvd_greater0_flag), whether the MVD absolute value is greater than 1, and information indicating the remainder of the MVD (abs_mvd_greater1_flag) may be signaled step by step. For example, only when the value of the flag information (abs_mvd_greater0_flag) indicating whether the MVD absolute value is greater than 0 is 1, the information indicating whether the MVD absolute value is greater than 1 (abs_mvd_greater1_flag) may be signaled.

[0206] For example, the information about the MVD is constructed by the syntax shown in Table 1 below, may be encoded by the encoding device 100 and signaled to the decoding device 200 .

[0207] [Table 1]

[0208]

[0209] For example, MVD[compIdx] may be derived based on abs_mvd_greater0_flag[compIdx] * (abs_mvd_minus2[compIdx] + 2) * (1 - 2 * mvd_sign_flag[compIdx]). Here, compIdx (or cpIdx) may represent an index of each component and may have a value of 0 or 1. compIdx 0 may indicate an x ​​component, and compIdx 1 may indicate a y component. However, this may be done by using a coordinate system other than the x and y coordinate system to indicate the value of each component as an example.

[0210] Meanwhile, the MVD (MVDL0) for L0 prediction and the MVD (MVDL1) for L1 prediction may be signaled separately, and the information about the MVD may include information about MVDL0 and / or information about MVDL1. For example, when the MVP mode is applied and bidirectional prediction is applied, both the information about MVDL0 and the information about MVDL1 may be signaled.

[0211] Symmetrical MVD (SMVD)

[0212] Fig.19 An example of a case where a symmetric motion vector difference (MVD) mode is applied according to an embodiment of the present disclosure is shown.

[0213] Meanwhile, when bidirectional prediction is applied, symmetric MVD (SMVD) may also be used by considering coding efficiency. In this case, some motion information may be omitted from being signaled. For example, when SMVD is applied to the current block, information about refidxL0, information about refidxL1, and information about MVDL1 may not be signaled from the encoding device 100 to the decoding device 200, but may be derived internally. For example, when the MVP mode and bidirectional prediction are applied to the current block, flag information indicating whether SMVD is applied (e.g., symmetric MVD flag information or sym_mvd_flag syntax element) may be signaled, and when the value of the flag information is 1, the decoding device 200 may determine that SMVD is applied to the current block.

[0214] When the SMVD mode is applied (i.e., when the value of the symmetric MVD flag information is 1), mvp_l0_lfag, mvp_l1_flag, and information about MVD0 may be explicitly signaled, and while signaling of the information about refidxL0, the information about refidxL1, and the information about MVDL1 is omitted, the information about refidxL0, the information about refidxL1, and the information about MVDL1 may be derived inside the decoder as described above. For example, refdIxL0 may be derived to indicate an index of a previous reference picture closest to the current picture in a picture order count (POC) order within a reference picture list 0 (which may be referred to as list 0, L0, or a first reference list). refidxL1 may be derived to indicate an index of a subsequent reference picture closest to the current picture in a POC order within a reference picture list 1 (which may be referred to as list 1, L1, or a second reference list). Furthermore, for example, each of refidxL0 and refidxL1 may be derived as 0, and further, for example, each of refidxL0 and refidxL1 may be derived as the minimum index having the same POC difference with respect to the current picture. As a more specific example, when it is assumed that [POC of the current picture] - [POC of the first reference picture indicated by refidxL0] is referred to as the first POC difference, and when [POC of the current picture] - [POC of the second reference picture indicated by refidxL1] is referred to as the second POC difference, only when the first POC difference and the second POC difference are equal to each other, the value of refidxL0 indicating the first reference picture may be derived as the value of refidxL0 of the current block, and the value of refidxL1 indicating the second reference picture may be derived as the value of refidxL1 of the current block. Furthermore, for example, when there are a plurality of sets in which the first POC difference and the second POC difference are equal to each other, refidxL0 and refidxL1 of a set having the smallest difference among the plurality of sets may be derived as refidxL0 and refidxL1 of the current block.

[0215] MVDL1 may be derived as -MVDL0. For example, the final MV of the current block may be derived as in Equation 1 below.

[0216] [Equation 1]

[0217]

[0218] In Equation 1, mvx0 and mvy0 represent the x and y components of the L0 directional motion vector of the current block, and mvx1 and mvy1 represent the x and y components of the motion vector for L0 directional prediction of the current block, and represent the x and y components of the motion vector for L1 directional prediction. mvp0 and mvp0 represent the motion vector of MVP for L0 directional prediction (L0 basic motion vector), and mvp1 and mvp1 represent the motion vector of MVP for L1 directional prediction (L1 basic motion vector). mvd0 and mvd0 represent the x and y components of MVD for L0 directional prediction. According to Equation 1, the MVD for L1 directional prediction has the same value as the L0 MVD, but has an opposite sign to the L0 MVD.

[0219] Affine Mode

[0220] The conventional video coding system uses one motion vector in order to express the motion of the coding block (using a translation motion model). However, in the method using one motion vector, the optimal motion can be expressed in units of blocks, but for each pixel, the corresponding motion is not actually the optimal motion. As a result, if the optimal motion vector is determined in units of pixels, the coding efficiency can be improved. For this reason, in an embodiment, a method of affine motion prediction for performing encoding / decoding by using an affine motion model is described. In the affine motion prediction method, a motion vector can be expressed in units of each pixel of a block by using 2, 3, or 4 motion vectors.

[0221] Fig. 20 An example of an affine motion model according to an embodiment of the present disclosure is shown.

[0222] The affine motion model can express four motions, such as Fig.16 As shown in . An affine motion model that represents three motions (translation, scaling, and rotation) among the motions that can be expressed by the affine motion model is called a similar (or simplified) affine motion model, and in the present disclosure, a method proposed based on the similar (or simplified) affine motion model is described. However, embodiments of the present invention are not limited to the similar (or simplified) affine motion model.

[0223] Fig.21a and 21b An example of a motion vector of each control point according to an embodiment of the present disclosure is shown.

[0224] like Fig.21a and 21b As shown, in affine motion prediction, a motion vector may be determined for each pixel position included in a block by using two or more control point motion vectors (CPMVs).

[0225] About the 4-parameter affine motion model ( Fig.21a), the motion vector at the sample position (x, y) can be derived as shown in Equation 2 below.

[0226] [Equation 2]

[0227]

[0228] About the 6-parameter affine motion model ( Figure 21b ), the motion vector at the sample position (x, y) can be derived as shown in Equation 3 below.

[0229] [Equation 3]

[0230]

[0231] Here, {v 0x , v 0y} represents the CPMV of the CP at the upper left corner of the coding block, {v 1x , v 1y} represents the CPMV of the CP at the upper right corner, {v 2x , v 2y} represents the CPMV of the CP at the lower left corner. In addition, W corresponds to the width of the current block, H corresponds to the height of the current block, and {v x , v y} represents the motion vector at the position {x, y}.

[0232] Fig. 22 An example of a motion vector of each sub-block according to an embodiment of the present disclosure is shown.

[0233] During the encoding / decoding process, an affine motion vector field (MVF) may be determined in units of pixels or in units of predefined sub-blocks. When the MVF is determined in units of pixels, a motion vector may be obtained based on each pixel value, and when the MVP is determined in units of sub-blocks, a motion vector of the corresponding block may be obtained based on the pixel value of the center of the sub-block (the lower right side of the center, i.e., the lower right sample of the four center samples). In the following description, the description is made by assuming that the affine MVF is determined in units of 4*4 sub-blocks. However, this is only for the convenience of description, and the size of the sub-block may be variously modified.

[0234] That is, when affine prediction is effective, the motion model that can be applied to the current block may include the following three motion models: translation motion model, 4-parameter affine motion model, and 6-parameter affine motion model. Here, the translation motion model may represent a model in which a conventional block unit motion vector is used, the 4-parameter affine motion model may represent a model in which two CPMVs are used, and the 6-parameter affine motion model may represent a model in which three CPMVs are used.

[0235] Affine motion prediction may include affine MVP (or affine inter) mode and affine merging. In affine motion prediction, the motion vector of the current block may be derived in units of samples or in units of sub-blocks.

[0236] Affine Merge

[0237] In affine merge mode, the CPMV can be determined based on the affine motion model of the neighboring block encoded by affine motion prediction. In the search order, the affine-coded neighboring blocks can be used for the affine merge mode. When one or more neighboring blocks are encoded by affine motion prediction, the current block can be encoded as AF_MERGE. That is, when the affine merge mode is applied, the CPMV of the current block can be derived by using the CPMV of the neighboring blocks. In this case, the CPMV of the neighboring block can be used as the CPMV of the current block as it is, and the CPMV of the neighboring block is modified based on the size of the neighboring block and the size of the current block to be used as the CPMV of the current block.

[0238] When the affine merge mode is applied, an affine merge candidate list may be constructed to derive the CPVM of the current block. The affine merge candidate list may include, for example, at least one of the following candidates.

[0239] 1) Inherited affine candidates

[0240] 2) Constructed affine candidates

[0241] 3) Zero MV candidates

[0242] Here, when the neighboring blocks are encoded in affine mode, the inherited affine candidate can be a candidate derived based on the CPMV of the neighboring blocks, the constructed affine candidate can be a candidate derived by constructing the CPMV based on the MV of the corresponding CP neighboring blocks in each CPMV unit, and the zero MV candidate can represent a candidate constructed by a CPMV with a value of 0.

[0243] Fig.23 An example of a flowchart for constructing an affine merge candidate list according to an embodiment of the present disclosure is shown.

[0244] Reference Fig.23 , the coding device (coding device or decoding device) may insert the inherited affine candidate into the candidate list (S2310), insert the constructed affine candidate into the affine candidate list (S2320), and insert the zero MV candidate into the affine candidate list (S2330). In an embodiment, when the number of candidates included in the candidate list is less than a reference number (e.g., 2), the coding device may insert the constructed affine candidate or the zero MV candidate.

[0245] Fig.24An example of a block for deriving an inherited affine motion predictor according to an embodiment of the present disclosure is shown, and Fig.25 An example of control point motion vectors for deriving an inherited affine motion predictor according to an embodiment of the present disclosure is shown.

[0246] There may be up to 2 inherited affine candidates (one from the left neighboring CU and one from the above neighboring CU), and the inherited affine candidates may be derived from the affine motion model of the neighboring blocks. Fig.24 , the candidate blocks are shown. The scan order of the left predictor is A0-A1, while the scan order of the upper predictor is B0-B1-B2. Only the first inherited candidate from each side surface is selected. No pruning check may be performed between two inherited candidates. When a neighboring affine CU is identified, the control point motion vector of the neighboring affine CU may be used to derive the control point motion vector predictor (CPMVP) from the affine merge list of the current CU. Fig.25 As shown, if the left neighboring block A is coded in affine mode, v2, v3, and v4 of the upper left, upper right, and lower left corners of the motion vector of the CU including block A are used. When block A is coded as a 4-parameter affine model, two CPMVs of the current CU are calculated based on v2 and v3. When block A is coded by a 6-parameter affine model, three CPMVs of the current CU are calculated based on v2, v3, and v4.

[0247] Fig.26 An example of a block for deriving a constructed affine merge candidate according to an embodiment of the present disclosure is shown.

[0248] The constructed affine merge represents the candidate constructed by combining the adjacent translation motion information of each control point. Fig.26 As shown, the motion information of the control point is derived from the specified spatial and temporal neighborhoods. CPMVk (k = 1, 2, 3, 4) represents the kth control point. Regarding the upper left corner CPMV1 (CP0), the blocks are checked in the order of B2-B3-A2, and the MV of the first valid block is used. For the upper right corner CPMV2 (CP1), the blocks are checked in the order of B1-B0, and for the lower left corner CMPV3 (CP2), the blocks are checked in the order of A1-A0. If valid, TMVP is used for the lower right corner CPMV4 (CP3).

[0249] When the MVs of the four control points are obtained, an affine merge candidate is constructed based on the motion information. The following combinations of control point MVs are used in order.

[0250] {CPMV1, CPMV2, CPMV3}, {CPMV1, CPMV2, CPMV4}, {CPMV1, CPMV3, CPMV4},

[0251] {CPMV2, CPMV3, CPMV4}, {CPMV1, CPMV2}, {CPMV1, CPMV3}

[0252] The combination of 3 CPMVs constructs a 6-parameter affine merge candidate, and the combination of 2 CPMVs constructs a 4-parameter affine merge candidate. To avoid motion scaling processing, the combination of related control point MVs is discarded if the reference indexes of the control points are different.

[0253] Affine MVP

[0254] Fig. 27 An example of a flowchart for constructing an affine MVP candidate list according to an embodiment of the present disclosure is shown.

[0255] In the affine MVP mode, after two or more control point motion vector predictions (CPMVP) and CPMV of a current block are determined, a control point motion vector difference (CPMVD) corresponding to a difference value is transmitted from the encoding device 100 to the decoding device 200 .

[0256] When the affine MVP mode is applied, an affine MVP candidate list may be constructed to derive the CPMV of the current block. For example, the affine MVP candidate list may include at least one of the following candidates. For example, the affine MVP candidate list may include up to n (eg, 2) candidates.

[0257] 1) Inherited affine MVP candidate extrapolated from the CPMV of the neighboring CU (S2710)

[0258] 2) Affine MVP candidate CPMVP constructed using translation MV of adjacent CU (S2720)

[0259] 3) Additional candidates based on translation MV from neighboring CU (S2730)

[0260] 4) Zero MV Candidate (S2740)

[0261] Here, when the neighboring blocks are encoded in affine mode, the inherited affine candidates may be candidates derived based on the CPMV of the neighboring blocks, the constructed affine candidates may be candidates derived by constructing the CPMV based on the MV of the corresponding CP neighboring blocks in each CPMV unit, and the zero MV candidate may represent a candidate constructed by a CPMV having a value of 0. When the maximum number of candidates in the affine MVP candidate list is 2, the candidates below clause 2) of the above order may be considered and added for the case where the number of current candidates is less than 2. In addition, additional candidates based on the translation MV from the neighboring CU may be derived in the following order.

[0262] 1) If the number of candidates is less than 2 and the CPMV0 of the constructed candidate is valid, CPMV0 is used as the affine MVP candidate. That is, all MVs of CP0, CP1, and CP2 are configured to be the same as the CPMV0 of the constructed candidate.

[0263] 2) If the number of candidates is less than 2 and the constructed candidate's CPMV1 is valid, CPMV1 is used as the affine MVP candidate. That is, all MVs of CP0, CP1, and CP2 are configured to be the same as the constructed candidate's CPMV1.

[0264] 3) If the number of candidates is less than 2 and the constructed candidate's CPMV2 is valid, CPMV2 is used as the affine MVP candidate. That is, all MVs of CP0, CP1, and CP2 are configured to be the same as the constructed candidate's CPMV2.

[0265] 4) If the number of candidates is less than 2, the temporal motion vector predictor (TMVP or mvCol) is used as the affine MVP candidate.

[0266] The affine MVP candidate list can be obtained by Fig. 27 The process shown in is derived.

[0267] The order of checking inherited MVP candidates is the same as that of checking inherited affine merge candidates. The difference is that, for MVP candidates, only affine CUs with the same reference picture as the current block are considered. When an inherited affine motion predictor is added to the candidate list, the pruning process is not applied.

[0268] from Fig.26 The MVP candidate constructed by deriving from the neighboring blocks shown. The same check order as the affine merge candidate is used. In addition, the reference picture index of the neighboring block is also checked. The first block is used, which is inter-coded in the check order and has the same reference picture as the current CU.

[0269] Adaptive Motion Vector Resolution (AMVR)

[0270] In the related art, when use_integer_mv_flag is 0 in the slice header, the motion vector difference (MVD) (between the predicted motion vector and the motion vector of the CU) can be signaled in quarter-luma-sample units. In the present disclosure, a CU-level AMVR scheme is introduced. AMVR may allow the MVD of the CU to be encoded in units of quarter-luma-sample, integer-luma-sample, or 4-luma-sample. In order for the current CT to have at least one non-zero MVD component, the CU-level MVD resolution indication is conditionally signaled. When all MVD components (i.e., the horizontal and vertical MVDs of reference list L0 and reference list L1) are 0, a quarter-luma-sample MVD resolution is inferred.

[0271] For a CU with at least one non-zero MVD component, a first flag is signaled to determine whether quarter-luma-sample MVD accuracy is applied to the CU. If the first flag is 0, no additional signaling is required, and quarter-luma-sample MVD accuracy is used for the current CU. Otherwise, a second flag is signaled to indicate whether integer-luma-sample or 4-luma-sample MVD accuracy is used. In order to ensure that the reconstructed MV guarantees a given accuracy (quarter-luma-sample, integer-luma-sample, or 4-luma-sample), the motion vector predictor of the CU can be rounded to have the same accuracy as the motion vector predictor previously added with the MVD. The motion vector predictor can be rounded to 0 (ie, a negative motion vector predictor is rounded to positive infinity, while a positive motion vector predictor is rounded to negative infinity). The encoder determines the motion vector resolution of the current CU by using an RD check. To avoid performing three CU-level RD checks consecutively for each MVD resolution, the RD check for 4-luma-sample MVD resolution can be conditionally called. The RD cost for quarter-sample MVD accuracy is first calculated. Then, to determine whether the RD cost for 4-luma-sample MVD accuracy needs to be checked, the RD cost for integer-luma-sample MVD accuracy is compared with the RD cost for quarter-luma-sample MVD accuracy. When the RD cost for quarter-luma-sample MVD accuracy is less than the RD cost for integer-luma-sample MVD accuracy, the RD cost for 4-sample MVD accuracy is omitted.

[0272] Sports Field Storage

[0273] In order to reduce the memory load, the motion information of the previously decoded reference picture can be stored in units of predetermined areas. This can be called temporal motion field storage, motion field compression, or motion data compression. In this case, the storage unit can be set differently depending on whether the affine mode is applied. In this case, among the motion vectors explicitly signaled, the motion vector with the highest accuracy is a quarter-luma-sample. In some inter-frame prediction modes such as the affine mode, the motion vector is derived with 1 / 16th-luma-sample accuracy, and motion compensated prediction is performed with 1 / 16th-luma-sample accuracy. In terms of internal motion field storage, all motion vectors are stored with 1 / 16th-luma-sample accuracy.

[0274] In the present disclosure, to store the temporal motion fields used by TMVP and ATMVP, motion field compression is performed at 8×8 granularity.

[0275] History-based Merge Candidate Derivation

[0276] History-based MVP (HMVP) merge candidates can be added to the merge list after spatial MVP and TMVP. In this method, the motion information of the previously coded block is stored in a table and used as the MVP of the current CU. During the encoding / decoding process, a table built from multiple HVMP candidates is maintained. When a new CTU row is used, the table is reset (empty). When there is a CU coded by inter-frame prediction instead of a sub-block, the relevant motion information is added to the last entry of the table as a new HMVP candidate.

[0277] In an embodiment, the HVMP table size S is set to 6, and this means that up to 6 HVMP candidates can be added to the table. When a new motion candidate is inserted into the table, a constrained first-in-first-out (FIFO) rule is used. Here, a redundancy check is first performed to check whether the same HVMP candidate as the HVMP candidate to be added exists in the table. When the same HVMP candidate exists, the existing same HVMP candidate is removed from the table, and all HMVP candidates are moved to the previous order.

[0278] HMVP candidates can be used in the merge candidate list construction process. In this table, the nearest HVMP candidate is checked and inserted into the merge candidate list in the subsequent order of the TMVP candidate. The redundancy of checking HMVP candidates is applied to spatial or temporal merge candidates.

[0279] To reduce the number of redundant checking operations performed, the following simplification can be used.

[0280] 1) The number of HMVP candidates used to generate the merge list is set to (N <= 4)? M: (8-N). Here, N represents the number of candidates present in the merge list, and M represents the number of HMVP candidates valid in the table.

[0281] 2) When the total number of valid merge candidates reaches a value obtained by subtracting 1 from the maximum number of allowed merge candidates, the merge candidate list construction process from HVMP is terminated.

[0282] Pairwise average merge candidate derivation

[0283] The pairwise average candidate is generated by the average of the predefined candidate pairs present in the merge candidate list. Here, the predefined pairs are defined as {(0, 1), (0, 2), (1, 2), (0, 3), (1, 3), (2, 3)}, and numbers such as 0, 1, 2, and 3 are merge indexes in the merge candidate list. The average of the motion vectors is calculated separately for each reference list. When two motion vectors are valid in one list, the average of the two motion vectors is used even if the two motion vectors are related to different reference pictures. If only one motion vector is valid, the valid motion vector is used immediately. When there is no valid motion vector, the list remains invalid.

[0284] When the merge list is not filled even after pairwise averaging of merge candidates, zero MVs are inserted until the maximum number of merge candidates is reached.

[0285] Prediction sample generation

[0286] The prediction block of the current block may be derived based on the motion information derived according to the prediction mode. The prediction block may include prediction samples (prediction sample array) of the current block. When the motion vector of the current block indicates a fractional sample unit, an interpolation process may be performed. The prediction samples of the current block may be derived from reference samples of fractional sample units within the reference picture through an interpolation process. When affine inter prediction is applied to the current block, the prediction samples may be generated based on the sample / sub-block unit motion vector. When bidirectional prediction is applied, the prediction samples derived by the weighted sum or weighted average of the prediction samples derived based on the L0 direction prediction (i.e., prediction using the reference picture in the L0 reference picture list and the L0 motion vector) and the prediction samples derived based on the L1 prediction (i.e., prediction using the reference picture in the L1 reference picture list and the L1 motion vector) (according to the phase) may be used as the prediction samples of the current block. When bidirectional prediction is applied, if the reference picture for L0 prediction and the reference picture for L1 prediction are located in different time directions based on the current picture (i.e., if the prediction corresponds to bidirectional prediction and bidirectional prediction), this may be referred to as true bidirectional prediction.

[0287] Reconstructed samples and reconstructed pictures may be generated based on the derived prediction samples, and thereafter, processes such as in-loop filtering may be performed as described above.

[0288] Bidirectional prediction using weighted average (BWA)

[0289] As described above, according to the present disclosure, when bidirectional prediction is applied to the current block, a prediction sample can be derived based on a weighted average. The bidirectional prediction signal (i.e., bidirectional prediction sample) can be derived by a simple average or a weighted average of an L0 prediction signal (L0 prediction sample) and an L1 prediction signal (L1 prediction sample). When the prediction sample derivation by a simple average is applied, the bidirectional prediction sample can be derived as an average of an L0 prediction sample based on an L0 reference picture and an L0 motion vector and an L1 prediction sample based on an L1 reference picture and an L1 motion vector. According to an embodiment of the present disclosure, when bidirectional prediction is applied, a bidirectional prediction signal (bidirectional prediction sample) can be derived by a weighted average of an L0 prediction signal and an L1 prediction signal, as shown in the following equation 4.

[0290] [Equation 4]

[0291]

[0292] In Equation 4, P bi-pred represents a bidirectional prediction sample value, L0 represents an L0 prediction sample value, P1 represents an L0 prediction sample value, and w represents a weight value.

[0293] In the weighted average bidirectional prediction, 5 weight values ​​w may be allowed, and the weight values ​​w may be -2, 3, 4, 5, and 10. For each CU to which bidirectional prediction is applied, the weight w may be determined by one of two methods.

[0294] 1) For non-merged CUs, the weight index is signaled after MVD.

[0295] 2) For the merged CU, the weight index is inferred from the neighboring blocks based on the merge candidate index.

[0296] Weighted and bidirectional prediction can be applied only to CUs with 256 or more luma samples (CUs where the product of CU width and CU height is equal to or greater than 256). For low-latency pictures, all 5 weights can be used. For non-low-latency pictures, only 3 weights (3, 4, and 5) can be used.

[0297] a) In the encoder, a fast search algorithm is applied in order to find the weight index without significantly increasing the encoder complexity. The algorithm is summarized as follows. When coupled to AMVR, the unequal weights of 1-pel and 4-pel motion vector accuracy are only conditionally checked if the current picture is a low-delay picture.

[0298] b) When coupled with affine, if the affine mode is selected as the current best mode, affine motion estimation (ME) will be performed with unequal weights.

[0299] c) When the two reference pictures are equal in bi-directional prediction, unequal weights are only checked conditionally.

[0300] e) According to the POC distance between the current picture and the reference picture, the coded quantization parameter QP and the temporal level, when specific conditions are not met, unequal weights are not searched.

[0301] Combined Inter and Intra Prediction (CIIP)

[0302] CIIP can be applied to the current CU. For example, when the CU is coded in merge mode, if the CU includes at least 64 luma samples (if the product of the CU width and the CU height is equal to or greater than 64), an additional flag can be signaled to indicate whether the CIIP mode is applied to the current CU. The CIIP mode may also be referred to as the multi-hypothesis mode or the inter / intra multi-hypothesis mode.

[0303] Intra-frame prediction mode derivation

[0304] Up to 4 intra prediction modes including DC, planar, horizontal, and vertical modes can be used to predict the luma component in CIIP mode. When the CU shape is very wide (e.g., when the width is twice or more than twice the height), the horizontal mode is not allowed. When the CU shape is very narrow (e.g., when the height is twice or more than the width), the vertical mode is not allowed. For this case, three intra prediction modes are allowed.

[0305] In CIIP mode, 3 Most Probable Modes (MPMs) are used for intra prediction. The CIIP MPM candidate list is formed as follows.

[0306] - The left neighboring block and the top neighboring block are configured as A and B respectively.

[0307] - The prediction modes of block A and block B are called IntraModeA and IntraModeB respectively and are derived as follows.

[0308] X is configured as either A or B.

[0309] IntraModeX is configured as DC if i) block X is invalid, ii) block X is not predicted by using CIIP mode, or iii) block B is located outside the current CTU.

[0310] Otherwise, i) if the intra prediction mode of block X is DC or planar, then IntraModeX is configured as DC or planar, ii) if the intra prediction mode of block X is a "vertical-like" direction mode (a mode greater than 34), then IntraModeX is configured as vertical, or iii) if the intra prediction mode of block X is a "horizontal-like" direction mode (a mode equal to or less than 34), then IntraModeX is configured as horizontal.

[0311] - If IntraModeA is equal to IntraModeB,

[0312] If IntraModeA is planar or DC, 3 MPMs are configured in the order of {planar, DC, vertical}.

[0313] Otherwise, 3 MPMs can be configured in the order of { IntraModeA, Planar, DC}.

[0314] - Otherwise (if IntraModeA is not equal to IntraModeB),

[0315] Additionally, 2 MPMs can be configured in the order of { IntraModeA, intraModeB}.

[0316] The first two MPM candidates are checked for planar, DC, and vertical uniqueness (redundancy), and when a unique (non-redundant) pattern is found, that unique pattern is added as the third MPM.

[0317] If the CU shape is very wide or very narrow, the MPM flag is inferred to be 1 without signaling. Otherwise, the MPM flag is signaled to indicate whether the CIIP intra prediction mode is one of the CIIP MPM candidate modes.

[0318] If the MPM flag is 1, the MPM index may be additionally signaled to indicate which of the MPM candidate modes is used for CIIP intra prediction. Otherwise, if the MPM flag is 0, the intra prediction mode is configured as a "missing" mode in the MPM candidate list. For example, if the plane mode does not exist in the MPM candidate list, the plane becomes a missing mode and the intra prediction mode is configured as a plane. Since 4 valid intra prediction modes are allowed in CIIP, the MPM candidate list only includes 3 intra prediction candidates. For chroma components, the DM mode is applied continuously without additional signaling. That is, the same prediction mode as the luminance component is used in the chroma component. The intra prediction mode of the CU coded with CIIP will be stored and used for intra mode coding of the adjacent CU.

[0319] Combining inter- and intra-prediction signals

[0320] The inter-frame prediction signal P in CIIP mode is derived by using the same inter-frame prediction process applied to the general merge mode inter , and derives an intra-frame prediction signal P by using the CIIP intra-frame prediction according to the intra-frame prediction process intra Then, the intra prediction signal and the inter prediction signal are combined using a weighted average, where the weight value depends on the intra prediction mode and the position of the sample in the coding block as follows.

[0321] If the intra prediction mode is DC or planar mode, or the width or height of the block is less than 4, the same weight is applied to the intra prediction signal and the inter prediction signal.

[0322] Otherwise, the weights are determined based on the intra prediction mode (in this case, horizontal mode or vertical mode) and the sample position in the block. The horizontal prediction mode is described as an example (the weights for the vertical mode are similar, but can be derived in the orthogonal direction). The width of the block is configured as W and the height of the block is configured as H. First, the coding block is partitioned into 4 equal area parts, and each size is (W / 4)×H. From the beginning of the part closest to the intra prediction reference sample to the end of the part farthest from the intra prediction sample, the weight wt for each of the 4 regions is set to 6, 5, 3 and 2. The final CIIP prediction signal can be derived as in Equation 5 below.

[0323] [Equation 5]

[0324]

[0325] In Equation 5, P CIIP represents the CIIP predicted sample value, P inter represents the inter-frame prediction sample value, P intra represents the intra prediction sample value, and wt represents the weight.

[0326] Example

[0327] Embodiments of the present invention relate to MVP prediction and symmetric MVD in an inter prediction method, and describe a motion information derivation method and a syntax signaling method for inter prediction.

[0328] When symmetric motion vector difference (SMVD) is applied, if a block coded in MVP mode is coded by bi prediction, the SMVD flag sym_mvd_flag indicating whether SMVD is applied is signaled to the decoder, and only the MVD for L0 direction prediction, the MVP index for L0 direction prediction, and the MVP index for L1 direction are sent to the decoder. The decoder can perform bidirectional prediction by deriving L0 and L1 reference picture indexes refidxL0 and refidxL1 and L1 MVD (MVDL1). refidxL0 can be referred to as refidxsymL0, and can be referred to as refidxsymL1.

[0329] Meanwhile, a flag mvd_l1_zero_flag indicating whether MVDL1 is 0 may be signaled. If mvd_l1_zero_flag is 0, coding (decoding) of MVDL1 is performed, and if mvd_l1_zero_flag is 1, coding (decoding) of MVDL1 is not performed.

[0330] For example, if the tile group type (picture type, slice type) of the current tile group (or picture, slice) including the current block is B (bidirectional prediction), mvd_l1_zero_flag may be signaled. That is, mvd_l1_zero_flag may be included in the coding information for a level (e.g., picture, slice, tile group) higher than the current block (coding unit) and signaled.

[0331] If mvd_l1_zero_flag is 1, using the SMVD method is inefficient when considering the MV determination method of the encoder. Therefore, an embodiment of the present disclosure provides a method for inferring that the value of sym_mvd_flag is 0 if mvd_l1_zero_flag is 1 without signaling (parsing) sym_mvd_flag.

[0332] The syntax structure for a compilation unit according to an embodiment of the present disclosure may be shown in Table 2.

[0333] [Table 2]

[0334]

[0335]

[0336]

[0337]

[0338] In Table 2, the decoder checks the flag mvd_l1_zero_flag indicating whether the L1 direction MVD is 0 as a condition for parsing the flag indicating whether SMVD is applied (ie, L0 (sym_mvd_flag)). That is, the decoder parses sym_mvd_flag based on mvd_l1_zero_flag. If sym_mvd_flag is 1, coding (parsing) information about the L0 reference picture (e.g., ref_idx_l0), information about the L1 reference picture (e.g., ref_idx_l1), and information about the L1 MVD (e.g., mvd_coding (x0, y0, 1, 0)) are omitted.

[0339] Fig.28 An example of a flowchart for deriving a motion vector according to an embodiment of the present disclosure is shown. Fig.28 The operation may be performed by the inter-frame predictor 260 of the decoding device 200 or the processor 510 of the video signal processing apparatus 500 . Fig.28 The flowchart can correspond to Fig.14 An example of step S1420 in .

[0340] First, the decoder checks whether the skip mode or merge mode is applied to the current block (S2805). For example, as shown in the syntax structure of Table 1, the decoder checks whether the skip mode is applied by using a flag cu_skip_flag indicating whether the skip mode is applied, and if the skip mode is not applied (cu_skip_flag = 0), the decoder checks whether the merge mode is applied by using a flag merge_flag indicating whether the merge mode is applied.

[0341] When skip mode or merge mode is applied, the decoder constructs a merge candidate (S2810) and derives a motion vector based on the merge index (S2815). If skip mode or merge mode is not applied, the decoder checks the index inter_pred_idc indicating the prediction type of the current block (S2820). Here, the prediction type may correspond to one of unidirectional prediction or bidirectional prediction. If the prediction type is unidirectional prediction, the decoder may construct an MVP[X] candidate list (X 0 or 1) (S2825), derive MVP[X] based on the MVP index mvp_idx[X] of the L0 or L1 direction, and derive a motion vector by adding MVD[X] and MVP[X] (S2830).

[0342] If the prediction type of the current block is bidirectional prediction, the decoder checks the flag Sym_mvd_flag indicating whether SMVD is applied (S2835). If SMVD is not applied, the decoder performs a motion vector derivation process for each of the L0 direction and the L1 direction (S2840), and the decoder constructs an MVP candidate for LX (S2870), derives an MVP motion vector based on the MVP index of LX (S2875), and derives a final motion vector by the sum of the MVP motion vector and MVD (S2880).

[0343] If SMVD is applied, the decoder constructs each of the MVP candidate list of L0 and the MVP candidate list of L1 (S2845 and S2850). Before constructing the MVP candidate list, the decoder may derive the reference picture index of the picture corresponding to the reference picture list closest to the current picture as the reference index for L0 and L1 (S2885). According to the SMVD, the decoder determines that the MVD for L1 (MVD [L1]) has the same magnitude as the MVD for L0 (MVD [L0]), but has a different sign from MVD [L0] (MVD [L1] = -1 * MVD [L0]). Thereafter, the decoder derives the final motion vector based on the MVP motion vectors corresponding to the MVD and MVP indexes of L0 and L1, respectively (S2860 and S2865).

[0344] Fig.29 An example of a flowchart for estimating motion according to an embodiment of the present disclosure is shown. Fig.29 The operation may be performed by the inter-frame predictor 180 of the encoding device 100 or the processor 510 of the video signal processing apparatus 500 . Fig.29 The flowchart can correspond to Fig.12 An example of step S1210 in .

[0345] First, the encoder builds MVP candidate lists for L0 and L1 (S2905 and S2910). Thereafter, the encoder checks whether L1 MVD is 0 (whether L1 MVD information is coded) in the tile group (or picture, slice) including the current block through mvd_l1_zero_flag (S2915). If L1 MVD is coded (if mvd_l1_zero_flag is 0), the encoder performs motion search for both L0 and L1 (S2920).

[0346] If L1 MVD is not coded (if mvd_l1_zero_flag is 1), the encoder fixes L1 MV to MVP motion vector (PMV) and brings L1 prediction block corresponding to L1 MV (MV[L1]) (S2930). Thereafter, the encoder performs motion vector search for L0 (S2935), performs motion search within the search range (S2940), determines the average of L0 predictor and L1 predictor (S2945), and determines the best L0 MV (S2950).

[0347] According to an embodiment of the present disclosure, in an encoder, if mvd_l1_zero_flag is 1 (if L1 MVD is not coded), when SMVD is applied in the process of performing motion prediction, it may be inefficient. Fig.29 1 shows the process of determining the best MV by performing bidirectional prediction by the encoder if mvd_l1_zero_flag is 1. Fig.29 As shown, if mvd_l1_zero_flag is 1, L0 motion search is performed. In this case, when SMVD is applied, each time in the process of determining the optimal MV[L0], MVD[L0] is mirrored and applied to L1, and then calculation is performed, and as a result, the motion search process may be very complicated. Therefore, an embodiment of the present disclosure is a method in which SMVD is not applied if mvd_l1_zero_flag is 1.

[0348] BitTorrent

[0349] Based on the above-mentioned embodiments of the present disclosure, the coding information (e.g., coded video / image information) derived by the coding device 100 can be output in the form of a bitstream. The coding information can be sent or stored in units of a network abstraction layer (NAS) in the form of a bitstream. The bitstream can be sent via a network or stored in a non-temporary digital storage medium. In addition, as described above, the bitstream is not sent directly from the coding device 100 to the decoding device 200, but can be subjected to a streaming / download service through an external server (e.g., a content streaming server). Here, the network may include a broadcast network and / or a communication network, and the digital storage medium may include various storage media, including USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc.

[0350] Fig.30 An example of a flow chart for encoding a video signal for inter-frame prediction according to an embodiment of the present disclosure is shown. Fig.30 The operation may be performed by the inter-frame predictor 180 of the encoding device 100 or the processor 510 of the video signal processing apparatus 500 . Fig.30 The flowchart can correspond to Fig.12An example of step S1230 in .

[0351] In step S3010, the encoder encodes first coding information of a first-level unit. Here, the first-level unit may correspond to a relatively high-level processing unit (eg, a picture, a slice, a tile group).

[0352] According to an embodiment of the present disclosure, the first coding information includes a first flag mvd_l1_zero_flag related to whether to code the second MVD information in the first MVD (L0 MVD) information for the first directional prediction (L0 prediction) and the second MVD (L1 MVD) information for the second directional prediction (L1 prediction). Here, the first MVD information and the second MVD information may be coded in the syntax structure shown in Table 1, and may be inferred to be 0 when the coding for the second MVD information is omitted according to the first flag mvd_l1_zero_flag. For example, if the first flag mvd_l1_zero_flag is 0, the coding of the second MVD information may be performed, and if the first flag mvd_l1_zero_flag is 1, the coding of the second MVD information may be omitted.

[0353] In step S3020, the encoder encodes second coding information for a second level unit lower than the first level unit. Here, the second level unit may correspond to a coding unit. Here, the second coding information includes a second flag sym_mvd_flag related to whether SMVD is applied to a current block corresponding to the second level unit.

[0354] According to an embodiment of the present disclosure, the second flag sym_mvd_flag is encoded based on the first flag mvd_l1_zero_flag. For example, if the first flag mvd_l1_zero_flag is 0, the encoder may encode the second flag based on a search process of a first motion vector (L0 motion vector) for first direction prediction and a second motion vector (L1 motion vector) for second direction prediction. If the first flag mvd_l1_zero_flag is 1, the encoder performs motion estimation while excluding the application of SMVD, and does not encode the second flag sym_mvd_flag.

[0355] Fig.31 An example of a decoding flowchart of a video signal for inter-frame prediction according to an embodiment of the present disclosure is shown. Fig.31 The operation may be performed by the inter-frame predictor 260 of the decoding device 200 or the processor 510 of the video signal processing apparatus 500 . Fig.31 Steps S3110 to S3150 in correspond to Fig.14 An example of step S1420 in , and Fig.31 Step S3160 in corresponds to Fig.14 An example of step S1430 in .

[0356] In step S3110, the decoder obtains a first flag mvd_l1_zero_flag related to whether the second MVD information (L1MVD information) is encoded between the first MVD (L0 MVD) information for the first directional prediction (L0 prediction) and the second MVD (L1 MVD) information for the second directional prediction (L1 prediction). The first level unit, which is a relatively high-level processing unit, may correspond to one of a picture, a slice, or a tile group. The first MVD information (L0 MVD information) and the second MVD information (L1MVD information) may be decoded by the syntax structure shown in Table 1.

[0357] For example, if the first flag mvd_l1_zero_flag is 0, decoding the second MVD information (L1MVD information) may not be performed, and if the first flag mvd_l1_zero_flag is 1, decoding the second MVD information (L1 MVD information) may be omitted. For example, in Table 2, if the first flag mvd_l1_zero_flag is 1, the second MVD value (MvdL1, MvdCpL1) is regarded as 0 without a coding process for the second MVD.

[0358] In step S3120, the encoder obtains a second flag sym_mvd_flag related to whether to apply SMVD to the current block based on the first flag mvd_l1_zero_flag from second coding information for a second level unit lower than the first level unit.

[0359] For example, if the first flag mvd_l1_zero_flag is 0 and the additional condition is met, the decoder can decode the second flag sym_mvd_flag, and if the first flag mvd_l1_zero_flag is 1, the decoder can infer that the second flag is 0 and does not decode the second flag. For example, in Table 2, as a condition for parsing the second flag sym_mvd_flag, the first flag mvd_l1_zero_flag is included to be 0.

[0360] In step S3130, the encoder determines a first MVD (L0 MVD) for the current block based on the first MVD information (L0 MVD information). For example, the encoder may determine the first MVD (L0 MVD) through the syntax structure shown in Table 1 after calling the mvd_coding process of Table 2.

[0361] In step S3140, the encoder determines the second MVD (L1 MVD) from the first MVD (L0 MVD) based on the second flag sym_mvd_flag. For example, if the second flag sym_mvd_flag is 0, the decoder may determine the second MVD (L1 MVD) from the second MVD information (L1MVD information), and if the second flag sym_mvd_flag is 1, the decoder may determine the second MVD (L1 MVD) from the first MVD (L0 MVD) based on the SMVD. For example, if the second flag sym_mvd_flag is 0, the decoder determines the second MVD (L1 MVD) through the syntax structure shown in Table 1 by calling the coding process (mvd_coding (x0, y0, 1, 0)) of the second MVD information, and if the second flag sym_mvd_flag is 1, the decoder determines the second MVD (L1 MVD) from the first MVD (L0 MVD). As shown in Table 2, if the second flag sym_mvd_flag is 1, the second MVD (L1 MVD) may have the same magnitude and opposite sign as the first MVD (L0 MVD). (MvdL1[ x0 ][ y0 ][ 0 ] = -MvdL0[ x0 ][ y0 ][ 0 ], MvdL1[ x0 ][ y0 ][ 1 ] = -MvdL0[ x0 ][ y0 ][ 1 ]).

[0362] In step S3150, the decoder determines a first motion vector (L0 motion vector) and a second motion vector (L1 motion vector) based on the first MVD (L0 MVD) and the second MVD (L1 MVD). For example, the decoder may obtain first MVP information (L0 MVP information) for first directional prediction (L0 prediction) (e.g., mvp_l0_flag of Table 2) and second MVP information (L1 MVP information) for second directional prediction (L1 prediction) (e.g., mvp_l1_flag of Table 2). Thereafter, the decoder may determine a first candidate motion vector (L0 candidate motion vector) corresponding to the first MVP information (L0 MVP information) in a first MVP candidate list (L0 MVP candidate list) for first directional prediction (L0 prediction) and a second candidate motion vector (L1 candidate motion vector) corresponding to the second MVP information (L1 MVP information) in a second MVP candidate list (L1MVP candidate list). In addition, the decoder can determine a first motion vector (L0 motion vector) by adding the first MVD (L0 MVD) to the first candidate motion vector (L0 candidate motion vector), and determine a second motion vector (L1 motion vector) by adding the second MVD (L1 MVD) to the second candidate motion vector (L1 candidate motion vector).

[0363] In step S3160, the decoder generates a prediction sample of the current block based on the first motion vector (L0 motion vector) and the second motion vector (L1 motion vector). For example, the decoder may determine a first reference picture (L0 reference picture) for first direction prediction (L0 prediction) and a second reference picture (L1 reference picture) for second direction prediction (L1 prediction 0), and generate the prediction sample of the current block based on a first reference sample (L0 reference sample) indicated by a first motion vector (L0 motion vector) in the first reference picture (L0 reference picture) and a second reference sample (L1 reference sample) indicated by a second motion vector (L1 motion vector 0) in the second reference picture (L1 reference picture). In one example, the reference sample may be derived by a weighted average of the first reference sample (L0 reference sample) and the second reference sample (L1 reference sample).

[0364] In an embodiment, the first reference picture (L0 reference picture) may correspond to a reference picture that is preceding and closest to the current picture in display order in a first reference picture list (L0 reference picture list) for first directional prediction (L0 prediction), and the second reference picture (L1 reference picture) may correspond to a reference picture that is following and closest to the current picture in display order in a second reference picture list (L1 reference picture list) for second directional prediction (L1 prediction).

[0365] As described above, the embodiments described in the present disclosure may be implemented and executed on a processor, a microprocessor, a controller or a chip. For example, the functional units shown in each figure may be implemented and executed on a computer, a processor, a microprocessor, a controller or a chip.

[0366] The video signal processing apparatus 500 according to an embodiment of the present disclosure may include a memory 520 storing a video signal and a processor 510 coupled to the memory 520 .

[0367] In order to encode a video signal, the processor 510 is configured to encode first coding information for a first level unit and encode second coding information for a second level unit lower than the first level unit. The first coding information includes a first flag as to whether the second MVD information is coded between the first MVD information for first direction prediction and the second MVD information for second direction prediction, and the second coding information includes a second flag as to whether a symmetric MVD (SMVD) is applied to a current block corresponding to the second level unit. The second flag is encoded based on the first flag.

[0368] In an embodiment, the first level unit may correspond to one of the following: a picture, a tile group, or a slice, and the second level unit may correspond to a coding unit.

[0369] In an embodiment, if the first flag is 0, encoding of the second MVD information may be performed, and if the first flag is 1, encoding of the second MVD information may be omitted.

[0370] In an embodiment, the processor 510 may be configured to encode the second flag based on a search process of a first motion vector for first direction prediction and a second motion vector for second direction prediction when the first flag is 0.

[0371] In order to decode a video signal, the processor 510 is configured to obtain a first flag related to whether second MVD information is compiled between first MVD information for first direction prediction and second MVD information for second direction prediction in a first level unit, obtain a second flag related to whether SMVD is applied to a current block corresponding to a second level unit lower than the first level unit based on the first flag, determine a first MVD for the current block based on the first MVD information, determine a second MVD based on the second flag, determine a first motion vector and a second motion vector based on the first MVD and the second MVD, and generate a prediction sample of the current block based on the first motion vector and the second motion vector.

[0372] In an embodiment, the first level unit may correspond to at least one of: a picture, a tile group, or a slice, and the second level unit may correspond to a coding unit.

[0373] In an embodiment, if the first flag is 0, decoding of the second MVD information is performed, and if the first flag is 1, decoding of the second MVD information may be omitted.

[0374] In an embodiment, in the process of obtaining the second flag, the processor 510 may be configured to decode the second flag when the first flag is 0 and the additional condition is satisfied, and infer that the second flag is 0 without decoding the second flag when the first flag is 1.

[0375] In an embodiment, in determining the second MVD, the processor 510 may be configured to determine the second MVD from the second MVD information when the second flag is 0, and determine the second MVD from the first MVD based on the SMVD when the second flag is 1.

[0376] In an embodiment, when the second flag is 1, the second MVD may have the same magnitude as the first MVD and an opposite sign than the first MVD.

[0377] In an embodiment, in the process of determining the first motion vector and the second motion vector, the processor 510 can be configured to obtain first MVP information for first direction prediction and second MVP information for second direction prediction, determine a first candidate motion vector corresponding to the first MVP information in a first MVP candidate list for first direction prediction and determine a second candidate motion vector corresponding to the second MVP information in a second MVP candidate list for second direction prediction, and determine the first motion vector by adding the first MVD to the first candidate motion vector, and determine the second motion vector by adding the second MVD to the second candidate motion vector.

[0378] In an embodiment, in the process of generating a prediction sample of a current block, the processor 510 may be configured to determine a first reference picture for first direction prediction and a second reference picture for second direction prediction, and generate a prediction sample of the current block based on a first reference sample indicated by a first motion vector in the first reference picture and a second reference sample indicated by a second motion vector in the second reference picture.

[0379] In an embodiment, the first reference picture may correspond to a reference picture that is preceding and closest to the current picture in the display order in a first reference picture list for first directional prediction, and the second reference picture may correspond to a reference picture that is following and closest to the current picture in the display order in a second reference picture list for second directional prediction.

[0380] In addition, the processing method of the present disclosure can be generated in the form of a program executed by a computer and can be stored in a computer-readable recording medium. Multimedia data having a data structure according to the present disclosure can also be stored in a computer-readable recording medium. Computer-readable recording media include all types of storage devices and distributed storage devices that store computer-readable data. Computer-readable recording media may include, for example, Blu-ray discs (BDs), universal serial buses (USBs), ROMs, PROMs, EPROMs, EEPROMs, RAMs, CD-ROMs, magnetic tapes, floppy disks, and optical data storage devices. In addition, computer-readable recording media include media implemented in the form of carrier waves (e.g., transmission via the Internet). In addition, the bit stream generated by the encoding method can be stored in a computer-readable recording medium or sent via a wired / wireless communication network.

[0381] In addition, the embodiments of the present disclosure may be implemented as a computer program product through a program code, and the program code may be executed on a computer through the embodiments of the present disclosure. The program code may be stored on a computer readable carrier.

[0382] A non-transitory computer-readable medium according to an embodiment of the present disclosure stores one or more instructions executed by one or more processors. In order to encode a video signal, the one or more instructions control the video signal processing device 500 (or the encoding device 100) to encode first coding information for a first level unit and encode second coding information for a second level unit lower than the first level unit. The first coding information includes a first flag related to whether the second MVD information is coded between the first MVD information for first direction prediction and the second MVD information for second direction prediction, and the second coding information includes a second flag related to whether a symmetric MVD (SMVD) is applied to a current block corresponding to the second level unit, and the second flag is encoded based on the first flag.

[0383] In an embodiment, the first level unit may correspond to at least one of: a picture, a tile group, or a slice, and the second level unit may correspond to a coding unit.

[0384] In an embodiment, if the first flag is 0, encoding of the second MVD information may be performed, and if the first flag is 1, encoding of the second MVD information may be omitted.

[0385] In an embodiment, one or more instructions may control the video signal processing apparatus 500 (or the encoding device 100 ) to encode the second flag based on a search process of a first motion vector for first direction prediction and a second motion vector for second direction prediction when the first flag is 0.

[0386] In addition, in order to decode the video signal, one or more instructions control the video signal processing device 500 (or the decoding device 200) to obtain a first flag about whether the second MVD information is encoded between the first MVD information for the first direction prediction and the second MVD information for the second direction prediction from the first coding information of the first level unit, obtain a second flag about whether SMVD is applied to the current block based on the first flag from the second coding information of the second level unit lower than the first level unit, determine the first MVD for the current block based on the first MVD information, determine the second MVD based on the second flag, determine the first motion vector and the second motion vector based on the first MVD and the second MVD, and generate a prediction sample of the current block based on the first motion vector and the second motion vector.

[0387] In an embodiment, the first level unit may correspond to at least one of: a picture, a tile group, or a slice, and the second level unit may correspond to a coding unit.

[0388] In an embodiment, if the first flag is 0, decoding of the second MVD information may be performed, and if the first flag is 1, decoding of the second MVD information may be omitted.

[0389] In an embodiment, in the process of obtaining the second flag, one or more instructions may control the video signal processing device 500 (or the decoding device 200) to decode the second flag when the first flag is 0 and the additional condition is satisfied, and to infer that the second flag is 0 without decoding the second flag when the first flag is 1.

[0390] In an embodiment, in the process of determining the second MVD, one or more instructions may control the video signal processing device 500 (or the decoding device 200) to determine the second MVD from the second MVD information when the second flag is 0, and determine the second MVD from the first MVD based on the SMVD when the second flag is 1.

[0391] In an embodiment, when the second flag is 1, the second MVD may have the same magnitude as the first MVD and an opposite sign than the first MVD.

[0392] In an embodiment, in the process of determining the first motion vector and the second motion vector, one or more instructions may control the video signal processing device 500 (or the decoding device 200) to obtain first MVP information for first direction prediction and second MVP information for second direction prediction, determine a first candidate motion vector corresponding to the first MVP information in a first MVP candidate list for first direction prediction and a second candidate motion vector corresponding to the second MVP information in a second MVP candidate list for second direction prediction, and determine the first motion vector by adding the first MVD to the first candidate motion vector, and determine the second motion vector by adding the second MVD to the second candidate motion vector.

[0393] In an embodiment, in the process of generating a prediction sample of a current block, one or more instructions may control the video signal processing device 500 (or the decoding device 200) to determine a first reference picture for first direction prediction and a second reference picture for second direction prediction, and generate a prediction sample of the current block based on a first reference sample indicated by a first motion vector in the first reference picture and a second reference sample indicated by a second motion vector in the second reference picture.

[0394] In an embodiment, the first reference picture may correspond to a reference picture that is preceding and closest to the current picture in the display order in a first reference picture list for first directional prediction, and the second reference picture may correspond to a reference picture that is following and closest to the current picture in the display order in a second reference picture list for second directional prediction.

[0395] In addition, the decoder and encoder of the present disclosure may be included in multimedia broadcast sending and receiving devices, mobile communication terminals, home theater video devices, digital theater video devices, surveillance cameras, video chat devices, real-time communication devices such as video communication, mobile streaming devices, storage media, cameras, video on demand (VoD) service providing devices, (over the top) OTT video devices, Internet streaming service providing devices, 3-dimensional (3D) video devices, video phone video devices, transportation terminals (e.g., vehicle terminals, aircraft terminals, ship terminals, etc.) and medical video devices, and may be used to process video signals or data signals. For example, over the top (OTT) video devices may include game consoles, Blu-ray players, Internet access TVs, home theater systems, smart phones, tablet PCs, digital video recorders (DVRs), etc.

[0396] In the above-mentioned embodiments, the components and features of the present disclosure are combined in a predetermined form. Unless otherwise clearly stated, each component or feature should be considered as an option. Each component or feature can be implemented as not associated with other components or features. In addition, the embodiments of the present disclosure can be configured by associating some components and / or features. The order of the operations described in the embodiments of the present disclosure can be changed. Some components or features of any embodiment can be included in another embodiment or replaced by components and features corresponding to another embodiment. Obviously, claims that are not clearly cited in the claims are combined to form embodiments, or are included in new claims by modifications after application.

[0397] The embodiments of the present disclosure may be implemented by hardware, firmware, software or a combination thereof. In the case of being implemented by hardware, according to the hardware implementation, the exemplary embodiments described herein may be implemented by using one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, etc.

[0398] In the case of implementation by firmware or software, the embodiments of the present disclosure may be implemented in the form of modules, procedures, functions, etc. to perform the above functions or operations. The software code may be stored in a memory and executed by a processor. The memory may be located inside or outside the processor and may send and receive data to / from the processor through various devices.

[0399] In the present disclosure, the terms " / " and "," should be interpreted as indicating "and / or". For example, the expression "A / B" may mean "A and / or B", and "A, B" may mean "A and / or B". In addition, "A / B / C" may mean "at least one of A, B, and / or C". In addition, "A / B / C" may mean "at least one of A, B, and / or C".

[0400] Furthermore, in the present disclosure, the term "or" should be interpreted as indicating "and / or". For example, the expression "A or B" can include 1) only A, 2) only B, and / or 3) both A and B. In other words, the term "or" in the present disclosure should be interpreted as indicating "additionally or alternatively".

[0401] It is obvious to those skilled in the art that the present disclosure may be implemented in other specific forms without departing from the basic features of the present disclosure. Therefore, the above detailed description should not be interpreted as being restrictive in all aspects and should be considered illustratively. The scope of the present disclosure should be determined by a reasonable interpretation of the attached claims, and all modifications within the equivalent scope of the present disclosure are included within the scope of the present disclosure.

[0402] Industrial Applicability

[0403] In the above, the preferred embodiments of the present disclosure are disclosed for illustrative purposes, and hereinafter, those skilled in the art will make modifications, changes, substitutions or additions to various other embodiments within the technical spirit and technical scope of the present disclosure disclosed in the appended claims.

Claims

1. A device for decoding an image, the device comprising: Memory; as well as at least one processor connected to the memory, the at least one processor configured to: Determine the prediction mode of the current block as the inter-frame prediction mode; Obtaining a first flag related to whether to encode L1 motion vector difference (MVD) information; Based on the first flag, obtaining a second flag related to whether to apply a symmetric MVD as an SMVD to the current block, the second flag being obtained based on a value of the first flag; Derivation of L0 MVD; deriving an L1 MVD based on the second marker; deriving an L0 motion vector and an L1 motion vector based on the L0 MVD and the L1 MVD; Generate a prediction sample of the current block based on the L0 motion vector and the L1 motion vector; and Generate a reconstructed sample based on the predicted sample, wherein the second flag is obtained based on the first flag indicating that the L1 MVD information is encoded, and Wherein, based on the first flag indicating that the L1 MVD information is not encoded, the second flag is not obtained and the value of the second flag is inferred to be zero.

2. A device for encoding an image, the device comprising: Memory; as well as at least one processor connected to the memory, the at least one processor configured to: generating a first flag related to whether to encode L1 motion vector difference (MVD) information; generating a second flag related to whether to apply the symmetric MVD as SMVD to the current block; wherein the second flag is encoded based on the value of the first flag, wherein the second flag is encoded based on the first flag indicating that the L1 MVD information is encoded, and Wherein, based on the first flag indicating that the L1 MVD information is not encoded, the second flag is not encoded.

3. A device for transmitting data for an image, the device comprising: at least one processor configured to obtain a bit stream for image information including a first marker and a second marker; as well as a transmitter configured to transmit the data including the bit stream, The first flag is related to whether to encode L1 motion vector difference MVD information. The second flag is related to whether the symmetric MVD as SMVD is applied to the current block, wherein the second flag is encoded based on the value of the first flag, wherein the second flag is encoded based on the first flag indicating that the L1 MVD information is encoded, and Wherein, based on the first flag indicating that the L1 MVD information is not encoded, the second flag is not encoded.

Images