Method for encoding image information, method for decoding image information, computer-readable recording medium, and method for transmitting image information

Abstract

A method according to an embodiment comprises the steps of: acquiring image information from a bitstream; deriving a prediction mode applied to the current block on the basis of the acquired image information; deriving, on the basis of the prediction mode, a reference picture set including a first reference picture and a second reference picture for the current block; deriving a plurality of motion vector differences including a first motion vector difference and a second motion vector difference respectively corresponding to the first reference picture and the second reference picture; and generating prediction samples for the current block on the basis of the plurality of reference pictures and the plurality of motion vector differences, wherein, in the step of deriving the plurality of motion vector differences, the first motion vector difference is derived from the image information, and the second motion vector difference is derived by shifting or scaling the first motion vector difference.

Description

Method for encoding image information, method for decoding image information, computer-readable storage medium and method for transmitting image information

[0001] The present disclosure relates to a method for decoding / encoding image information, a computer-readable storage medium for storing image information, and a method for transmitting image information.

[0002] Recently, the demand for high-resolution, high-quality images, such as HD (High Definition) images and UHD (Ultra High Definition) images, is increasing in various application fields, and accordingly, high-efficiency image compression technologies are being discussed.

[0003] There are various technologies such as inter prediction technology that predicts pixel values ​​included in the current picture from pictures before or after the current picture, intra prediction technology that predicts pixel values ​​included in the current picture using pixel information within the current picture, and entropy coding technology that assigns short codes to values ​​with high frequency of appearance and long codes to values ​​with low frequency of appearance, and these video compression technologies can be used to effectively compress and transmit or store video data.

[0004] Accordingly, a highly efficient image compression technology is required to effectively transmit, store, and play high-resolution, high-quality image information.

[0005] The present disclosure provides a method for more efficiently selecting a reference picture when bi-prediction is applied in an inter prediction mode.

[0006] The present disclosure provides a method for more efficiently signaling and deriving relevant information when bi-prediction is applied in inter-prediction mode.

[0007] According to one embodiment, a method includes the steps of: obtaining image information from a bitstream; deriving a prediction mode to be applied to a current block based on the obtained image information; deriving a reference picture set including a first reference picture and a second reference picture for the current block based on the prediction mode; deriving a plurality of motion vector differences including a first motion vector difference and a second motion vector difference corresponding to the first reference picture and the second reference picture, respectively; and generating prediction samples for the current block based on the plurality of reference pictures and the plurality of motion vector differences; wherein the step of deriving the plurality of motion vector differences includes deriving the first motion vector difference from the image information and deriving the second motion vector difference by shifting or scaling the first motion vector difference.

[0008] According to one embodiment, a method includes: obtaining first information from a bitstream; deriving information indicating whether a symmetric motion vector difference (SMVD) mode is applied to a current block based on the obtained first information; and obtaining second information from the bitstream based on the fact that the SMVD mode is not applied to the current block; and determining whether bi-prediction is applied to the current block based on the second information.

[0009] According to one embodiment, a method includes the steps of: obtaining image information from a bitstream; deriving a prediction mode to be applied to a current block based on the obtained image information; deriving a plurality of motion vector predictors including a first motion vector predictor and a second motion vector predictor for the current block based on the prediction mode; deriving a plurality of motion vector differentials including a first motion vector differential and a second motion vector differential for the current block; and generating prediction samples for the current block based on the plurality of motion vector predictors and the plurality of motion vector differentials; wherein the step of deriving the plurality of motion vector predictors includes deriving the first motion vector predictor from the image information and the second motion vector predictor from the first motion vector predictor, and the step of deriving the plurality of motion vector differentials includes deriving the first motion vector differential from the image information and the second motion vector differential from the first motion vector differential.

[0010] According to one embodiment, a method includes the steps of: determining a prediction mode to be applied to a current block; deriving a reference picture set including a first reference picture and a second reference picture for the current block based on the prediction mode; deriving a first motion vector difference and a second motion vector difference corresponding to the first reference picture and the second reference picture, respectively; and encoding image information including information about the prediction mode and information about the first motion vector difference; wherein the second motion vector difference can be derived by shifting or scaling the first motion vector difference.

[0011] A computer-readable storage medium storing a bitstream generated by an encoding method according to one embodiment, the encoding method comprising: a step of determining a prediction mode to be applied to a current block; a step of deriving a reference picture set including a first reference picture and a second reference picture for the current block based on the prediction mode; a step of deriving a first motion vector difference and a second motion vector difference corresponding to the first reference picture and the second reference picture, respectively; and a step of encoding image information including information about the prediction mode and information about the first motion vector difference; wherein the second motion vector difference can be derived by shifting or scaling the first motion vector difference.

[0012] According to one embodiment, a method comprises the steps of: generating a bitstream regarding the image information; and transmitting data including the bitstream; wherein the bitstream is generated by: determining a prediction mode to be applied to a current block; deriving a reference picture set including a first reference picture and a second reference picture for the current block based on the prediction mode; deriving a first motion vector difference and a second motion vector difference corresponding to the first reference picture and the second reference picture, respectively; and encoding image information including information regarding the prediction mode and information regarding the first motion vector difference; wherein the second motion vector difference can be derived by shifting or scaling the first motion vector difference.

[0013] According to the present disclosure, when bi-prediction is applied in inter prediction mode, inter-screen prediction performance can be improved by selecting reference pictures more efficiently.

[0014] According to the present disclosure, when pair prediction is applied in inter prediction mode, inter-screen prediction performance can be improved by more efficiently signaling and deriving relevant information.

[0015] The effects that can be obtained from the present disclosure are not limited to the effects mentioned above, and other effects that are not mentioned will be clearly understood by a person having ordinary skill in the art to which the present disclosure pertains from the description below.

[0016] FIG. 1 illustrates a video / image coding system according to the present disclosure.

[0017] FIG. 2 is a schematic block diagram of an encoding device to which an embodiment of the present disclosure can be applied and in which encoding of a video / image signal is performed.

[0018] FIG. 3 is a schematic block diagram of a decoding device to which an embodiment of the present disclosure can be applied and in which decoding of a video / image signal is performed.

[0019] FIG. 4 illustrates an example of a video / image decoding method to which an embodiment of the present disclosure can be applied.

[0020] FIG. 5 illustrates an example of a video / image encoding method to which an embodiment of the present disclosure can be applied.

[0021] FIG. 6 and FIG. 7 illustrate examples of inter-prediction based video / image encoding methods to which embodiments of the present disclosure can be applied.

[0022] FIG. 8 and FIG. 9 illustrate examples of inter-prediction based video / image decoding methods to which embodiments of the present disclosure can be applied.

[0023] FIG. 10 exemplarily illustrates an inter prediction procedure to which an embodiment of the present disclosure can be applied.

[0024] FIG. 11 is a flowchart illustrating a method for generating a prediction sample according to one embodiment.

[0025] FIG. 12 and FIG. 13 are schematic diagrams illustrating a process for inducing MVD according to one embodiment.

[0026] FIG. 14 and FIG. 15 are diagrams showing examples of a method for inducing MVD according to one embodiment.

[0027] FIG. 16 and FIG. 17 are drawings showing another example of a method for deriving MVD according to one embodiment.

[0028] Figure 18 is a diagram showing signaling bits used to signal information indicating whether AMVP mode, affine mode, and SMVD mode are applied.

[0029] FIG. 19 is a diagram illustrating signaling bits used to signal information indicating whether AMVP mode, affine mode, and SMVD mode are applied, according to one embodiment.

[0030] FIG. 20 is a diagram illustrating an example of a paired MVP candidate list according to one embodiment.

[0031] FIG. 21 is a diagram illustrating an example of a content streaming system to which an embodiment according to the present disclosure can be applied.

[0032] The present disclosure may be modified in various ways and encompasses numerous embodiments. Specific embodiments are illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the present disclosure to specific embodiments, but rather to encompass all modifications, equivalents, and alternatives falling within the spirit and technical scope of the present disclosure. Throughout the description of each drawing, similar reference numerals have been used to designate similar components.

[0033] While terms such as "first" and "second" may be used to describe various components, these components should not be limited by these terms. These terms are used solely to distinguish one component from another. For example, without departing from the scope of the present disclosure, a first component could be referred to as a "second component," and similarly, a second component could also be referred to as a "first component." The term "and / or" includes a combination of multiple related items described herein or any of multiple related items described herein.

[0034] When a component is referred to as being "connected" or "connected" to another component, it should be understood that it may be directly connected or connected to that other component, but that there may be other components intervening. Conversely, when a component is referred to as being "directly connected" or "connected" to another component, it should be understood that there are no other components intervening.

[0035] The terminology used in this application is only used to describe specific embodiments and is not intended to limit the present disclosure. The singular expression includes the plural expression unless the context clearly indicates otherwise. In this application, it should be understood that the terms "comprise" or "have" indicate the presence of a feature, number, step, operation, component, part, or combination thereof described in the specification, but do not preclude the possibility of the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

[0036] The present disclosure relates to video / image coding. For example, the methods / embodiments disclosed herein may be applied to methods disclosed in various video / image coding standards.

[0037] This specification discloses various embodiments or examples relating to video / image coding, and unless otherwise stated, the disclosed embodiments or examples can be performed in combination with each other.

[0038] In this specification, video may refer to a set of images over time. A picture generally refers to a unit representing one image at a specific time point, and a slice / tile is a unit that constitutes part of a picture in coding. A slice / tile may include one or more coding tree units (CTUs). A picture may be composed of one or more slices / tiles. A tile is a rectangular area consisting of multiple CTUs within a specific tile column and a specific tile row of a picture. A tile column is a rectangular area of ​​CTUs that has a height equal to the height of the picture and a width specified by the syntax requirements of the picture parameter set. A tile row is a rectangular area of ​​CTUs that has a height specified by the picture parameter set and a width equal to the width of the picture. CTUs within a tile are arranged consecutively according to the CTU raster scan, while tiles within a picture may be arranged consecutively according to the tile raster scan. A slice may contain an integer number of complete tiles or an integer number of contiguous complete CTU rows within a picture, which may be exclusively contained within a single NAL unit. Meanwhile, a picture may be divided into two or more subpictures. A subpicture may be a rectangular region of one or more slices within a picture.

[0039] A pixel, or pel, can refer to the smallest unit that constitutes a picture (or image). Additionally, the term "sample" can be used as a counterpart to a pixel. A sample can generally represent a pixel or a pixel value, and can represent only the pixel / pixel value of the luminance component, or only the pixel / pixel value of the chrominance component.

[0040] A unit may represent a basic unit of image processing. A unit may include at least one of a specific region of a picture and information related to the region. One unit may include one luma block and two chroma (e.g., cb, cr) blocks. In some cases, the term "unit" may be used interchangeably with terms such as "block" or "area." In general, an MxN block may include a set (or array) of samples (or sample array) or transform coefficients consisting of M columns and N rows.

[0041] As used herein, “A or B” can mean “only A,” “only B,” or “both A and B.” In other words, as used herein, “A or B” can be interpreted as “A and / or B.” For example, as used herein, “A, B or C” can mean “only A,” “only B,” “only C,” or “any combination of A, B, and C.”

[0042] As used herein, a slash ( / ) or a comma can mean "and / or." For example, "A / B" can mean "A and / or B." Accordingly, "A / B" can mean "only A," "only B," or "both A and B." For example, "A, B, C" can mean "A, B, or C."

[0043] In this specification, “at least one of A and B” may mean “only A,” “only B,” or “both A and B.” Additionally, in this specification, the expressions “at least one of A or B” or “at least one of A and / or B” may be interpreted identically to “at least one of A and B.”

[0044] Additionally, in this specification, “at least one of A, B and C” can mean “only A,” “only B,” “only C,” or “any combination of A, B and C.” Additionally, “at least one of A, B or C” or “at least one of A, B and / or C” can mean “at least one of A, B and C.”

[0045] Additionally, parentheses used herein may mean "for example." Specifically, when "prediction (intra-prediction)" is indicated, "intra-prediction" may be suggested as an example of "prediction." In other words, "prediction" in this specification is not limited to "intra-prediction," and "intra-prediction" may be suggested as an example of "prediction." Furthermore, even when "prediction (i.e., intra-prediction)" is indicated, "intra-prediction" may be suggested as an example of "prediction."

[0046] Technical features individually described in a single drawing in this specification may be implemented individually or simultaneously.

[0047] FIG. 1 illustrates a video / image coding system according to the present disclosure.

[0048] Referring to FIG. 1, a video / image coding system may include a first device (source device) and a second device (receiving device).

[0049] A source device can transmit encoded video / image information or data to a receiving device via a digital storage medium or a network in the form of a file or streaming. The source device may include a video source, an encoding device, and a transmitting device. The receiving device may include a receiving device, a decoding device, and a renderer. The encoding device may be referred to as a video / image encoding device, and the decoding device may be referred to as a video / image decoding device. The transmitter may be included in the encoding device. The receiver may be included in the decoding device. The renderer may include a display unit, and the display unit may be configured as a separate device or an external component.

[0050] A video source may obtain video / images through a process of capturing, synthesizing, or generating video / images. The video source may include a video / image capture device and / or a video / image generation device. The video / image capture device may include one or more cameras, a video / image archive containing previously captured video / images, etc. The video / image generation device may include a computer, a tablet, a smartphone, etc., and may (electronically) generate video / images. For example, a virtual video / image may be generated through a computer, etc., in which case the video / image capture process may be replaced by a process of generating related data.

[0051] An encoding device can encode input video / images. The encoding device can perform a series of procedures, such as prediction, transformation, and quantization, to improve compression and coding efficiency. The encoded data (encoded video / image information) can be output in the form of a bitstream.

[0052] The transmission unit can transmit encoded video / image information or data output in the form of a bitstream to the receiving unit of a receiving device via a digital storage medium or a network in the form of a file or streaming. The digital storage medium can include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. The storage medium can be a computer-readable storage medium and can store data non-transitory. The transmission unit can include an element for generating a media file via a predetermined file format and an element for transmission via a broadcasting / communication network. The receiving unit can receive / extract the bitstream and transmit it to a decoding device.

[0053] The decoding device can decode the video / image by performing a series of procedures such as inverse quantization, inverse transformation, and prediction corresponding to the operation of the encoding device.

[0054] The renderer can render decoded video / images. The rendered video / images can be displayed through the display unit.

[0055] FIG. 2 is a schematic block diagram of an encoding device to which an embodiment of the present disclosure can be applied and in which encoding of a video / image signal is performed.

[0056] Referring to FIG. 2, the encoding device (200) may be configured to include an image partitioner (210), a prediction unit (predictor) 220, a residual processor (residual processor) 230, an entropy encoder (entropy encoder) 240, an adder (adder) 250, a filter (filter) 260, and a memory (memory) 270. The prediction unit (220) may include an inter prediction unit (221) and an intra prediction unit (222). The residual processor (230) may include a transformer (transformer) 232, a quantizer (quantizer) 233, a dequantizer (dequantizer) 234, and an inverse transformer (inverse transformer) 235. The residual processing unit (230) may further include a subtractor (231). The addition unit (250) may be called a reconstructor or a recontructed block generator. The image segmentation unit (210), the prediction unit (220), the residual processing unit (230), the entropy encoding unit (240), the addition unit (250), and the filtering unit (260) described above may be configured by one or more hardware components (e.g., an encoding device chipset or processor) according to an embodiment. In addition, the memory (270) may include a decoded picture buffer (DPB) and may be configured by a digital storage medium. The hardware component may further include the memory (270) as an internal / external component.

[0057] The encoding device (200) may include at least one processor and at least one memory. At least one processor may perform the operations of the encoding device (200) described below, and at least one memory may be electrically connected to at least one processor and store programs and various data necessary to perform the operations of the encoding device (200).

[0058] The image segmentation unit (210) can segment an input image (or picture, frame) input to the encoding device (200) into one or more processing units (PUs). For example, the processing units may be called coding units (CUs). In this case, the coding units may be recursively segmented from a coding tree unit (CTU) or a largest coding unit (LCU) according to a QTBTTT (Quad-Tree Binary-Tree Ternary-Tree) structure.

[0059] For example, a single coding unit may be split into multiple coding units with deeper depths based on a quad-tree structure, a binary tree structure, and / or a ternary structure. In this case, for example, the quad-tree structure may be applied first, and the binary tree structure and / or the ternary structure may be applied later. Alternatively, the binary tree structure may be applied before the quad-tree structure. The coding procedure according to the present specification may be performed based on the final coding unit that is no longer split. In this case, based on coding efficiency according to image characteristics, etc., the largest coding unit may be used directly as the final coding unit, or, if necessary, the coding unit may be recursively split into coding units of lower depths, and the coding unit with the optimal size may be used as the final coding unit. Here, the coding procedure may include procedures such as prediction, transformation, and restoration, which will be described later.

[0060] As another example, the processing unit may further include a prediction unit (PU) or a transform unit (TU). In this case, the prediction unit and the transform unit may each be split or partitioned from the final coding unit described above. The prediction unit may be a unit of sample prediction, and the transform unit may be a unit for deriving a transform coefficient and / or a unit for deriving a residual signal from a transform coefficient.

[0061] The term "unit" may be used interchangeably with terms such as "block" or "area" depending on the case. In general, an MxN block can represent a set of samples or transform coefficients consisting of M columns and N rows. A sample can generally represent a pixel or a pixel value, and can represent only a pixel / pixel value of a luminance component or only a pixel / pixel value of a chrominance component. A sample can be used as a term corresponding to a pixel or pel of a picture (or image).

[0062] The encoding device (200) can generate a residual signal (residual block, residual sample array) by subtracting a prediction signal (prediction block, prediction sample array) output from an inter prediction unit (221) or an intra prediction unit (222) from an input video signal (original block, original sample array), and the generated residual signal is transmitted to a conversion unit (232). In this case, a unit that subtracts a prediction signal (prediction block, prediction sample array) from an input video signal (original block, original sample array) within the encoding device (200) may be called a subtraction unit (231).

[0063] The prediction unit (220) can perform prediction on a block to be processed (hereinafter, referred to as a current block) to generate prediction samples, and can generate a predicted block including the prediction samples for the current block. The prediction unit (220) can determine whether intra prediction or inter prediction is applied to the current block or CU unit. The prediction unit (220) can generate various information regarding prediction, such as prediction mode information, as described later in the description of each prediction mode, and transmit the information to the entropy encoding unit (240). The information regarding prediction can be encoded by the entropy encoding unit (240) and output in the form of a bitstream.

[0064] The intra prediction unit (222) can predict the current block by referring to samples within the current picture. The referenced samples, i.e., the reference samples, may be located in the neighborhood of the current block or may be located a certain distance away from the current block depending on the prediction mode. In intra prediction, the prediction modes may include one or more non-directional modes and multiple directional modes. The non-directional mode may include at least one of the DC mode or the planar mode. The directional mode may include 33 directional modes or 65 directional modes depending on the degree of detail in the prediction direction. However, this is merely an example, and a greater or lesser number of directional modes may be used depending on the settings. The intra prediction unit (222) may also determine the prediction mode applied to the current block by using the prediction mode applied to the neighboring blocks.

[0065] The inter prediction unit (221) can derive a prediction block for the current block based on a reference block (reference sample array) specified by a motion vector on a reference picture. At this time, in order to reduce the amount of motion information transmitted in the inter prediction mode, the motion information can be predicted in units of blocks, sub-blocks, or samples based on the correlation of the motion information between the neighboring blocks and the current block. The motion information can include a motion vector and a reference picture index. The motion information can further include inter prediction direction information (L0 prediction, L1 prediction, Bi prediction, etc.). In the case of inter prediction, the neighboring block can include a spatial neighboring block existing in the current picture and a temporal neighboring block existing in the reference picture. The reference picture including the reference block and the reference picture including the temporal neighboring block may be the same or different. Temporal neighboring blocks may be called collocated reference blocks, collocated CUs (colCUs), etc., and reference pictures including temporal neighboring blocks may be called collocated pictures (colPic). For example, the inter prediction unit (221) may construct a motion information candidate list based on neighboring blocks, and generate information indicating which candidate is used to derive the motion vector and / or reference picture index of the current block. Inter prediction may be performed based on various prediction modes, and for example, in the case of skip mode and merge mode, the inter prediction unit (221) may use the motion information of neighboring blocks as the motion information of the current block. In the case of skip mode, unlike the merge mode, a residual signal may not be transmitted.In the motion vector prediction (MVP) mode, the motion vector of the surrounding blocks is used as a motion vector predictor, and the motion vector of the current block can be indicated by signaling the motion vector difference.

[0066] The prediction unit (220) can generate a prediction signal based on various prediction methods described below. For example, the prediction unit can apply intra prediction or inter prediction for prediction of a single block, and can also apply intra prediction and inter prediction simultaneously. This can be called combined inter and intra prediction (CIIP) mode. In addition, the prediction unit can be based on an intra block copy (IBC) prediction mode or a palette mode for prediction of a block. The IBC prediction mode or palette mode can be used for content image / video coding such as games, such as screen content coding (SCC). IBC basically performs prediction within the current picture, but can be performed similarly to inter prediction in that it derives a reference block within the current picture. That is, IBC can utilize at least one of the inter prediction techniques described herein. Palette mode can be viewed as an example of intra coding or intra prediction. When the palette mode is applied, sample values ​​within a picture can be signaled based on information about the palette table and palette index. The prediction signal generated through the prediction unit (220) can be used to generate a restoration signal or a residual signal.

[0067] The transform unit (232) can apply a transform technique to the residual signal to generate transform coefficients. For example, the transform technique can include at least one of a Discrete Cosine Transform (DCT), a Discrete Sine Transform (DST), a Karhunen-Loeve Transform (KLT), a Graph-Based Transform (GBT), or a Conditionally Non-linear Transform (CNT). Here, GBT refers to a transform obtained from a graph when the relationship information between pixels is expressed as a graph. CNT refers to a transform obtained based on generating a prediction signal using all previously restored pixels. In addition, the transform process can be applied to a pixel block having a square size and the same size, or can be applied to a block of a non-square variable size.

[0068] The quantization unit (233) quantizes the transform coefficients and transmits them to the entropy encoding unit (240), and the entropy encoding unit (240) can encode the quantized signal (information about the quantized transform coefficients) and output it as a bitstream. The information about the quantized transform coefficients can be called residual information. The quantization unit (233) can rearrange the quantized transform coefficients in a block form into a one-dimensional vector form based on the coefficient scan order, and can also generate information about the quantized transform coefficients based on the quantized transform coefficients in the one-dimensional vector form.

[0069] The entropy encoding unit (240) can perform various encoding methods such as exponential Golomb, context-adaptive variable length coding (CAVLC), context-adaptive binary arithmetic coding (CABAC), etc. The entropy encoding unit (240) can also encode information necessary for video / image restoration (e.g., values ​​of syntax elements, etc.) together or separately from quantized transform coefficients.

[0070] Encoded information (e.g., encoded video / image information) can be transmitted or stored in the form of a bitstream in units of NAL (network abstraction layer) units. The video / image information may further include information on various parameter sets, such as an adaptation parameter set (APS), a picture parameter set (PPS), a sequence parameter set (SPS), or a video parameter set (VPS). In addition, the video / image information may further include general constraint information. In the present specification, information and / or syntax elements transmitted / signaled from an encoding device to a decoding device may be included in the video / image information. The video / image information may be encoded through the above-described encoding procedure and included in the bitstream. The bitstream may be transmitted via a network or stored in a digital storage medium. Here, the network may include a broadcasting network and / or a communication network, and the digital storage medium may include various storage media, such as a USB, SD, CD, DVD, Blu-ray, HDD, or SSD. A transmission unit (not shown) for transmitting a signal output from an entropy encoding unit (240) and / or a storage unit (not shown) for storing the signal may be configured as an internal / external element of the encoding device (200), or the transmission unit may be included in the entropy encoding unit (240).

[0071] The quantized transform coefficients output from the quantization unit (233) can be used to generate a prediction signal. For example, by applying inverse quantization and inverse transformation to the quantized transform coefficients through the inverse quantization unit (234) and the inverse transform unit (235), a residual signal (residual block or residual samples) can be reconstructed. The addition unit (250) can generate a reconstructed signal (reconstructed picture, reconstructed block, reconstructed sample array) by adding the reconstructed residual signal to the prediction signal output from the inter prediction unit (221) or the intra prediction unit (222). When there is no residual for the block to be processed, such as when skip mode is applied, the predicted block can be used as a reconstructed block. The addition unit (250) may be called a reconstructor or a reconstructed block generation unit. The generated restoration signal can be used for intra prediction of the next processing target block within the current picture, and can also be used for inter prediction of the next picture after filtering as described below. Meanwhile, LMCS (luma mapping with chroma scaling) may be applied during the picture encoding and / or restoration process.

[0072] The filtering unit (260) can improve subjective / objective picture quality by applying filtering to the restoration signal. For example, the filtering unit (260) can apply various filtering methods to the restoration picture to generate a modified restoration picture, and store the modified restoration picture in the memory (270), specifically, in the DPB of the memory (270). The various filtering methods can include deblocking filtering, sample adaptive offset, adaptive loop filter, bilateral filter, etc. The filtering unit (260) can generate various information regarding filtering and transmit it to the entropy encoding unit (240). The information regarding filtering can be encoded by the entropy encoding unit (240) and output in the form of a bitstream.

[0073] The modified restored picture transmitted to the memory (270) can be used as a reference picture in the inter prediction unit (221). Through this, when inter prediction is applied, the encoding device can avoid prediction mismatch between the encoding device (200) and the decoding device, and can also improve encoding efficiency.

[0074] The DPB of the memory (270) can store the modified restored picture to be used as a reference picture in the inter prediction unit (221). The memory (270) can store motion information of a block from which motion information in the current picture is derived (or encoded) and / or motion information of blocks in a picture that has already been restored. The stored motion information can be transferred to the inter prediction unit (221) to be used as motion information of a spatial neighboring block or motion information of a temporal neighboring block. The memory (270) can store restored samples of restored blocks in the current picture and transfer them to the intra prediction unit (222).

[0075] Image information output in the form of a bitstream from the encoding device (200) can be transmitted to the decoding device (300).

[0076] FIG. 3 is a schematic block diagram of a decoding device to which an embodiment of the present disclosure can be applied and in which decoding of a video / image signal is performed.

[0077] Image information transmitted in the form of a bitstream from the encoding device (200) can be received by the decoding device (300).

[0078] Referring to FIG. 3, the decoding device (300) may be configured to include an entropy decoder (310), a residual processor (320), a predictor (330), an adder (340), a filter (350), and a memory (360). The predictor (330) may include an inter-prediction unit (332) and an intra-prediction unit (331). The residual processor (320) may include a dequantizer (321) and an inverse transformer (321).

[0079] The entropy decoding unit (310), residual processing unit (320), prediction unit (330), addition unit (340), and filtering unit (350) described above may be configured by a single hardware component (e.g., a decoding device chipset or processor) depending on the embodiment. In addition, the memory (360) may include a decoded picture buffer (DPB) and may be configured by a digital storage medium. The hardware component may further include the memory (360) as an internal / external component.

[0080] The decoding device (300) may include at least one processor and at least one memory. At least one processor may perform the operations of the encoding device (300) described below, and at least one memory may be electrically connected to at least one processor and store programs and various data necessary to perform the operations of the encoding device (300).

[0081] When a bitstream including video / image information is input, the decoding device (300) can restore the image corresponding to the process in which the video / image information is processed in the encoding device of FIG. 2. For example, the decoding device (300) can derive units / blocks based on block division related information obtained from the bitstream. The decoding device (300) can perform decoding using a processing unit applied in the encoding device. Accordingly, the processing unit of decoding may be a coding unit, and the coding unit may be divided from a coding tree unit or a maximum coding unit according to a quad tree structure, a binary tree structure, and / or a ternary tree structure. One or more transform units may be derived from the coding unit. Then, the restored image signal decoded and output by the decoding device (300) can be reproduced through a reproduction device.

[0082] The decoding device (300) can receive a signal output from the encoding device of FIG. 2 in the form of a bitstream, and the received signal can be decoded through the entropy decoding unit (310). For example, the entropy decoding unit (310) can parse the bitstream to derive information (e.g., video / image information) necessary for image restoration (or picture restoration). The video / image information may further include information on various parameter sets, such as an adaptation parameter set (APS), a picture parameter set (PPS), a sequence parameter set (SPS), or a video parameter set (VPS). In addition, the video / image information may further include general constraint information. The decoding device can decode the picture further based on the information on the parameter set and / or the general constraint information. The signaling / received information and / or syntax elements described later in this specification can be decoded and obtained from the bitstream through the decoding procedure. For example, the entropy decoding unit (310) can decode information in a bitstream based on a coding method such as exponential Golomb coding, CAVLC, or CABAC, and output the values ​​of syntax elements required for image restoration and the quantized values ​​of transform coefficients for residuals. More specifically, the CABAC entropy decoding method receives a bin corresponding to each syntax element in the bitstream, determines a context model using information of the syntax element to be decoded and decoding information of the surrounding and decoding target blocks or information of symbols / bins decoded in the previous step, and predicts the occurrence probability of the bin according to the determined context model to perform arithmetic decoding of the bin to generate a symbol corresponding to the value of each syntax element.At this time, the CABAC entropy decoding method can update the context model using the information of the decoded symbol / bin for the context model of the next symbol / bin after determining the context model. Information regarding prediction among the information decoded by the entropy decoding unit (310) is provided to the prediction unit (inter prediction unit (332) and intra prediction unit (331)), and residual values ​​on which entropy decoding is performed by the entropy decoding unit (310), i.e., quantized transform coefficients and related parameter information, can be input to the residual processing unit (320). The residual processing unit (320) can derive a residual signal (residual block, residual samples, residual sample array). In addition, information regarding filtering among the information decoded by the entropy decoding unit (310) can be provided to the filtering unit (350). Meanwhile, a receiving unit (not shown) that receives a signal output from an encoding device may be further configured as an internal / external element of a decoding device (300), or the receiving unit may be a component of an entropy decoding unit (310).

[0083] Meanwhile, a decoding device according to the present specification may be called a video / video / picture decoding device, and the decoding device may be divided into an information decoding device (video / video / picture information decoding device) and a sample decoding device (video / video / picture sample decoding device). The information decoding device may include the entropy decoding unit (310), and the sample decoding device may include at least one of the inverse quantization unit (321), the inverse transformation unit (322), the addition unit (340), the filtering unit (350), the memory (360), the inter prediction unit (332), and the intra prediction unit (331).

[0084] The inverse quantization unit (321) can inverse quantize the quantized transform coefficients and output the transform coefficients. The inverse quantization unit (321) can rearrange the quantized transform coefficients into a two-dimensional block form. In this case, the rearrangement can be performed based on the coefficient scanning order performed in the encoding device. The inverse quantization unit (321) can perform inverse quantization on the quantized transform coefficients using quantization parameters (e.g., quantization step size information) and obtain transform coefficients.

[0085] In the inverse transform unit (322), the transform coefficients are inversely transformed to obtain a residual signal (residual block, residual sample array).

[0086] The prediction unit (320) can perform a prediction on the current block and generate a predicted block including prediction samples for the current block. The prediction unit (320) can determine whether intra-prediction or inter-prediction is applied to the current block based on the information regarding the prediction output from the entropy decoding unit (310), and can determine a specific intra / inter-prediction mode.

[0087] The prediction unit (320) can generate a prediction signal based on various prediction methods described below. For example, the prediction unit (320) can apply intra prediction or inter prediction for prediction of a single block, and can also apply intra prediction and inter prediction simultaneously. This can be called combined inter and intra prediction (CIIP) mode. In addition, the prediction unit can be based on an intra block copy (IBC) prediction mode or a palette mode for prediction of a block. The IBC prediction mode or palette mode can be used for content image / video coding such as games, such as screen content coding (SCC). IBC basically performs prediction within the current picture, but can be performed similarly to inter prediction in that it derives a reference block within the current picture. That is, IBC can utilize at least one of the inter prediction techniques described herein. The palette mode can be viewed as an example of intra coding or intra prediction. When palette mode is applied, information about the palette table and palette index may be signaled and included in the video / image information.

[0088] The intra prediction unit (331) can predict the current block by referring to samples within the current picture. The referenced samples may be located in the neighborhood of the current block, or may be located a certain distance away from the current block, depending on the prediction mode. In intra prediction, the prediction modes may include one or more non-directional modes and multiple directional modes. The intra prediction unit (331) may also determine the prediction mode applied to the current block by using the prediction mode applied to the neighboring blocks.

[0089] The inter prediction unit (332) can derive a prediction block for the current block based on a reference block (reference sample array) specified by a motion vector on a reference picture. At this time, in order to reduce the amount of motion information transmitted in the inter prediction mode, the motion information can be predicted in units of blocks, sub-blocks, or samples based on the correlation of the motion information between the neighboring blocks and the current block. The motion information can include a motion vector and a reference picture index. The motion information can further include inter prediction direction information (L0 prediction, L1 prediction, Bi prediction, etc.). In the case of inter prediction, the neighboring blocks can include spatial neighboring blocks existing in the current picture and temporal neighboring blocks existing in the reference picture. For example, the inter prediction unit (332) can construct a motion information candidate list based on the neighboring blocks, and derive the motion vector and / or reference picture index of the current block based on the received candidate selection information. Inter prediction can be performed based on various prediction modes, and information about the prediction can include information indicating an inter prediction mode for the current block.

[0090] The addition unit (340) can generate a restoration signal (restored picture, restoration block, restoration sample array) by adding the acquired residual signal to the prediction signal (prediction block, prediction sample array) output from the prediction unit (including the inter-prediction unit (332) and / or intra-prediction unit (331)). When there is no residual for the block to be processed, such as when skip mode is applied, the prediction block can be used as the restoration block.

[0091] The addition unit (340) may be referred to as a restoration unit or restoration block generation unit. The generated restoration signal may be used for intra prediction of the next processing target block within the current picture, may be output after filtering as described below, or may be used for inter prediction of the next picture. Meanwhile, LMCS (luma mapping with chroma scaling) may be applied during the picture decoding process.

[0092] The filtering unit (350) can improve subjective / objective image quality by applying filtering to the restored signal. For example, the filtering unit (350) can apply various filtering methods to the restored picture to generate a modified restored picture, and transmit the modified restored picture to the memory (360), specifically, to the DPB of the memory (360). The various filtering methods can include deblocking filtering, sample adaptive offset, adaptive loop filter, bilateral filter, etc.

[0093] The (modified) reconstructed picture stored in the DPB of the memory (360) can be used as a reference picture in the inter prediction unit (332). The memory (360) can store motion information of a block from which motion information in the current picture is derived (or decoded) and / or motion information of blocks in an already reconstructed picture. The stored motion information can be transmitted to the inter prediction unit (332) to be used as motion information of a spatial neighboring block or motion information of a temporal neighboring block. The memory (360) can store reconstructed samples of reconstructed blocks in the current picture and transmit them to the intra prediction unit (331).

[0094] In this specification, the embodiments described in the filtering unit (260), the inter prediction unit (221), and the intra prediction unit (222) of the encoding device (200) can be applied to the filtering unit (350), the inter prediction unit (332), and the intra prediction unit (331) of the decoding device (300) in the same or corresponding manner, respectively.

[0095] FIG. 4 illustrates an example of a video / image decoding method to which an embodiment of the present disclosure can be applied.

[0096] In image / video coding, the pictures that make up an image / video can be encoded / decoded according to a series of decoding orders. The picture order corresponding to the output order of the decoded pictures can be set differently from the decoding order, and based on this, not only forward prediction but also backward prediction can be performed during inter prediction.

[0097] In FIG. 4, S400 may be performed in the entropy decoding unit (310) of the aforementioned decoding device (300), S410 may be performed in the prediction unit (330), S420 may be performed in the residual processing unit (320), S430 may be performed in the addition unit (340), and S440 may be performed in the filtering unit (350). S400 may include a decoding procedure according to the present disclosure, S410 may include an inter / intra prediction procedure according to the present disclosure, S420 may include a residual processing procedure according to the present disclosure, S430 may include a block / picture restoration procedure according to the present disclosure, and S440 may include an in-loop filtering procedure according to the present disclosure.

[0098] Referring to FIG. 4, the decoding device obtains image / video information from a bitstream (S400), performs prediction based on the obtained image / video information (S410), and restores a picture through residual processing (S420, inverse quantization for quantized transform coefficients, inverse transformation) (S430).

[0099] A modified restored picture can be generated by applying an in-loop filtering procedure (S440) to a restored picture generated through the above restoration procedure, and the modified restored picture can be output as a decoded picture and can be stored in a buffer or memory of a decoding device to be used as a reference picture in an inter prediction procedure when decoding a next picture. In some cases, the in-loop filtering procedure can be omitted, in which case the restored picture can be output as a decoded picture and can be stored in a buffer or memory of a decoding device to be used as a reference picture in an inter prediction procedure when decoding a next picture.

[0100] The in-loop filtering procedure (S440) may include a deblocking filtering procedure, a sample adaptive offset (SAO) procedure, an adaptive loop filter (ALF) procedure, and / or a bi-lateral filter procedure, and some or all of them may be omitted. In addition, one or some of the deblocking filtering procedure, the sample adaptive offset (SAO) procedure, the adaptive loop filter (ALF) procedure, and the bi-lateral filter procedure may be sequentially applied, or all of them may be sequentially applied. For example, the SAO procedure may be performed after the deblocking filtering procedure is applied to the restored picture. Or, for example, the ALF procedure may be performed after the deblocking filtering procedure is applied to the restored picture. This may also be performed in an encoding device.

[0101] FIG. 5 illustrates an example of a video / image encoding method to which an embodiment of the present disclosure can be applied.

[0102] In FIG. 5, the prediction step (S500) may be performed in the prediction unit (220) of the encoding device (200) described above, residual processing (S510) based on the prediction result may be performed in the residual processing unit (230), and the step (S520) of encoding image information including prediction information and residual information may be performed in the entropy encoding unit (240). S500 may include an inter / intra prediction procedure according to the present disclosure, S510 may include a residual processing procedure according to the present disclosure, and S520 may include an encoding procedure according to the present disclosure.

[0103] The encoding procedure may optionally include a procedure for encoding information for picture restoration (e.g., prediction information, residual information, partitioning information, etc.) and outputting it in the form of a bitstream, as well as a procedure for generating a restored picture for the current picture and a procedure for applying in-loop filtering to the restored picture.

[0104] The encoding device (200) can derive (corrected) residual samples from the quantized transform coefficients through the inverse quantization unit (234) and the inverse transformation unit (235), and can generate a restored picture based on the prediction samples and (corrected) residual samples, which are outputs of S500. The restored picture generated in this way can be the same as the restored picture generated by the decoding device (300) described above. A modified restored picture can be generated through an in-loop filtering procedure for the restored picture, which can be stored in a buffer or memory, and, as in the case of the decoding device, can be used as a reference picture in the inter prediction procedure when encoding a subsequent picture.

[0105] As described above, some or all of the in-loop filtering procedure may be omitted in some cases. When the in-loop filtering procedure is performed, (in-loop) filtering-related information (parameters) may be encoded by the entropy encoding unit (240) and output in the form of a bitstream, and the decoding device (300) may perform the in-loop filtering procedure in the same manner as the encoding device based on the filtering-related information.

[0106] Through this in-loop filtering procedure, noise occurring during image / video coding, such as blocking artifacts and ringing artifacts, can be reduced, and subjective / objective image quality can be improved. In addition, by performing the in-loop filtering procedure in both the encoding device (200) and the decoding device (300), the same prediction results can be derived from the encoding device (200) and the decoding device (300), thereby increasing the reliability of picture coding and reducing the amount of data that must be transmitted for picture coding.

[0107] As described above, the picture restoration procedure can be performed not only in the decoding device (300) but also in the encoding device (200). A restoration block can be generated based on intra-prediction / inter-prediction for each block, and a restoration picture including the restoration blocks can be generated. If the current picture / slice / tile group is an I picture / slice / tile group, the blocks included in the current picture / slice / tile group can be restored based only on intra-prediction. On the other hand, if the current picture / slice / tile group is a P or B picture / slice / tile group, the blocks included in the current picture / slice / tile group can be restored based on intra-prediction or inter-prediction. In this case, inter-prediction may be applied to some blocks in the current picture / slice / tile group, and intra-prediction may be applied to some remaining blocks.

[0108] The color component of a picture may include a luma component and a chroma component, and unless explicitly limited in the present disclosure, embodiments according to the present disclosure may be applied to the luma component and the chroma component.

[0109] Meanwhile, when intra prediction is performed, the prediction unit (220, 330) of the encoding device (200) / decoding device (300) can derive a reference sample according to the intra prediction mode of the current block among the surrounding samples of the current block, and can generate a prediction sample of the current block based on the reference sample.

[0110] For example, (i) the prediction sample can be derived based on the average or interpolation of neighboring reference samples of the current block, and (ii) the prediction sample can be derived based on a reference sample existing in a specific (prediction) direction with respect to the prediction sample among the neighboring reference samples of the current block. The case of (i) can be called a non-directional mode or a non-angular mode, and the case of (ii) can be called a directional mode or an angular mode.

[0111] Additionally, linear interpolation intra prediction (LIP) may be applied to perform intra prediction on the current block by linearly interpolating prediction sample values ​​generated based on the intra prediction mode of the current block.

[0112] Additionally, a temporary prediction sample of the current block may be derived based on filtered peripheral reference samples, and a prediction sample of the current block may be derived by weighting at least one reference sample derived according to an intra prediction mode among existing peripheral reference samples, i.e., unfiltered peripheral reference samples, and the temporary prediction sample. Such prediction may be referred to as Position Dependent Intra Prediction Combination (PDPC).

[0113] In addition, intra prediction encoding can be performed by selecting a reference sample line with the highest prediction accuracy among the surrounding multiple reference sample lines of the current block, deriving a prediction sample using the reference sample located in the prediction direction of the selected line, and then instructing (signaling) the used reference sample line to the decoding device. This case can be referred to as multi-reference line intra prediction (MRL) or MRL-based intra prediction.

[0114] Additionally, the current block can be divided into vertical or horizontal subpartitions, and intra prediction can be performed based on the same intra prediction mode, while peripheral reference samples can be derived and utilized for each subpartition. In other words, in this case, the intra prediction mode for the current block is applied equally to the subpartitions, but peripheral reference samples can be derived and utilized for each subpartition, thereby improving intra prediction performance in some cases. This prediction method can be called intra subpartitions (ISP) or ISP-based intra prediction.

[0115] Additionally, if the prediction direction based on the prediction sample points between surrounding reference samples, i.e., if the prediction direction points to a fractional sample location, the value of the prediction sample can be derived through interpolation of multiple reference samples located around the prediction direction (around the fractional sample location).

[0116] Meanwhile, when inter prediction is applied, the prediction unit of the encoding device / decoding device can perform inter prediction on a block-by-block basis to derive prediction samples. Inter prediction can refer to a prediction derived in a manner dependent on data elements (e.g., sample values, or motion information) of pictures other than the current picture. When inter prediction is applied to the current block, a predicted block (prediction sample array) for the current block can be derived based on a reference block (reference sample array) specified by a motion vector on a reference picture pointed to by a reference picture index.

[0117] At this time, in order to reduce the amount of motion information transmitted in the inter prediction mode, the motion information of the current block can be predicted in units of blocks, sub-blocks, or samples based on the correlation of the motion information between the neighboring blocks and the current block. The motion information may include a motion vector and / or a reference picture index. The motion information may further include information on the inter prediction type (L0 prediction, L1 prediction, Bi prediction, etc.). When inter prediction is applied, the neighboring blocks may include spatial neighboring blocks existing in the current picture and temporal neighboring blocks existing in the reference picture.

[0118] The reference picture including the above reference block and the reference picture including the temporal neighboring block may be the same or different. The temporal neighboring block may be called a collocated reference block, a collocated CU (colCU), etc., and the reference picture including the temporal neighboring block may be called a collocated picture (colPic). For example, a motion information candidate list may be constructed based on the neighboring blocks of the current block, and a flag or index information indicating which candidate is selected (used) to derive the motion vector and / or reference picture index of the current block may be signaled.

[0119] Inter prediction can be performed based on various prediction modes. For example, in the case of skip mode and merge mode, the motion information of the current block may be the same as the motion information of the selected neighboring block. In the case of skip mode, unlike the merge mode, a residual signal may not be transmitted. In the case of motion vector prediction (MVP) mode, the motion vector of the selected neighboring block may be used as a motion vector predictor, and the motion vector difference may be signaled. In this case, the motion vector of the current block can be derived using the sum of the motion vector predictor and the motion vector difference.

[0120] The above motion information may include L0 motion information and / or L1 motion information depending on the inter prediction type (L0 prediction, L1 prediction, Bi prediction, etc.). The L0 motion vector may be called L0 motion vector or MVL0, and the L1 motion vector may be called L1 motion vector or MVL1. Prediction based on an L0 motion vector may be called L0 prediction, prediction based on an L1 motion vector may be called L1 prediction, and prediction based on both the L0 motion vector and the L1 motion vector may be called bi-prediction. Here, the L0 motion vector may indicate a motion vector associated with an L0 reference picture list, and the L1 motion vector may indicate a motion vector associated with an L1 reference picture list. The L0 reference picture list may include pictures that are earlier in output order than the current picture as reference pictures, and the L1 reference picture list may include pictures that are later in output order than the current picture. The above previous pictures may be called forward pictures, and the above subsequent pictures may be called backward pictures.

[0121] The above L0 reference picture list may include pictures subsequent to the current picture in output order as reference pictures. For example, the previous pictures may be indexed first and the subsequent pictures may be indexed next within the L0 reference picture list. The above L1 reference picture list may include pictures previous to the current picture in output order as reference pictures. For example, the subsequent pictures may be indexed first and the previous pictures may be indexed next within the L1 reference picture list. Here, the output order may correspond to a POC (picture order count) order.

[0122] That is, in the present embodiment, the distinction between L0 and L1 does not necessarily limit the output order of reference pictures. As described above, the L0 reference picture list may also include backward reference pictures, the L1 reference picture list may also include forward reference pictures, and it is also possible for the L0 reference picture list and the L1 reference picture list to include the same reference picture list. L0 and L1 may simply be used to distinguish between different lists or between reference pictures.

[0123] FIG. 6 and FIG. 7 illustrate examples of inter-prediction based video / image encoding methods to which embodiments of the present disclosure can be applied.

[0124] Referring to FIG. 6, the encoding device (200) can perform inter prediction for the current block (S600). The encoding device can determine or derive the inter prediction mode and motion information of the current block, and generate prediction samples of the current block. Here, the inter prediction mode determination, motion information derivation, and prediction sample generation procedures may be performed simultaneously, or one procedure may be performed before the other. For example, as illustrated in FIG. 7, the inter prediction unit (221) of the encoding device (200) may include a prediction mode determination unit (221a), a motion information derivation unit (221b), and a prediction sample derivation unit (221c), and the prediction mode determination unit (221a) may determine the prediction mode for the current block, the motion information derivation unit (221b) may derive motion information of the current block, and the prediction sample derivation unit (221c) may derive prediction samples of the current block.

[0125] For example, the inter prediction unit of the encoding device can search for a block similar to the current block within a certain area (search area) of reference pictures through motion estimation, and derive a reference block whose difference from the current block is minimal or below a certain standard. Based on this, a reference picture index indicating a reference picture where the reference block is located can be derived, and a motion vector can be derived based on the positional difference between the reference block and the current block. The encoding device can determine a mode to be applied to the current block among various prediction modes. The encoding device can compare RD costs for the various prediction modes and determine an optimal prediction mode for the current block.

[0126] For example, when skip mode or merge mode is applied to the current block, the encoding device can construct a merge candidate list and derive a reference block among the reference blocks indicated by the merge candidates included in the merge candidate list, the reference block having a sample difference from the current block, i.e., a difference in SAD or SATD, that is, a minimum or a predetermined standard. In this case, a merge candidate associated with the derived reference block is selected, and merge index information indicating the selected merge candidate can be generated and signaled to the decoding device. Motion information of the current block can be derived using motion information of the selected merge candidate.

[0127] As another example, when the (A)MVP mode is applied to the current block, the encoding device may configure an (A)MVP candidate list described below, and use the motion vector of a motion vector predictor (mvp) candidate selected from among motion vector predictor (mvp) candidates included in the (A)MVP candidate list as the motion vector predictor of the current block. In this case, for example, a motion vector pointing to a reference block derived by the above-described motion estimation may be used as the motion vector of the current block, and a motion vector predictor candidate having a motion vector with the smallest difference from the motion vector of the current block among the motion vector predictor candidates may be the selected motion vector predictor candidate. A Motion Vector Difference (MVD), which is the difference obtained by subtracting the motion vector predictor from the motion vector of the current block, may be derived. In this case, information about the MVD may be signaled to the decoding device. In addition, when the (A)MVP mode is applied, the value of the reference picture index may be configured with reference picture index information and signaled separately to the decoding device.

[0128] The encoding device can derive residual samples based on predicted samples (S610). The encoding device can derive residual samples by comparing the original samples of the current block with the predicted samples.

[0129] An encoding device can encode image information including prediction information and residual information (S620). The encoding device can output the encoded image information in the form of a bitstream. The prediction information is information about prediction and may include prediction mode information (e.g., skip flag, merge flag, or merge index, etc.) and / or motion information. The motion information may include candidate selection information (e.g., merge index, mvp flag, or mvp index), which is information for deriving a motion vector. In addition, the motion information may include information about the above-described MVD and / or reference picture index information. In addition, the motion information may include information indicating whether L0 prediction, L1 prediction, or bi-prediction is applied. The residual information is information about residual samples. The residual information may include information about quantized transform coefficients for the residual samples.

[0130] The output bitstream may be stored on a (digital) storage medium and transmitted to a decoding device, or may be transmitted to a decoding device via a network. Here, the storage medium may be a computer-readable storage medium capable of storing data non-temporarily.

[0131] Meanwhile, as described above, the encoding device can generate a reconstructed picture (including reconstructed samples and reconstructed blocks) based on reference samples and residual samples. This is to derive the same prediction result as that performed by the decoding device from the encoding device, thereby improving coding efficiency. Accordingly, the encoding device can store the reconstructed picture (or reconstructed samples, reconstructed blocks) in memory and use it as a reference picture for inter prediction. As described above, an in-loop filtering procedure, etc. can be further applied to the reconstructed picture.

[0132] FIG. 8 and FIG. 9 illustrate examples of inter-prediction based video / image decoding methods to which embodiments of the present disclosure can be applied.

[0133] A video / image decoding procedure based on inter prediction may roughly include, for example:

[0134] Referring to FIGS. 8 and 9, the decoding device (300) can perform an operation corresponding to the operation performed in the encoding device (200). The decoding device can perform a prediction on the current block based on the received prediction information and derive prediction samples.

[0135] Specifically, the decoding device can determine a prediction mode for the current block based on the received prediction information (S700). The prediction mode determination unit (332a) of the decoding device (300) can determine or derive which inter prediction mode is applied to the current block based on the prediction mode information in the prediction information.

[0136] For example, based on the merge flag, it can be determined whether the current block is subject to merge mode or (A)MVP mode. Alternatively, one of various inter prediction mode candidates can be selected based on the mode index. The inter prediction mode candidates can include skip mode, merge mode, and / or (A)MVP mode, or can include various inter prediction modes described below.

[0137] The decoding device can derive motion information of the current block based on the determined inter prediction mode (S710). For example, when skip mode or merge mode is applied to the current block, the motion information derivation unit (332b) of the decoding device (300) can construct a merge candidate list described below and select one merge candidate from among the merge candidates included in the merge candidate list. This selection can be performed based on the above-described selection information (merge index). Motion information of the current block can be derived using motion information of the selected merge candidate. Motion information of the selected merge candidate can be used as motion information of the current block.

[0138] As another example, when the (A)MVP mode is applied to the current block, the decoding device may construct an (A)MVP candidate list described below, and use the motion vector of an MVP candidate selected from among the MVP (motion vector predictor) candidates included in the (A)MVP candidate list as the MVP of the current block. This selection may be performed based on the selection information (mvp flag or mvp index) described above. In this case, the MVD of the current block may be derived based on information about the MVD, and the motion vector of the current block may be derived based on the MVP and MVD of the current block. In addition, the reference picture index of the current block may be derived based on reference picture index information. A picture indicated by a reference picture index within the reference picture list for the current block may be derived as a reference picture referenced for inter prediction of the current block.

[0139] Meanwhile, the motion information of the current block may be derived without constructing a candidate list, in which case the motion information of the current block may be derived according to the procedure initiated in the prediction mode. In this case, the candidate list construction described above may be omitted.

[0140] The decoding device can generate prediction samples for the current block based on the motion information of the current block (S720). In this case, the prediction sample derivation unit (332c) of the decoding device (300) can derive a reference picture based on the reference picture index of the current block, and derive prediction samples of the current block using samples of the reference block pointed to by the motion vector of the current block on the reference picture. In this case, as described below, a prediction sample filtering procedure may be further performed on all or part of the prediction samples of the current block, depending on the case.

[0141] In other words, the inter prediction unit (332) of the decoding device (300) may include a prediction mode determination unit (332a), a motion information derivation unit (332b), and a prediction sample derivation unit (332c), and may determine a prediction mode for the current block based on the prediction mode information received from the prediction mode determination unit (332a), derive motion information (motion vector and / or reference picture index, etc.) of the current block based on the information about motion information received from the motion information derivation unit (332b), and derive or generate prediction samples of the current block from the prediction sample derivation unit (332c).

[0142] The decoding device derives residual samples for the current block based on the received residual information (S730). The decoding device (300) generates restoration samples for the current block based on the prediction samples and residual samples (S740), and can generate a restoration picture based on these. As described above, in-loop filtering procedures, etc. may be further applied to the restoration picture.

[0143] FIG. 10 exemplarily illustrates an inter prediction procedure to which an embodiment of the present disclosure can be applied.

[0144] Referring to FIG. 10, the inter prediction procedure (S600) described above may include an inter prediction mode determination step, a motion information derivation step according to the determined prediction mode, and a prediction performance (prediction sample generation) step based on the derived motion information. The inter prediction procedure may be performed in an encoding device and a decoding device as described above. In this document, a coding device may include an encoding device and / or a decoding device.

[0145] Referring to FIG. 10, the coding device determines an inter prediction mode for a current block (S800). Various inter prediction modes may be used to predict the current block within a picture. For example, various modes may be used, such as merge mode, skip mode, MVP (Motion Vector Prediction) mode, affine mode, sub-block merge mode, MMVD (Merge with MVD) mode, and SMVD (Symmetric Motion Vector Difference) mode. Decoder side Motion Vector Refinement (DMVR) mode, Adaptive Motion Vector Resolution (AMVR) mode, Bi-prediction with CU-level Weight (BCW), Bi-Directional Optical Flow (BDOF), etc. may also be used as auxiliary modes.

[0146] In the present disclosure, the affine mode may be referred to as the affine motion prediction mode. In addition, the MVP mode may be referred to as the Advanced Motion Vector Prediction (AMVP) mode. In the present disclosure, motion information candidates derived from some modes and / or some modes may be included as one of the motion information-related candidates of other modes. For example, an HMVP candidate may be added as a merge candidate of the merge / skip mode, or may be added as a motion vector predictor candidate of the AMVP mode. When an HMVP candidate is used as a motion information candidate of the merge mode or the skip mode, the HMVP candidate may be referred to as an HMVP merge candidate.

[0147] Prediction mode information indicating the inter-prediction mode of the current block can be signaled from the encoding device to the decoding device. The prediction mode information can be included in the bitstream and received by the decoding device. The prediction mode information can include index information indicating one of multiple candidate modes. Alternatively, the inter-prediction mode can be indicated through hierarchical signaling of flag information.

[0148] In this case, the prediction mode information may include one or more flags. For example, a skip flag may be signaled to indicate whether skip mode is applied, a merge flag may be signaled to indicate whether merge mode is applied when skip mode is not applied, and MVP mode may be indicated to be applied when merge mode is not applied, or additional flags may be signaled for additional distinction. Affine mode may be signaled as an independent mode, or as a mode dependent on merge mode or MVP mode. For example, an affine mode may include an affine merge mode and an affine MVP mode.

[0149] The coding device can derive motion information for the current block (S810). The motion information can be derived based on the inter-prediction mode determined in the aforementioned step. The coding device can perform inter-prediction using the motion information of the current block. The encoding device can derive optimal motion information for the current block through a motion estimation procedure.

[0150] For example, an encoding device can search for a similar reference block with a high correlation in a fractional pixel unit within a predetermined search range within the reference picture using an original block within the original picture for the current block, and thereby derive motion information. The similarity of blocks can be derived based on the difference in phase-based sample values. For example, the similarity of blocks can be calculated based on the SAD between the current block (or a template of the current block) and the reference block (or a template of the reference block). In this case, motion information can be derived based on the reference block with the smallest SAD within the search range. The derived motion information can be encoded in various ways based on an inter prediction mode and transmitted to a decoding device in the form of a bitstream.

[0151] The decoding device can derive the motion information from the bitstream.

[0152] The coding device can perform inter prediction based on motion information for the current block to generate prediction samples (S820). The current block containing the prediction samples may be referred to as a prediction block.

[0153] Meanwhile, information indicating whether the above-described List0 (L0) prediction, List1 (L1) prediction, or bi-prediction is used for the current block (current coding unit) can be signaled. This information may be called motion prediction direction information, inter-prediction direction information, or inter-prediction indication information, and may be configured / encoded / signaled, for example, in the form of an inter_pred_idc syntax element. That is, the inter_pred_idc syntax element can indicate whether the above-described List0 (L0) prediction, List1 (L1) prediction, or bi-prediction is used for the current block (current coding unit). In this document, for the convenience of explanation, the inter-prediction type (L0 prediction, L1 prediction, or BI prediction) indicated by the inter_pred_idc syntax element may be indicated as motion prediction direction. L0 prediction may be represented as pred_L0, L1 prediction as pred_L1, and bi-prediction as pred_BI. For example, depending on the value of the inter_pred_idc syntax element, the prediction type can be indicated as in Table 1 below.

[0154] [Table 1]

[0155]

[0156] As described above, a picture may include one or more slices. A slice may have one of the following types: intra (I) slice, predictive (P) slice, and bi-predictive (B) slice. The slice type may be indicated based on slice type information. For blocks within an I slice, inter prediction is not used for prediction, and only intra prediction can be used. Of course, even in this case, the original sample values ​​can be coded and signaled without prediction. For blocks within a P slice, either intra prediction or inter prediction can be used, and when inter prediction is used, only uni prediction can be used. On the other hand, for blocks within a B slice, either intra prediction or inter prediction can be used, and when inter prediction is used, up to bi prediction can be used.

[0157] L0 and L1 may include reference pictures encoded / decoded before the current picture. For example, L0 may include reference pictures that are before and / or after the current picture in POC order, and L1 may include reference pictures that are after and / or before the current picture in POC order. For example, L0 may be assigned a relatively lower reference picture index to reference pictures that are before the current picture in POC order, and L1 may be assigned a relatively lower reference picture index to reference pictures that are after the current picture in POC order. For B slices, bi-prediction may be applied, and in this case, either uni-directional bi-prediction or bi-directional bi-prediction may be applied. Bi-directional bi-prediction may be called true bi-prediction.

[0158] A predicted block for a current block can be derived based on motion information derived according to a prediction mode. The predicted block can include prediction samples (prediction sample array) of the current block. If the motion vector of the current block points to fractional sample units, an interpolation procedure can be performed, through which prediction samples of the current block can be derived based on reference samples of fractional sample units within a reference picture.

[0159] When affine inter prediction is applied to the current block, prediction samples can be generated based on sample / subblock unit MV. When bi-prediction is applied, prediction samples derived from L0 prediction (i.e., prediction using reference pictures in the reference picture list L0 and MVL0) and L1 prediction (i.e., prediction using reference pictures in the reference picture list L1 and MVL1) can be used as prediction samples of the current block through a weighted sum or weighted average (according to phase). As described above, when the reference pictures used for L0 prediction and the reference pictures used for L1 prediction are located in different temporal directions with respect to the current picture (i.e., bi-prediction and bidirectional prediction), this can be called true bi-prediction.

[0160] As described above, restoration samples and restoration pictures can be generated based on the derived prediction samples, and then procedures such as in-loop filtering can be performed.

[0161] Inter prediction can be performed using motion information of the current block. The encoding device can derive optimal motion information for the current block through a motion estimation procedure. For example, the encoding device can search for similar reference blocks with high correlation within a predetermined search range within the reference picture using the original block within the original picture for the current block, in fractional pixel units, and thereby derive motion information. The similarity between blocks can be derived based on the difference in phase-based sample values. For example, the similarity between blocks can be calculated based on the SAD between the current block (or a template of the current block) and the reference block (or a template of the reference block). In this case, motion information can be derived based on the reference block with the smallest SAD within the search range. The derived motion information can be signaled to the decoding device in various ways based on the inter prediction mode.

[0162] The MVP (Motion Vector Prediction) mode may also be referred to as the AMVP (advanced motion vector prediction) mode. When the MVP mode is applied, a motion vector predictor (mvp) candidate list can be generated using the motion vectors of the reconstructed spatial neighboring blocks and / or the motion vectors corresponding to the temporal neighboring blocks (or Col blocks). That is, the motion vectors of the reconstructed spatial neighboring blocks and / or the motion vectors corresponding to the temporal neighboring blocks can be used as motion vector predictor candidates. When paired prediction is applied, an mvp candidate list for deriving L0 motion information and an mvp candidate list for deriving L1 motion information can be generated and used separately.

[0163] The above-described prediction information (or information regarding prediction) may include selection information (e.g., MVP flag or MVP index) indicating an optimal motion vector predictor candidate selected from among the motion vector predictor candidates included in the list. In this case, the prediction unit of the decoding device may use the selection information to select a motion vector predictor of the current block from among the motion vector predictor candidates included in the motion vector candidate list.

[0164] The prediction unit of the encoding device can obtain a motion vector difference (MVD) between the motion vector of the current block and the motion vector predictor, and can encode and output it in the form of a bitstream. That is, the MVD can be obtained as a value obtained by subtracting the motion vector predictor from the motion vector of the current block. At this time, the prediction unit of the decoding device can obtain the motion vector difference included in the information about the prediction (e.g., motion information), and derive the motion vector of the current block by adding the motion vector difference and the motion vector predictor. The prediction unit of the decoding device can obtain or derive a reference picture index indicating a reference picture, etc. from the information about the prediction. For example, a motion vector predictor candidate list can be configured as follows:

[0165] - Search for spatial candidate blocks for motion vector prediction and insert them into the prediction candidate list.

[0166] - Check if the number of spatial candidate blocks is less than 2

[0167] - If the number of spatial candidate blocks is less than 2, search for temporal candidate blocks and insert them into the prediction candidate list.

[0168] - If no temporal candidate block is available, use zero motion vector.

[0169] - If the number of spatial candidate blocks is not less than 2, the construction of the motion vector predictor candidate list is terminated.

[0170] Meanwhile, when the MVP mode is applied, the reference picture index can be explicitly signaled. In this case, the reference picture index for L0 prediction (refidxL0) and the reference picture index for L1 prediction (refidxL1) can be signaled separately. For example, when the MVP mode is applied and bi-prediction (BI prediction) is applied, information about both refidxL0 and refidxL1 can be signaled.

[0171] When the MVP mode is applied, information about the MVD derived from the encoding device as described above can be signaled or encoded and transmitted to the decoding device. The information about the MVD can include, for example, information indicating x, y components for the MVD absolute value and sign. In this case, information indicating whether the MVD absolute value is greater than 0 and greater than 1, and the MVD remainder can be signaled in stages. For example, information indicating whether the MVD absolute value is greater than 1 can be signaled only when the value of the flag information indicating whether the MVD absolute value is greater than 0 is 1.

[0172] For example, information about an MVD can be encoded in an encoding device and signaled to a decoding device using the following syntax:

[0173] [Table 2]

[0174]

[0175] For example, MVD[compIdx] can be derived based on abs_mvd_greater0_flag[compIdx] * ( abs_mvd_minus2[compIdx] + 2 ) * ( 1 - 2 * mvd_sign_flag[compIdx]). Here, compIdx (or cpIdx) represents the index of each component and can have the value 0 or 1. A compIdx value of 0 can represent the x component, and a compIdx value of 1 can represent the y component. However, this is just an example, and values ​​for each component can be represented using a coordinate system other than the x, y coordinate system.

[0176] Meanwhile, MVD for L0 prediction (MVDL0) and MVD for L1 prediction (MVDL1) may be signaled separately, and information about MVD may include information about MVDL0 and / or information about MVDL1. For example, if MVP mode is applied to the current block and BI prediction is applied, information about MVDLO and information about MVDL1 may both be signaled. In addition, as described below, information about MVDL1 may be derived from information about MVDL0, or information about MVDL0 may be derived from information about MVDL1.

[0177] Meanwhile, when BI prediction is applied, symmetric MVD mode may be used considering coding efficiency. In this case, signaling of some of the motion information may be omitted. For example, when symmetric MVD mode is applied to the current block, information about refidxL0, information about refidxL1, and information about MVDL1 may not be signaled from the encoding device to the decoding device, but may be derived internally. For example, when MVP mode and BI prediction are applied to the current block, flag information indicating whether symmetric MVD mode is applied (e.g., symmetric MVD flag information or sym_mvd_flag syntax element) may be signaled, and when the value of the flag information is 1, the decoding device may determine that symmetric MVD mode is applied to the current block.

[0178] When the symmetric MVD mode is applied (i.e., when the value of the symmetric MVD flag information is 1), information about mvp_l0_flag, mvp_l1_flag, and MVDL0 may be explicitly signaled, and signaling of information about refidxL0, information about refidxL1, and information about MVDL1 may be omitted and derived internally as described above. For example, refidxL0 may be derived as an index pointing to a previous reference picture that is closest to the current picture in POC order within reference picture list 0 (which may be referred to as list 0 or L0). refidxL1 may be derived as an index pointing to a subsequent reference picture that is closest to the current picture in POC order within reference picture list 1 (which may be referred to as list 1 or L1). Or, for example, both refidxL0 and refidxL1 may be derived as 0. Or, for example, the refidxL0 and refidxL1 may be derived as the minimum indices having the same POC difference in relation to the current picture. As a specific example, when [POC of the current picture] - [POC of the first reference picture indicated by refidxL0] is referred to as the first POC difference, and [POC of the second reference picture indicated by refidxL1] is referred to as the second POC difference, only when the first POC difference and the second POC difference are the same, the value of refidxL0 pointing to the first reference picture may be derived as the value of refidxL0 of the current block, and the value of refidxL1 pointing to the second reference picture may be derived as the value of refidxL1 of the current block.Also, for example, if there are multiple sets in which the first POC difference and the second POC difference are the same, refidxL0 and refidxL1 of the set with the minimum difference can be derived as refidxL0 and refidxL1 of the current block.

[0179] MVDL1 can be derived from -MVDL0. For example, the final MV for the current block can be derived as shown in Equation 1 below.

[0180] [Formula 1]

[0181]

[0182] As described above, the SMVD mode can be used when there are a forward reference picture and a backward reference picture temporally relative to the current picture, i.e., when the True-Bi prediction condition is satisfied, and the closest reference picture in each direction can be used without signaling the reference picture index. In the disclosed embodiment, the forward reference picture may also be referred to as a forward reference picture or a previous reference picture, and the backward reference picture may also be referred to as a backward reference picture or a later reference picture.

[0183] Meanwhile, predictable motion can be achieved not only between reference pictures in a symmetrical relationship but also between reference pictures in an asymmetrical relationship. Therefore, in the disclosed embodiment, when applying the MVP mode, not only when using the previous and subsequent reference pictures based on the current picture, but also when using reference pictures in the same direction, it is possible to derive another MVD from one MVD. According to the disclosed embodiment, when pair prediction is applied, deriving another MVD from one MVD while using reference pictures in the same direction can be regarded as an operation included in the SMVD mode. Alternatively, the MVP mode that derives another MVD from one MVD can also be referred to as a derived MVD mode regardless of the relative direction between the reference pictures. In this case, among the derived MVD modes, a case in which reference pictures in different directions are used can be referred to as an SMVD mode. Regardless of the name used to designate a prediction mode, if the operations described below are performed in that prediction mode, it may be included in the scope of the SMVD mode or the derived MVD mode of the disclosed embodiment.

[0184] FIG. 11 is a flowchart illustrating a method for generating a prediction sample according to one embodiment, and FIGS. 12 and 13 are schematic diagrams illustrating a process for deriving an MVD according to one embodiment.

[0185] The method for generating a prediction sample of FIG. 11 may be included in a decoding method according to an embodiment or an encoding method according to an embodiment. When performing the decoding method according to an embodiment, the decoding device (300) described above may be used, and when performing the encoding method according to an embodiment, the encoding device (200) described above may be used. Therefore, the contents regarding the decoding device (300) and the contents regarding the encoding device (200) described above may be applied to performing the method according to the example of FIG. 11 without separate mention. In addition, the descriptions regarding FIGS. 4 to 10 described above and the descriptions regarding the inter prediction mode may be equally applied as long as they do not conflict with the embodiments described below.

[0186] According to the method of FIG. 11, an inter prediction mode for a current block can be derived (S1110), a reference picture set for the current block can be derived based on the derived inter prediction mode (S1120), MVP information and MVD information for the current block can be derived (S1130), and prediction samples for the current block can be generated (S1140).

[0187] When the example of FIG. 11 is applied to a decoding method, a step of acquiring image information from a bitstream may be performed first. The acquired image information may include prediction information and residual information, and the prediction information, as information regarding prediction, may include prediction mode information and / or motion information.

[0188] Prediction mode information may include information regarding the prediction mode applied to the current block, and the prediction mode applied to the current block may be derived from the prediction mode information. For example, the prediction mode information may include flag information or index information indicating the prediction mode applied to the current block.

[0189] The motion information may include candidate selection information, which is information for deriving a motion vector. In addition, the motion information may include information about the MVD described above and / or reference picture index information. Alternatively, the reference picture index information may not be included in the motion information but may be derived by the decoding device. In addition, the motion information may include information indicating whether L0 prediction, L1 prediction, or bi-prediction is applied. The residual information is information about residual samples. The residual information may include information about quantized transform coefficients for the residual samples.

[0190] The step of deriving an inter prediction mode (S1110) may include deriving a prediction mode applied to a current block based on acquired image information as a prediction mode using MVD, for example, a mode using derived MVD or an SMVD mode.

[0191] The step of deriving a reference picture set (S1120) may include deriving a reference picture set composed of multiple reference pictures for the current block.

[0192] The step (S1130) of deriving MVP information and MVD information for the current block may include deriving multiple MVPs and multiple MVDs corresponding to multiple reference pictures. The multiple MVPs and multiple MVDs may be explicitly signaled and included in the bitstream, or some may be derived from the decoding device.

[0193] The step of generating prediction samples for the current block (S1140) may include generating prediction samples for the current block based on a plurality of reference pictures and the plurality of motion vector differences.

[0194] Here, the step of deriving a reference picture set may include deriving a reference picture set composed of a plurality of reference pictures in temporally different directions with respect to the current block or a reference picture set composed of a plurality of reference pictures in the same direction.

[0195] In addition, deriving a plurality of MVDs may include deriving a first MVD among the plurality of MVDs from the image information, and deriving a second MVD among the plurality of MVDs from the first MVD.

[0196] When the example of FIG. 11 is applied to an encoding method, the step of deriving a prediction mode (S1110) may include determining a prediction mode to be applied to the current block.

[0197] The step of deriving a reference picture set (S1120) may include deriving a reference picture set composed of multiple reference pictures for the current block based on the determined prediction mode. The reference picture set may be composed of multiple reference pictures in the same temporal direction relative to the current block, or may be composed of multiple reference pictures in different temporal directions.

[0198] The step (1130) of deriving MVP information and MVD information may include deriving multiple MVPs and multiple MVDs corresponding to multiple reference pictures. For example, a motion vector (MV) may be derived (e.g., calculated) based on the current block and the reference block, and an MVD may be derived (e.g., calculated) based on the MV and MVP.

[0199] In addition, the encoding method according to one embodiment may further include a step of encoding image information including information regarding a prediction mode and information regarding one MVD among the plurality of MVDs. That is, a first MVD among the plurality of MVDs may be included in the image information to be encoded, and a second MVD among the plurality of MVDs may be derived from the first MVD during a decoding process.

[0200] For convenience, FIG. 11 illustrates the process of deriving a prediction mode (S1110), deriving a reference picture set (S1120), and deriving MVP information and MVD information (S1130) in that order. However, the disclosed embodiment is not limited to this order. Some of these processes may be reversed in order, performed simultaneously, or may influence each other.

[0201] The above-described operation will be described in more detail. When the prediction mode applied to the current block is derived as the MVP mode or the prediction mode using MVD and bi-prediction is applied to the current block, when deriving a reference picture set, not only a reference picture set composed of reference pictures in different directions with respect to the current picture, but also a reference picture set composed of reference pictures in the same direction can be derived. In the disclosed embodiment, the reference picture set may mean a set composed of a plurality of reference pictures for use in bi-prediction. For example, it may mean a set composed of two reference pictures (a first reference picture and a second reference picture).

[0202] In deriving MVD information during the decoding process, information on the first MVD corresponding to the first reference picture, i.e., the first MVD information (MVDL0), can be derived from image information, and information on the second MVD corresponding to the second reference picture, i.e., the second MVD information (MVDL1), can be derived from the first MVD information. For example, the second MVD information can be derived by copying or mirroring the first MVD information depending on the relative direction (whether in the same direction or in both directions) of the first reference picture and the second reference picture with respect to the current picture.

[0203] In deriving MVD information during the encoding process, the first MVD information may be derived and signaled and encoded, and the second MVD information may be derived from the first MVD during the decoding process without signaling.

[0204] In this way, even when using reference pictures in the same direction for pair prediction, compression performance can be improved by deriving one MVD from another MVD without individually signaling / deriving the MVDs.

[0205] MVP (Motion Vector Predictor) information can be derived from motion information included in image information. For example, the MVP information can include first MVP information corresponding to a first reference picture and second MVP information corresponding to a second reference picture. A motion vector (MV) for a current block can be derived based on the MVP information and the MVD information, and the derived motion vector can include a first motion vector corresponding to the first reference picture and a second motion vector corresponding to the second reference picture. The first motion vector can be derived by adding the first MVP and the second MVD, and the second motion vector can be derived by adding the second MVP and the second MVD.

[0206] Here, the ordinal numbers 1 and 2 used when referring to reference pictures, MVPs, MVDs, motion vectors, etc. are only for distinguishing between multiple reference pictures, multiple MVPs, multiple MVDs, and multiple motion vectors, and the ordinal numbers do not limit the direction (backward, forward) of the reference pictures with respect to the current picture.

[0207] According to the above-described embodiment, when applying the SMVD mode, not only reference pictures of different directions but also reference pictures of the same direction can be used. That is, the reference picture set used in the SMVD mode may be composed of reference pictures of the same direction.

[0208] For example, when the current slice is a B slice, reference pictures for the SMVD mode can be determined before the decoding process, and when there are bidirectional reference pictures, a flag that determines whether the SMVD mode is applied can be transmitted per CU. When the SMVD mode is applied, as illustrated in FIG. 12, the first MVD (MvdL0) can be signaled and parsed, and the second MVD (MvdL1) can be derived by mirroring the first MVD such that MvdL1 = -MvdL0.

[0209] In addition, when using reference pictures in the same direction, as illustrated in Fig. 13, the first MVD (MvdL0) can be signaled and parsed, and the second MVD (MvdL1) can be derived by simply copying the first MVD such that MvdL1 = MvdL0.

[0210] FIG. 14 and FIG. 15 are diagrams showing examples of a method for inducing MVD according to one embodiment.

[0211] Referring to Fig. 14, when deriving the second MVD (MvdL1), a scale value derived based on the POC difference between the current picture and the first reference picture (L0 reference picture) and the POC difference between the current picture and the second reference picture (L1 reference picture) can be applied. In this case, a more accurate MVD can be derived by considering the distance between the current picture and the reference picture. Here, the distance between pictures can mean the temporal distance, i.e., the POC difference between pictures.

[0212] These examples do not impose any restrictions on the orientation of the first reference picture and the second reference picture, and can be applied in the same manner even when the first reference picture and the second reference picture have different orientations relative to the current picture.

[0213] Specifically, as illustrated in FIG. 15, when the distance between the current picture and the first reference picture (L0 reference picture) is td and the distance between the current picture and the second reference picture (L1 reference picture) is tb, the second MVD (MvdL1) can be calculated by scaling the first MVD (MvdL0) using the scale value of tb / td.

[0214] At this time, tb / td can be calculated by applying scaling and offset as in Equation 2 below to reduce truncation error.

[0215] [Formula 2]

[0216]

[0217] The Clip3() operation used in the above equation 2 can be defined as in equation 3 below.

[0218] [Formula 3]

[0219]

[0220] FIG. 16 and FIG. 17 are drawings showing another example of a method for deriving MVD according to one embodiment.

[0221] The method illustrated in Figures 16 and 17 is a more practical method that considers the distance between the current picture and the reference picture while reducing complexity. Specifically, a second MVD can be derived by applying bit shifting to the first MVD based on the distance between the current picture and the first reference picture and the distance between the current picture and the second reference picture.

[0222] The method illustrated in Fig. 16 can be applied when the first reference picture (reference picture of L0) is closer to the current picture than the second reference picture (reference picture of L1). Since the first reference picture is temporally closer to the current picture, it can be assumed that the reliability of L0 prediction is high, and the MVD obtained from the L0 prediction can be expanded and applied. For example, the second MVD (MvdL1) can be derived by applying a left shift of 1 to the x and y elements of the first MVD (MvdL0). That is, the first MVD can be expanded by a factor of 2 by shifting it to the left by 1 bit.

[0223] The method illustrated in Fig. 17 can be applied when the second reference picture (reference picture of L1) is closer to the current picture than the first reference picture (reference picture of L0). Since the second reference picture is temporally closer to the current picture, it can be assumed that the reliability of L0 prediction is relatively low, and the MVD obtained from L0 prediction can be reduced. For example, the second MVD (MvdL1) can be derived by applying a right shift of 1 to the x and y elements of the first MVD (MvdL0). In other words, the first MVD can be reduced by half by shifting it to the right by 1 bit.

[0224] The aforementioned method for deriving the second MVD (MvdL1) can be adaptively used depending on the direction and distance of the reference picture. For example, when True-Bi prediction is applied, i.e., when reference pictures in different directions are used, the second MVD can be derived using the mirroring method, and when only reference pictures in the same direction exist, the second MVD can be derived by applying shifting or scaling.

[0225] As another example, even when True-Bi prediction is applied, it is possible to derive the second MVD by applying shifting or scaling as well as mirroring based on the distance between the current picture and the first reference picture and the distance between the current picture and the second reference picture.

[0226] As another example, it is also possible to selectively apply shifts and scaling based on the distance between the current picture and the first reference picture, and the distance between the current picture and the second reference picture. When the distance between the current picture and the first reference picture (L0 reference picture) is T1, and the distance between the current picture and the second reference picture (L1 reference picture) is T2, if T1 < T2 and T2 is less than a threshold value (e.g., 4), a shift is applied to derive 'MvdL1 = MvdL0 << 1', and if it is equal to or greater than the threshold value, a scaling is applied to derive 'MvdL1 = MvdL0 * scale', and so on. In other words, the MVD can be derived by adaptively using scaling or '<<1' or '>>1' operations depending on the difference between the POCs.

[0227] According to one embodiment, accuracy can be improved by signaling information indicating which of the aforementioned MVD derivation methods is to be applied to the current block. Specifically, when the derived MVD is applied to the current block (e.g., when sym_mvd_flag is True), information indicating the MVD derivation method can be signaled. The information indicating the MVD derivation method can be flag or index information, and is represented as sym_mvd_derivation_idx in the present example. However, even if represented differently, any information indicating an MVD derivation method according to the disclosed embodiment can be included in the scope of the disclosed embodiment.

[0228] When N MVD derivation methods (N is a natural number) are applicable, information indicating the MVD derivation method can be used to determine which of the N methods to apply. For example, as shown in Table 3 below, when three methods of copy / mirroring, shift, and scale are applicable, the sym_mvd_derivation_idx value can be signaled as one of 0, 1, and 2. When reference pictures in different directions are used, index 0 can indicate mirroring, and when reference pictures in the same direction are used, index 0 can indicate copying. Of course, when N is 2, it can be indicated in the form of a flag (e.g., sym_mvd_derivation_flag) instead of an index.

[0229] [Table 3]

[0230]

[0231] Specifically, when there is a low-delay condition, sym_mvd_derivation_idx can be defined as follows. In this case, the low-delay condition can indicate a case where only forward-direction reference pictures are included in the reference pictures that the current picture can have.

[0232] - - When sym_mvd_derivation_idx is 0, MvdL1 = MvdL0 can be determined.

[0233] - - When sym_mvd_derivation_idx is 1, MvdL1 = MvdL0 << 1 or MvdL1 = MvdL0 >> 1 can be determined. The direction of each shift can be determined using the distance between the current picture and the L0 reference picture and the distance between the current picture and the L1 reference picture. When the distance to the L0 reference picture is close, MvdL1 = MvdL0 << 1 can be calculated, and when the distance to the L1 reference picture is closer, MvdL1 = MvdL0 >> 1 can be calculated.

[0234] - - When sym_mvd_derivaiton_idx is 2, MvdL1 can be determined as MvdL0 * scale. In this case, scale has a value greater than 0.

[0235] Also, when it is a True-Bi Condition, sym_mvd_derivation_idx can be defined as follows. In this case, the True-Bi Condition can indicate the case where each reference picture of the current picture is in a different direction in time order.

[0236] - - When sym_mvd_derivation_idx is 0, MvdL1 can be determined as -MvdL0.

[0237] - - When sym_mvd_derivation_idx is 1, MvdL1 = -MvdL0 << 1 or MvdL1 = -MvdL0 >> 1 can be determined. The direction of each shift can be determined using the distance between the current picture and the L0 reference picture and the distance between the current picture and the L1 reference picture. When the distance to the L0 reference picture is close, MvdL1 = -MvdL0 << 1 can be calculated, and when the distance to the L1 reference picture is closer, MvdL1 = -MvdL0 >> 1 can be calculated.

[0238] - - When sym_mvd_derivaiton_idx is 2, MvdL1 can be determined as MvdL0 * scale. At this time, scale indicates a negative value depending on the direction of each reference picture.

[0239] As described above, the SMVD mode is a bidirectional prediction mode that signals MVD in one direction and induces MVD in the other direction. When it is determined that bidirectional prediction is applied based on information (inter_pred_idc) indicating whether bidirectional prediction is applied as shown in Table 4 below, information (sym_mvd_flag) that determines whether SMVD is applied is signaled.

[0240] [Table 4]

[0241]

[0242]

[0243] Figure 18 is a diagram showing signaling bits used to signal information indicating whether AMVP mode, affine mode, and SMVD mode are applied.

[0244] According to Table 4 above, AMVP mode, affine mode, and SMVD mode have bit usage as shown in Fig. 18 depending on whether they are bi-predicted or uni-predicted. Specifically, AMVP mode (normal Inter mode) requires 3 bits for both uni-predicted and bi-predicted, affine mode requires 2 bits for bi-predicted and 3 bits for uni-predicted, and SMVD mode requires 3 bits.

[0245] Although SMVD mode already refers to the biprediction mode, Table 4 determines whether bipredation is applied based on inter_pred_idc before determining whether SMVD mode is applied. Therefore, in one embodiment, the signaling / parsing order can be changed as follows to eliminate semantic redundancy and reduce signaling bits.

[0246] FIG. 19 is a diagram illustrating signaling bits used to signal information indicating whether AMVP mode, affine mode, and SMVD mode are applied, according to one embodiment.

[0247] Referring to Fig. 19, information indicating whether affine mode is applied (inter_affine_flag) is first signaled / parsed, and then information indicating whether pair prediction is applied (inter_pred_idx) is signaled / parsed. Accordingly, the signaling bit for affine mode does not change. When information indicating whether affine mode is applied (inter_affine_flag) is False, information indicating whether SMVD is applied (sym_mvd_flag) can be signaled / parsed.

[0248] Therefore, information indicating whether SMVD is applied (sym_mvd_flag) can be signaled regardless of information indicating whether bi-prediction is performed (inter_pred_idc), and since 2 bits are used to determine whether SMVD is applied, 1 bit can be saved. When information indicating whether SMVD is applied (sym_mvd_flag) is False, bi-prediction or uni-prediction of AMVP can be determined according to information indicating whether bi-prediction is performed (inter_pred_idx). The bits used to indicate bi-prediction are the same, and the bits used to indicate uni-prediction can increase by 1 bit. Since bi-prediction in inter-picture prediction plays a role in increasing the compression ratio, bit savings of SMVD that performs bi-prediction can more effectively increase the compression performance. The signaling / parsing order described above can be expressed in the syntax of Table 5 below.

[0249] [Table 5]

[0250]

[0251]

[0252] Meanwhile, in SMVD mode, reference pictures are derived in slice units, MVP indices in each direction (first direction and second direction) are signaled in CU units, MVD in the first direction can be signaled, and MVD in the second direction can be derived. Here, the first direction and the second direction can be different or the same. According to one embodiment, in order to reduce signaling information for SMVD mode, it is also possible to derive MVP information in the second direction (MVP index of L1) in the decoding device without signaling it.

[0253] According to the typical AMVP candidate construction method, one MVP candidate can be selected from the two blocks to the left of the current block, and one MVP candidate can be selected from the three blocks above the current block. Furthermore, if the maximum number of MVPs has not been established, the MVP candidate list can be constructed in the following order: TMVP candidate, HMVP candidate, and Zero Vector candidate.

[0254] In inter mode, an MVP candidate list is constructed for a reference picture in each direction, and since the construction order of the candidates is the same, the first MVP candidate list (the MVP candidate list of L0) and the second MVP candidate list (the MVP candidate list of L1) may have similar characteristics. Therefore, according to one embodiment, the second MVP index (the L1 MVP index) can be derived from the first MVP index (the L0 MVP index). For example, when the MVP index of L0 is 0, the MVP index of L1 often also has a value of 0. Therefore, signaling for the MVP index of L1 is omitted, and it can be derived from the MVP index of L0.

[0255] According to one embodiment, when applying the SMVD mode, the second MVP index (MVP index of L1) can be induced to have the same value as the first MVP index (MVP index of L0) without signaling. For example, when the SMVD mode is applied, not only the information about the second MVD (MvdL1), but also the information about the second MVP (mvp_l1_flag) can be derived from the information about the first MVP (mvp_l0_flag) as expressed in Equation 4 below without being separately signaled. In Equation 4, the information about the MVP is expressed as a flag, but it is of course also possible to express it as an index depending on the number of MVP candidates.

[0256] [Formula 4]

[0257] mvp_l1_flag[x0][y0] = mvp_l0_flag[x0][y0]

[0258] The syntax reflecting this is described in Table 6 below.

[0259] [Table 6]

[0260]

[0261]

[0262] Alternatively, it is also possible to modify it as shown in Table 7 below. The syntax in Table 7 below explicitly determines when the SMVD mode is not applied, and signals information about the second MVP (mvp_l1_flag) based on the determination result.

[0263] [Table 7]

[0264]

[0265]

[0266] Alternatively, it is possible to use a fixed value (e.g., 0) as in Equation 5 below without signaling information about the second MVP (mvp_l1_flag).

[0267] [Formula 5]

[0268] mvp_l1_flag[ x0 ][ y0 ] = 0

[0269] The above is reflected in the syntax structure in Table 8 below.

[0270] [Table 8]

[0271]

[0272]

[0273] Alternatively, as another example for signaling MVP information used in SMVD mode, various combinations can be configured by pairing the MVP candidate lists of L0 and L1. FIG. 20 is a diagram illustrating an example of a paired MVP candidate list, according to one embodiment.

[0274] Referring to FIG. 20, when two candidate lists containing two MVP candidates are formed, each MVP candidate can be paired to ultimately generate four sets of MVP candidates. That is, by pairing the MVP candidates included in the first MVP candidate list (L0 MVP candidate list) with the MVP candidates included in the second MVP candidate list (L1 MVP candidate list), an MVP candidate set in which two MVP candidates are paired together can be generated. The paired MVP candidate list can be composed of paired MVP candidate sets.

[0275] Applying this example, when the SMVD mode is applied, the information about the first MVP (mvp_l0_flag) and the information about the second MVP (mvp_l1_flag) can be replaced with a single MVP index (mvp_idx) instead of being signaled separately, as described in Table 9 below.

[0276] [Table 9]

[0277]

[0278]

[0279] Meanwhile, in order to reduce the number of signaling bits for information related to the SMVD mode in the embodiment described above, an operation of changing the signaling / parsing order of some information has been described. When the signaling / parsing order is changed as in the syntax of Table 5 described above, the MVP index (mvp_idx) can be signaled / parsed separately from the first motion information (L0 motion information) and the second motion information (L1 motion information). The syntax structure related to this is described in Table 10 below.

[0280] [Table 10]

[0281]

[0282]

[0283] Referring to Table 10 above, when the affine mode is not applied, information indicating whether the SMVD mode is applied (sym_mvd_flag) is signaled / parsed, and when sym_mvd_flag is True, i.e., when the SMVD mode is applied, the MVP index (mvp_idx) can be signaled / parsed separately from the L0 motion information / L1 motion information. However, the syntax of Table 10 above is only an example, and if the condition that the signaling / parsing process of the MVP index (mvp_idx) and the L0 motion information / L1 motion information are separated is satisfied, it is of course possible for the MVP index (mvp_idx) to be signaled / parsed at another location within the coding_unit() syntax.

[0284] As another example, instead of constructing separate MVP candidate lists for L0 and L1 to signal a single MVP index for SMVD mode, an MVP candidate that includes motion vectors from both L0 and L1 can be constructed, similar to constructing a merge candidate list. In this case, the priorities within the lists can change when the motion vectors of adjacent blocks include bidirectional motion vectors, and motion vector candidates can be constructed similarly to AMVP mode and merge mode by utilizing neighboring blocks. The aforementioned method can be applied to signal the MVP index.

[0285] FIG. 21 is a diagram illustrating an example of a content streaming system to which an embodiment according to the present disclosure can be applied.

[0286] Referring to FIG. 21, a content streaming system to which the embodiment(s) of the present specification are applied may largely include an encoding server, a streaming server, a web server, a media storage, a user device, and a multimedia input device.

[0287] The encoding server compresses content input from multimedia input devices such as smartphones, cameras, and camcorders into digital data, generates a bitstream, and transmits it to the streaming server. Alternatively, if multimedia input devices such as smartphones, cameras, and camcorders directly generate bitstreams, the encoding server may be omitted.

[0288] The above bitstream can be generated by an encoding method or a bitstream generation method to which the embodiment(s) of the present specification are applied, and the streaming server can temporarily store the bitstream during the process of transmitting or receiving the bitstream.

[0289] The streaming server transmits multimedia data to a user device based on a user request via a web server, and the web server acts as an intermediary to inform the user of available services. When a user requests a desired service from the web server, the web server transmits the request to the streaming server, and the streaming server transmits the multimedia data to the user. At this time, the content streaming system may include a separate control server, in which case the control server controls commands / responses between each device within the content streaming system.

[0290] The streaming server can receive content from a media repository and / or an encoding server. For example, when receiving content from the encoding server, the content can be received in real time. In this case, to provide a smooth streaming service, the streaming server can store the bitstream for a certain period of time.

[0291] Examples of the user devices may include mobile phones, smart phones, laptop computers, digital broadcasting terminals, personal digital assistants (PDAs), portable multimedia players (PMPs), navigation devices, slate PCs, tablet PCs, ultrabooks, wearable devices (e.g., smartwatches, smart glasses, HMDs), digital TVs, desktop computers, digital signage, etc.

[0292] Each server within the above content streaming system can be operated as a distributed server, in which case data received from each server can be processed in a distributed manner.

[0293] The claims set forth in this specification may be combined in various ways. For example, the technical features of the method claims of this specification may be combined and implemented as a device, and the technical features of the device claims of this specification may be combined and implemented as a method. Furthermore, the technical features of the method claims and the technical features of the device claims of this specification may be combined and implemented as a device, and the technical features of the method claims and the technical features of the device claims of this specification may be combined and implemented as a method.

[0294] Embodiments according to the present disclosure can be used to encode / decode images.

Claims

1. A step of obtaining image information from a bitstream; A step of deriving a prediction mode for the current block based on the above image information; A step of deriving a reference picture set consisting of a first reference picture and a second reference picture for the current block; A step of deriving a plurality of motion vector differences including a first motion vector difference and a second motion vector difference corresponding to the first reference picture and the second reference picture, respectively; and A step of generating prediction samples for the current block based on the plurality of reference pictures and the plurality of motion vector differences; The step of deriving the above plurality of motion vector differences is: A method comprising: deriving the first motion vector difference from the image information; and deriving the second motion vector difference by shifting or scaling the first motion vector difference.

2. In paragraph 1, Deriving the above plurality of motion vector differences is: A method comprising left-shifting or right-shifting the first motion vector difference based on the distance between the current picture and the first reference picture and the distance between the current picture and the second reference picture.

3. In paragraph 1, Deriving the above plurality of motion vector differences is: A method comprising determining a scaling value based on a distance between a current picture and the first reference picture and a distance between the current picture and the second reference picture.

4. In paragraph 1, Deriving the above plurality of motion vector differences is: A method comprising determining whether to shift or scale the first motion vector difference based on a distance between the current picture and the first reference picture and a distance between the current picture and the second reference picture.

5. In paragraph 1, The above video information is, A method further comprising information regarding a method for deriving the second motion vector difference.

6. In paragraph 5, The above reference picture set is composed of multiple reference pictures in different temporal directions based on the current block, Deriving the above plurality of motion vector differences is: A method comprising mirroring, shifting, or scaling the first motion vector difference based on information regarding a method for deriving the second motion vector difference.

7. In paragraph 5, The above reference picture set is composed of multiple reference pictures in the same temporal direction based on the current block, Deriving the above plurality of motion vector differences is: A method comprising copying, shifting, or scaling the first motion vector difference based on information regarding a method for deriving the second motion vector difference.

8. Step of obtaining first information from a bitstream; A step of determining whether SMVD (Symmetric Motion Vector Difference) mode is applied to the current block based on the above first information; A step of obtaining second information from the bitstream based on the fact that the SMVD mode is not applied to the current block; and A method comprising: a step of determining whether bi-prediction is applied to the current block based on the second information; 9. Step of obtaining image information from a bitstream; A step of deriving a prediction mode for the current block based on the above image information; A step of deriving a plurality of motion vector predictors including a first motion vector predictor and a second motion vector predictor for the current block based on the above prediction mode; A step of deriving a plurality of motion vector differences including a first motion vector difference and a second motion vector difference for the current block; and A step of generating prediction samples for the current block based on the plurality of motion vector predictors and the plurality of motion vector differences; The step of deriving the above plurality of motion vector predictors is: The first motion vector predictor is derived from the image information, and the second motion vector predictor is derived from the first motion vector predictor, The step of deriving the above plurality of motion vector differences is: A method comprising: deriving the first motion vector difference from the image information, and deriving the second motion vector difference from the first motion vector difference.

10. A step for determining the prediction mode applied to the current block; A step of deriving a reference picture set consisting of a first reference picture and a second reference picture for the current block based on the above prediction mode; A step of deriving a first motion vector difference and a second motion vector difference corresponding to the first reference picture and the second reference picture, respectively; and A step of encoding image information including information about the prediction mode and information about the first motion vector difference; A method in which the second motion vector difference is derived by shifting or scaling the first motion vector difference.

11. In paragraph 10, The above video information is, A method further comprising information regarding a method for deriving the second motion vector difference.

12. In a computer-readable storage medium storing a bitstream generated by an encoding method, The above encoding method is, A step of determining the prediction mode applied to the current block; A step of deriving a reference picture set consisting of a first reference picture and a second reference picture for the current block based on the above prediction mode; A step of deriving a first motion vector difference and a second motion vector difference corresponding to the first reference picture and the second reference picture, respectively; and A step of encoding image information including information about the prediction mode and information about the first motion vector difference; A storage medium in which the second motion vector difference is derived by shifting or scaling the first motion vector difference.

13. A step of generating a bitstream regarding the above image information; and A step of transmitting data including the bitstream; The above bitstream is, A step of determining a prediction mode to be applied to a current block; a step of deriving a reference picture set consisting of a first reference picture and a second reference picture for the current block based on the prediction mode; a step of deriving a first motion vector difference and a second motion vector difference corresponding to the first reference picture and the second reference picture, respectively; and a step of encoding image information including information about the prediction mode and information about the first motion vector difference; generated by, A method in which the second motion vector difference is derived by shifting or scaling the first motion vector difference.

Images