Method, computer-readable storage medium, and method for transmitting image information

Abstract

A method, according to one embodiment, comprises the steps of: acquiring image information from a bitstream; deriving vector information for the current block on the basis of vector difference information and adaptive motion vector resolution (AMVR) information included in the image information; and generating a prediction block of the current block on the basis of the vector information, wherein the AMVR information includes AMVR index information indicating at least one AMVR candidate in an AMVR candidate set configured on the basis of first information and second information, the first information including information related to a prediction mode applied to the current block, and the second information being information different from the first information.

Description

Method, computer-readable storage medium and method for transmitting image information

[0001] The present disclosure relates to a method for decoding and encoding image information, a computer-readable storage medium for storing image information, and a method for transmitting image information.

[0002] Recently, the demand for high-resolution, high-quality images, such as HD (High Definition) images and UHD (Ultra High Definition) images, is increasing in various application fields, and accordingly, high-efficiency image compression technologies are being discussed.

[0003] There are various technologies such as inter prediction technology that predicts pixel values ​​included in the current picture from pictures before or after the current picture, intra prediction technology that predicts pixel values ​​included in the current picture using pixel information within the current picture, and entropy coding technology that assigns short codes to values ​​with high frequency of appearance and long codes to values ​​with low frequency of appearance, and these video compression technologies can be used to effectively compress and transmit or store video data.

[0004] Accordingly, a highly efficient image compression technology is required to effectively transmit, store, and play high-resolution, high-quality image information.

[0005] The present disclosure provides a method for determining a more appropriate resolution by reflecting not only the prediction mode but also the characteristics of an image in the process of determining the resolution of motion information, thereby saving signaling bits of motion information and improving compression performance.

[0006] According to one embodiment, a method includes the steps of: obtaining image information from a bitstream; deriving vector information for a current block based on vector difference information and Adaptive Motion Vector Resolution (AMVR) information included in the image information; and generating a prediction block of the current block based on the vector information; wherein the AMVR information includes AMVR index information indicating at least one AMVR candidate within an AMVR candidate set configured based on first information and second information, the first information includes information regarding a prediction mode applied to the current block, and the second information is information different from the first information.

[0007] According to one embodiment, a method includes the steps of: determining a prediction mode of a current block; deriving vector information of the current block based on the prediction mode; deriving a vector difference of the current block based on the vector information of the current block; and encoding image information including the vector difference of the current block and Adaptive Motion Vector Resolution (AMVR) information; wherein the AMVR information includes AMVR index information indicating at least one AMVR candidate within an AMVR candidate set configured based on first information and second information, the first information includes information regarding a prediction mode applied to the current block, and the second information is information different from the first information.

[0008] In one embodiment, a computer-readable storage medium for non-temporarily storing a bitstream generated by an encoding method, the encoding method comprising: determining a prediction mode of a current block; deriving vector information of the current block based on the prediction mode; deriving a vector difference of the current block based on the vector information of the current block; and encoding image information including the vector difference of the current block and Adaptive Motion Vector Resolution (AMVR) information; wherein the AMVR information includes AMVR index information indicating at least one AMVR candidate within an AMVR candidate set configured based on first information and second information, the first information includes information regarding a prediction mode applied to the current block, and the second information is information different from the first information.

[0009] According to one embodiment, a method includes the steps of: generating a bitstream relating to image information; and transmitting data including the bitstream; wherein the bitstream is generated by the steps of: determining a prediction mode of a current block; deriving vector information of the current block based on the prediction mode; deriving a vector differential of the current block based on the vector information of the current block; and encoding image information including the vector differential of the current block and Adaptive Motion Vector Resolution (AMVR) information, wherein the AMVR information includes AMVR index information indicating at least one AMVR candidate within an AMVR candidate set constructed based on first information and second information, wherein the first information includes information regarding a prediction mode applied to the current block, and the second information is information different from the first information.

[0010] According to one embodiment, in the process of determining the resolution of motion information, a more appropriate resolution can be determined by reflecting not only the prediction mode but also the characteristics of the image, thereby saving signaling bits of motion information and improving compression performance.

[0011] The effects that can be obtained from the present disclosure are not limited to the effects mentioned above, and other effects that are not mentioned will be clearly understood by a person having ordinary skill in the art to which the present disclosure pertains from the description below.

[0012] FIG. 1 illustrates a video / image coding system applicable to one embodiment.

[0013] FIG. 2 is a schematic block diagram of an encoding device to which one embodiment can be applied and in which encoding of a video / image signal is performed.

[0014] FIG. 3 is a schematic block diagram of a decoding device to which one embodiment can be applied and in which decoding of a video / image signal is performed.

[0015] Figure 4 illustrates an example of a video / image decoding method to which one embodiment can be applied.

[0016] Figure 5 illustrates an example of a video / image encoding method to which one embodiment may be applied.

[0017] Figures 6 and 7 illustrate examples of an inter prediction based video / image encoding method to which one embodiment can be applied.

[0018] Figures 8 and 9 illustrate examples of an inter prediction based video / image decoding method to which one embodiment can be applied.

[0019] Figure 10 illustrates an example of an inter prediction procedure to which one embodiment can be applied.

[0020] FIG. 11 is a diagram showing examples of blocks used as spatial merge candidates in an encoding method or a decoding method according to one embodiment.

[0021] Fig. 12 is a flowchart illustrating an example of a process for constructing a merge candidate list in an encoding method or a decoding method according to one embodiment.

[0022] Figure 13 is a diagram showing four movements expressed in the affine movement model.

[0023] Figure 14 is a diagram showing motion vectors used in affine motion prediction.

[0024] Figure 15 is a diagram for a case where the affine MVF is determined in 4*4 subblock units.

[0025] Figure 16 is a diagram showing the inheritance of control point motion vectors.

[0026] Figure 17 is a diagram showing an example of the location of a candidate for the configured affine merge mode.

[0027] Figure 18 is a diagram showing padding candidates for replacing zero vectors in the IBC list.

[0028] Figure 19 is a diagram showing an example of another IBC reference area at the current CU location.

[0029] Figure 20 is a diagram showing an example of a reference area when a CTU is coded.

[0030] Fig. 21 is a flowchart showing a process of parsing AMVR index information in a decoding method according to one embodiment.

[0031] Fig. 22 is a flowchart showing a process of parsing AMVR index information according to a prediction direction in a decoding method according to one embodiment.

[0032] FIG. 23 is a flowchart showing another example of a process for parsing AMVR index information in a decoding method according to one embodiment.

[0033] Figure 24 is a flowchart illustrating a decoding method according to one embodiment.

[0034] Fig. 25 is a flowchart illustrating an encoding method according to one embodiment.

[0035] FIG. 26 is a diagram illustrating an example of a content streaming system to which an embodiment according to the present disclosure can be applied.

[0036] The present disclosure may be subject to various modifications and embodiments. Therefore, specific embodiments are illustrated and described in detail in the drawings. However, this is not intended to limit the present disclosure to specific embodiments, but rather to encompass all modifications, equivalents, and alternatives falling within the spirit and technical scope of the present disclosure. Similar reference numerals have been used to designate similar components throughout the description of each drawing.

[0037] While terms such as "first" and "second" may be used to describe various components, these components should not be limited by these terms. These terms are used solely to distinguish one component from another. For example, without departing from the scope of the present disclosure, a first component could be referred to as a "second component," and similarly, a second component could also be referred to as a "first component." The term "and / or" includes a combination of multiple related items described herein or any of multiple related items described herein.

[0038] When a component is referred to as being "connected" or "connected" to another component, it should be understood that it may be directly connected or connected to that other component, but that there may be other components intervening. Conversely, when a component is referred to as being "directly connected" or "connected" to another component, it should be understood that there are no other components intervening.

[0039] The terminology used in this application is only used to describe specific embodiments and is not intended to limit the present disclosure. The singular expression includes the plural expression unless the context clearly indicates otherwise. In this application, it should be understood that the terms "comprise" or "have" indicate the presence of a feature, number, step, operation, component, part, or combination thereof described in the specification, but do not preclude the possibility of the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

[0040] The present disclosure relates to video / image coding. For example, the methods / embodiments disclosed herein may be applied to methods disclosed in various video / image coding standards.

[0041] This specification discloses various embodiments or examples relating to video / image coding, and unless otherwise stated, the disclosed embodiments or examples can be performed in combination with each other.

[0042] In this specification, video may refer to a set of images over time. A picture generally refers to a unit representing one image at a specific time point, and a slice / tile is a unit that constitutes part of a picture in coding. A slice / tile may include one or more coding tree units (CTUs). A picture may be composed of one or more slices / tiles. A tile is a rectangular area consisting of multiple CTUs within a specific tile column and a specific tile row of a picture. A tile column is a rectangular area of ​​CTUs that has a height equal to the height of the picture and a width specified by the syntax requirements of the picture parameter set. A tile row is a rectangular area of ​​CTUs that has a height specified by the picture parameter set and a width equal to the width of the picture. CTUs within a tile are arranged consecutively according to the CTU raster scan, while tiles within a picture may be arranged consecutively according to the tile raster scan. A slice may contain an integer number of complete tiles or an integer number of contiguous complete CTU rows within a picture, which may be exclusively contained within a single NAL unit. Meanwhile, a picture may be divided into two or more subpictures. A subpicture may be a rectangular region of one or more slices within a picture.

[0043] A pixel, or pel, can refer to the smallest unit that constitutes a picture (or image). Additionally, the term "sample" can be used as a counterpart to a pixel. A sample can generally represent a pixel or a pixel value, and can represent only the pixel / pixel value of the luminance component, or only the pixel / pixel value of the chrominance component.

[0044] A unit may represent a basic unit of image processing. A unit may include at least one of a specific region of a picture and information related to the region. One unit may include one luma block and two chroma (e.g., cb, cr) blocks. In some cases, the term "unit" may be used interchangeably with terms such as "block" or "area." In general, an MxN block may include a set (or array) of samples (or sample array) or transform coefficients consisting of M columns and N rows.

[0045] In this specification, “A or B” can mean “only A,” “only B,” or “both A and B.” In other words, “A or B” in this specification can be interpreted as “A and / or B.” For example, “A, B or C” in this specification can mean “only A,” “only B,” “only C,” or “any combination of A, B, and C.”

[0046] As used herein, a slash ( / ) or a comma can mean "and / or." For example, "A / B" can mean "A and / or B." Accordingly, "A / B" can mean "only A," "only B," or "both A and B." For example, "A, B, C" can mean "A, B, or C."

[0047] In this specification, “at least one of A and B” may mean “only A,” “only B,” or “both A and B.” Additionally, in this specification, the expressions “at least one of A or B” or “at least one of A and / or B” may be interpreted identically to “at least one of A and B.”

[0048] Additionally, in this specification, “at least one of A, B and C” can mean “only A,” “only B,” “only C,” or “any combination of A, B and C.” Additionally, “at least one of A, B or C” or “at least one of A, B and / or C” can mean “at least one of A, B and C.”

[0049] Additionally, parentheses used herein may mean "for example." Specifically, when "prediction (intra-prediction)" is indicated, "intra-prediction" may be suggested as an example of "prediction." In other words, "prediction" in this specification is not limited to "intra-prediction," and "intra-prediction" may be suggested as an example of "prediction." Furthermore, even when "prediction (i.e., intra-prediction)" is indicated, "intra-prediction" may be suggested as an example of "prediction."

[0050] Technical features individually described in a single drawing in this specification may be implemented individually or simultaneously.

[0051] FIG. 1 illustrates a video / image coding system applicable to one embodiment.

[0052] Referring to FIG. 1, a video / image coding system may include a first device (source device) and a second device (receiving device).

[0053] A source device can transmit encoded video / image information or data to a receiving device via a digital storage medium or a network in the form of a file or streaming. The source device may include a video source, an encoding device, and a transmitting device. The receiving device may include a receiving device, a decoding device, and a renderer. The encoding device may be referred to as a video / image encoding device, and the decoding device may be referred to as a video / image decoding device. The transmitter may be included in the encoding device. The receiver may be included in the decoding device. The renderer may include a display unit, and the display unit may be configured as a separate device or an external component.

[0054] A video source may obtain video / images through a process of capturing, synthesizing, or generating video / images. The video source may include a video / image capture device and / or a video / image generation device. The video / image capture device may include one or more cameras, a video / image archive containing previously captured video / images, etc. The video / image generation device may include a computer, a tablet, a smartphone, etc., and may (electronically) generate video / images. For example, a virtual video / image may be generated through a computer, etc., in which case the video / image capture process may be replaced by a process of generating related data.

[0055] An encoding device can encode input video / images. The encoding device can perform a series of procedures, such as prediction, transformation, and quantization, to improve compression and coding efficiency. The encoded data (encoded video / image information) can be output in the form of a bitstream.

[0056] The transmission unit can transmit encoded video / image information or data output in the form of a bitstream to the receiving unit of a receiving device via a digital storage medium or a network in the form of a file or streaming. The digital storage medium can include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. The storage medium can be a computer-readable storage medium and can store data non-transitory. The transmission unit can include an element for generating a media file via a predetermined file format and an element for transmission via a broadcasting / communication network. The receiving unit can receive / extract the bitstream and transmit it to a decoding device.

[0057] The decoding device can decode the video / image by performing a series of procedures such as inverse quantization, inverse transformation, and prediction corresponding to the operation of the encoding device.

[0058] The renderer can render decoded video / images. The rendered video / images can be displayed through the display unit.

[0059] FIG. 2 is a schematic block diagram of an encoding device to which one embodiment can be applied and in which encoding of a video / image signal is performed.

[0060] Referring to FIG. 2, the encoding device (200) may be configured to include an image partitioner (210), a prediction unit (predictor) 220, a residual processor (residual processor) 230, an entropy encoder (entropy encoder) 240, an adder (adder) 250, a filter (filter) 260, and a memory (memory) 270. The prediction unit (220) may include an inter prediction unit (221) and an intra prediction unit (222). The residual processor (230) may include a transformer (transformer) 232, a quantizer (quantizer) 233, a dequantizer (dequantizer) 234, and an inverse transformer (inverse transformer) 235. The residual processing unit (230) may further include a subtractor (231). The addition unit (250) may be called a reconstructor or a recontructed block generator. The image segmentation unit (210), the prediction unit (220), the residual processing unit (230), the entropy encoding unit (240), the addition unit (250), and the filtering unit (260) described above may be configured by one or more hardware components (e.g., an encoding device chipset or processor) according to an embodiment. In addition, the memory (270) may include a decoded picture buffer (DPB) and may be configured by a digital storage medium. The hardware component may further include the memory (270) as an internal / external component.

[0061] The encoding device (200) may include at least one processor and at least one memory. At least one processor may perform the operations of the encoding device (200) described below, and at least one memory may be electrically connected to at least one processor and store programs and various data necessary to perform the operations of the encoding device (200).

[0062] The image segmentation unit (210) can segment an input image (or picture, frame) input to the encoding device (200) into one or more processing units (PUs). For example, the processing units may be called coding units (CUs). In this case, the coding units may be recursively segmented from a coding tree unit (CTU) or a largest coding unit (LCU) according to a QTBTTT (Quad-Tree Binary-Tree Ternary-Tree) structure.

[0063] For example, a single coding unit may be split into multiple coding units with deeper depths based on a quad-tree structure, a binary tree structure, and / or a ternary structure. In this case, for example, the quad-tree structure may be applied first, and the binary tree structure and / or the ternary structure may be applied later. Alternatively, the binary tree structure may be applied before the quad-tree structure. The coding procedure according to the present specification may be performed based on the final coding unit that is no longer split. In this case, based on coding efficiency according to image characteristics, etc., the largest coding unit may be used directly as the final coding unit, or, if necessary, the coding unit may be recursively split into coding units of lower depths, and the coding unit with the optimal size may be used as the final coding unit. Here, the coding procedure may include procedures such as prediction, transformation, and restoration, which will be described later.

[0064] As another example, the processing unit may further include a prediction unit (PU) or a transform unit (TU). In this case, the prediction unit and the transform unit may each be split or partitioned from the final coding unit described above. The prediction unit may be a unit of sample prediction, and the transform unit may be a unit for deriving a transform coefficient and / or a unit for deriving a residual signal from a transform coefficient.

[0065] The term "unit" may be used interchangeably with terms such as "block" or "area" depending on the case. In general, an MxN block can represent a set of samples or transform coefficients consisting of M columns and N rows. A sample can generally represent a pixel or a pixel value, and can represent only a pixel / pixel value of a luminance component or only a pixel / pixel value of a chrominance component. A sample can be used as a term corresponding to a pixel or pel of a picture (or image).

[0066] The encoding device (200) can generate a residual signal (residual block, residual sample array) by subtracting a prediction signal (prediction block, prediction sample array) output from an inter prediction unit (221) or an intra prediction unit (222) from an input video signal (original block, original sample array), and the generated residual signal is transmitted to a conversion unit (232). In this case, a unit that subtracts a prediction signal (prediction block, prediction sample array) from an input video signal (original block, original sample array) within the encoding device (200) may be called a subtraction unit (231).

[0067] The prediction unit (220) can perform prediction on a block to be processed (hereinafter, referred to as a current block) to generate prediction samples, and can generate a predicted block including the prediction samples for the current block. The prediction unit (220) can determine whether intra prediction or inter prediction is applied to the current block or CU unit. The prediction unit (220) can generate various information regarding prediction, such as prediction mode information, as described later in the description of each prediction mode, and transmit the information to the entropy encoding unit (240). The information regarding prediction can be encoded by the entropy encoding unit (240) and output in the form of a bitstream.

[0068] The intra prediction unit (222) can predict the current block by referring to samples within the current picture. The referenced samples, i.e., the reference samples, may be located in the neighborhood of the current block or may be located a certain distance away from the current block depending on the prediction mode. In intra prediction, the prediction modes may include one or more non-directional modes and multiple directional modes. The non-directional mode may include at least one of the DC mode or the planar mode. The directional mode may include 33 directional modes or 65 directional modes depending on the degree of detail in the prediction direction. However, this is merely an example, and a greater or lesser number of directional modes may be used depending on the settings. The intra prediction unit (222) may also determine the prediction mode applied to the current block by using the prediction mode applied to the neighboring blocks.

[0069] The inter prediction unit (221) can derive a prediction block for the current block based on a reference block (reference sample array) specified by a motion vector on a reference picture. At this time, in order to reduce the amount of motion information transmitted in the inter prediction mode, the motion information can be predicted in units of blocks, sub-blocks, or samples based on the correlation of the motion information between the neighboring blocks and the current block. The motion information can include a motion vector and a reference picture index. The motion information can further include inter prediction direction information (L0 prediction, L1 prediction, Bi prediction, etc.). In the case of inter prediction, the neighboring block can include a spatial neighboring block existing in the current picture and a temporal neighboring block existing in the reference picture. The reference picture including the reference block and the reference picture including the temporal neighboring block may be the same or different. Temporal neighboring blocks may be called collocated reference blocks, collocated CUs (colCUs), etc., and reference pictures including temporal neighboring blocks may be called collocated pictures (colPic). For example, the inter prediction unit (221) may construct a motion information candidate list based on neighboring blocks, and generate information indicating which candidate is used to derive the motion vector and / or reference picture index of the current block. Inter prediction may be performed based on various prediction modes, and for example, in the case of skip mode and merge mode, the inter prediction unit (221) may use the motion information of neighboring blocks as the motion information of the current block. In the case of skip mode, unlike the merge mode, a residual signal may not be transmitted.In the motion vector prediction (MVP) mode, the motion vector of the surrounding blocks is used as a motion vector predictor, and the motion vector of the current block can be indicated by signaling the motion vector difference.

[0070] The prediction unit (220) can generate a prediction signal based on various prediction methods described below. For example, the prediction unit can apply intra prediction or inter prediction for prediction of a single block, and can also apply intra prediction and inter prediction simultaneously. This can be called combined inter and intra prediction (CIIP) mode. In addition, the prediction unit can be based on an intra block copy (IBC) prediction mode or a palette mode for prediction of a block. The IBC prediction mode or palette mode can be used for content image / video coding such as games, such as screen content coding (SCC). IBC basically performs prediction within the current picture, but can be performed similarly to inter prediction in that it derives a reference block within the current picture. That is, IBC can utilize at least one of the inter prediction techniques described herein. Palette mode can be viewed as an example of intra coding or intra prediction. When the palette mode is applied, sample values ​​within a picture can be signaled based on information about the palette table and palette index. The prediction signal generated through the prediction unit (220) can be used to generate a restoration signal or a residual signal.

[0071] The transform unit (232) can apply a transform technique to the residual signal to generate transform coefficients. For example, the transform technique can include at least one of a Discrete Cosine Transform (DCT), a Discrete Sine Transform (DST), a Karhunen-Loeve Transform (KLT), a Graph-Based Transform (GBT), or a Conditionally Non-linear Transform (CNT). Here, GBT refers to a transform obtained from a graph when the relationship information between pixels is expressed as a graph. CNT refers to a transform obtained based on generating a prediction signal using all previously restored pixels. In addition, the transform process can be applied to a pixel block having a square size and the same size, or can be applied to a block of a non-square variable size.

[0072] The quantization unit (233) quantizes the transform coefficients and transmits them to the entropy encoding unit (240), and the entropy encoding unit (240) can encode the quantized signal (information about the quantized transform coefficients) and output it as a bitstream. The information about the quantized transform coefficients can be called residual information. The quantization unit (233) can rearrange the quantized transform coefficients in a block form into a one-dimensional vector form based on the coefficient scan order, and can also generate information about the quantized transform coefficients based on the quantized transform coefficients in the one-dimensional vector form.

[0073] The entropy encoding unit (240) can perform various encoding methods such as exponential Golomb, context-adaptive variable length coding (CAVLC), context-adaptive binary arithmetic coding (CABAC), etc. The entropy encoding unit (240) can also encode information necessary for video / image restoration (e.g., values ​​of syntax elements, etc.) together or separately from quantized transform coefficients.

[0074] Encoded information (e.g., encoded video / image information) can be transmitted or stored in the form of a bitstream in units of NAL (network abstraction layer) units. The video / image information may further include information on various parameter sets, such as an adaptation parameter set (APS), a picture parameter set (PPS), a sequence parameter set (SPS), or a video parameter set (VPS). In addition, the video / image information may further include general constraint information. In the present specification, information and / or syntax elements transmitted / signaled from an encoding device to a decoding device may be included in the video / image information. The video / image information may be encoded through the above-described encoding procedure and included in the bitstream. The bitstream may be transmitted via a network or stored in a digital storage medium. Here, the network may include a broadcasting network and / or a communication network, and the digital storage medium may include various storage media, such as a USB, SD, CD, DVD, Blu-ray, HDD, or SSD. A transmission unit (not shown) for transmitting a signal output from an entropy encoding unit (240) and / or a storage unit (not shown) for storing the signal may be configured as an internal / external element of the encoding device (200), or the transmission unit may be included in the entropy encoding unit (240).

[0075] The quantized transform coefficients output from the quantization unit (233) can be used to generate a prediction signal. For example, by applying inverse quantization and inverse transformation to the quantized transform coefficients through the inverse quantization unit (234) and the inverse transform unit (235), a residual signal (residual block or residual samples) can be reconstructed. The addition unit (250) can generate a reconstructed signal (reconstructed picture, reconstructed block, reconstructed sample array) by adding the reconstructed residual signal to the prediction signal output from the inter prediction unit (221) or the intra prediction unit (222). When there is no residual for the block to be processed, such as when skip mode is applied, the predicted block can be used as a reconstructed block. The addition unit (250) may be called a reconstructor or a reconstructed block generation unit. The generated restoration signal can be used for intra prediction of the next processing target block within the current picture, and can also be used for inter prediction of the next picture after filtering as described below. Meanwhile, LMCS (luma mapping with chroma scaling) may be applied during the picture encoding and / or restoration process.

[0076] The filtering unit (260) can improve subjective / objective picture quality by applying filtering to the restoration signal. For example, the filtering unit (260) can apply various filtering methods to the restoration picture to generate a modified restoration picture, and store the modified restoration picture in the memory (270), specifically, in the DPB of the memory (270). The various filtering methods can include deblocking filtering, sample adaptive offset, adaptive loop filter, bilateral filter, etc. The filtering unit (260) can generate various information regarding filtering and transmit it to the entropy encoding unit (240). The information regarding filtering can be encoded by the entropy encoding unit (240) and output in the form of a bitstream.

[0077] The modified restored picture transmitted to the memory (270) can be used as a reference picture in the inter prediction unit (221). Through this, when inter prediction is applied, the encoding device can avoid prediction mismatch between the encoding device (200) and the decoding device, and can also improve encoding efficiency.

[0078] The DPB of the memory (270) can store the modified restored picture to be used as a reference picture in the inter prediction unit (221). The memory (270) can store motion information of a block from which motion information in the current picture is derived (or encoded) and / or motion information of blocks in a picture that has already been restored. The stored motion information can be transferred to the inter prediction unit (221) to be used as motion information of a spatial neighboring block or motion information of a temporal neighboring block. The memory (270) can store restored samples of restored blocks in the current picture and transfer them to the intra prediction unit (222).

[0079] Image information output in the form of a bitstream from the encoding device (200) can be transmitted to the decoding device (300).

[0080] FIG. 3 is a schematic block diagram of a decoding device to which one embodiment can be applied and in which decoding of a video / image signal is performed.

[0081] Image information transmitted in the form of a bitstream from the encoding device (200) can be received by the decoding device (300).

[0082] Referring to FIG. 3, the decoding device (300) may be configured to include an entropy decoder (310), a residual processor (320), a predictor (330), an adder (340), a filter (350), and a memory (360). The predictor (330) may include an inter-prediction unit (332) and an intra-prediction unit (331). The residual processor (320) may include a dequantizer (321) and an inverse transformer (321).

[0083] The entropy decoding unit (310), residual processing unit (320), prediction unit (330), addition unit (340), and filtering unit (350) described above may be configured by a single hardware component (e.g., a decoding device chipset or processor) depending on the embodiment. In addition, the memory (360) may include a decoded picture buffer (DPB) and may be configured by a digital storage medium. The hardware component may further include the memory (360) as an internal / external component.

[0084] The decoding device (300) may include at least one processor and at least one memory. At least one processor may perform the operations of the encoding device (300) described below, and at least one memory may be electrically connected to at least one processor and store programs and various data necessary to perform the operations of the encoding device (300).

[0085] When a bitstream including video / image information is input, the decoding device (300) can restore the image corresponding to the process in which the video / image information is processed in the encoding device of FIG. 2. For example, the decoding device (300) can derive units / blocks based on block division related information obtained from the bitstream. The decoding device (300) can perform decoding using a processing unit applied in the encoding device. Accordingly, the processing unit of decoding may be a coding unit, and the coding unit may be divided from a coding tree unit or a maximum coding unit according to a quad tree structure, a binary tree structure, and / or a ternary tree structure. One or more transform units may be derived from the coding unit. Then, the restored image signal decoded and output by the decoding device (300) can be reproduced through a reproduction device.

[0086] The decoding device (300) can receive a signal output from the encoding device of FIG. 2 in the form of a bitstream, and the received signal can be decoded through the entropy decoding unit (310). For example, the entropy decoding unit (310) can parse the bitstream to derive information (e.g., video / image information) necessary for image restoration (or picture restoration). The video / image information may further include information on various parameter sets, such as an adaptation parameter set (APS), a picture parameter set (PPS), a sequence parameter set (SPS), or a video parameter set (VPS). In addition, the video / image information may further include general constraint information. The decoding device can decode the picture further based on the information on the parameter set and / or the general constraint information. The signaling / received information and / or syntax elements described later in this specification can be decoded and obtained from the bitstream through the decoding procedure. For example, the entropy decoding unit (310) can decode information in a bitstream based on a coding method such as exponential Golomb coding, CAVLC, or CABAC, and output the values ​​of syntax elements required for image restoration and the quantized values ​​of transform coefficients for residuals. More specifically, the CABAC entropy decoding method receives a bin corresponding to each syntax element in the bitstream, determines a context model using information of the syntax element to be decoded and decoding information of the surrounding and decoding target blocks or information of symbols / bins decoded in the previous step, and predicts the occurrence probability of the bin according to the determined context model to perform arithmetic decoding of the bin to generate a symbol corresponding to the value of each syntax element.At this time, the CABAC entropy decoding method can update the context model using the information of the decoded symbol / bin for the context model of the next symbol / bin after determining the context model. Information regarding prediction among the information decoded by the entropy decoding unit (310) is provided to the prediction unit (inter prediction unit (332) and intra prediction unit (331)), and residual values ​​on which entropy decoding is performed by the entropy decoding unit (310), i.e., quantized transform coefficients and related parameter information, can be input to the residual processing unit (320). The residual processing unit (320) can derive a residual signal (residual block, residual samples, residual sample array). In addition, information regarding filtering among the information decoded by the entropy decoding unit (310) can be provided to the filtering unit (350). Meanwhile, a receiving unit (not shown) that receives a signal output from an encoding device may be further configured as an internal / external element of a decoding device (300), or the receiving unit may be a component of an entropy decoding unit (310).

[0087] Meanwhile, a decoding device according to the present specification may be called a video / video / picture decoding device, and the decoding device may be divided into an information decoding device (video / video / picture information decoding device) and a sample decoding device (video / video / picture sample decoding device). The information decoding device may include the entropy decoding unit (310), and the sample decoding device may include at least one of the inverse quantization unit (321), the inverse transformation unit (322), the addition unit (340), the filtering unit (350), the memory (360), the inter prediction unit (332), and the intra prediction unit (331).

[0088] The inverse quantization unit (321) can inverse quantize the quantized transform coefficients and output the transform coefficients. The inverse quantization unit (321) can rearrange the quantized transform coefficients into a two-dimensional block form. In this case, the rearrangement can be performed based on the coefficient scanning order performed in the encoding device. The inverse quantization unit (321) can perform inverse quantization on the quantized transform coefficients using quantization parameters (e.g., quantization step size information) and obtain transform coefficients.

[0089] In the inverse transform unit (322), the transform coefficients are inversely transformed to obtain a residual signal (residual block, residual sample array).

[0090] The prediction unit (320) can perform a prediction on the current block and generate a predicted block including prediction samples for the current block. The prediction unit (320) can determine whether intra-prediction or inter-prediction is applied to the current block based on the information regarding the prediction output from the entropy decoding unit (310), and can determine a specific intra / inter-prediction mode.

[0091] The prediction unit (320) can generate a prediction signal based on various prediction methods described below. For example, the prediction unit (320) can apply intra prediction or inter prediction for prediction of a single block, and can also apply intra prediction and inter prediction simultaneously. This can be called combined inter and intra prediction (CIIP) mode. In addition, the prediction unit can be based on an intra block copy (IBC) prediction mode or a palette mode for prediction of a block. The IBC prediction mode or palette mode can be used for content image / video coding such as games, such as screen content coding (SCC). IBC basically performs prediction within the current picture, but can be performed similarly to inter prediction in that it derives a reference block within the current picture. That is, IBC can utilize at least one of the inter prediction techniques described herein. The palette mode can be viewed as an example of intra coding or intra prediction. When palette mode is applied, information about the palette table and palette index may be signaled and included in the video / image information.

[0092] The intra prediction unit (331) can predict the current block by referring to samples within the current picture. The referenced samples may be located in the neighborhood of the current block, or may be located a certain distance away from the current block, depending on the prediction mode. In intra prediction, the prediction modes may include one or more non-directional modes and multiple directional modes. The intra prediction unit (331) may also determine the prediction mode applied to the current block by using the prediction mode applied to the neighboring blocks.

[0093] The inter prediction unit (332) can derive a prediction block for the current block based on a reference block (reference sample array) specified by a motion vector on a reference picture. At this time, in order to reduce the amount of motion information transmitted in the inter prediction mode, the motion information can be predicted in units of blocks, sub-blocks, or samples based on the correlation of the motion information between the neighboring blocks and the current block. The motion information can include a motion vector and a reference picture index. The motion information can further include inter prediction direction information (L0 prediction, L1 prediction, Bi prediction, etc.). In the case of inter prediction, the neighboring blocks can include spatial neighboring blocks existing in the current picture and temporal neighboring blocks existing in the reference picture. For example, the inter prediction unit (332) can construct a motion information candidate list based on the neighboring blocks, and derive the motion vector and / or reference picture index of the current block based on the received candidate selection information. Inter prediction can be performed based on various prediction modes, and information about the prediction can include information indicating an inter prediction mode for the current block.

[0094] The addition unit (340) can generate a restoration signal (restored picture, restoration block, restoration sample array) by adding the acquired residual signal to the prediction signal (prediction block, prediction sample array) output from the prediction unit (including the inter-prediction unit (332) and / or intra-prediction unit (331)). When there is no residual for the block to be processed, such as when skip mode is applied, the prediction block can be used as the restoration block.

[0095] The addition unit (340) may be referred to as a restoration unit or restoration block generation unit. The generated restoration signal may be used for intra prediction of the next processing target block within the current picture, may be output after filtering as described below, or may be used for inter prediction of the next picture. Meanwhile, LMCS (luma mapping with chroma scaling) may be applied during the picture decoding process.

[0096] The filtering unit (350) can improve subjective / objective image quality by applying filtering to the restored signal. For example, the filtering unit (350) can apply various filtering methods to the restored picture to generate a modified restored picture, and transmit the modified restored picture to the memory (360), specifically, to the DPB of the memory (360). The various filtering methods can include deblocking filtering, sample adaptive offset, adaptive loop filter, bilateral filter, etc.

[0097] The (modified) reconstructed picture stored in the DPB of the memory (360) can be used as a reference picture in the inter prediction unit (332). The memory (360) can store motion information of a block from which motion information in the current picture is derived (or decoded) and / or motion information of blocks in an already reconstructed picture. The stored motion information can be transmitted to the inter prediction unit (332) to be used as motion information of a spatial neighboring block or motion information of a temporal neighboring block. The memory (360) can store reconstructed samples of reconstructed blocks in the current picture and transmit them to the intra prediction unit (331).

[0098] In this specification, the embodiments described in the filtering unit (260), the inter prediction unit (221), and the intra prediction unit (222) of the encoding device (200) can be applied to the filtering unit (350), the inter prediction unit (332), and the intra prediction unit (331) of the decoding device (300) in the same or corresponding manner, respectively.

[0099] Figure 4 illustrates an example of a video / image decoding method to which one embodiment can be applied.

[0100] In image / video coding, the pictures that make up an image / video can be encoded / decoded according to a series of decoding orders. The picture order corresponding to the output order of the decoded pictures can be set differently from the decoding order, and based on this, not only forward prediction but also backward prediction can be performed during inter prediction.

[0101] In FIG. 4, S400 may be performed in the entropy decoding unit (310) of the aforementioned decoding device (300), S410 may be performed in the prediction unit (330), S420 may be performed in the residual processing unit (320), S430 may be performed in the addition unit (340), and S440 may be performed in the filtering unit (350). S400 may include a decoding procedure according to the present disclosure, S410 may include an inter / intra prediction procedure according to the present disclosure, S420 may include a residual processing procedure according to the present disclosure, S430 may include a block / picture restoration procedure according to the present disclosure, and S440 may include an in-loop filtering procedure according to the present disclosure.

[0102] Referring to FIG. 4, the decoding device obtains image / video information from a bitstream (S400), performs prediction based on the obtained image / video information (S410), and restores a picture through residual processing (S420, inverse quantization for quantized transform coefficients, inverse transformation) (S430).

[0103] A modified restored picture can be generated by applying an in-loop filtering procedure (S440) to a restored picture generated through the above restoration procedure, and the modified restored picture can be output as a decoded picture and can be stored in a buffer or memory of a decoding device to be used as a reference picture in an inter prediction procedure when decoding a next picture. In some cases, the in-loop filtering procedure can be omitted, in which case the restored picture can be output as a decoded picture and can be stored in a buffer or memory of a decoding device to be used as a reference picture in an inter prediction procedure when decoding a next picture.

[0104] The in-loop filtering procedure (S440) may include a deblocking filtering procedure, a sample adaptive offset (SAO) procedure, an adaptive loop filter (ALF) procedure, and / or a bi-lateral filter procedure, and some or all of them may be omitted. In addition, one or some of the deblocking filtering procedure, the sample adaptive offset (SAO) procedure, the adaptive loop filter (ALF) procedure, and the bi-lateral filter procedure may be sequentially applied, or all of them may be sequentially applied. For example, the SAO procedure may be performed after the deblocking filtering procedure is applied to the restored picture. Or, for example, the ALF procedure may be performed after the deblocking filtering procedure is applied to the restored picture. This may also be performed in an encoding device.

[0105] FIG. 5 illustrates an example of a video / image encoding method to which an embodiment of the present disclosure can be applied.

[0106] In FIG. 5, the prediction step (S500) may be performed in the prediction unit (220) of the encoding device (200) described above, residual processing (S510) based on the prediction result may be performed in the residual processing unit (230), and the step (S520) of encoding image information including prediction information and residual information may be performed in the entropy encoding unit (240). S500 may include an inter / intra prediction procedure according to the present disclosure, S510 may include a residual processing procedure according to the present disclosure, and S520 may include an encoding procedure according to the present disclosure.

[0107] The encoding procedure may optionally include a procedure for encoding information for picture restoration (e.g., prediction information, residual information, partitioning information, etc.) and outputting it in the form of a bitstream, as well as a procedure for generating a restored picture for the current picture and a procedure for applying in-loop filtering to the restored picture.

[0108] The encoding device (200) can derive (corrected) residual samples from the quantized transform coefficients through the inverse quantization unit (234) and the inverse transformation unit (235), and can generate a restored picture based on the prediction samples and (corrected) residual samples, which are outputs of S500. The restored picture generated in this way can be the same as the restored picture generated by the decoding device (300) described above. A modified restored picture can be generated through an in-loop filtering procedure for the restored picture, which can be stored in a buffer or memory, and, as in the case of the decoding device, can be used as a reference picture in the inter prediction procedure when encoding a subsequent picture.

[0109] As described above, some or all of the in-loop filtering procedure may be omitted in some cases. When the in-loop filtering procedure is performed, (in-loop) filtering-related information (parameters) may be encoded by the entropy encoding unit (240) and output in the form of a bitstream, and the decoding device (300) may perform the in-loop filtering procedure in the same manner as the encoding device based on the filtering-related information.

[0110] Through this in-loop filtering procedure, noise occurring during image / video coding, such as blocking artifacts and ringing artifacts, can be reduced, and subjective / objective image quality can be improved. In addition, by performing the in-loop filtering procedure in both the encoding device (200) and the decoding device (300), the same prediction results can be derived from the encoding device (200) and the decoding device (300), thereby increasing the reliability of picture coding and reducing the amount of data that must be transmitted for picture coding.

[0111] As described above, the picture restoration procedure can be performed not only in the decoding device (300) but also in the encoding device (200). A restoration block can be generated based on intra-prediction / inter-prediction for each block, and a restoration picture including the restoration blocks can be generated. If the current picture / slice / tile group is an I picture / slice / tile group, the blocks included in the current picture / slice / tile group can be restored based only on intra-prediction. On the other hand, if the current picture / slice / tile group is a P or B picture / slice / tile group, the blocks included in the current picture / slice / tile group can be restored based on intra-prediction or inter-prediction. In this case, inter-prediction may be applied to some blocks in the current picture / slice / tile group, and intra-prediction may be applied to some remaining blocks.

[0112] The color component of a picture may include a luma component and a chroma component, and unless explicitly limited in the present disclosure, embodiments according to the present disclosure may be applied to the luma component and the chroma component.

[0113] Meanwhile, when intra prediction is performed, the prediction unit (220, 330) of the encoding device (200) / decoding device (300) can derive a reference sample according to the intra prediction mode of the current block among the surrounding samples of the current block, and can generate a prediction sample of the current block based on the reference sample.

[0114] For example, (i) the prediction sample can be derived based on the average or interpolation of neighboring reference samples of the current block, and (ii) the prediction sample can be derived based on a reference sample existing in a specific (prediction) direction with respect to the prediction sample among the neighboring reference samples of the current block. The case of (i) can be called a non-directional mode or a non-angular mode, and the case of (ii) can be called a directional mode or an angular mode.

[0115] Additionally, linear interpolation intra prediction (LIP) may be applied to perform intra prediction on the current block by linearly interpolating prediction sample values ​​generated based on the intra prediction mode of the current block.

[0116] Additionally, a temporary prediction sample of the current block may be derived based on filtered peripheral reference samples, and a prediction sample of the current block may be derived by weighting at least one reference sample derived according to an intra prediction mode among existing peripheral reference samples, i.e., unfiltered peripheral reference samples, and the temporary prediction sample. Such prediction may be referred to as Position Dependent Intra Prediction Combination (PDPC).

[0117] In addition, intra prediction encoding can be performed by selecting a reference sample line with the highest prediction accuracy among the surrounding multiple reference sample lines of the current block, deriving a prediction sample using the reference sample located in the prediction direction of the selected line, and then instructing (signaling) the used reference sample line to the decoding device. This case can be referred to as multi-reference line intra prediction (MRL) or MRL-based intra prediction.

[0118] Additionally, the current block can be divided into vertical or horizontal subpartitions, and intra prediction can be performed based on the same intra prediction mode, while peripheral reference samples can be derived and utilized for each subpartition. In other words, in this case, the intra prediction mode for the current block is applied equally to the subpartitions, but peripheral reference samples can be derived and utilized for each subpartition, thereby improving intra prediction performance in some cases. This prediction method can be called intra subpartitions (ISP) or ISP-based intra prediction.

[0119] Additionally, if the prediction direction based on the prediction sample points between surrounding reference samples, i.e., if the prediction direction points to a fractional sample location, the value of the prediction sample can be derived through interpolation of multiple reference samples located around the prediction direction (around the fractional sample location).

[0120] Meanwhile, when inter prediction is applied, the prediction unit of the encoding device / decoding device can perform inter prediction on a block-by-block basis to derive prediction samples. Inter prediction can refer to a prediction derived in a manner dependent on data elements (e.g., sample values, or motion information) of pictures other than the current picture. When inter prediction is applied to the current block, a predicted block (prediction sample array) for the current block can be derived based on a reference block (reference sample array) specified by a motion vector on a reference picture pointed to by a reference picture index.

[0121] At this time, in order to reduce the amount of motion information transmitted in the inter prediction mode, the motion information of the current block can be predicted in units of blocks, sub-blocks, or samples based on the correlation of the motion information between the neighboring blocks and the current block. The motion information may include a motion vector and / or a reference picture index. The motion information may further include information on the inter prediction type (L0 prediction, L1 prediction, Bi prediction, etc.). When inter prediction is applied, the neighboring blocks may include spatial neighboring blocks existing in the current picture and temporal neighboring blocks existing in the reference picture.

[0122] The reference picture including the above reference block and the reference picture including the temporal neighboring block may be the same or different. The temporal neighboring block may be called a collocated reference block, a collocated CU (colCU), etc., and the reference picture including the temporal neighboring block may be called a collocated picture (colPic). For example, a motion information candidate list may be constructed based on the neighboring blocks of the current block, and a flag or index information indicating which candidate is selected (used) to derive the motion vector and / or reference picture index of the current block may be signaled.

[0123] Inter prediction can be performed based on various prediction modes. For example, in the case of skip mode and merge mode, the motion information of the current block may be the same as the motion information of the selected neighboring block. In the case of skip mode, unlike the merge mode, a residual signal may not be transmitted. In the case of motion vector prediction (MVP) mode, the motion vector of the selected neighboring block may be used as a motion vector predictor, and the motion vector difference may be signaled. In this case, the motion vector of the current block can be derived using the sum of the motion vector predictor and the motion vector difference.

[0124] The motion information may include L0 motion information and / or L1 motion information depending on the inter prediction type (L0 prediction, L1 prediction, Bi prediction, etc.). The L0 motion vector may be called L0 motion vector or MVL0, and the L1 motion vector may be called L1 motion vector or MVL1. Prediction based on an L0 motion vector may be called L0 prediction, prediction based on an L1 motion vector may be called L1 prediction, and prediction based on both the L0 motion vector and the L1 motion vector may be called bi(Bi) prediction. Here, the L0 motion vector may indicate a motion vector associated with an L0 reference picture list, and the L1 motion vector may indicate a motion vector associated with an L1 reference picture list. The L0 reference picture list may include pictures that are earlier in output order than the current picture as reference pictures, and the L1 reference picture list may include pictures that are later in output order than the current picture. Previous pictures may be called forward pictures, and subsequent pictures may be called backward pictures.

[0125] The L0 reference picture list may include pictures that are later in output order than the current picture as reference pictures. For example, in the L0 reference picture list, previous pictures may be indexed first and subsequent pictures may be indexed next. The L1 reference picture list may include pictures that are earlier in output order than the current picture as reference pictures. For example, in the L1 reference picture list, the subsequent pictures may be indexed first and subsequent pictures may be indexed next. Here, the output order may correspond to the POC (picture order count) order.

[0126] That is, in the present embodiment, the distinction between L0 and L1 does not necessarily limit the output order of reference pictures. As described above, the L0 reference picture list may also include backward reference pictures, the L1 reference picture list may also include forward reference pictures, and it is also possible for the L0 reference picture list and the L1 reference picture list to include the same reference picture list. L0 and L1 may simply be used to distinguish between different lists or between reference pictures.

[0127] Figures 6 and 7 illustrate examples of an inter prediction based video / image encoding method to which one embodiment can be applied.

[0128] Referring to FIG. 6, the encoding device (200) can perform inter prediction for the current block (S600). The encoding device can determine or derive the inter prediction mode and motion information of the current block, and generate prediction samples of the current block. Here, the inter prediction mode determination, motion information derivation, and prediction sample generation procedures may be performed simultaneously, or one procedure may be performed before the other. For example, as illustrated in FIG. 7, the inter prediction unit (221) of the encoding device (200) may include a prediction mode determination unit (221a), a motion information derivation unit (221b), and a prediction sample derivation unit (221c), and the prediction mode determination unit (221a) may determine the prediction mode for the current block, the motion information derivation unit (221b) may derive motion information of the current block, and the prediction sample derivation unit (221c) may derive prediction samples of the current block.

[0129] For example, the inter prediction unit of the encoding device can search for a block similar to the current block within a certain area (search area) of reference pictures through motion estimation, and derive a reference block whose difference from the current block is minimal or below a certain standard. Based on this, a reference picture index indicating a reference picture where the reference block is located can be derived, and a motion vector can be derived based on the positional difference between the reference block and the current block. The encoding device can determine a mode to be applied to the current block among various prediction modes. The encoding device can compare RD costs for the various prediction modes and determine an optimal prediction mode for the current block.

[0130] For example, when the skip mode or merge mode is applied to the current block, the encoding device (200) may configure a merge candidate list and derive a reference block among the reference blocks indicated by the merge candidates included in the merge candidate list, wherein the difference between samples from the current block, i.e., the difference in SAD or SATD, is minimal or below a certain standard. In this case, a merge candidate associated with the derived reference block is selected, and merge index information indicating the selected merge candidate may be generated and signaled to the decoding device. Motion information of the current block may be derived using motion information of the selected merge candidate.

[0131] As another example, when the (A)MVP mode is applied to the current block, the encoding device (200) may configure an (A)MVP candidate list described below, and use the motion vector of a motion vector predictor (mvp) candidate selected from among motion vector predictor (mvp) candidates included in the (A)MVP candidate list as the motion vector predictor of the current block. In this case, for example, a motion vector pointing to a reference block derived by the above-described motion estimation may be used as the motion vector of the current block, and a motion vector predictor candidate having a motion vector with the smallest difference from the motion vector of the current block among the motion vector predictor candidates may be the selected motion vector predictor candidate. A Motion Vector Difference (MVD), which is the difference obtained by subtracting the motion vector predictor from the motion vector of the current block, may be derived. In this case, information about the MVD may be signaled to the decoding device. Additionally, when the (A)MVP mode is applied, the value of the reference picture index can be configured as reference picture index information and signaled separately to the decoding device.

[0132] The encoding device (200) can derive residual samples based on predicted samples (S610). The encoding device can derive residual samples by comparing the original samples of the current block with the predicted samples.

[0133] The encoding device (200) can encode image information including prediction information and residual information (S620). The encoding device can output the encoded image information in the form of a bitstream. The prediction information is information about prediction and may include prediction mode information (e.g., skip flag, merge flag, or merge index, etc.) and / or motion information. The motion information may include candidate selection information (e.g., merge index, mvp flag, or mvp index), which is information for deriving a motion vector. In addition, the motion information may include information about the above-described MVD and / or reference picture index information. In addition, the motion information may include information indicating whether L0 prediction, L1 prediction, or bi-prediction is applied. The residual information is information about residual samples. The residual information may include information about quantized transform coefficients for the residual samples.

[0134] The output bitstream may be stored in a (digital) storage medium and transmitted to a decoding device (300), or may be transmitted to a decoding device (300) via a network. Here, the storage medium may be a computer-readable storage medium capable of storing data non-temporarily.

[0135] Meanwhile, as described above, the encoding device (200) can generate a restored picture (including restored samples and restored blocks) based on reference samples and residual samples. This is to derive the same prediction result as that performed by the decoding device (300) from the encoding device, thereby increasing coding efficiency. Accordingly, the encoding device (200) can store the restored picture (or restored samples, restored blocks) in memory and utilize it as a reference picture for inter prediction. As described above, an in-loop filtering procedure, etc. can be further applied to the restored picture.

[0136] FIG. 8 and FIG. 9 illustrate examples of inter-prediction based video / image decoding methods to which embodiments of the present disclosure can be applied.

[0137] A video / image decoding procedure based on inter prediction may roughly include, for example:

[0138] Referring to FIGS. 8 and 9, the decoding device (300) can perform an operation corresponding to the operation performed in the encoding device (200). The decoding device can perform a prediction on the current block based on the received prediction information and derive prediction samples.

[0139] Specifically, the decoding device (300) can determine a prediction mode for the current block based on the received prediction information (S700). The prediction mode determination unit (332a) of the decoding device (300) can determine or derive which inter prediction mode is applied to the current block based on the prediction mode information in the prediction information.

[0140] For example, based on the merge flag, it can be determined whether the current block is subject to merge mode or (A)MVP mode. Alternatively, one of various inter prediction mode candidates can be selected based on the mode index. The inter prediction mode candidates can include skip mode, merge mode, and / or (A)MVP mode, or can include various inter prediction modes described below.

[0141] The decoding device (300) can derive motion information of the current block based on the determined inter prediction mode (S710). For example, when skip mode or merge mode is applied to the current block, the motion information derivation unit (332b) of the decoding device (300) can construct a merge candidate list described below and select one merge candidate from among the merge candidates included in the merge candidate list. This selection can be performed based on the above-described selection information (merge index). Motion information of the current block can be derived using motion information of the selected merge candidate. Motion information of the selected merge candidate can be used as motion information of the current block.

[0142] As another example, when the (A)MVP mode is applied to the current block, the decoding device (300) may construct an (A)MVP candidate list described below, and use the motion vector of an MVP candidate selected from among the MVP (motion vector predictor) candidates included in the (A)MVP candidate list as the MVP of the current block. This selection may be performed based on the selection information (mvp flag or mvp index) described above. In this case, the MVD of the current block may be derived based on information about the MVD, and the motion vector of the current block may be derived based on the MVP and MVD of the current block. In addition, the reference picture index of the current block may be derived based on reference picture index information. A picture indicated by a reference picture index within the reference picture list for the current block may be derived as a reference picture referenced for inter prediction of the current block.

[0143] Meanwhile, the motion information of the current block may be derived without constructing a candidate list, in which case the motion information of the current block may be derived according to the procedure initiated in the prediction mode. In this case, the candidate list construction described above may be omitted.

[0144] The decoding device (300) can generate prediction samples for the current block based on the motion information of the current block (S720). In this case, the prediction sample derivation unit (332c) of the decoding device (300) can derive a reference picture based on the reference picture index of the current block, and derive prediction samples of the current block using samples of a reference block pointed to by the motion vector of the current block on the reference picture. In this case, as described below, a prediction sample filtering procedure may be further performed on all or part of the prediction samples of the current block, depending on the case.

[0145] In other words, the inter prediction unit (332) of the decoding device (300) may include a prediction mode determination unit (332a), a motion information derivation unit (332b), and a prediction sample derivation unit (332c), and may determine a prediction mode for the current block based on the prediction mode information received from the prediction mode determination unit (332a), derive motion information (motion vector and / or reference picture index, etc.) of the current block based on the information about motion information received from the motion information derivation unit (332b), and derive or generate prediction samples of the current block from the prediction sample derivation unit (332c).

[0146] The decoding device (300) derives residual samples for the current block based on the received residual information (S730). The decoding device (300) generates restoration samples for the current block based on the prediction samples and residual samples (S740), and can generate a restoration picture based on these. As described above, in-loop filtering procedures, etc. may be further applied to the restoration picture.

[0147] FIG. 10 exemplarily illustrates an inter prediction procedure to which an embodiment of the present disclosure can be applied.

[0148] As described above, the inter prediction procedure (S600) may include an inter prediction mode determination step, a motion information derivation step according to the determined prediction mode, and a prediction performance (prediction sample generation) step based on the derived motion information. The inter prediction procedure may be performed in the encoding device (200) and the decoding device (300) as described above. In this document, the coding device may include an encoding device and / or a decoding device.

[0149] Referring to FIG. 10, the coding device determines an inter prediction mode for a current block (S800). Various inter prediction modes may be used to predict the current block within a picture. For example, various modes may be used, such as merge mode, skip mode, MVP (Motion Vector Prediction) mode, affine mode, sub-block merge mode, MMVD (Merge with MVD) mode, and SMVD (Symmetric Motion Vector Difference) mode. Decoder side Motion Vector Refinement (DMVR) mode, Adaptive Motion Vector Resolution (AMVR) mode, Bi-prediction with CU-level Weight (BCW), Bi-Directional Optical Flow (BDOF), etc. may also be used as auxiliary modes.

[0150] In the present disclosure, the affine mode may be referred to as the affine motion prediction mode. In addition, the MVP mode may be referred to as the Advanced Motion Vector Prediction (AMVP) mode. In the present disclosure, motion information candidates derived from some modes and / or some modes may be included as one of the motion information-related candidates of other modes. For example, an HMVP candidate may be added as a merge candidate of the merge / skip mode, or may be added as a motion vector predictor candidate of the AMVP mode. When an HMVP candidate is used as a motion information candidate of the merge mode or the skip mode, the HMVP candidate may be referred to as an HMVP merge candidate.

[0151] Prediction mode information indicating the inter-prediction mode of the current block can be signaled from the encoding device to the decoding device. The prediction mode information can be included in the bitstream and received by the decoding device. The prediction mode information can include index information indicating one of multiple candidate modes. Alternatively, the inter-prediction mode can be indicated through hierarchical signaling of flag information.

[0152] In this case, the prediction mode information may include one or more flags. For example, a skip flag may be signaled to indicate whether skip mode is applied, a merge flag may be signaled to indicate whether merge mode is applied when skip mode is not applied, and MVP mode may be indicated to be applied when merge mode is not applied, or additional flags may be signaled for additional distinction. Affine mode may be signaled as an independent mode, or as a mode dependent on merge mode or MVP mode. For example, an affine mode may include an affine merge mode and an affine MVP mode.

[0153] The coding device can derive motion information for the current block (S810). The motion information can be derived based on the inter-prediction mode determined in the aforementioned step. The coding device can perform inter-prediction using the motion information of the current block. The encoding device can derive optimal motion information for the current block through a motion estimation procedure.

[0154] For example, the encoding device (200) can search for a similar reference block with a high correlation in units of fractional pixels within a predetermined search range within the reference picture using an original block within the original picture for the current block, and thereby derive motion information. The similarity of blocks can be derived based on the difference in phase-based sample values. For example, the similarity of blocks can be calculated based on the SAD between the current block (or the template of the current block) and the reference block (or the template of the reference block). In this case, motion information can be derived based on the reference block with the smallest SAD within the search range. The derived motion information can be encoded in various ways based on an inter prediction mode and transmitted to a decoding device in the form of a bitstream.

[0155] The decoding device (300) can derive the motion information from the bitstream.

[0156] The coding device can perform inter prediction based on motion information for the current block to generate prediction samples (S820). The current block containing the prediction samples may be referred to as a prediction block.

[0157] Meanwhile, information indicating whether the above-described List0 (L0) prediction, List1 (L1) prediction, or bi-prediction is used for the current block (current coding unit) can be signaled. This information may be called motion prediction direction information, inter-prediction direction information, or inter-prediction indication information, and may be configured / encoded / signaled, for example, in the form of an inter_pred_idc syntax element. That is, the inter_pred_idc syntax element can indicate whether the above-described List0 (L0) prediction, List1 (L1) prediction, or bi-prediction is used for the current block (current coding unit). In this document, for the convenience of explanation, the inter-prediction type (L0 prediction, L1 prediction, or BI prediction) indicated by the inter_pred_idc syntax element may be indicated as motion prediction direction. L0 prediction may be represented as pred_L0, L1 prediction as pred_L1, and bi-prediction as pred_BI. For example, depending on the value of the inter_pred_idc syntax element, the prediction type can be indicated as in Table 1 below.

[0158] [Table 1]

[0159]

[0160] As described above, a picture may include one or more slices. A slice may have one of the following slice types: intra (I) slice, predictive (P) slice, and bi-predictive (B) slice. The slice type may be indicated based on slice type information. For blocks within an I slice, inter prediction is not used for prediction, and only intra prediction can be used. Of course, even in this case, the original sample values ​​can be coded and signaled without prediction. For blocks within a P slice, either intra prediction or inter prediction can be used, and when inter prediction is used, only uni prediction can be used. On the other hand, for blocks within a B slice, either intra prediction or inter prediction can be used, and when inter prediction is used, up to bi prediction can be used.

[0161] L0 and L1 may include reference pictures encoded / decoded before the current picture. For example, L0 may include reference pictures that are before and / or after the current picture in POC order, and L1 may include reference pictures that are after and / or before the current picture in POC order. For example, L0 may be assigned a relatively lower reference picture index to reference pictures that are before the current picture in POC order, and L1 may be assigned a relatively lower reference picture index to reference pictures that are after the current picture in POC order. For B slices, bi-prediction may be applied, and in this case, either uni-directional bi-prediction or bi-directional bi-prediction may be applied. Bi-directional bi-prediction may be called true bi-prediction.

[0162] For example, information about the inter-prediction mode of the current block can be coded and signaled at the CU (CU syntax) level, or implicitly determined based on conditions. In this case, some modes may be explicitly signaled, while others may be implicitly derived.

[0163] For example, the CU syntax can carry information about (inter)prediction mode, etc.

[0164] [Table 2]

[0165]

[0166]

[0167]

[0168]

[0169] Here, cu_skip_flag can indicate whether skip mode is applied to the current block (CU). If pred_mode_flag is 0, it indicates that the current coding unit is encoded in inter prediction mode. If pred_mode_flag is 1, it indicates that the current coding unit is encoded in intra prediction mode. If pred_mode_ibc_flag is 1, it indicates that the current coding unit is encoded in IBC prediction mode (Intra Block Copy prediction mode). If pred_mode_ibc_flag is 0, it indicates that the current coding unit is not encoded in IBC prediction mode. When pcm_flag[x0][y0] is 1, it indicates that the pcm_sample( ) syntax structure exists and the transform_tree( ) syntax structure does not exist in the coding unit containing the luma coding block at position (x0, y0). When pcm_flag[x0][y0] is 0, it indicates that the pcm_sample( ) syntax structure does not exist. That is, pcm_flag can indicate whether the pulse coding modulation (PCM) mode is applied to the current block. When the PCM mode is applied to the current block, prediction / transformation / quantization, etc. are not applied, and the values ​​of the original samples in the current block can be coded and signaled.

[0170] If intra_mip_flag[x0][y0] is 1, it indicates that the intra prediction type for the luma sample is Matrix-based Intra Prediction (MIP). If intra_mip_flag[x0][y0] is 0, it indicates that the intra prediction type for the luma sample is not MIP. In other words, intra_mip_flag can indicate whether the MIP prediction mode (type) is applied to the current block (its luma sample).

[0171] intra_chroma_pred_mode[x0][y0] indicates the intra prediction mode for the chroma samples of the current block. general_merge_flag[x0][y0] indicates whether the inter prediction parameters of the current coding unit are inferred from the surrounding inter-predicted partitions. That is, general_merge_flag can indicate that general merge is available, and when the value of general_merge_flag is 1, regular merge mode, MMVD mode, and merge subblock mode can be available. For example, when the value of general_merge_flag is 1, the merge data syntax can be parsed from the encoded video / image information (or bitstream), and the merge data syntax can be configured / coded to include information as shown in Table 3 below.

[0172] [Table 3]

[0173]

[0174] Here, if regular_merge_flag[x0][y0] is 1, it indicates that the regular merge mode is used to generate the inter prediction parameters of the current coding unit. That is, regular_merge_flag indicates whether the regular merge mode is applied to the current block. If mmvd_merge_flag[x0][y0] is 1, it indicates that the Merge Mode with Motion Vector Difference (MMVD) is used to generate the inter prediction parameters of the current coding unit. That is, mmvd_merge_flag indicates whether MMVD is applied to the current block.

[0175] mmvd_cand_flag[x0][y0] indicates whether the first candidate (0) or the second candidate (1) is selected in the merge candidate list, and the selected candidate is constructed using the motion vector difference (MVD) derived from mmvd_distance_idx[x0][y0] and mmvd_direction_idx[x0][y0]. mmvd_distance_idx[x0][y0] indicates the index used to derive MmvdDistance[x0][y0]. mmvd_direction_idx[x0][y0] indicates the index used to derive MmvdSign[x0][y0].

[0176] merge_subblock_flag[x0][y0] indicates whether subblock-based inter prediction parameters are used for the current coding unit. That is, merge_subblock_flag can indicate whether subblock merge mode (or affine merge mode) is applied to the current block. merge_subblock_idx[x0][y0] indicates the index of the merge candidate selected from the subblock-based merge candidate list.

[0177] ciip_flag[x0][y0] indicates whether Combined Inter-picture and Intra-picture Prediction (CIIP) is applied to the current coding unit.

[0178] merge_triangle_idx0[x0][y0] represents the index of the first selected merge candidate in the triangle shape-based motion compensation candidate list. merge_triangle_idx1[x0][y0] represents the index of the second selected merge candidate in the triangle shape-based motion compensation candidate list.

[0179] merge_idx[x0][y0] represents the index of the merge candidate selected from the merge candidate list.

[0180] Meanwhile, referring back to the CU syntax, mvp_l0_flag can indicate a candidate to be selected from the MVP candidate list 0 for deriving the MVP of the current block when the MVP mode is applied. mvp_l1_flag[x0][y0] has the same meaning as mvp_l0_flag, but in this case, l0 and list 0 are replaced with l1 and list 1, respectively.

[0181] inter_pred_idc[x0][y0] indicates whether list 0 (list0), list 1 (list1), or bi-prediction is used for the current coding unit.

[0182] When sym_mvd_flag[x0][y0] is 1, it indicates that the syntax elements ref_idx_l0[x0][y0], ref_idx_l1[x0][y0], and the mvd_coding(x0, y0, refList, cpIdx) syntax structure when refList is 1 do not exist. That is, sym_mvd_flag indicates whether symmetric MVD is used in mvd coding.

[0183] ref_idx_l0[ x0 ][ y0 ] specifies the list 0 reference picture index for the current coding unit. ref_idx_l1[ x0 ][ y0 ] has the same meaning as ref_idx_l0, except that l0, L0, and list 0 are replaced by l1, L1, and list 1, respectively.

[0184] If inter_affine_flag[ x0 ][ y0 ] is 1, it indicates that affine model-based motion compensation is used to generate prediction samples for the current coding unit when decoding a P slice or a B slice.

[0185] If cu_affine_type_flag[ x0 ][ y0 ] is 1, it indicates that the prediction sample is generated using the 6-parameter affine model-based motion compensation for the current coding unit when decoding a P slice or a B slice.

[0186] If cu_affine_type_flag[ x0 ][ y0 ] is 0, it indicates that the prediction samples are generated using a 4-parameter affine model-based motion compensation.

[0187] amvr_flag[ x0 ][ y0 ] specifies the precision of the motion vector difference. The array indices x0, y0 represent the position (x0, y0) of the upper left luma sample of the target coding block with respect to the upper left luma sample of the picture. If amvr_flag[ x0 ][ y0 ] is 0, it indicates that the precision of the motion vector difference is 1 / 4 of a luma sample. If amvr_flag[ x0 ][ y0 ] is 1, it indicates that the precision of the motion vector difference is additionally specified by amvr_precision_flag[ x0 ][ y0 ]. If amvr_precision_flag[ x0 ][ y0 ] is 0, the precision of the motion vector difference is one integer luma sample if inter_affine_flag[ x0 ][ y0 ] is 0, otherwise it is 1 / 16 luma sample. If amvr_precision_flag[ x0 ][ y0 ] is 1, the precision of the motion vector difference is 4 luma samples if inter_affine_flag[ x0 ][ y0 ] is 0, otherwise it is one integer luma sample. The array indices x0, y0 indicate the position (x0, y0) of the upper left luma sample of the target coding block relative to the upper left luma sample of the picture.

[0188] bcw_idx[ x0 ][ y0 ] specifies the weight index in bidirectional prediction using CU weights.

[0189] A predicted block for a current block can be derived based on motion information derived according to a prediction mode. The predicted block can include prediction samples (prediction sample array) of the current block. If the motion vector of the current block points to fractional sample units, an interpolation procedure can be performed, through which prediction samples of the current block can be derived based on reference samples of fractional sample units within a reference picture.

[0190] When affine inter prediction is applied to the current block, prediction samples can be generated based on sample / subblock unit MV. When bi-prediction is applied, prediction samples derived from L0 prediction (i.e., prediction using reference pictures in the reference picture list L0 and MVL0) and L1 prediction (i.e., prediction using reference pictures in the reference picture list L1 and MVL1) can be used as prediction samples of the current block through a weighted sum or weighted average (according to phase). As described above, when the reference pictures used for L0 prediction and the reference pictures used for L1 prediction are located in different temporal directions with respect to the current picture (i.e., bi-prediction and bidirectional prediction), this can be called true bi-prediction.

[0191] As described above, restoration samples and restoration pictures can be generated based on the derived prediction samples, and then procedures such as in-loop filtering can be performed.

[0192] Inter prediction can be performed using motion information of the current block. The encoding device can derive optimal motion information for the current block through a motion estimation procedure. For example, the encoding device can search for similar reference blocks with high correlation within a predetermined search range within the reference picture using the original block within the original picture for the current block, in fractional pixel units, and thereby derive motion information. The similarity between blocks can be derived based on the difference in phase-based sample values. For example, the similarity between blocks can be calculated based on the SAD between the current block (or a template of the current block) and the reference block (or a template of the reference block). In this case, motion information can be derived based on the reference block with the smallest SAD within the search range. The derived motion information can be signaled to the decoding device in various ways based on the inter prediction mode.

[0193] When merge mode is applied, the motion information of the current prediction block is not directly transmitted, but rather the motion information of the surrounding prediction blocks is used to derive the motion information of the current prediction block. Accordingly, the motion information of the current prediction block can be indicated by transmitting flag information indicating that merge mode is used and a merge index indicating which surrounding prediction block was used. The above merge mode may be referred to as regular merge mode.

[0194] In order to perform merge mode, the encoder must search for merge candidate blocks used to derive motion information of the current prediction block. For example, up to five merge candidate blocks may be used, but the present invention is not limited thereto. In addition, the maximum number of merge candidate blocks may be transmitted in a slice header or a tile group header, but the present invention is not limited thereto. After finding the merge candidate blocks, the encoder can generate a merge candidate list, and select the merge candidate block with the lowest cost among them as the final merge candidate block.

[0195] The present invention provides various embodiments for merge candidate blocks constituting the merge candidate list.

[0196] FIG. 11 is a diagram showing examples of blocks used as spatial merge candidates in an encoding method or a decoding method according to one embodiment, and FIG. 12 is a flowchart showing an example of a process for constructing a merge candidate list in an encoding method or a decoding method according to one embodiment.

[0197] The above merge candidate list may utilize, for example, five merge candidate blocks. For example, four spatial merge candidates and one temporal merge candidate may be utilized. As a specific example, in the case of spatial merge candidates, the blocks illustrated in FIG. 11 may be utilized as spatial merge candidates. Hereinafter, the spatial merge candidates or the spatial MVP candidates described below may be referred to as SMVPs, and the temporal merge candidates or the temporal MVP candidates described below may be referred to as TMVPs.

[0198] The merge candidate list for the current block above can be constructed based on the procedure illustrated in FIG. 12, for example.

[0199] A coding device (encoding device or decoding device) searches the spatial neighboring blocks of the current block and inserts the derived spatial merge candidates into a merge candidate list (S910). For example, the spatial neighboring blocks may include a lower left corner neighboring block, a left neighboring block, an upper right corner neighboring block, an upper neighboring block, and an upper left corner neighboring block of the current block. However, this is merely an example, and in addition to the above-described spatial neighboring blocks, additional neighboring blocks such as a right neighboring block, a lower neighboring block, and a lower right neighboring block may also be used as spatial neighboring blocks. The coding device may search the spatial neighboring blocks based on priorities to detect available blocks, and derive motion information of the detected blocks as the spatial merge candidates. For example, the coding device may search the five blocks illustrated in FIG. 11 in the order of A1, B1, B0, A0, and B2, and sequentially index the available candidates to form a merge candidate list.

[0200] The coding device searches for temporal neighboring blocks of the current block and inserts the derived temporal merge candidates into the merge candidate list (S920). The temporal neighboring blocks may be located on a reference picture that is a different picture from the current picture where the current block is located. The reference picture where the temporal neighboring blocks are located may be called a collocated picture or col picture. The temporal neighboring blocks may be searched in the order of the lower right corner neighboring blocks and the lower right center block of the co-located block for the current block on the col picture. Meanwhile, when motion data compression is applied, specific motion information can be stored as representative motion information for each storage unit in the col picture. In this case, there is no need to store motion information for all blocks within a certain storage unit, and thus the effect of motion data compression can be achieved. In this case, the predetermined storage unit may be predetermined, for example, as a 16x16 sample unit, or an 8x8 sample unit, or size information for the predetermined storage unit may be signaled from the encoding device to the decoding device.

[0201] When motion data compression is applied, the motion information of temporal neighboring blocks can be replaced with representative motion information of a certain storage unit where the temporal neighboring blocks are located. That is, in this case, from an implementation perspective, a temporal merge candidate can be derived based on the motion information of a prediction block that covers a position that is arithmetically shifted to the right by a certain value based on the coordinates of the temporal neighboring block (upper left sample position), rather than a prediction block located at the coordinates of the temporal neighboring block.

[0202] For example, if the schedule storage unit is 2 n x2 nIn the case of sample units, if the coordinates of the temporal surrounding block are (xTnb, yTnb), then the modified position is ((xTnb>>n)<<n), (yTnb> >n)< <n))에 위치하는 예측 블록의 움직임 정보가 시간적 머지 후보를 위하여 사용될 수 있다. 구체적으로 상기 일정 저장 단위가 16x16 샘플 단위인 경우, 시간적 주변 블록의 좌표가 (xTnb, yTnb)라 하면, 수정된 위치인 ((xTnb> >4)<<4), (yTnb>>4)<<4)) motion information of the prediction block located at can be used for the temporal merge candidate. Alternatively, if the unit of storage is an 8x8 sample unit and the coordinates of the temporal surrounding block are (xTnb, yTnb), the motion information of the prediction block located at the modified position ((xTnb>>3)<<3), (yTnb>>3)<<3)) can be used for the temporal merge candidate.

[0203] The encoding device can determine whether the current number of merge candidates is less than the maximum number of merge candidates (S930). The maximum number of merge candidates may be predefined or signaled from the encoding device (200) to the decoding device (300). For example, the encoding device (200) may generate information regarding the maximum number of merge candidates, encode the information, and transmit it to the decoding device (300) in the form of a bitstream. If the maximum number of merge candidates is filled, the subsequent candidate addition process may not be performed.

[0204] If the number of current merge candidates is smaller than the maximum number of merge candidates as a result of the verification, the coding device inserts an additional merge candidate into the merge candidate list (S940). The additional merge candidate may include, for example, at least one of a history-based merge candidate(s), a pair-wise average merge candidate(s), an ATMVP, a combined bi-predictive merge candidate (when the slice / tile group type of the current slice / tile group is type B), and / or a zero-vector merge candidate.

[0205] If the number of current merge candidates is not less than the maximum number of merge candidates as a result of the verification, the coding device can terminate the construction of the merge candidate list. In this case, the encoding device (200) can select an optimal merge candidate among the merge candidates constituting the merge candidate list based on RD (rate-distortion) cost, and can signal selection information (e.g., merge index) indicating the selected merge candidate to the decoding device (300). The decoding device (300) can select an optimal merge candidate based on the merge candidate list and the selection information.

[0206] As described above, the motion information of the selected merge candidate can be used as the motion information of the current block, and prediction samples of the current block can be derived based on the motion information of the current block. The encoding device (200) can derive residual samples of the current block based on the prediction samples, and signal residual information about the residual samples to the decoding device (300). As described above, the decoding device (300) can generate restoration samples based on the residual samples and the prediction samples derived based on the residual information, and generate a restoration picture based on the same.

[0207] When skip mode is applied, the motion information of the current block can be derived in the same manner as when merge mode is applied. However, when skip mode is applied, the residual signal for the corresponding block is omitted, and the predicted samples can be directly used as restored samples.

[0208] The MVP (Motion Vector Prediction) mode may also be referred to as the AMVP (advanced motion vector prediction) mode. When the MVP mode is applied, a motion vector predictor (mvp) candidate list can be generated using the motion vectors of the reconstructed spatial neighboring blocks and / or the motion vectors corresponding to the temporal neighboring blocks (or Col blocks). That is, the motion vectors of the reconstructed spatial neighboring blocks and / or the motion vectors corresponding to the temporal neighboring blocks can be used as motion vector predictor candidates. When paired prediction is applied, an mvp candidate list for deriving L0 motion information and an mvp candidate list for deriving L1 motion information can be generated and used separately.

[0209] The above-described prediction information (or information regarding prediction) may include selection information (e.g., MVP flag or MVP index) indicating an optimal motion vector predictor candidate selected from among the motion vector predictor candidates included in the list. At this time, the prediction unit (330) of the decoding device (300) may use the selection information to select a motion vector predictor of the current block from among the motion vector predictor candidates included in the motion vector candidate list.

[0210] The prediction unit (220) of the encoding device (200) can obtain a motion vector difference (MVD) between the motion vector of the current block and the motion vector predictor, and can encode and output it in the form of a bitstream. That is, the motion vector difference can be obtained as a value obtained by subtracting the motion vector predictor from the motion vector of the current block. At this time, the prediction unit (330) of the decoding device (300) can obtain the motion vector difference included in the information regarding the prediction (e.g., motion information), and derive the motion vector of the current block through the addition of the motion vector difference and the motion vector predictor. The prediction unit (330) of the decoding device (300) can obtain or derive a reference picture index indicating a reference picture, etc. from the information regarding the prediction. For example, a motion vector predictor candidate list can be configured as follows:

[0211] Search for spatial candidate blocks for motion vector prediction and insert them into the prediction candidate list.

[0212] Check if the number of spatial candidate blocks is less than 2

[0213] If the number of spatial candidate blocks is less than 2, search for temporal candidate blocks and insert them into the prediction candidate list.

[0214] If no temporal candidate block is available, use zero motion vector.

[0215] If the number of spatial candidate blocks is not less than 2, the construction of the motion vector predictor candidate list is terminated.

[0216] Meanwhile, when the MVP mode is applied, the reference picture index can be explicitly signaled. In this case, the reference picture index for L0 prediction (refidxL0) and the reference picture index for L1 prediction (refidxL1) can be signaled separately. For example, when the MVP mode is applied and bi-prediction (BI prediction) is applied, information about both refidxL0 and refidxL1 can be signaled.

[0217] When the MVP mode is applied, information about the MVD derived from the encoding device (200) as described above may be signaled or encoded and transmitted to the decoding device (300). The information about the MVD may include, for example, information indicating x and y components for the MVD absolute value and sign. In this case, information indicating whether the MVD absolute value is greater than 0 and greater than 1, and the MVD remainder may be signaled in stages. For example, information indicating whether the MVD absolute value is greater than 1 may be signaled only when the value of the flag information indicating whether the MVD absolute value is greater than 0 is 1.

[0218] For example, information about an MVD can be encoded in an encoding device and signaled to a decoding device using the following syntax:

[0219] [Table 4]

[0220]

[0221] For example, MVD[compIdx] can be derived based on abs_mvd_greater0_flag[compIdx] * ( abs_mvd_minus2[compIdx] + 2 ) * ( 1 2 * mvd_sign_flag[compIdx]). Here, compIdx (or cpIdx) represents the index of each component and can have the value 0 or 1. A compIdx value of 0 can represent the x component, and a compIdx value of 1 can represent the y component. However, this is just an example, and values ​​for each component can be represented using a coordinate system other than the x, y coordinate system.

[0222] Meanwhile, MVD for L0 prediction (MVDL0) and MVD for L1 prediction (MVDL1) may be signaled separately, and information about MVD may include information about MVDL0 and / or information about MVDL1. For example, if MVP mode is applied to the current block and BI prediction is applied, information about MVDLO and information about MVDL1 may both be signaled. In addition, information about MVDL1 may be derived from information about MVDL0, or information about MVDL0 may be derived from information about MVDL1.

[0223] Meanwhile, when BI prediction is applied, symmetric MVD mode may be used considering coding efficiency. In this case, signaling of some of the motion information may be omitted. For example, when symmetric MVD mode is applied to the current block, information about refidxL0, information about refidxL1, and information about MVDL1 may not be signaled from the encoding device to the decoding device, but may be derived internally. For example, when MVP mode and BI prediction are applied to the current block, flag information indicating whether symmetric MVD mode is applied (e.g., symmetric MVD flag information or sym_mvd_flag syntax element) may be signaled, and when the value of the flag information is 1, the decoding device may determine that symmetric MVD mode is applied to the current block.

[0224] When the symmetric MVD mode is applied (i.e., when the value of the symmetric MVD flag information is 1), information about mvp_l0_flag, mvp_l1_flag, and MVDL0 may be explicitly signaled, and signaling of information about refidxL0, information about refidxL1, and information about MVDL1 may be omitted and derived internally as described above. For example, refidxL0 may be derived as an index pointing to a previous reference picture that is closest to the current picture in POC order within reference picture list 0 (which may be referred to as list 0 or L0). refidxL1 may be derived as an index pointing to a subsequent reference picture that is closest to the current picture in POC order within reference picture list 1 (which may be referred to as list 1 or L1).

[0225] Or, for example, both refidxL0 and refidxL1 may be derived as 0. Or, for example, the refidxL0 and refidxL1 may be derived as the minimum indices having the same POC difference in relation to the current picture. As a specific example, when [POC of the current picture] - [POC of the first reference picture indicated by refidxL0] is referred to as the first POC difference, and [POC of the second reference picture indicated by refidxL1] is referred to as the second POC difference, only when the first POC difference and the second POC difference are the same, the value of refidxL0 pointing to the first reference picture may be derived as the value of refidxL0 of the current block, and the value of refidxL1 pointing to the second reference picture may be derived as the value of refidxL1 of the current block. Also, for example, if there are multiple sets in which the first POC difference and the second POC difference are the same, refidxL0 and refidxL1 of the set with the minimum difference can be derived as refidxL0 and refidxL1 of the current block.

[0226] MVDL1 can be derived from -MVDL0. For example, the final MV for the current block can be derived as shown in Equation 1 below.

[0227] [Formula 1]

[0228]

[0229] As described above, the SMVD mode can be used when there are reference pictures temporally ahead (forward direction) and backward (backward direction) of the current picture, that is, when the True-Bi prediction condition is satisfied, and the closest reference picture in each direction can be used without signaling the reference picture index.

[0230] When only one motion vector (using a translaton motion model) is used to represent the motion of an encoded block, this motion vector may not actually represent the optimal motion of each pixel. Determining the optimal motion vector at the pixel level can improve encoding efficiency. To this end, we describe an affine motion prediction method that uses an affine motion model to encode. Affine motion prediction can represent the motion vector at the pixel level of a block using two, three, or four motion vectors.

[0231] Figure 13 is a diagram showing four motions expressed in an affine motion model, and Figure 14 is a diagram showing motion vectors used for affine motion prediction.

[0232] The affine motion model can express the four motions shown in Fig. 13. The affine motion model that expresses three of the motions (translation, scale, and rotation) that the affine motion model can express is called the similarity (or simplified) affine motion model, and the following description will be based on the similarity (or simplified) affine motion model. However, the disclosed embodiment is not limited to the motion model.

[0233] As illustrated in Fig. 14, affine motion prediction can determine the motion vector of the pixel position included in the current block (Cur) using two or more control point motion vectors (CPMV). At this time, the set of motion vectors is called an affine motion vector field (MVF) and can be determined by the following equation 2.

[0234] For the 4-parameter affine motion model, the motion vector at sample location (x, y) can be derived according to the following equation.

[0235] [Formula 2]

[0236]

[0237] For the 6-parameter affine motion model, the motion vector at sample location (x, y) can be derived according to the following equation.

[0238] [Formula 3]

[0239]

[0240] Figure 15 is a diagram for a case where the affine MVF is determined in 4*4 subblock units.

[0241] Referring to Figure 15, is the CPMV of the CP at the top-left corner position of the coding block. is the CPMV of the CP at the top-right corner location, is the CPMV of the CP at the bottom-left corner location. And W corresponds to the width of the current block, H corresponds to the height of the current block, is the motion vector at position {x, y}.

[0242] During the encoding / decoding process, the affine MVF can be determined on a pixel-by-pixel basis or on a pre-defined subblock basis. When determined on a pixel-by-pixel basis, a motion vector is obtained based on each pixel value, and when determined on a subblock-by-subblock basis, the motion vector of the corresponding block is obtained based on the pixel value at the center of the subblock (the lower right side of the center, i.e., the lower right sample among the four central samples). In the above example, it is assumed and explained that the affine MVF is determined on a 4*4 subblock basis, but this is for convenience of explanation, and the size of the subblock can be varied.

[0243] That is, when affine prediction is available, the motion models applicable to the current block may include a translational motion model, a 4-parameter affine motion model, and a 6-parameter affine motion model. Here, the translational motion model may represent a model in which existing block-unit motion vectors are used, the 4-parameter affine motion model may represent a model in which two CPMVs are used, and the 6-parameter affine motion model may represent a model in which three CPMVs are used.

[0244] Affine motion prediction may include affine MVP (or affine inter) mode and affine merge. In affine motion prediction, the motion vector of the current block can be derived on a sample-by-sample or sub-block-by-subblock basis.

[0245] In affine merge mode, the control point motion vector (CPMV) can be determined based on the affine motion model of the neighboring blocks encoded using affine motion prediction. Neighboring blocks encoded using affine motion prediction in the search order can be used in affine merge mode. If one or more neighboring blocks are encoded using affine motion prediction, the current block can be encoded using AF_MERGE.

[0246] That is, when the affine merge mode is applied, the CPMVs of the current block can be derived using the CPMVs of the surrounding blocks. In this case, the CPMVs of the surrounding blocks can be used as the CPMVs of the current block as they are, or the CPMVs of the surrounding blocks can be modified based on the size of the surrounding blocks and the size of the current block and then used as the CPMVs of the current block.

[0247] Meanwhile, in the case of affine merge where MV is derived in units of subblocks, it can be called subblock merge mode, and this can be indicated based on merge_subblock_flag (value 1). In this case, the affine merging candidate list described later can also be called a subblock merging candidate list. In this case, the subblock merging candidate list can further include a candidate derived by SbTMVP described later. In this case, the candidate derived by sbTMVP can be used as a candidate for index 0 of the subblock merge candidate list. In other words, the candidate derived by sbTMVP can be positioned before the inherited affine candidates and constructed affine candidates described later in the subblock merge candidate list.

[0248] When the affine merge mode is applied, an affine merge candidate list may be constructed to derive CPMVs for the current block. The affine merge candidate list may include, for example, at least one of the following candidates:

[0249] - Inherited affine candidates

[0250] - Constructed affine candidates

[0251] - Zero MV candidates

[0252] Here, the inherited affine candidates are candidates derived based on the CPMVs of the surrounding blocks when the surrounding blocks are coded in the affine mode, the constructed affine candidates are candidates derived by constructing CPMVs based on the MVs of the corresponding CP surrounding blocks for each CPMV unit, and the zero MV candidate can represent a candidate composed of CPMVs whose value is 0.

[0253] The list of affine merge candidates could be structured, for example, as follows:

[0254] - Insert inherited candidates into the candidate list

[0255] - Insert candidates into the candidate list

[0256] - Insert zero MV candidates into the candidate list

[0257] Here, a zero MV candidate can be optionally inserted, for example, into the candidate list when the number of current candidates is less than the number of maximum candidates.

[0258] There can be at most two inherited affine candidates, derived from the affine motion models of the left and upper surrounding blocks, respectively: one from the left surrounding CU, and the other from the upper surrounding CU.

[0259] The positions of the inherited affine moving predictors can be explained with reference to the aforementioned Figure 11.

[0260] Referring back to Figure 11, for the left predictor, the scan order is A0 --> A1, and for the upper predictor, the scan order is B0 --> B1 --> B2. Only the first detected inheritance candidate on each side is selected. No pruning is performed between two inheritance candidates. Once a neighboring affine CU is identified, the CPMVP candidate in the affine merge list of the current CU is derived using the control point motion vectors of the CU.

[0261] Figure 16 is a diagram showing the inheritance of control point motion vectors.

[0262] Referring to the example of Fig. 16, when block A at the lower left is encoded in affine mode, motion vectors v2, v3, and v4 corresponding to the upper left, upper right, and lower left positions of the CU including block A are acquired. When block A is encoded with a 4-parameter affine model, the two CPMVs of the current CU are calculated based on v2 and v3. When block A is encoded with a 6-parameter affine model, the three CPMVs of the current CU are calculated based on v2, v3, and v4.

[0263] Figure 17 is a diagram showing an example of the location of a candidate for the configured affine merge mode.

[0264] Constructed affine candidate refers to a method of constructing a candidate by combining the translational motion information of each control point. The motion information for the control point is derived from specific spatial and temporal neighboring blocks as shown in Fig. 17. CPMV (k=1, 2, 3, 4) represents the kth position of the control point. For CPMV1, the order of blocks B2 --> B3 --> A2 is checked, and the MV of the first available block is used. CPMV2 is checked in the order of blocks B1 --> B0, and CPMV3 is checked in the order of blocks A1 --> A0. CPMV4 uses Temporal Motion Vector Predictor (TMVP) if available.

[0265] After acquiring the four control point motion vectors, affine merge candidates are constructed based on the motion information. The control point MVs are combined in the following order:

[0266] {CPMV1, CPMV₂, CPMV₃},

[0267] {CPMV1, CPMV₂, CPMV₄},

[0268] {CPMV1, CPMV₃, CPMV₄},

[0269] {CPMV2, CPMV₃, CPMV₄},

[0270] {CPMV1, CPMV₂},

[0271] {CPMV1, CPMV₃}

[0272] A combination using three CPMVs forms a six-parameter affine merge candidate, and a combination using two CPMVs forms a four-parameter affine merge candidate. To prevent motion scaling, if the reference indices between control points are different, the control point MV combination is discarded.

[0273] In the affine MVP mode, after determining two or more CPMVPs and CPMVs for the current block, the control point motion vector difference (CPMVD) corresponding to the difference value is transmitted from the encoding device (200) to the decoding device (300).

[0274] For example, if the value of affine merge flag or merge_subblock_flag is 0, the affine MVP mode can be applied. Or, for example, if the value of inter_affine_flg is 1, the affine MVP mode can be applied. The affine MVP mode may also be called the affine CP MVP mode. Alternatively, the affine MVP mode may be called the affine inter mode or the inter-affine mode. The affine MVP candidate list described below may be called the control point motion vector predictor candidate list (CPMVP candidate list).

[0275] When the affine MVP mode is applied, an affine MVP candidate list can be constructed to derive CPMVs for the current block. The affine MVP candidate list may include, for example, at least one of the following candidates. For example, the affine MVP candidate list may include at most n candidates (e.g., 2).

[0276] - Inherited affine MVP candidates derived by extrapolation from the control point motion vectors (CPMVs) of surrounding CUs.

[0277] - Affine MVP candidates derived using the translational motion vectors (MVs) of surrounding CUs.

[0278] - Additional candidates based on translational motion vectors from surrounding CUs

[0279] - Zero Motion Vector (Zero MV) Candidate

[0280] Here, the inherited affine MVP candidate is a candidate derived based on the CPMVs of the surrounding blocks when the surrounding blocks are coded in the affine mode, the constructed affine MVP candidate is a candidate derived by constructing CPMVs based on the MVs of the corresponding CP surrounding blocks for each CPMV unit, and the zero motion vector candidate may represent a candidate composed of CPMVs whose value is 0. When the maximum number of candidates for the affine MVP candidate list is 2, the candidates below 2) above may be considered and added when the current number of candidates is less than 2. In addition, additional candidates based on the translational motion vectors from the surrounding CUs may be derived in the following order.

[0281] - If the number of candidates is less than 2 and the CPMV0 of the constructed candidate is valid, CPMV0 is used as the affine MVP candidate. That is, the candidate whose MVs of CP0, CP1, and CP2 are all equal to the CPMV0 of the constructed candidate is considered.

[0282] - If the number of candidates is less than 2 and the CPMV1 of the constructed candidate is valid, CPMV1 is used as the affine MVP candidate. That is, the candidate whose MVs of CP0, CP1, and CP2 are all equal to the CPMV1 of the constructed candidate is considered.

[0283] - If the number of candidates is less than 2 and the CPMV2 of the constructed candidate is valid, CPMV2 is used as the affine MVP candidate. That is, the candidate whose MVs of CP0, CP1, and CP2 are all equal to the CPMV2 of the constructed candidate is considered.

[0284] - If the number of candidates is less than 2, TMVP is used as the affine MVP candidate.

[0285] The above affine MVP candidates can be composed, for example, as follows:

[0286] - Insert inherited candidates into the candidate list

[0287] - Insert the configured candidates into the candidate list

[0288] - Insert additional candidates based on the translation MVs from surrounding CUs.

[0289] - Insert zero MVs candidates

[0290] However, inserting additional candidates based on the translational MVs from the surrounding CUs and inserting zero MVs candidates can be optionally performed when the current number of candidates is less than 2.

[0291] The examination order of inherited affine MVP candidates is identical to that of inherited affine merge candidates. The only difference is that for MVP candidates, only affine CUs with the same reference picture as the current block are considered. No pruning is applied when inserting an inherited affine motion predictor into the candidate list.

[0292] The constructed MVP candidates are derived only from specific spatially neighboring blocks. The same inspection order as used to construct affine merge candidates is used. Additionally, the reference picture indices of the neighboring blocks are also inspected. The block detected first in the inspection order, encoded in inter mode and having the same reference picture as the current CU, is used.

[0293] The control point motion vector (CPMV) of an affine CU can be stored in a separate buffer. The stored CPMV is only used to generate CPMVPs inherited from affine merge mode and affine AMVP mode for subsequently encoded CUs. Sub-block motion vectors (MVs) derived from the CPMV are used for motion compensation, MV derivation within the merge / AMVP list in translational mode, and deblocking.

[0294] To avoid using the picture line buffer for additional CPMVs, inheritance of affine motion data from a CU located in an above CTU is handled differently from inheritance from a normal neighboring CU. If a candidate CU for affine motion data inheritance exists in an above CTU line, the motion vectors of the lower left and lower right subblocks in that line buffer are used to derive the affine MVP instead of the CPMV. In this way, the CPMV is stored only in the local buffer. If the candidate CU is encoded with a 6-parameter affine, the affine model is degraded to a 4-parameter model.

[0295] Previously, when use_integer_mv_flag was set to 0 in the slice header, the difference between the motion vector of a CU and the predicted motion vector (MVD) was signaled in units of quarter-luma samples. In the disclosed embodiment, a CU-level Adaptive Motion Vector Resolution (AMVR) scheme is introduced. AMVR allows encoding the MVD of a CU with different precisions. Depending on the current CU mode (normal AMVP mode or affine AMVP mode), the MVD of the CU can be adaptively selected from the following precisions:

[0296] - Normal AMVP mode: quarter luma sample, integer luma sample, or four luma sample

[0297] - Affine AMVP mode: quarter luma samples, integer luma samples, or 1 / 16 luma samples

[0298] The CU-level MVD resolution indication is conditionally signaled only if the current CU contains at least one non-zero MVD component. If all MVD components (i.e., horizontal and vertical MVDs for reference lists L0 and L1) are zero, quarter luma sample MVD resolution is automatically inferred. If the CU has at least one non-zero MVD component, a first flag is signaled to indicate whether an MVD with quarter luma sample precision is used for the CU. If the first flag is zero, no further signaling is required, and an MVD with quarter luma sample precision is used for the current CU. Otherwise, a second flag is signaled to indicate whether an integer luma sample or a 4 luma sample MVD precision is used for regular AMVP CUs. The same second flag is also used for affine AMVP CUs to indicate either integer luma sample or 1 / 16 luma sample MVD precision.

[0299] To ensure that the reconstructed motion vectors have the intended precision (quarter luma samples, integer luma samples, or 4 luma samples), the motion vector predictor (MVP) of the corresponding CU is rounded to the same precision as the MVD before being added to the MVD. The motion vector predictor is rounded towards zero, meaning that a negative MVP is rounded towards positive infinity, and a positive MVP is rounded towards negative infinity.

[0300] The encoding device (200) determines the optimal motion vector precision for the current CU through rate-distortion (RD) cost evaluation. To avoid performing three CU-level RD checks for each MVD precision, in one example, the RD check for MVD precision excluding quarter luma samples can be conditionally performed.

[0301] In the normal AMVP mode, the RD cost of quarter luma samples and integer luma sample precision is first calculated. Then, the RD cost of integer luma sample precision is compared to whether it is smaller than that of quarter luma samples to determine whether the RD cost of 4 luma sample MVD precision needs to be further evaluated. If the RD cost of quarter luma samples is significantly smaller than that of integer luma samples, the RD evaluation for 4 luma sample MVD precision is omitted.

[0302] For affine AMVP mode, after RD cost evaluation for affine merge / skip mode, merge / skip mode, regular AMVP mode with quarter luma sample precision, and affine AMVP mode with quarter luma sample precision, if affine inter mode is not selected, affine inter mode with 1 / 16 luma sample or integer sample precision is not evaluated.

[0303] Additionally, the affine parameters obtained in the affine inter mode with quarter luma sample precision are used as a starting point for exploration in the affine inter mode with 1 / 16 luma sample and integer luma sample precision.

[0304] Meanwhile, to reduce memory load, motion information from previously decoded reference pictures can be stored in fixed-area units. This can be called temporal motion field storage, motion field compression, or motion data compression. In this case, the storage unit for motion information can be set differently depending on whether affine mode is applied.

[0305] For example, the highest precision for explicitly signaled motion vectors is quarter luma samples. In some inter-prediction modes, such as affine mode, motion vectors are derived with 1 / 16th luma sample precision, and motion-compensated prediction is performed with 1 / 16th sample precision. In terms of internal motion field storage, all motion vectors are stored with 1 / 16th luma sample precision.

[0306] For example, for storing the temporal motion field used in TMVP and ATMVP, motion field compression can be performed with 8Х8 granularity.

[0307] Decoder-side Motion Vector Refinement (DMVR) is a method for motion prediction that refines the motion information of surrounding blocks at the decoder side. When DMVR is applied, the decoder can derive improved motion information through cost comparison based on templates generated using motion information from surrounding blocks in merge / skip mode. This improves motion prediction accuracy and compression performance without requiring additional signaling information.

[0308] Here, for convenience of explanation, the description focuses on the decoding device (300), but DMVR can also be performed in the same manner in the encoding device (200).

[0309] In pairwise prediction, improved MVs are searched for each of the reference picture list L0 and the reference picture list L1 within the search region around the initial MV.

[0310] The Block Matching (BM) method calculates the distortion between two candidate blocks in the reference picture list L0 and list L1. For each MV candidate within the search range surrounding the initial MV, the Sum of Absolute Differences (SAD) between the differential MVs is calculated.

[0311] The MV candidate with the smallest SAD becomes the improved MV, which is used to generate the biprediction signal.

[0312] The decoding device (300) may invoke the DMVR process to improve the accuracy of the initial motion compensation prediction (i.e., motion compensation prediction via the conventional merge / skip mode). For example, the decoding device (300) may perform the DMVR process when the prediction mode of the current block is the merge mode or the skip mode, and bidirectional bi-prediction is applied to the current block in which the bidirectional reference pictures are in opposite directions relative to the current picture in display order.

[0313] For example, DMVR can be applied to CUs encoded with the following modes and conditions:

[0314] - Merge mode at CU level with bidirectional MV

[0315] - One reference image is in the past, and the other reference image is in the future, based on the current image.

[0316] - The distance between the two reference images and the current image (i.e., the POC difference) is the same.

[0317] - CU contains more than 64 luma samples

[0318] - The height and width of the CU are both greater than 8 luma samples.

[0319] - BCW (weighted bi-prediction) weight index indicates equal weights

[0320] - WP (weighted prediction) is not applied to the current block.

[0321] The improved MV derived from the DMVR process is used to generate inter-prediction samples and also to generate temporal MVPs for future image encoding. Meanwhile, the original MV is used for deblocking processing and also to generate spatial MVPs for future CU encoding.

[0322] Meanwhile, the IBC (Intra Block Copy) mode can apply similar concepts to the motion vector, motion vector difference, and motion vector predictor used in the inter prediction mode. This will be described in detail below.

[0323] The IBC mode is implemented as a block-level encoding mode, and the encoding device (200) performs block matching (BM) to find an optimal block vector (or motion vector) for each coding unit (CU). Here, the block vector represents the displacement from the current block to a reference block already restored within the same image.

[0324] Luma block vectors of CUs encoded with IBC have integer precision, and chroma block vectors are also rounded to integers. When combined with AMVR, IBC mode can optionally use motion vector precision of 1 pixel (pel) or 4 pixels.

[0325] CUs encoded with IBC can be treated as a third prediction mode, separate from the intra or inter prediction modes. IBC mode is applicable to CUs whose luma block width and height are each 64 pixels or less.

[0326] The encoding device (200) performs hash-based motion estimation for IBC. In particular, RD-based judgment is performed for blocks with either a width or height of 16 pixels or less. In non-merge mode, hash-based search is performed first, and if no valid candidate is found, local block matching-based search is additionally performed.

[0327] In hash-based search, the hash key (32-bit CRC) matching between the current block and the reference block is extended to all allowed block sizes. For every location in the current image, the hash key is calculated based on the 4X4 subblock. If the current block is larger, the hash key of the reference block is determined to match if the hash keys of all 4X4 subblocks constituting the current block all match the hash key of the reference location. If multiple reference blocks have the same hash key, the block vector cost is calculated for each reference block, and the reference block with the minimum cost is selected.

[0328] In block matching search, the search range is set to include both the previous and current CTUs. At the CU level, the IBC mode is signaled via a flag, which can be either IBC AMVP mode or IBC Skip / Merge mode.

[0329] - IBC Skip / Merge Mode: A merge candidate index is used to indicate which block vector from the list of candidate block vectors generated from surrounding IBC-encoded blocks to use to predict the current block. The merge list consists of spatial candidates, history-based motion prediction (HMVP) candidates, and pairwise candidates.

[0330] - IBC AMVP mode: The block vector difference is encoded in the same way as the motion vector difference (MVD). The block vector prediction method uses one prediction candidate each from the left and upper neighboring blocks (if encoded with IBC). If either neighboring block is unavailable, the default block vector is used as the predictor. A flag is signaled to indicate the index of the block vector predictor candidate used.

[0331] The IBC Merge / AMVP list configuration can be modified as follows:

[0332] - Only if an IBC merge / AMVP candidate is valid, can the candidate be inserted into the IBC merge / AMVP candidate list.

[0333] - The above-right, bottom-left, and above-left spatial candidates and one pairwise average candidate can be added to the IBC merge / AMVP candidate list.

[0334] - Template-based adaptive reordering (ARMC-TM: Adaptive Reordering based on Matching Cost - Template Matching) is applied to the IBC merge list.

[0335] The size of the HMVP table for IBC can be increased to 25. After a maximum of 20 IBC merge candidates are derived through full pruning, these candidates are reordered together. After reordering, the six candidates with the lowest template matching cost are selected as candidates for the final IBC merge list.

[0336] The zero vector candidates used to populate the IBC merge / AMVP list are replaced with a set of Block Vector Predictor (BVP) candidates located within the IBC reference area. The zero vector is not considered a valid block vector in IBC merge mode and is therefore discarded as a BVP from the IBC candidate list.

[0337] Figure 18 is a diagram showing padding candidates for replacing zero vectors in the IBC list.

[0338] Referring to Figure 18, three candidates are located at the nearest corners of the reference area, and three additional candidates are determined at the centers of three sub-areas (A, B, C). The coordinates of each candidate are determined based on the width and height of the current block, and the △X and △Y parameters.

[0339] Template matching can be used in both IBC merge mode and IBC AMVP mode. The IBC-TM merge list is modified compared to the list used in regular IBC merge mode, and candidates are selected based on a pruning method based on the motion distance between candidates, similar to regular TM merge mode. The last zero motion candidate is replaced with the motion vectors left (-W, 0), up (0, -H), and up-left (-W, -H), where W represents the width of the current CU and H represents the height.

[0340] In IBC-TM merge mode, selected candidates are refined through Rate-Distortion Optimization (RDO) or template matching techniques prior to decoding. IBC-TM merge mode competes with the standard IBC merge mode, and the TM-merge flag is signaled.

[0341] In IBC-TM AMVP mode, up to three candidates are selected from the IBC-TM merge list. Each of these three candidates is refined using a template matching technique and sorted based on the template matching cost of the refined results. Only the top two candidates are then considered in the motion estimation process, as in the conventional method.

[0342] Figure 19 is a diagram showing an example of another IBC reference area at the current CU location.

[0343] Template matching improvement is straightforward in both IBC-TM merge and AMVP modes. This is because the IBC motion vectors must (i) be integers and (ii) lie within the reference region depicted in Figure 19. Therefore, in IBC-TM merge mode, all improvements are performed with integer precision, while in IBC-TM AMVP mode, they are performed with either integer or 4-pixel precision, depending on the AMVR value. This improvement is performed by directly accessing samples without interpolation. In both cases, the improved motion vectors and the templates used in each improvement step must satisfy the constraints of the reference region.

[0344] Figure 20 is a diagram illustrating an example of a reference area when a CTU is coded. The area highlighted in bold in Figure 20 is the reference area, and the white area is an invalid area.

[0345] The reference region of IBC can extend upwards by two CTU rows. Fig. 20 shows the reference region for CTU (m, n). Specifically, when coding CTU (m, n), the reference region includes CTUs with indices (m-2, n-2)...(W, n-2), (0, n-1)...(W, n-1), (0, n)...(m, n), where W represents the maximum horizontal index within the current tile, slice, or picture. When the CTU size is 256, the reference region is limited to one CTU row upwards. The search range of the pixel-wise block vector search (also called local search) is limited to [-(C << 1), C >> 2] in the horizontal direction and [-C, C >> 2] in the vertical direction to match the reference region extension, where C represents the CTU size.

[0346] As described above, AMVR mode can allow different MV resolutions depending on each prediction mode. However, conventional AMVR has a problem in that it cannot reflect image characteristics other than mode-specific characteristics because the candidate MV precision is fixed according to the mode (normal mode, affine mode, IBC mode, etc.). The disclosed embodiment provides an improved AMVR that takes into account image characteristics. Since image characteristics vary, applying various AMVRs suited to the image characteristics can improve compression performance.

[0347] According to one embodiment, the AMVR mode may have various AMVR candidates considering the characteristics of the image. For example, the characteristic of the image may be the resolution of the image, and one of two or more AMVR candidate sets may be selected based on the resolution. As another example, the characteristic of the image may be the size of a coding unit (or coding block), and one of two or more AMVR candidate sets may be selected based on the size of the coding unit. As another example, the characteristic of the image may be the magnitude of a motion vector of an adjacent block, and one of two or more AMVR candidate sets may be selected based on this. As another example, the characteristic of the image may be the distance between the current picture and a reference picture, and one of two or more AMVR candidate sets may be selected based on the distance. As another example, the characteristic of the image may be the AMVR selection rate of a previously encoded picture, and an AMVR candidate set including candidates with a high selection rate may be configured and one of the multiple candidate sets may be selected. The method of selecting AMVR candidates considering the characteristics of the image may also be applied to the affine mode and the IBC mode.

[0348] The various AMVR candidates described above can be applied separately on a prediction direction basis. That is, when in bidirectional prediction mode, the AMVR candidates of L0 and L1 can be different. Bidirectional prediction can be limited to blocks with non-zero motion vectors. In addition, bidirectional prediction can be limited to cases where ph_mvd_l1_zero_flag is not TRUE, i.e., when the motion vector difference of L1 is not a zero vector. Different AMVR candidates for each prediction direction (L0 or L1) may or may not be included in a specific AMVR set depending on the characteristics of the image.

[0349] The process of selecting one of the various AMVR candidate sets can be performed through signaling. When two or more AMVR candidate sets exist, an index indicating the corresponding set can be signaled. When two AMVR candidate sets exist, a flag can be signaled. Furthermore, the process of selecting one of the various AMVR candidate sets can be determined based on various conditions without signaling.

[0350] Hereinafter, the application method of various MV resolutions according to the prediction mode (first information) such as normal inter mode, affine inter mode, IBC mode, etc. and the characteristics of the image (second information) will be specifically described. In the present disclosure, normal inter mode and affine inter mode may be used with the same meaning as normal mode and affine mode, respectively. In the present disclosure, normal mode may mean at least one mode among various modes that perform inter prediction using motion vectors other than affine mode and IBC mode.

[0351] Different MV resolutions can be allowed depending on each mode. Specifically, normal inter mode can have motion vectors (MV) in {1 / 4, 1 / 2, 1, 4}-pel units, affine inter mode can have motion vectors (MV) in {1 / 14, 1 / 16, 1}-pel units, and IBC mode can have block vectors (BV) in {1, 4}-pel units. Table 4 below briefly describes the derivation process of the final motion (block) vector using the allowed MV (or BV) resolutions. In Table 5, MVD (Motion Vector Difference), MVP (Motion Ventor Predictor), and MV (Motion Vector) can be calculated for the x-component or the y-component, respectively.

[0352] [Table 5]

[0353]

[0354] According to Table 5 above, the signaled MVD information is scaled using the AmvrShift value, and the MVP derived through the MVP derivation process is rounded using the AmvrShift value. Finally, the MV can be calculated using the rounded MVP and the scaled MVD. At this time, the AmvrShift value can be determined according to the prediction mode and AMVR precision index, as shown in Table 6 below.

[0355] [Table 6]

[0356]

[0357] According to Table 6 above, AMVR index 0..3 can be derived through AMVR flag (amvr_flag) and AMVR precision index (amvr_precision_idx), and the number and Amvr shift value can vary depending on the prediction mode. In general, normal mode tends to have relatively large MVD, such as 1 Pel and 4 Pel units, compared to affine mode. Therefore, the MVD resolution of normal mode can be adjusted relatively low compared to affine mode. Here, a low MVD resolution means a low precision of MVD, and it means a case where the value of luma sample or pel indicating MVD resolution or precision is large, such as 4 luma samples.

[0358] An AMVR index can designate a specific AMVR candidate within an AMVR candidate set consisting of multiple AMVR candidates. An AMVR candidate can be expressed in terms of an MVD resolution or an AMVR shift value, and an MVD resolution and a corresponding AMVR shift value are assigned to each other. For example, if the MVD resolution is 1 / 4 luma sample (or pel), the corresponding AMVR shift value is 2, if the MVD resolution is 1 / 16 luma sample, the corresponding AMVR shift value is 0, if the MVD resolution is 1 / 2 luma sample, the corresponding AMVR shift value is 3, and if the MVD resolution is 4 luma samples, the corresponding AMVR shift value is 6. The AMVR flag (amvr_flag) indicates the resolution of the MVD. If the value of the AMVR flag is 0, the resolution of the MVD is 1 / 4 pel (or luma sample). If the value of the AMVR flag is 1, the resolution of the MVD can be specified by the AMVR precision index (amvr_precision_idx).

[0359] Therefore, the AMVR index can be derived based on the AMVR flag and the AMVR precision index. According to the example in Table 6 above, when in affine mode (inter_affine_flag == 1), the AMVR index 0..2 can be derived through the AMVR flag and the AMVR precision index, and the values ​​of AmvrShift {2, 0, 4} are assigned for each {1 / 4, 1 / 16, 1}-pel resolution.

[0360] Additionally, when in IBC mode (CuPredMode == MODE_IBC), AMVR indices 1 and 2 can be derived through AMVR flags and AMVR precision indices, and the values ​​of AmvrShift {4, 6} are assigned for each {1, 4}-pel resolution.

[0361] Finally, when in normal inter mode (inter_affine_flag == 0 && CuPredMode != MODE_IBC), the AMVR index 0..3 can be derived through the AMVR flag and AMVR precision index, and the values ​​of AmvrShift {2, 3, 4, 6} are assigned for each {1 / 4, 1 / 2, 1, 4}-pel resolution.

[0362] Even within the same prediction mode, the optimal AMVR may vary depending on factors such as image resolution and motion speed. In one embodiment, various AMVR candidate sets can be constructed by considering not only the prediction mode but also the characteristics of the image. In this embodiment, the AMVR candidate set and AMVR candidates may have the same meaning as the AMVR candidate mode set and AMVR candidate modes, respectively.

[0363] Here, the characteristics of the image may include at least one of the resolution of the image, the size of the coding unit, the motion vector of the current block and the adjacent block, the distance between the current picture and the reference picture, the AMVR candidate selection rate of the picture decoded before the current picture, or the applicability and application of a decoder-side motion compensation (e.g., DMVR or TM-based enhancement) technology to a prediction block in the current picture or the current block.

[0364] In this example, AMVR candidates can be expressed in terms of MVD resolution (or precision) in units of luma samples or pels. For example, AMVR candidates can be selected from the {1 / 16, 1 / 8, 1 / 4, 1 / 2, 1, 2, 4, 8, 16}-pel candidates. However, this is merely an example, and AMVR candidates can also be expressed in terms of AMVR shift values.

[0365] The AMVR candidate set for normal mode may include some or all of {1 / 16, 1 / 8, 1 / 4, 1 / 2, 1, 2, 4, 8, 16}-Pel.

[0366] First, the AMVR candidate set for normal mode may include different AMVR candidates depending on the resolution of the image. For example, as the resolution of the image increases, the motion vector tends to increase. Therefore, the AMVR candidate set may be configured in the direction that the MVD resolution or precision decreases as the resolution of the image increases. Specifically, if the width and / or height of the picture or its area is greater than a predefined threshold, the AMVR candidate set may include {1 / 2, 1, 2, 4, 8}-pel, and if it is equal to or less than the threshold, the AMVR candidate set may include {1 / 4, 1 / 2, 1, 2, 4}-pel. In this case, 1280(width) x 720(height) or 1920(width) x 1080(height) may be considered as an example of the threshold. In addition, the AMVR candidate set is an example, and the type, number, arrangement order, etc. of AMVR candidates within the AMVR candidate set may be changed. It is also possible to construct two or more AMVR candidate sets based on multiple thresholds.

[0367] Alternatively, the AMVR candidate set for normal mode may include different AMVR candidates depending on the size of the coding unit (CU). For example, if the width and / or height or the area of ​​the CU is greater than a predefined threshold, {1 / 2, 1, 2, 4, 8}-pel may be included, and if it is equal to or less than the threshold, {1 / 4, 1 / 2, 1, 2, 4}-pel may be included. In this case, 16 (width) x 16 (height) may be considered as an example of the threshold. In addition, the AMVR candidate set is an example, and the types, number, arrangement order, etc. of AMVR candidates in the AMVR candidate set may be changed. In addition, two or more AMVR candidate sets may be configured based on multiple thresholds.

[0368] Alternatively, the AMVR candidate set for normal mode may include different AMVR candidates depending on the size of the coding unit (CU). For example, if the width and / or height or the area of ​​the CU is greater than a predefined threshold, {1 / 2, 1, 2, 4, 8}-pel may be included, and if it is equal to or less than the threshold, {1 / 4, 1 / 2, 1, 2, 4}-pel may be included. In this case, 16 (width) x 16 (height) may be considered as an example of the threshold. In addition, the AMVR candidate set is an example, and the types, number, arrangement order, etc. of AMVR candidates in the AMVR candidate set may be changed. In addition, two or more AMVR candidate sets may be configured based on multiple thresholds.

[0369] Alternatively, the AMVR candidate set for normal mode may include different AMVR candidates by considering the motion vectors of the current block and its adjacent blocks. For example, if the value derived from the motion vector of the adjacent block is greater than a predefined threshold, {1 / 2, 1, 2, 4, 8}-pel may be included, and if the value is equal to or less than the threshold, {1 / 4, 1 / 2, 1, 2, 4}-pel may be included. For example, when a motion vector is expressed as (mv_x, mv_y), the absolute value (or square value) of mv_x (or mv_y) may be compared with a predefined threshold, and an example of the threshold may be considered as 4-pel. Alternatively, the sum of the absolute value (or square value) of mv_x and the absolute value (or square value) of mv_y may be compared with a predefined threshold, and an example of the threshold may be considered as 8-pel. At this time, the above AMVR candidate set is an example, and the type and number of AMVR candidates in the AMVR candidate set, arrangement order, etc. can be changed.

[0370] Additionally, the motion vector of an adjacent block may be a motion vector of a block contained in a predefined location, or a representative motion vector derived from the motion vectors of multiple adjacent blocks. In this case, the representative motion vector may refer to the average or median of the motion vectors of each block. Furthermore, two or more AMVR candidate sets may be constructed based on multiple thresholds.

[0371] In the above example, the motion vectors of the blocks adjacent to the current block are considered, but the motion vectors of blocks not adjacent to the current block may also be considered as targets for including other AMVR candidates. As another example, while scanning the blocks adjacent to the current block in a specific, predefined order, candidates that do not overlap with existing AMVR candidates may be added, and the total number of candidates including the added candidates may be limited to a specific value. That is, when the AMVR candidates are filled to the total number of predefined candidates, the scanning of the motion vectors of the neighboring blocks may stop. The neighboring blocks of the current block may include only blocks adjacent to the current block, or may include blocks adjacent to the current block and non-adjacent blocks.

[0372] Alternatively, the AMVR candidate set for normal mode may include different AMVR candidates depending on the distance between the current picture and the reference picture. For example, the distance between the current picture and the reference picture may be derived as the difference between the picture order count (POC) of the current picture and the POC of the reference picture. Specifically, when the distance between the current picture and the reference picture is defined as diffPOC, if the value is greater than a predefined threshold, {1 / 2, 1, 2, 4, 8}-pel may be included, and if the value is equal to or less than the threshold, {1 / 4, 1 / 2, 1, 2, 4}-pel may be included. In this case, the threshold may be 8, etc. In addition, in the case of a block to which bi-prediction is applied, the AMVR candidate set may be formed by considering all or part of the distance between the current picture and the reference picture in each direction. In this case, the AMVR candidate set is only one example, and the type, number, and arrangement order of AMVR candidates within the AMVR candidate set may be changed. Additionally, two or more AMVR candidate sets can be constructed based on multiple thresholds.

[0373] Alternatively, the AMVR candidate set for normal mode may include different AMVR candidates depending on the AMVR candidate selection rates of pictures decoded before the current picture. For example, the AMVR candidate set may include n candidates with high selection rates among the AMVR candidates of pictures decoded before the current picture, and a default candidate set. The n may be an integer greater than or equal to 1. Alternatively, the AMVR candidate with high selection rates among the AMVR candidates of pictures decoded before the current picture may be placed at the front of the AMVR candidate set to reduce the amount of signaling information.

[0374] Alternatively, the AMVR candidate set for normal mode may include different AMVR candidates depending on the applicability and application of decoder-side motion compensation (e.g., decoder-side motion vector refinement or template matching based refinement) technology to the prediction block in the current picture or the current block. For example, when the decoder-side motion compensation technology for the prediction block target in the current picture or the current block is applied to increase the precision of the motion vector, {1 / 2, 1, 2, 4, 8}-pel may be included, and when the decoder-side motion compensation technology is not applied, {1 / 4, 1 / 2, 1, 2, 4}-pel may be included. In this case, the AMVR candidate set is only one example, and the type, number, arrangement order, etc. of the AMVR candidates may be changed. In addition, the candidates may also vary, such as configuring two or more AMVR candidate sets, depending on the precision of the motion vector used in the decoder-side motion compensation technology.

[0375] Additionally, it is possible to construct an AMVR candidate set by combining two or more examples of the methods listed above.

[0376] The following describes the composition of the AMVR candidate set for affine mode. The AMVR candidate set for affine mode may include some or all of {1 / 16, 1 / 8, 1 / 4, 1 / 2, 1, 2, 4, 8, 16}-pel.

[0377] An AMVR candidate set for affine mode may include different AMVR candidates depending on the resolution of the image. For example, if the width and / or height of the picture is greater than a predefined threshold, {1 / 8, 1 / 4, 1 / 2, 1, 2}-pel may be included, and if the width and / or height of the picture is equal to or less than the threshold, {1 / 16, 1 / 8, 1 / 4, 1 / 2, 1}-pel may be included. In this case, an example of the threshold may be 1280(width) x 720(height) or 1920(width) x 1080(height). In addition, the AMVR candidate set is an example, and the type, number, arrangement order, etc. of AMVR candidates in the AMVR candidate set may be changed. In addition, two or more AMVR candidate sets may be configured based on multiple thresholds.

[0378] Alternatively, the AMVR candidate set for the affine mode may include different AMVR candidates depending on the size of the coding unit (CU). For example, if the width and / or height of the CU are greater than a predefined threshold, {1 / 8, 1 / 4, 1 / 2, 1, 2}-pel may be included, and if the width and / or height of the CU are equal to or less than the threshold, {1 / 16, 1 / 8, 1 / 4, 1 / 2, 1}-pel may be included. In this case, 16 (width) x 16 (height) may be considered as an example of the threshold. In addition, the AMVR candidate set is an example, and the types, number, arrangement order, etc. of AMVR candidates in the AMVR candidate set may be changed. In addition, two or more AMVR candidate sets may be configured based on multiple thresholds.

[0379] Alternatively, the AMVR candidate set for affine mode may include different AMVR candidates by considering the motion vectors of the current block and its adjacent blocks. For example, if the value derived from the motion vector of the adjacent block is greater than a predefined threshold, {1 / 8, 1 / 4, 1 / 2, 1, 2}-pel may be included, and if the value is equal to or less than the threshold, {1 / 16, 1 / 8, 1 / 4, 1 / 2, 1}-pel may be included. When the motion vector is expressed as (mv_x, mv_y), the absolute value (or square value) of mv_x (or mv_y) may be compared with a predefined threshold, such as 2-pel. Alternatively, the sum of the absolute value (or square value) of mv_x and the absolute value (or square value) of mv_y may be compared with a predefined threshold, such as 4-pel may be considered. At this time, the above AMVR candidate set is an example, and the type and number of AMVR candidates in the AMVR candidate set, arrangement order, etc. can be changed.

[0380] Additionally, the motion vector of an adjacent block may be a motion vector of a block contained in a predefined location, or a representative motion vector derived from the motion vectors of multiple adjacent blocks. In this case, the representative motion vector may refer to the average or median of the motion vectors of each block. Furthermore, two or more AMVR candidate sets may be constructed based on multiple thresholds.

[0381] In the above example, the motion vectors of the blocks adjacent to the current block are considered, but the motion vectors of blocks not adjacent to the current block may also be considered as targets for including other AMVR candidates. As another example, while scanning the blocks adjacent to the current block in a specific, predefined order, candidates that do not overlap with existing AMVR candidates may be added, and the total number of candidates including the added candidates may be limited to a specific value. That is, when the AMVR candidates are filled to the total number of predefined candidates, the scanning of the motion vectors of the neighboring blocks may stop. The neighboring blocks of the current block may include only blocks adjacent to the current block, or may include blocks adjacent to the current block and non-adjacent blocks.

[0382] Alternatively, the AMVR candidate set for affine mode may include different AMVR candidates depending on the distance between the current picture and the reference picture. For example, the distance between the current picture and the reference picture may be derived as the difference between the POC (picture order count) of the current picture and the POC of the reference picture. For example, when the distance between the current picture and the reference picture is defined as diffPOC, if the value is greater than a predefined threshold, {1 / 8, 1 / 4, 1 / 2, 1, 2}-pel may be included, and if the value is equal to or less than the threshold, {1 / 16, 1 / 8, 1 / 4, 1 / 2, 1}-pel may be included. In this case, a threshold value such as 8 may be considered. In addition, in the case of a block to which paired prediction is applied, the AMVR candidate set may be configured by considering all or part of the distance between the current picture and the reference picture in each direction. In this case, the AMVR candidate set is just one example, and the type, number, arrangement order, etc. of AMVR candidates within the AMVR candidate set may be changed. Additionally, two or more AMVR candidate sets can be constructed based on multiple thresholds.

[0383] Alternatively, the AMVR candidate set for the affine mode may include different AMVR candidates depending on the AMVR candidate selection rates of pictures decoded before the current picture. For example, the AMVR candidate set may include n candidates with high selection rates among the AMVR candidates of pictures decoded before the current picture, and a default candidate set. The n may be an integer greater than or equal to 1. Alternatively, the AMVR candidate with a high selection rate among the pictures decoded before the current picture may be placed at the front of the AMVR candidate set to reduce the amount of signaling information.

[0384] Alternatively, the AMVR candidate set for the affine mode may include different AMVR candidates depending on the applicability and application of a decoder-side motion compensation (e.g., decoder-side motion vector refinement or template matching based refinement) technology to a prediction block in the current picture or the current block. For example, when the decoder-side motion compensation technology for the prediction block target in the current picture or the current block is applied to increase the precision of the motion vector, it may include {1 / 2, 1, 2, 4, 8}-pel, and when the decoder-side motion compensation technology is not applied, it may include {1 / 4, 1 / 2, 1, 2, 4}-pel. In this case, the AMVR candidate set is just one example, and the type, number, arrangement order, etc. of the AMVR candidates may be changed. In addition, the candidates may also vary, such as configuring two or more AMVR candidate sets, depending on the precision of the motion vector used in the decoder-side motion compensation technology.

[0385] Additionally, it is possible to construct an AMVR candidate set by combining two or more examples of the methods listed above.

[0386] The following describes the composition of the AMVR candidate set for IBC mode. The AMVR candidate set for IBC mode may include some or all of {1 / 16, 1 / 8, 1 / 4, 1 / 2, 1, 2, 4, 8, 16}-pel.

[0387] An AMVR candidate set for IBC mode may include different AMVR candidates depending on the resolution of the image. For example, if the width and / or height of the picture is greater than a predefined threshold, {1 / 2, 1, 2, 4, 8}-pel may be included, and if the width and / or height of the picture is equal to or less than the threshold, {1 / 4, 1 / 2, 1, 2, 4}-pel may be included. In this case, an example of the threshold may be 1280(width) x 720(height) or 1920(width) x 1080(height). In addition, the AMVR candidate set is an example, and the type, number, arrangement order, etc. of AMVR candidates in the AMVR candidate set may be changed. In addition, two or more AMVR candidate sets may be configured based on multiple thresholds.

[0388] Alternatively, the AMVR candidate set for the IBC mode may include different AMVR candidates depending on the size of the coding unit (CU). For example, if the width and / or height of the CU are greater than a predefined threshold, {1 / 2, 1, 2, 4, 8}-pel may be included, and if the width and / or height of the CU are equal to or less than the threshold, {1 / 4, 1 / 2, 1, 2, 4}-pel may be included. In this case, 16 (width) x 16 (height) may be considered as an example of the threshold. In addition, the AMVR candidate set is an example, and the types, number, arrangement order, etc. of AMVR candidates in the AMVR candidate set may be changed. In addition, two or more AMVR candidate sets may be configured based on multiple thresholds.

[0389] Alternatively, the AMVR candidate set for IBC mode may include different AMVR candidates by considering the block vectors of the current block and its adjacent blocks. For example, if the value derived from the block vector of the adjacent block is greater than a predefined threshold, {1 / 4, 1 / 2, 1, 2, 4}-pel may be included, and if the value is equal to or less than the threshold, {1 / 2, 1, 2, 4, 8}-pel may be included. When a block vector is expressed as (bv_x, bv_y), the absolute value (or square value) of bv_x (or bv_y) may be compared with a predefined threshold, and an example of the threshold may be considered as 2-pel. Alternatively, the sum of the absolute value (or square value) of bv_x and the absolute value (or square value) of bv_y may be compared with a predefined threshold, and an example of the threshold may be considered as 4-pel. At this time, the above AMVR candidate set is an example, and the type and number of AMVR candidates in the AMVR candidate set, arrangement order, etc. can be changed.

[0390] Additionally, the block vector of an adjacent block may be a block vector of a block included in a predefined location, or a representative block vector derived from the block vectors of multiple adjacent blocks. In this case, the representative block vector may mean the average or median of the block vectors of each block. Furthermore, two or more AMVR candidate sets may be constructed based on multiple thresholds.

[0391] In the above example, the block vectors of blocks adjacent to the current block are considered, but the block vectors of blocks not adjacent to the current block may also be considered as targets for including other AMVR candidates. As another example, while scanning the blocks adjacent to the current block in a specific, predefined order, candidates that do not overlap with existing AMVR candidates may be added, and the total number of candidates including the added candidates may be limited to a specific value. That is, when the AMVR candidates are filled to the total number of predefined candidates, the scanning of the block vectors of the neighboring blocks may stop. The neighboring blocks of the current block may include only blocks adjacent to the current block, or may include blocks adjacent to the current block and non-adjacent blocks.

[0392] Alternatively, the AMVR candidate set for the IBC mode may include different AMVR candidates depending on the AMVR candidate selection rates of pictures decoded before the current picture. For example, the AMVR candidate set may include n candidates with high selection rates among the AMVR candidates of pictures decoded before the current picture, and a default candidate set. The n may be an integer greater than or equal to 1. Alternatively, the AMVR candidate with a high selection rate among the pictures decoded before the current picture may be placed at the front of the AMVR candidate set to reduce the amount of signaling information.

[0393] Alternatively, the AMVR candidate set for the IBC mode may include different AMVR candidates depending on whether a decoder-side motion compensation (e.g., template matching based refinement) technique is applied to a prediction block in the current picture or the current block. For example, if the decoder-side motion compensation technique is applied to the prediction block target in the current picture or the current block to increase the precision of the block vector, {1 / 2, 1, 2, 4, 8}-pel may be included, and if the decoder-side motion compensation technique is not applied, {1 / 4, 1 / 2, 1, 2, 4}-pel may be included. In this case, the AMVR set is only one example, and the type, number, arrangement order, etc. of the AMVR candidates may be changed. In addition, the candidates may also vary, such as configuring two or more AMVR candidate sets, depending on the precision of the block vector used in the decoder-side motion compensation technique.

[0394] Additionally, it is possible to construct an AMVR candidate set by combining two or more examples of the methods listed above.

[0395] The AMVR candidate set, considering the characteristics of the above video, can be defined on a block-by-block basis. Alternatively, it can be defined on a slice-by-slice or picture-by-picture basis. For example, a set of available AMVR candidates can be configured for each slice type, and can be selectively applied on a slice-by-slice / picture-by-CU basis using the methods listed above.

[0396] According to the examples described above, if {1 / 4, 1 / 2, 1, 2, 4, 8}-pel is allowed in normal mode, AmvrShift in Table 6 can be changed as shown in Table 7 below. In this case, the allocation of indexes can be changed.

[0397] [Table 7]

[0398]

[0399] According to the examples described above, if {1 / 16, 1 / 4, 1 / 2, 1, 2}-pel is allowed in the affine mode, the AmvrShift of Table 6 can be changed as shown in Table 8 below. In this case, the allocation of the index can be changed.

[0400] [Table 8]

[0401]

[0402] According to the examples described above, if {1 / 4, 1, 4, 8}-pel is allowed in IBC mode, AmvrShift in Table 6 can be changed as in Table 9 below. In this case, the allocation of indexes can be changed.

[0403] [Table 9]

[0404]

[0405] Additionally, the composition of the AMVR candidate set for each prediction mode according to the example described above can be changed and applied simultaneously, as shown in Table 10 below.

[0406] [Table 10]

[0407]

[0408] According to the above-described embodiment, since the AMVR candidate set is adaptively configured according to the characteristics of the image, a more appropriate AMVR can be selected, thereby improving the compression efficiency of the image without increasing the signaling bits.

[0409] Meanwhile, the configuration of the aforementioned AMVR candidate set can also be applied to different prediction directions. That is, when bi-prediction is applied, the AMVR candidates in the L0 direction and the AMVR candidates in the L1 direction may be different. The method described in this embodiment can be applied to normal inter mode and affine inter mode, and of course, it can also be applied to IBC mode if the IBC mode supports bi-prediction (when two block vectors are allowed).

[0410] Table 11 below shows the bidirectional combinations for each CU when the AMVR candidate set is {1 / 4, 1 / 2, 1, 2, 4, 8}-pel. Various combinations can be applied for each prediction direction. Naturally, the number and order of combinations may vary depending on the number of AMVR candidates allowed and the number of candidates.

[0411] [Table 11]

[0412]

[0413] The AMVR precision index or AMVR index can indicate a combination of AMVR candidates for each direction. For example, some of the candidate combinations may be included in consideration of the signaling bits, and the optimal candidate combination can be selected by considering the resolution of the image, the size of the block, the degree of motion of the adjacent block, and the selectivity in the previous picture as described above. For example, when the resolution of the image is greater than a specific threshold, the AMVR combination (pair) of L0 and L1 may be {(1 / 4, 1 / 4), (1 / 2, 1 / 2), (1, 1), (4, 4), (4, 8), (8, 4)}-Pel, and when it is equal to or less than the threshold, {(1 / 4, 1 / 4), (1 / 2, 1 / 2), (1 / 4, 1 / 2), (1 / 2, 1 / 4), (1, 1), (4, 4)}-Pel. The above set of candidate combinations is an example, and similar methods can be applied to affine mode and IBC mode as well.

[0414] Additionally, it is also possible to use separate AMVR index information to indicate AMVR candidates for each direction. As in the compensator example, when {(1 / 4, 1 / 4), (1 / 2, 1 / 2), (1 / 4, 1 / 2), (1 / 2, 1 / 4), (1, 1), (4, 4)}-Pel is allowed, the AMVR indices assigned are shown in Table 12. The AMVR precision index can be signaled on a per-CU basis, and the AMVR index (index in Table 12 below), which indicates the MV resolution assigned to each prediction direction, can be derived based on the AMVR flag and the AMVR precision index. When applying different AMVR candidates for each prediction direction, an alternative interpolation filter that is only applied to Half-pel (1 / 2-pel) can be applied for each prediction direction.

[0415] [Table 12]

[0416]

[0417] The above AMVR candidate combinations by prediction direction can be applied in the bi-prediction mode, but if the motion vector difference in the L1 direction is 0, only the AMVR candidates separately defined for blocks to which uni-directional prediction is applied can be used. For example, whether the motion vector difference in the L1 direction is 0 can be specified by ph_mvd_l1_zero_flag. If this flag does not exist, the value of ph_mvd_l1_zero_flag is inferred to be 1. If ph_mvd_l1_zero_flag is 1, the mvd_coding( x0, y0, 1, cpIdx ) syntax structure is not parsed, and MvdL1[ x0 ][ y0 ][ compIdx ] and MvdCpL1[ x0 ][ y0 ][ cpIdx ][ compIdx ] are set to 0 for compIdx = 0..1 and cpIdx = 0..2.

[0418] If ph_mvd_l1_zero_flag is 0, the mvd_coding( x0, y0, 1, cpIdx ) syntax structure is parsed.

[0419] In addition, it is possible to derive the AMVR of L1 based on the AMVR of L0, or to derive the AMVR of L0 based on the AMVR of L1. In particular, when all reference pictures are temporally prior to the current picture, the AMVR of L1 can be derived using the signaled AMVR of L0. For example, the AMVR candidate set of L1 can be composed of scaled AMVR candidates in which each AMVR candidate included in the AMVR candidate set of L0 is scaled by the same value. For example, when the AMVR candidate set of L0 is {1 / 4, 1 / 2, 1, 2}-pel, the AMVR of L1 can be composed of candidates that are '<<1', such as {1 / 2, 1, 2, 4}-pel. The scaling method is not limited to the above example, and "<<n" 또는 "> >n" can be used for scaling. In this case, n can be an integer greater than or equal to 1. "<<n"을 이용하여 스케일링할지 또는 "> Whether to scale using ">n" can be determined based on the temporal order of the L0 reference picture and the L1 reference picture, and the value of "n" can be derived based on the distance from the current picture to the L0 reference picture and the L1 reference picture.

[0420] According to the present embodiment, since AMVR can be independently determined for each direction of pair prediction, effective prediction is possible, and the effect of improving compression efficiency can be expected.

[0421] According to the above-described embodiment, AMVR index information can be signaled / parsed to expand AMVR candidates and select AMVRs by prediction direction.

[0422] The AMVR index described above is intended to represent each AMVR candidate within the AMVR candidate set, and can be derived from an AMVR flag and an AMVR precision index. The AMVR index information may include an AMVR precision index, or an AMVR flag and an AMVR precision index. Accordingly, signaling / parsing the AMVR index information may include signaling / parsing an AMVR precision index, or signaling / parsing an AMVR flag and an AMVR precision index.

[0423] For example, AMVR index information can be signaled / parsed if either the motion vector difference in the L0 direction or the motion vector difference in the L1 direction of the current block is not 0.

[0424] Additionally, as described above, when applying one of multiple AMVR candidate sets depending on the characteristics of the image, a set index for indicating each of the multiple AMVR candidate sets can be signaled / parsed.

[0425] Fig. 21 is a flowchart showing a process of parsing AMVR index information in a decoding method according to one embodiment.

[0426] Referring to Fig. 21, if the MVD condition is satisfied as a non-zero value (example of S1100), the set index is parsed (S1110). That is, the set index can be obtained if the motion vector difference of the current block is not 0. If the paired prediction is applied to the current block, the non-zero MVD condition can be satisfied if either the motion vector difference in the L0 direction or the motion vector difference in the L1 direction is not 0. The set index can indicate one AMVR candidate set among two or more AMVR candidate sets. Therefore, the set index can be signaled / parsed when two or more AMVR candidate sets exist.

[0427] When the set index is 0 (S1120-1), AMVR index information for AMVR candidate set 1 is parsed (S1130-1), when the set index is 1 (S1120-2), AMVR index information for AMVR candidate set 2 is parsed (S1130-2), and in the same manner, when the set index is N-1 (N is an integer greater than or equal to 2) (S1120-N), AMVR index information for AMVR candidate set N is parsed (S1130-N). For example, AMVR candidate set 1 may include {1 / 2, 1, 2}-pel, AMVR candidate set 2 may include {4, 8, 16}-pel, and AMVR candidate set N may include {1 / 16, 1 / 8, 1 / 4}-pel. A set index can be used to indicate one of N AMVR candidate sets, and AMVR index information can be used to indicate one AMVR candidate within each AMVR candidate set. In this case, the AMVR index information can be signaled / parsed on a CU basis.

[0428] An AMVR shift value corresponding to the AMVR index information is derived (S1140). For example, if the value of the AMVR flag is 0, an AMVR shift value of 2 can be derived, and if the value of the AMVR flag is 1, an AMVR shift value assigned to the AMVR precision index can be derived. As described above, the derived AMVR shift value can be applied to a motion vector difference and a motion vector predictor or a block vector difference and a block vector predictor to derive a motion vector or a block vector.

[0429] If different AMVR index information is allowed for each prediction direction in a set of N AMVR candidates, AMVR information can be signaled / parsed separately for each L0 direction and L1 direction. That is, when pair prediction is applied, each AMVR index information, rather than a single AMVR index information, can be signaled / parsed for each prediction direction, and vector information for each direction can be derived by applying an AMVR shift value derived from each AMVR index information. Alternatively, as described above, it is also possible for a single AMVR index information to indicate a combination of an L0 direction AMVR candidate and an L1 direction AMVR candidate.

[0430] Fig. 22 is a flowchart showing a process of parsing AMVR index information according to a prediction direction in a decoding method according to one embodiment.

[0431] Referring to Fig. 22, if the non-zero MVD condition is satisfied (example of S1200), it is determined whether the set condition is satisfied (S1210). If either the L0 direction motion vector difference or the L1 direction motion vector difference of the current block is not 0, the non-zero MVD condition is satisfied. If both the L0 direction motion vector difference and the L1 direction motion vector difference of the current block are 0, the non-zero MVD condition is not satisfied and the AMVR index information is not parsed.

[0432] The conditions judged in step S1210 are as follows:

[0433] - interDir == 3 is satisfied. At this time, interDir indicates the prediction direction, and when the value is 1, it means L0 direction prediction, when it is 2, it means L1 direction prediction, and when it is 3, it means bi-prediction. In other words, interDir = 3 can mean that bi-prediction is performed for the current block.

[0434] - ph_mvd_l1_zero_flag == FALSE is satisfied. This flag induces the motion vector difference in the L1 direction to a zero vector, and ph_mvd_l1_zero_flag = FALSE can mean that the motion vector difference in the L1 direction is not 0. If this flag is TRUE, even if pair prediction is performed, only L0 has a non-zero vector, so the AMVR index information of L1 is meaningless. Therefore, when this condition is satisfied, the index pointing to the AMVR candidate of the bidirectional block can be parsed.

[0435] If the given condition is satisfied (yes of S1210), AMVR index information indicating a candidate within the AMVR candidate set for bi-pred can be parsed (S1220), and if the given condition is not satisfied (no of S1210), AMVR index information indicating a candidate within the AMVR candidate set for uni-pred can be parsed (S1230).

[0436] AMVR index information indicating a candidate within an AMVR candidate set for pair prediction can be signaled / parsed on a CU basis, signaled / parsed for each prediction direction, or one AMVR index information can indicate a combination of an L0 direction AMVR candidate and an L1 direction AMVR candidate.

[0437] AMVR index information indicating candidates within the AMVR candidate set for unidirectional prediction can also be signaled / parsed on a per-CU basis.

[0438] An AMVR shift value corresponding to the AMVR index information is derived (S1240). For example, if the value of the AMVR flag is 0, an AMVR shift value of 2 can be derived, and if the value of the AMVR flag is 1, an AMVR shift value assigned to the AMVR precision index can be derived. As described above, the derived AMVR shift value can be applied to a motion vector difference and a motion vector predictor or a block vector difference and a block vector predictor to derive a motion vector or a block vector.

[0439] If different AMVR index information is allowed for each prediction direction in the set of N AMVR candidates, AMVR index information can be signaled / parsed separately for each L0 direction and L1 direction. That is, when pair prediction is applied, each AMVR index information, rather than a single AMVR index information, can be signaled / parsed for each prediction direction. Alternatively, as described above, it is also possible for a single AMVR index information to indicate a combination of L0 direction AMVR candidates and L1 direction AMVR candidates.

[0440] FIG. 23 is a flowchart showing another example of a process for parsing AMVR index information in a decoding method according to one embodiment.

[0441] Referring to Fig. 23, if the non-zero MVD condition is satisfied (example of S1300), it is determined whether the set condition is satisfied (S1310). If either the L0 direction motion vector difference or the L1 direction motion vector difference of the current block is not 0, the non-zero MVD condition is satisfied. If both the L0 direction motion vector difference and the L1 direction motion vector difference of the current block are 0, the non-zero MVD condition is not satisfied and the AMVR index information is not parsed.

[0442] In the example described above, a set index was signaled / parsed to indicate one of multiple AMVR candidate sets. However, in this example, the AMVR candidate set is determined conditionally without signaling / parsing the set index.

[0443] Based on the conditions determined in step S1310, one of N sets of AMVR candidates (where N is an integer greater than or equal to 2) can be derived without signaling / parsing. The conditions may be conditions related to the resolution of the image, the size of the coding unit, the magnitude of the motion vector of an adjacent block, the distance between the current picture and a reference picture, the AMVR candidate selection rate of a picture decoded before the current picture, and the like. Examples thereof are as follows:

[0444] - Different AMVR candidate sets can be derived depending on the resolution of the video.

[0445] - Different AMVR candidate sets can be derived depending on the size of the coding unit.

[0446] - Another set of AMVR candidates can be derived by considering the motion vector magnitudes of adjacent blocks.

[0447] - Different AMVR candidate sets can be derived depending on the distance between the current picture and the reference picture.

[0448] - A different AMVR candidate set can be derived based on the AMVR candidate selection rate of the picture decoded before the current picture.

[0449] An AMVR candidate set can be determined based on a satisfied condition. For example, if condition 0 is satisfied (S1320-1), AMVR index information for AMVR candidate set 1 is parsed (S1330-1), if the set index is 1 (S1320-2), AMVR index information for AMVR candidate set 2 is parsed (S1330-2), and in the same manner, if the set index is N (S1320-N), AMVR index information for AMVR candidate set N is parsed (S1330-N). For example, AMVR candidate set 1 may include {1 / 2, 1, 2}-pel, AMVR candidate set 2 may include {4, 8, 16}-pel, and AMVR candidate set N may include {1 / 16, 1 / 8, 1 / 4}-pel. AMVR index information may be used to indicate one AMVR candidate within each AMVR candidate set. At this time, AMVR index information can be signaled / parsed on a per CU basis.

[0450] An AMVR shift value corresponding to the AMVR index information is derived (S1340). For example, when the value of the AMVR flag is 0, an AMVR shift value 2 can be derived, and when the value of the AMVR flag is 1, an AMVR shift value assigned to the AMVR precision index can be derived. As described above, the derived AMVR shift value can be applied to a motion vector difference and a motion vector predictor or a block vector difference and a block vector predictor to derive a motion vector or a block vector. When paired prediction is applied, each AMVR index information, not a single AMVR index information, can be signaled / parsed for each prediction direction, and vector information for each direction can be derived by applying an AMVR shift value derived from each AMVR index information. Alternatively, as described above, it is also possible for a single AMVR index information to indicate a combination of an L0 direction AMVR candidate and an L1 direction AMVR candidate.

[0451] The explanations above, which focus on motion vectors, can be applied to block vectors in the same manner. Furthermore, the examples above can be applied not only to normal mode, affine mode, and IBC mode, but also to other prediction modes that use motion vectors or block vectors.

[0452] Additionally, the description of parsing information such as AMVR index information and set index among the descriptions above can be equally applied to signaling of the corresponding information.

[0453] Figure 24 is a flowchart illustrating a decoding method according to one embodiment. As described above, various AMVR candidates can be applied to the decoding method according to the embodiment, taking into account the characteristics of the image.

[0454] A decoding method according to one embodiment can be performed by the decoding device (300) described above. In the embodiment described below, descriptions are made focusing on contents not described above to avoid redundant descriptions, and the description described below does not solely support the embodiment of the decoding method. The description of the operation of the decoding device (300) described above, the description of the decoding method (e.g., FIGS. 3, 4, 8 to 10, etc.), and the descriptions of various prediction modes or prediction types (FIGS. 11 to 19) can be equally applied to the embodiment described below as long as they do not conflict with each other.

[0455] Referring to FIG. 24, a decoding method according to one embodiment includes a step of obtaining image information from a bitstream (S1400), a step of deriving vector information for the current block based on the image information (S1410), and a step of generating a prediction block of the current block based on the vector information (S1420).

[0456] The image information acquired in step S1400 includes prediction information and residual information as described above. The prediction information may include prediction mode information indicating the prediction mode applied to the current block and motion information. For example, motion information may be included when inter prediction is applied to the current block. The motion information may include candidate selection information (e.g., merge index, MVP flag, or MVP index), which is information for deriving a motion vector. In addition, the motion information may include information about the MVD described above and / or reference picture index information. In addition, the motion information may include information indicating whether L0 prediction, L1 prediction, or bi-prediction is applied. In addition, motion information may be included in the image information even when inter prediction is not applied to the current block. In this case, the motion information may include information about a block vector used in the IBC mode. For example, information about a block vector used in the IBC mode may include candidate selection information expressed as an index, etc., and information about a block vector differential (BVD).

[0457] In the present embodiment, it is assumed that vector information can refer to both a motion vector and a block vector. That is, the vector information can represent a motion vector, a block vector, or both a motion vector and a block vector. In addition, the vector difference can refer to both a motion vector difference and a block vector difference. That is, the vector difference can represent a motion vector difference, a block vector difference, or both a motion vector difference and a block vector difference.

[0458] Meanwhile, the image information may include AMVR information, and the AMVR information may include AMVR index information. The AMVR index information may include an AMVR precision index (amvr_precision_idx) indicating at least one candidate within an AMVR candidate set, or may include an AMVR flag and an AMVR precision index. Here, the AMVR candidate set may be configured differently depending on the first information and the second information (different from the first information). As described above, the AMVR candidate set may be configured by considering not only the prediction mode but also the characteristics of the image. Here, the information regarding the prediction mode is referred to as the first information, and the information regarding the characteristics of the image is referred to as the second information.

[0459] Information about the characteristics of the image may include information about at least one of the resolution of the image, the size of the coding unit, the motion vector of the current block and the adjacent block, the distance between the current picture and the reference picture, the AMVR candidate selection rate of the picture decoded before the current picture, or the possibility and whether to apply motion compensation on the decoder side to the prediction block in the current picture or the current block.

[0460] The AMVR index information may include L0-direction AMVR index information and L1-direction AMVR index information based on whether bi-prediction is applied to the current block. Alternatively, the AMVR index information may indicate a combination of L0-direction AMVR candidates and L1-direction AMVR candidates based on whether bi-prediction is applied to the current block. Alternatively, it is possible to derive an L1-direction AMVR candidate set from an L0-direction AMVR candidate set based on whether bi-prediction is applied to the current block. Alternatively, it is possible to apply the same AMVR candidate to both the L0-direction and the L1-direction. Alternatively, it is also possible for the AMVR shift value to be derived according to a rule predetermined by the encoding device (200) and the decoding device (300) without signaling / parsing the AMVR index information.

[0461] Additionally, the AMVR information may include a set index indicating one of multiple AMVR candidate sets, as previously described in Figure 21. Alternatively, it is also possible to derive one of multiple AMVR candidate sets based on whether a condition related to the second information is satisfied, as previously described in Figure 23.

[0462] The step of obtaining vector information for the current block (S1410) includes deriving vector information for the current block based on a vector difference for the current block and an Adaptive Motion Vector Resolution (AMVR) candidate set.

[0463] For example, a motion vector for the current block can be derived based on the motion vector differential for the current block and a set of AMVR candidates. Specifically, the motion vector can be derived by summing the motion vector differential and the motion vector predictor. The process of deriving the motion vector predictor for the current block is the same as described above, so its description is omitted here.

[0464] Additionally, a block vector for the current block can be derived based on the block vector differential for the current block and the AMVR candidate set. Specifically, the block vector can be derived by summing the block vector differential and the block vector predictor. The process of deriving the block vector predictor for the current block is similar to the process of deriving the motion vector predictor, and can utilize the block vectors of neighboring blocks of the current block.

[0465] When deriving a motion vector or block vector for the current block, an AMVR candidate included in the AMVR candidate set can be applied. An AMVR candidate can directly or indirectly correspond to an AMVR shift value. That is, an AMVR candidate can represent an AMVR shift value or an MVD resolution. Even when an AMVR candidate represents an MVD resolution, an AMVR shift value corresponding to the MVD resolution can be matched with the AMVR candidate.

[0466] For example, the AMVR candidate applied to the current block can be specified by the AMVR index information, and the AMVR index information can be included in the AMVR information. As described above, the motion vector predictor is rounded according to the AMVR shift value indicated by the AMVR index information, the motion vector difference is scaled, and then the rounded motion vector predictor and the scaled motion vector difference are added to derive a motion vector having the intended precision. Of course, the block vector can also be derived in the same manner.

[0467] The step (S1420) of generating a prediction block of the current block based on vector information includes deriving a reference block based on the previously derived vector information and generating a prediction block based on the derived reference block. Since the description related to the generation of the prediction block overlaps with the previously described content (e.g., the description regarding step S720 of FIG. 8), a detailed description will be omitted here.

[0468] In addition, although not illustrated in FIG. 24, it is to be understood that the decoding method according to one embodiment may further include at least some of the steps of deriving residual samples for a current block based on residual information included in image information, generating reconstructed samples for the current block based on the prediction block and residual samples, and generating a reconstructed picture based thereon. Since the description related thereto overlaps with the previously described content (e.g., FIG. 8), a detailed description thereof will be omitted here.

[0469] Figure 25 is a flowchart illustrating an encoding method according to one embodiment. As described above, various AMVR candidates that take into account the characteristics of the image can be applied to the encoding method according to the embodiment.

[0470] An encoding method according to one embodiment can be performed by the encoding device (200) described above. In the embodiment described below, descriptions are made focusing on contents not described above to avoid redundant descriptions, and the description described below does not solely support the embodiment of the encoding method. The descriptions regarding the operation of the encoding device (200) described above, the descriptions regarding the decoding method (e.g., FIGS. 2, 5, 6, 7, 10, etc.), and the descriptions regarding various prediction modes or prediction types (FIGS. 11 to 19) can be equally applied to the embodiment described below as long as they do not conflict with each other.

[0471] Referring to FIG. 25, an encoding method according to one embodiment includes a step of determining a prediction mode of a current block (S1500), a step of deriving vector information of a current block based on the prediction mode (S1510), a step of deriving a vector difference of the current block based on the vector information of the current block (S1520), and a step of encoding image information including the vector difference and AMVR information of the current block (S1530). The encoded image information may be output in the form of a bitstream and transmitted to a decoding device (300).

[0472] Although not illustrated in FIG. 25, an encoding method according to one embodiment can generate a prediction block of a current block and generate residual information based on the generated prediction block. The generation of the prediction block and the generation of residual information are described above with reference to FIG. 6.

[0473] The step of deriving vector information of the current block (S1510) may include deriving a motion vector or block vector of the current block based on whether the prediction mode of the current block is an inter prediction mode (normal mode or affine mode) or an IBC mode. The motion vector or block vector of the current block may be derived based on a positional difference between a reference picture or a reference block within the current picture and the current block.

[0474] The step of deriving the vector difference of the current block (S1520) may include deriving the vector difference of the current block based on the difference between the motion vector and the motion vector predictor of the current block or the difference between the block vector and the block vector predictor of the current block. Here, the vector difference of the current block may include the motion vector difference or the block vector difference.

[0475] The AMVR information encoded in step S1530 may include AMVR index information. The AMVR index information may include an AMVR precision index indicating at least one AMVR candidate within an AMVR candidate set constructed based on the first information and the second information. Furthermore, the AMVR index information may further include an AMVR flag. The first information is information regarding the prediction mode applied to the current block, and the second information is information different from the first information and is information regarding the characteristics of the image.

[0476] The AMVR index information may include L0-direction AMVR index information and L1-direction AMVR index information based on whether bi-prediction is applied to the current block. Alternatively, the AMVR index information may indicate a combination of L0-direction AMVR candidates and L1-direction AMVR candidates based on whether bi-prediction is applied to the current block. Alternatively, the decoding device (300) may derive the L1-direction AMVR candidate set from the L0-direction AMVR candidate set based on whether bi-prediction is applied to the current block. Alternatively, the same AMVR candidate may be applied to the L0-direction and the L1-direction. Alternatively, the AMVR shift value may be derived according to a rule predetermined by the encoding device (200) and the decoding device (300) without signaling / parsing the AMVR index information.

[0477] Additionally, the AMVR information may include a set index indicating one of multiple AMVR candidate sets, as previously described in FIG. 21. Alternatively, one of multiple AMVR candidate sets may be derived by the decoding device (300) based on whether a condition related to the second information is satisfied. This is as previously described in FIG. 23.

[0478] In addition, the vector difference of the current block encoded in S1530 may be a vector difference scaled by the AMVR shift value of the current block. Specifically, an AMVR candidate specified by the AMVR index information may be equally applied to the encoding process. For example, if the AMVR candidate specified by the AMVR index information corresponds to the AMVR shift value 1, the motion vector difference of the current block is scaled by applying a right shift operation >> 1, and when the scaled motion vector difference is encoded and output in the form of a bitstream, the decoding device (300) scales it by applying a left shift operation << 1 as described in Table 5 above.

[0479] Additionally, it should be noted that the image information encoded in step S1530 may include prediction information and residual information. Details regarding the image information are as described in the preceding embodiment of the decoding method.

[0480] The AMVR described above is applicable to both the decoding method and the encoding method according to one embodiment. That is, the AMVR used in the decoding method and the encoding method according to one embodiment is an AMVR included in the AMVR candidate set configured according to the method described above. The description related to parsing the AMVR index information (e.g., AMVR precision index) disclosed above may be applied to the process of obtaining AMVR information in the decoding method according to one embodiment. The description related to signaling the AMVR index information (e.g., AMVR precision index) disclosed above may be applied to the process of encoding the AMVR information in the encoding method according to one embodiment.

[0481] The bitstream generated by the encoding method described above can be temporarily stored in a computer-readable storage medium. In addition, the bitstream can be generated according to the encoding method described above and then transmitted to a decoding device (300), etc. A transmission device that transmits the bitstream can include at least one processor that generates the bitstream according to the encoding method, and a transmitter that transmits the generated bitstream.

[0482] FIG. 26 is a diagram illustrating an example of a content streaming system to which an embodiment according to the present disclosure can be applied.

[0483] Referring to FIG. 26, a content streaming system to which the embodiment(s) of the present specification are applied may largely include an encoding server, a streaming server, a web server, a media storage, a user device, and a multimedia input device.

[0484] The encoding server compresses content input from multimedia input devices such as smartphones, cameras, and camcorders into digital data, generates a bitstream, and transmits it to the streaming server. Alternatively, if multimedia input devices such as smartphones, cameras, and camcorders directly generate bitstreams, the encoding server may be omitted.

[0485] The above bitstream can be generated by an encoding method or a bitstream generation method to which the embodiment(s) of the present specification are applied, and the streaming server can temporarily store the bitstream during the process of transmitting or receiving the bitstream.

[0486] The streaming server transmits multimedia data to a user device based on a user request via a web server, and the web server acts as an intermediary to inform the user of available services. When a user requests a desired service from the web server, the web server transmits the request to the streaming server, and the streaming server transmits the multimedia data to the user. At this time, the content streaming system may include a separate control server, in which case the control server controls commands / responses between each device within the content streaming system.

[0487] The streaming server can receive content from a media repository and / or an encoding server. For example, when receiving content from the encoding server, the content can be received in real time. In this case, to provide a smooth streaming service, the streaming server can store the bitstream for a certain period of time.

[0488] Examples of the user devices may include mobile phones, smart phones, laptop computers, digital broadcasting terminals, personal digital assistants (PDAs), portable multimedia players (PMPs), navigation devices, slate PCs, tablet PCs, ultrabooks, wearable devices (e.g., smartwatches, smart glasses, HMDs), digital TVs, desktop computers, digital signage, etc.

[0489] Each server within the above content streaming system can be operated as a distributed server, in which case data received from each server can be processed in a distributed manner.

[0490] The claims set forth in this specification may be combined in various ways. For example, the technical features of the method claims of this specification may be combined and implemented as a device, and the technical features of the device claims of this specification may be combined and implemented as a method. Furthermore, the technical features of the method claims and the technical features of the device claims of this specification may be combined and implemented as a device, and the technical features of the method claims and the technical features of the device claims of this specification may be combined and implemented as a method.

[0491] Embodiments according to the present disclosure can be used to encode / decode images.

Claims

1. A step of obtaining image information from a bitstream; A step of deriving vector information for the current block based on vector difference information and AMVR (Adaptive Motion Vector Resolution) information included in the above image information; and A step of generating a prediction block of the current block based on the vector information; The above AMVR information is, Includes AMVR index information indicating at least one AMVR candidate within an AMVR candidate set constructed based on the first information and the second information, The above first information includes information about the prediction mode applied to the current block, A method wherein the second information is different from the first information.

2. In paragraph 1, The vector information for the current block above is: A method comprising at least one of a motion vector for the current block or a block vector for the current block.

3. In paragraph 1, The above second information is, A method comprising information about the characteristics of an image.

4. In paragraph 3, Information about the characteristics of the above video, A method comprising information about at least one of: resolution of an image, size of a coding unit, motion vectors of blocks adjacent to the current block, distance between the current picture and a reference picture, AMVR candidate selection rate of a picture decoded before the current picture, or applicability and application of motion compensation on the decoder side to a prediction block in the current picture or the current block.

5. In paragraph 1, The above AMVR index information is, A method comprising L0 direction AMVR index information and L1 direction AMVR index information based on bi-prediction being applied to the current block.

6. In paragraph 1, The above AMVR index information is, A method for representing a combination of L0-direction AMVR candidates and L1-direction AMVR candidates based on bi-prediction being applied to the current block.

7. In paragraph 1, A method further comprising the step of deriving an AMVR candidate set in the L1 direction from an AMVR candidate set in the L0 direction based on the bi-prediction being applied to the current block.

8. In paragraph 1, The above AMVR information is, A method, wherein the AMVR candidate set includes a set index indicating one AMVR candidate set among the plurality of AMVR candidate sets, based on the number of AMVR candidate sets.

9. In paragraph 1, A method further comprising: a step of deriving one AMVR candidate set from among the plurality of AMVR candidate sets based on whether a condition related to the second information is satisfied, based on the fact that there are multiple AMVR candidate sets.

10. Step of determining the prediction mode of the current block; A step of deriving vector information of the current block based on the above prediction mode; A step of deriving a vector difference of the current block based on the vector information of the current block; and A step of encoding image information including vector difference of the current block and AMVR (Adaptive Motion Vector Resolution) information; The above AMVR information includes AMVR index information indicating at least one AMVR candidate within an AMVR candidate set configured based on the first information and the second information, The above first information includes information about the prediction mode applied to the current block, A method wherein the second information is different from the first information.

11. In paragraph 10, The vector information for the current block above is: A method comprising at least one of a motion vector for the current block or a block vector for the current block.

12. In paragraph 10, The above second information is, A method comprising information about at least one of: resolution of an image, size of a coding unit, motion vectors of blocks adjacent to the current block, distance between the current picture and a reference picture, AMVR candidate selection rate of a picture decoded before the current picture, or applicability and application of motion compensation on the decoder side to a prediction block in the current picture or the current block.

13. In paragraph 10, The above AMVR index information is, A method comprising L0 direction AMVR index information and L1 direction AMVR index information based on bi-prediction being applied to the current block.

14. In paragraph 10, The above AMVR index information is, A method for indicating a combination of L0-direction AMVR candidates and L1-direction AMVR candidates based on bi-prediction being applied to the current block.

15. In paragraph 10, The above AMVR information is, A method, wherein the AMVR candidate set includes a set index indicating one AMVR candidate set among the plurality of AMVR candidate sets, based on the number of AMVR candidate sets.

16. In a computer-readable storage medium that non-temporarily stores a bitstream generated by an encoding method, The above encoding method is, A step of determining the prediction mode of the current block; A step of deriving vector information of the current block based on the above prediction mode; A step of deriving a vector difference of the current block based on the vector information of the current block; and A step of encoding image information including vector difference of the current block and AMVR (Adaptive Motion Vector Resolution) information; The above AMVR information includes AMVR index information indicating at least one AMVR candidate within an AMVR candidate set configured based on the first information and the second information, The above first information includes information about the prediction mode applied to the current block, A storage medium in which the second information is different from the first information.

17. A step of generating a bitstream regarding image information; and A step of transmitting data including the bitstream; The above bitstream is, It is generated by a step of determining a prediction mode of a current block, a step of deriving vector information of the current block based on the prediction mode, a step of deriving a vector difference of the current block based on the vector information of the current block, and a step of encoding image information including the vector difference of the current block and AMVR (Adaptive Motion Vector Resolution) information. The above AMVR information includes AMVR index information indicating at least one AMVR candidate within an AMVR candidate set configured based on the first information and the second information, The above first information includes information about the prediction mode applied to the current block, A method wherein the second information is different from the first information.

Images