Method and computer-readable storage medium

Abstract

A method according to an embodiment comprises the steps of: acquiring image information from a bitstream; configuring a motion vector predictor (MVP) candidate list including MVPs for a current block on the basis of the image information; and generating a prediction block for the current block on the basis of at least one MVP candidate in the MVP candidate list, wherein the step of configuring the MVP candidate list includes adding motion information stored in a history-based motion vector predictor (HMVP) buffer to the MVP candidate list, a predetermined number of pieces of motion information of a decoded block are stored in the HMVP buffer, and when a predetermined condition is satisfied, the stored motion information is deleted and default motion information is stored.

Description

Method and computer-readable storage medium

[0001] The present disclosure relates to a method for decoding and encoding image information, a computer-readable storage medium for storing image information, and a method for transmitting image information.

[0002] Recently, the demand for high-resolution, high-quality images, such as HD (High Definition) images and UHD (Ultra High Definition) images, is increasing in various application fields, and accordingly, high-efficiency image compression technologies are being discussed.

[0003] There are various technologies for image compression, such as inter prediction technology that predicts pixel values ​​included in the current picture from pictures before or after the current picture, intra prediction technology that predicts pixel values ​​included in the current picture using pixel information within the current picture, and entropy coding technology that assigns short codes to values ​​with high frequency of appearance and long codes to values ​​with low frequency of appearance, and these technologies can be used to effectively compress and transmit or store image data.

[0004] Accordingly, a highly efficient image compression technology is required to effectively transmit, store, and play high-resolution, high-quality image information.

[0005] The present disclosure provides a method for improving compression performance by using default motion information as an HMVP candidate when initializing motion information stored in an HMVP buffer.

[0006] According to one embodiment, a method includes the steps of: obtaining image information from a bitstream; constructing an MVP candidate list including MVPs (Motion Vector Predictors) for a current block based on the image information; and generating a prediction block for the current block based on at least one MVP candidate in the MVP candidate list; wherein the step of constructing the MVP candidate list includes adding motion information stored in an HMVP (History-based Motion Vector Predictor) buffer to the MVP candidate list, and a predetermined number of motion information of a decoded block is stored in the HMVP buffer, and when a predetermined condition is satisfied, the stored motion information is deleted and default motion information is stored.

[0007] According to one embodiment, a method includes the steps of: determining a prediction mode of a current block; constructing an MVP candidate list including MVP candidates for the current block based on the prediction mode; and generating residual samples for the current block based on at least one MVP candidate in the MVP candidate list; wherein the step of constructing the MVP candidate list includes adding motion information stored in a History-based Motion Vector Predictor (HMVP) buffer to the MVP candidate list, and a predetermined number of motion information of a decoded block is stored in the HMVP buffer, and when a predetermined condition is satisfied, the stored motion information is deleted and default motion information is stored.

[0008] In one embodiment, a computer-readable storage medium storing a bitstream generated by an encoding method, the encoding method comprising: determining a prediction mode of a current block; constructing an MVP candidate list including MVP candidates for the current block based on the prediction mode; and generating residual samples for the current block based on at least one MVP candidate in the MVP candidate list; wherein the constructing the MVP candidate list includes adding motion information stored in an HMVP (History-based Motion Vector Predictor) buffer to the MVP candidate list, and a predetermined number of motion information of a decoded block is stored in the HMVP buffer, and when a predetermined condition is satisfied, the stored motion information is deleted and default motion information is stored.

[0009] According to one embodiment, a method for transmitting data for an image includes: obtaining a bitstream for the image, wherein the bitstream is generated based on the steps of: determining a prediction mode of a current block; constructing an MVP candidate list including MVP candidates for the current block based on the prediction mode; and generating residual samples for the current block based on at least one MVP candidate in the MVP candidate list; and transmitting the data including the bitstream; wherein the step of constructing the MVP candidate list includes adding motion information stored in an HMVP (History-based Motion Vector Predictor) buffer to the MVP candidate list, and wherein the HMVP buffer stores a predetermined number of motion information of a decoded block, and when a predetermined condition is satisfied, the stored motion information is deleted and default motion information is stored.

[0010] According to one embodiment, compression performance can be improved by using default motion information as an HMVP candidate when initializing motion information stored in an HMVP buffer.

[0011] The effects that can be obtained from the present disclosure are not limited to the effects mentioned above, and other effects that are not mentioned will be clearly understood by a person having ordinary skill in the art to which the present disclosure pertains from the description below.

[0012] FIG. 1 illustrates a video / image coding system applicable to one embodiment.

[0013] FIG. 2 is a schematic block diagram of an encoding device to which one embodiment can be applied and in which encoding of a video / image signal is performed.

[0014] FIG. 3 is a schematic block diagram of a decoding device to which one embodiment can be applied and in which decoding of a video / image signal is performed.

[0015] Figure 4 illustrates an example of a video / image decoding method to which one embodiment can be applied.

[0016] Figure 5 illustrates an example of a video / image encoding method to which one embodiment may be applied.

[0017] Figures 6 and 7 illustrate examples of an inter prediction based video / image encoding method to which one embodiment can be applied.

[0018] Figures 8 and 9 illustrate examples of an inter prediction based video / image decoding method to which one embodiment can be applied.

[0019] Figure 10 illustrates an example of an inter prediction procedure to which one embodiment can be applied.

[0020] FIG. 11 is a diagram showing examples of blocks used as spatial merge candidates in an encoding method or a decoding method according to one embodiment.

[0021] Fig. 12 is a flowchart illustrating an example of a process for constructing a merge candidate list in an encoding method or a decoding method according to one embodiment.

[0022] Figure 13 is a diagram showing four movements expressed in the affine movement model.

[0023] Figure 14 is a diagram showing motion vectors used in affine motion prediction.

[0024] Figure 15 is a diagram for a case where the affine MVF is determined in 4*4 subblock units.

[0025] Figure 16 is a diagram showing the inheritance of control point motion vectors.

[0026] Figure 17 is a diagram showing an example of the location of a candidate for the configured affine merge mode.

[0027] Figure 18 shows the procedure of SbTMVP.

[0028] Figure 19 illustrates GPM segments grouped at the same angle as an example.

[0029] Figure 20 illustrates neighboring blocks used in CIIP weight derivation.

[0030] Figure 21 illustrates available IPM candidates for GPM including inter and intra prediction.

[0031] Figure 22 is a diagram illustrating a method for deriving a motion vector from template matching.

[0032] Figure 23 is a diagram showing padding candidates for replacing zero vectors in the IBC list.

[0033] Figure 24 is a diagram showing an example of another IBC reference area at the current CU location.

[0034] Figure 25 is a diagram showing an example of a reference area when a CTU is coded.

[0035] Fig. 26 is a flowchart illustrating an example of a decoding method according to one embodiment.

[0036] Fig. 27 is a flowchart illustrating an encoding method according to one embodiment.

[0037] Figure 28 is a diagram showing the distance between the current picture and reference pictures in each direction.

[0038] Figure 29 is a drawing showing an example of a method for applying scaling values ​​by section.

[0039] Figure 30 is a diagram showing an example of a combination of signs that a motion vector of a decoded picture can have.

[0040] FIG. 31 is a diagram illustrating an example of a content streaming system to which an embodiment according to the present disclosure can be applied.

[0041] The present disclosure may be subject to various modifications and embodiments. Therefore, specific embodiments are illustrated and described in detail in the drawings. However, this is not intended to limit the present disclosure to specific embodiments, but rather to encompass all modifications, equivalents, and alternatives falling within the spirit and technical scope of the present disclosure. Similar reference numerals have been used to designate similar components throughout the description of each drawing.

[0042] While terms such as "first" and "second" may be used to describe various components, these components should not be limited by these terms. These terms are used solely to distinguish one component from another. For example, without departing from the scope of the present disclosure, a first component could be referred to as a "second component," and similarly, a second component could also be referred to as a "first component." The term "and / or" includes a combination of multiple related items described herein or any of multiple related items described herein.

[0043] When a component is referred to as being "connected" or "connected" to another component, it should be understood that it may be directly connected or connected to that other component, but that there may be other components intervening. Conversely, when a component is referred to as being "directly connected" or "connected" to another component, it should be understood that there are no other components intervening.

[0044] The terminology used in this application is only used to describe specific embodiments and is not intended to limit the present disclosure. The singular expression includes the plural expression unless the context clearly indicates otherwise. In this application, it should be understood that the terms "comprise" or "have" indicate the presence of a feature, number, step, operation, component, part, or combination thereof described in the specification, but do not preclude the possibility of the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

[0045] The present disclosure relates to video / image coding. For example, the methods / embodiments disclosed herein may be applied to methods disclosed in various video / image coding standards.

[0046] This specification discloses various embodiments or examples relating to video / image coding, and unless otherwise stated, the disclosed embodiments or examples can be performed in combination with each other.

[0047] In this specification, video may refer to a set of images over time. A picture generally refers to a unit representing one image at a specific time point, and a slice / tile is a unit that constitutes part of a picture in coding. A slice / tile may include one or more coding tree units (CTUs). A picture may be composed of one or more slices / tiles. A tile is a rectangular area consisting of multiple CTUs within a specific tile column and a specific tile row of a picture. A tile column is a rectangular area of ​​CTUs that has a height equal to the height of the picture and a width specified by the syntax requirements of the picture parameter set. A tile row is a rectangular area of ​​CTUs that has a height specified by the picture parameter set and a width equal to the width of the picture. CTUs within a tile are arranged consecutively according to the CTU raster scan, while tiles within a picture may be arranged consecutively according to the tile raster scan. A slice may contain an integer number of complete tiles or an integer number of contiguous complete CTU rows within a picture, which may be exclusively contained within a single NAL unit. Meanwhile, a picture may be divided into two or more subpictures. A subpicture may be a rectangular region of one or more slices within a picture.

[0048] A pixel, or pel, can refer to the smallest unit that constitutes a picture (or image). Additionally, the term "sample" can be used as a counterpart to a pixel. A sample can generally represent a pixel or a pixel value, and can represent only the pixel / pixel value of the luminance component, or only the pixel / pixel value of the chrominance component.

[0049] A unit may represent a basic unit of image processing. A unit may include at least one of a specific region of a picture and information related to the region. One unit may include one luma block and two chroma (e.g., cb, cr) blocks. In some cases, the term "unit" may be used interchangeably with terms such as "block" or "area." In general, an MxN block may include a set (or array) of samples (or sample array) or transform coefficients consisting of M columns and N rows.

[0050] In this specification, “A or B” can mean “only A,” “only B,” or “both A and B.” In other words, “A or B” in this specification can be interpreted as “A and / or B.” For example, “A, B or C” in this specification can mean “only A,” “only B,” “only C,” or “any combination of A, B, and C.”

[0051] As used herein, a slash ( / ) or a comma can mean "and / or." For example, "A / B" can mean "A and / or B." Accordingly, "A / B" can mean "only A," "only B," or "both A and B." For example, "A, B, C" can mean "A, B, or C."

[0052] In this specification, “at least one of A and B” may mean “only A,” “only B,” or “both A and B.” Additionally, in this specification, the expressions “at least one of A or B” or “at least one of A and / or B” may be interpreted identically to “at least one of A and B.”

[0053] Additionally, in this specification, “at least one of A, B and C” can mean “only A,” “only B,” “only C,” or “any combination of A, B and C.” Additionally, “at least one of A, B or C” or “at least one of A, B and / or C” can mean “at least one of A, B and C.”

[0054] Additionally, parentheses used herein may mean "for example." Specifically, when "prediction (intra-prediction)" is indicated, "intra-prediction" may be suggested as an example of "prediction." In other words, "prediction" in this specification is not limited to "intra-prediction," and "intra-prediction" may be suggested as an example of "prediction." Furthermore, even when "prediction (i.e., intra-prediction)" is indicated, "intra-prediction" may be suggested as an example of "prediction."

[0055] Technical features individually described in a single drawing in this specification may be implemented individually or simultaneously.

[0056] FIG. 1 illustrates a video / image coding system applicable to one embodiment.

[0057] Referring to FIG. 1, a video / image coding system may include a first device (source device) and a second device (receiving device).

[0058] A source device can transmit encoded video / image information or data to a receiving device via a digital storage medium or a network in the form of a file or streaming. The source device may include a video source, an encoding device, and a transmitting device. The receiving device may include a receiving device, a decoding device, and a renderer. The encoding device may be referred to as a video / image encoding device, and the decoding device may be referred to as a video / image decoding device. The transmitter may be included in the encoding device. The receiver may be included in the decoding device. The renderer may include a display unit, and the display unit may be configured as a separate device or an external component.

[0059] A video source may obtain video / images through a process of capturing, synthesizing, or generating video / images. The video source may include a video / image capture device and / or a video / image generation device. The video / image capture device may include one or more cameras, a video / image archive containing previously captured video / images, etc. The video / image generation device may include a computer, a tablet, a smartphone, etc., and may (electronically) generate video / images. For example, a virtual video / image may be generated through a computer, etc., in which case the video / image capture process may be replaced by a process of generating related data.

[0060] An encoding device can encode input video / images. The encoding device can perform a series of procedures, such as prediction, transformation, and quantization, to improve compression and coding efficiency. The encoded data (encoded video / image information) can be output in the form of a bitstream.

[0061] The transmission unit can transmit encoded video / image information or data output in the form of a bitstream to the receiving unit of a receiving device via a digital storage medium or a network in the form of a file or streaming. The digital storage medium can include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. The storage medium can be a computer-readable storage medium and can store data non-transitory. The transmission unit can include an element for generating a media file via a predetermined file format and an element for transmission via a broadcasting / communication network. The receiving unit can receive / extract the bitstream and transmit it to a decoding device.

[0062] The decoding device can decode the video / image by performing a series of procedures such as inverse quantization, inverse transformation, and prediction corresponding to the operation of the encoding device.

[0063] The renderer can render decoded video / images. The rendered video / images can be displayed through the display unit.

[0064] FIG. 2 is a schematic block diagram of an encoding device to which one embodiment can be applied and in which encoding of a video / image signal is performed.

[0065] Referring to FIG. 2, the encoding device (200) may be configured to include an image partitioner (210), a prediction unit (predictor) 220, a residual processor (residual processor) 230, an entropy encoder (entropy encoder) 240, an adder (adder) 250, a filter (filter) 260, and a memory (memory) 270. The prediction unit (220) may include an inter prediction unit (221) and an intra prediction unit (222). The residual processor (230) may include a transformer (transformer) 232, a quantizer (quantizer) 233, a dequantizer (dequantizer) 234, and an inverse transformer (inverse transformer) 235. The residual processing unit (230) may further include a subtractor (231). The addition unit (250) may be called a reconstructor or a recontructed block generator. The image segmentation unit (210), the prediction unit (220), the residual processing unit (230), the entropy encoding unit (240), the addition unit (250), and the filtering unit (260) described above may be configured by one or more hardware components (e.g., an encoding device chipset or processor) according to an embodiment. In addition, the memory (270) may include a decoded picture buffer (DPB) and may be configured by a digital storage medium. The hardware component may further include the memory (270) as an internal / external component.

[0066] The encoding device (200) may include at least one processor and at least one memory. At least one processor may perform the operations of the encoding device (200) described below, and at least one memory may be electrically connected to at least one processor and store programs and various data necessary to perform the operations of the encoding device (200).

[0067] The image segmentation unit (210) can segment an input image (or picture, frame) input to the encoding device (200) into one or more processing units (PUs). For example, the processing units may be called coding units (CUs). In this case, the coding units may be recursively segmented from a coding tree unit (CTU) or a largest coding unit (LCU) according to a QTBTTT (Quad-Tree Binary-Tree Ternary-Tree) structure.

[0068] For example, a single coding unit may be split into multiple coding units with deeper depths based on a quad-tree structure, a binary tree structure, and / or a ternary structure. In this case, for example, the quad-tree structure may be applied first, and the binary tree structure and / or the ternary structure may be applied later. Alternatively, the binary tree structure may be applied before the quad-tree structure. The coding procedure according to the present specification may be performed based on the final coding unit that is no longer split. In this case, based on coding efficiency according to image characteristics, etc., the largest coding unit may be used directly as the final coding unit, or, if necessary, the coding unit may be recursively split into coding units of lower depths, and the coding unit with the optimal size may be used as the final coding unit. Here, the coding procedure may include procedures such as prediction, transformation, and restoration, which will be described later.

[0069] As another example, the processing unit may further include a prediction unit (PU) or a transform unit (TU). In this case, the prediction unit and the transform unit may each be split or partitioned from the final coding unit described above. The prediction unit may be a unit of sample prediction, and the transform unit may be a unit for deriving a transform coefficient and / or a unit for deriving a residual signal from a transform coefficient.

[0070] The term "unit" may be used interchangeably with terms such as "block" or "area" depending on the case. In general, an MxN block can represent a set of samples or transform coefficients consisting of M columns and N rows. A sample can generally represent a pixel or a pixel value, and can represent only a pixel / pixel value of a luminance component or only a pixel / pixel value of a chrominance component. A sample can be used as a term corresponding to a pixel or pel of a picture (or image).

[0071] The encoding device (200) can generate a residual signal (residual block, residual sample array) by subtracting a prediction signal (prediction block, prediction sample array) output from an inter prediction unit (221) or an intra prediction unit (222) from an input video signal (original block, original sample array), and the generated residual signal is transmitted to a conversion unit (232). In this case, a unit that subtracts a prediction signal (prediction block, prediction sample array) from an input video signal (original block, original sample array) within the encoding device (200) may be called a subtraction unit (231).

[0072] The prediction unit (220) can perform prediction on a block to be processed (hereinafter, referred to as a current block) to generate prediction samples, and can generate a predicted block including the prediction samples for the current block. The prediction unit (220) can determine whether intra prediction or inter prediction is applied to the current block or CU unit. The prediction unit (220) can generate various information regarding prediction, such as prediction mode information, as described later in the description of each prediction mode, and transmit the information to the entropy encoding unit (240). The information regarding prediction can be encoded by the entropy encoding unit (240) and output in the form of a bitstream.

[0073] The intra prediction unit (222) can predict the current block by referring to samples in the current picture. The referenced samples, i.e., the reference samples, may be located in the neighborhood of the current block or may be located a certain distance away from the current block depending on the prediction mode. In intra prediction, the prediction modes may include one or more non-directional modes and multiple directional modes. The non-directional mode may include at least one of the DC mode or the planar mode. The directional mode may include 33 directional modes or 65 directional modes depending on the degree of detail in the prediction direction. However, this is only an example, and a greater or lesser number of directional modes may be used depending on the settings. The intra prediction unit (222) may also determine the prediction mode applied to the current block by using the prediction mode applied to the neighboring blocks.

[0074] The inter prediction unit (221) can derive a prediction block for the current block based on a reference block (reference sample array) specified by a motion vector on a reference picture. At this time, in order to reduce the amount of motion information transmitted in the inter prediction mode, the motion information can be predicted in units of blocks, sub-blocks, or samples based on the correlation of the motion information between the neighboring blocks and the current block. The motion information can include a motion vector and a reference picture index. The motion information can further include inter prediction direction information (L0 prediction, L1 prediction, Bi prediction, etc.). In the case of inter prediction, the neighboring block can include a spatial neighboring block existing in the current picture and a temporal neighboring block existing in the reference picture. The reference picture including the reference block and the reference picture including the temporal neighboring block may be the same or different. Temporal neighboring blocks may be called collocated reference blocks, collocated CUs (colCUs), etc., and reference pictures including temporal neighboring blocks may be called collocated pictures (colPic). For example, the inter prediction unit (221) may construct a motion information candidate list based on neighboring blocks, and generate information indicating which candidate is used to derive the motion vector and / or reference picture index of the current block. Inter prediction may be performed based on various prediction modes, and for example, in the case of skip mode and merge mode, the inter prediction unit (221) may use the motion information of neighboring blocks as the motion information of the current block. In the case of skip mode, unlike the merge mode, a residual signal may not be transmitted.In the motion vector prediction (MVP) mode, the motion vector of the surrounding blocks is used as a motion vector predictor, and the motion vector of the current block can be indicated by signaling the motion vector difference.

[0075] The prediction unit (220) can generate a prediction signal based on various prediction methods described below. For example, the prediction unit can apply intra prediction or inter prediction for prediction of a single block, and can also apply intra prediction and inter prediction simultaneously. This can be called combined inter and intra prediction (CIIP) mode. In addition, the prediction unit can be based on an intra block copy (IBC) prediction mode or a palette mode for prediction of a block. The IBC prediction mode or palette mode can be used for content image / video coding such as games, such as screen content coding (SCC). IBC basically performs prediction within the current picture, but can be performed similarly to inter prediction in that it derives a reference block within the current picture. That is, IBC can utilize at least one of the inter prediction techniques described herein. Palette mode can be viewed as an example of intra coding or intra prediction. When the palette mode is applied, sample values ​​within a picture can be signaled based on information about the palette table and palette index. The prediction signal generated through the prediction unit (220) can be used to generate a restoration signal or a residual signal.

[0076] The transform unit (232) can apply a transform technique to the residual signal to generate transform coefficients. For example, the transform technique can include at least one of a Discrete Cosine Transform (DCT), a Discrete Sine Transform (DST), a Karhunen-Loeve Transform (KLT), a Graph-Based Transform (GBT), or a Conditionally Non-linear Transform (CNT). Here, GBT refers to a transform obtained from a graph when the relationship information between pixels is expressed as a graph. CNT refers to a transform obtained based on generating a prediction signal using all previously restored pixels. In addition, the transform process can be applied to a pixel block having a square size and the same size, or can be applied to a block of a non-square variable size.

[0077] The quantization unit (233) quantizes the transform coefficients and transmits them to the entropy encoding unit (240), and the entropy encoding unit (240) can encode the quantized signal (information about the quantized transform coefficients) and output it as a bitstream. The information about the quantized transform coefficients can be called residual information. The quantization unit (233) can rearrange the quantized transform coefficients in a block form into a one-dimensional vector form based on the coefficient scan order, and can also generate information about the quantized transform coefficients based on the quantized transform coefficients in the one-dimensional vector form.

[0078] The entropy encoding unit (240) can perform various encoding methods such as exponential Golomb, context-adaptive variable length coding (CAVLC), context-adaptive binary arithmetic coding (CABAC), etc. The entropy encoding unit (240) can also encode information necessary for video / image restoration (e.g., values ​​of syntax elements, etc.) together or separately from quantized transform coefficients.

[0079] Encoded information (e.g., encoded video / image information) can be transmitted or stored in the form of a bitstream in units of NAL (network abstraction layer) units. The video / image information may further include information on various parameter sets, such as an adaptation parameter set (APS), a picture parameter set (PPS), a sequence parameter set (SPS), or a video parameter set (VPS). In addition, the video / image information may further include general constraint information. In the present specification, information and / or syntax elements transmitted / signaled from an encoding device to a decoding device may be included in the video / image information. The video / image information may be encoded through the above-described encoding procedure and included in the bitstream. The bitstream may be transmitted via a network or stored in a digital storage medium. Here, the network may include a broadcasting network and / or a communication network, and the digital storage medium may include various storage media, such as a USB, SD, CD, DVD, Blu-ray, HDD, or SSD. A transmission unit (not shown) for transmitting a signal output from an entropy encoding unit (240) and / or a storage unit (not shown) for storing the signal may be configured as an internal / external element of the encoding device (200), or the transmission unit may be included in the entropy encoding unit (240).

[0080] The quantized transform coefficients output from the quantization unit (233) can be used to generate a prediction signal. For example, by applying inverse quantization and inverse transformation to the quantized transform coefficients through the inverse quantization unit (234) and the inverse transform unit (235), a residual signal (residual block or residual samples) can be reconstructed. The addition unit (250) can generate a reconstructed signal (reconstructed picture, reconstructed block, reconstructed sample array) by adding the reconstructed residual signal to the prediction signal output from the inter prediction unit (221) or the intra prediction unit (222). When there is no residual for the block to be processed, such as when skip mode is applied, the predicted block can be used as a reconstructed block. The addition unit (250) may be called a reconstructor or a reconstructed block generation unit. The generated restoration signal can be used for intra prediction of the next processing target block within the current picture, and can also be used for inter prediction of the next picture after filtering as described below. Meanwhile, LMCS (luma mapping with chroma scaling) may be applied during the picture encoding and / or restoration process.

[0081] The filtering unit (260) can improve subjective / objective picture quality by applying filtering to the restoration signal. For example, the filtering unit (260) can apply various filtering methods to the restoration picture to generate a modified restoration picture, and store the modified restoration picture in the memory (270), specifically, in the DPB of the memory (270). The various filtering methods can include deblocking filtering, sample adaptive offset, adaptive loop filter, bilateral filter, etc. The filtering unit (260) can generate various information regarding filtering and transmit it to the entropy encoding unit (240). The information regarding filtering can be encoded by the entropy encoding unit (240) and output in the form of a bitstream.

[0082] The modified restored picture transmitted to the memory (270) can be used as a reference picture in the inter prediction unit (221). Through this, when inter prediction is applied, the encoding device can avoid prediction mismatch between the encoding device (200) and the decoding device, and can also improve encoding efficiency.

[0083] The DPB of the memory (270) can store the modified restored picture to be used as a reference picture in the inter prediction unit (221). The memory (270) can store motion information of a block from which motion information in the current picture is derived (or encoded) and / or motion information of blocks in a picture that has already been restored. The stored motion information can be transferred to the inter prediction unit (221) to be used as motion information of a spatial neighboring block or motion information of a temporal neighboring block. The memory (270) can store restored samples of restored blocks in the current picture and transfer them to the intra prediction unit (222).

[0084] Image information output in the form of a bitstream from the encoding device (200) can be transmitted to the decoding device (300).

[0085] FIG. 3 is a schematic block diagram of a decoding device to which one embodiment can be applied and in which decoding of a video / image signal is performed.

[0086] Image information transmitted in the form of a bitstream from the encoding device (200) can be received by the decoding device (300).

[0087] Referring to FIG. 3, the decoding device (300) may be configured to include an entropy decoder (310), a residual processor (320), a predictor (330), an adder (340), a filter (350), and a memory (360). The predictor (330) may include an inter-prediction unit (332) and an intra-prediction unit (331). The residual processor (320) may include a dequantizer (321) and an inverse transformer (321).

[0088] The entropy decoding unit (310), residual processing unit (320), prediction unit (330), addition unit (340), and filtering unit (350) described above may be configured by a single hardware component (e.g., a decoding device chipset or processor) depending on the embodiment. In addition, the memory (360) may include a decoded picture buffer (DPB) and may be configured by a digital storage medium. The hardware component may further include the memory (360) as an internal / external component.

[0089] The decoding device (300) may include at least one processor and at least one memory. At least one processor may perform the operations of the encoding device (300) described below, and at least one memory may be electrically connected to at least one processor and store programs and various data necessary to perform the operations of the encoding device (300).

[0090] When a bitstream including video / image information is input, the decoding device (300) can restore the image corresponding to the process in which the video / image information is processed in the encoding device of FIG. 2. For example, the decoding device (300) can derive units / blocks based on block division related information obtained from the bitstream. The decoding device (300) can perform decoding using a processing unit applied in the encoding device. Accordingly, the processing unit of decoding may be a coding unit, and the coding unit may be divided from a coding tree unit or a maximum coding unit according to a quad tree structure, a binary tree structure, and / or a ternary tree structure. One or more transform units may be derived from the coding unit. Then, the restored image signal decoded and output by the decoding device (300) can be reproduced through a reproduction device.

[0091] The decoding device (300) can receive a signal output from the encoding device of FIG. 2 in the form of a bitstream, and the received signal can be decoded through the entropy decoding unit (310). For example, the entropy decoding unit (310) can parse the bitstream to derive information (e.g., video / image information) necessary for image restoration (or picture restoration). The video / image information may further include information on various parameter sets, such as an adaptation parameter set (APS), a picture parameter set (PPS), a sequence parameter set (SPS), or a video parameter set (VPS). In addition, the video / image information may further include general constraint information. The decoding device can decode the picture further based on the information on the parameter set and / or the general constraint information. The signaling / received information and / or syntax elements described later in this specification can be decoded and obtained from the bitstream through the decoding procedure. For example, the entropy decoding unit (310) can decode information in a bitstream based on a coding method such as exponential Golomb coding, CAVLC, or CABAC, and output the values ​​of syntax elements required for image restoration and the quantized values ​​of transform coefficients for residuals. More specifically, the CABAC entropy decoding method receives a bin corresponding to each syntax element in the bitstream, determines a context model using information of the syntax element to be decoded and decoding information of the surrounding and decoding target blocks or information of symbols / bins decoded in the previous step, and predicts the occurrence probability of the bin according to the determined context model to perform arithmetic decoding of the bin to generate a symbol corresponding to the value of each syntax element.At this time, the CABAC entropy decoding method can update the context model using the information of the decoded symbol / bin for the context model of the next symbol / bin after determining the context model. Information regarding prediction among the information decoded by the entropy decoding unit (310) is provided to the prediction unit (inter prediction unit (332) and intra prediction unit (331)), and residual values ​​on which entropy decoding is performed by the entropy decoding unit (310), i.e., quantized transform coefficients and related parameter information, can be input to the residual processing unit (320). The residual processing unit (320) can derive a residual signal (residual block, residual samples, residual sample array). In addition, information regarding filtering among the information decoded by the entropy decoding unit (310) can be provided to the filtering unit (350). Meanwhile, a receiving unit (not shown) that receives a signal output from an encoding device may be further configured as an internal / external element of a decoding device (300), or the receiving unit may be a component of an entropy decoding unit (310).

[0092] Meanwhile, a decoding device according to the present specification may be called a video / video / picture decoding device, and the decoding device may be divided into an information decoding device (video / video / picture information decoding device) and a sample decoding device (video / video / picture sample decoding device). The information decoding device may include the entropy decoding unit (310), and the sample decoding device may include at least one of the inverse quantization unit (321), the inverse transformation unit (322), the addition unit (340), the filtering unit (350), the memory (360), the inter prediction unit (332), and the intra prediction unit (331).

[0093] The inverse quantization unit (321) can inverse quantize the quantized transform coefficients and output the transform coefficients. The inverse quantization unit (321) can rearrange the quantized transform coefficients into a two-dimensional block form. In this case, the rearrangement can be performed based on the coefficient scanning order performed in the encoding device. The inverse quantization unit (321) can perform inverse quantization on the quantized transform coefficients using quantization parameters (e.g., quantization step size information) and obtain transform coefficients.

[0094] In the inverse transform unit (322), the transform coefficients are inversely transformed to obtain a residual signal (residual block, residual sample array).

[0095] The prediction unit (320) can perform a prediction on the current block and generate a predicted block including prediction samples for the current block. The prediction unit (320) can determine whether intra-prediction or inter-prediction is applied to the current block based on the information regarding the prediction output from the entropy decoding unit (310), and can determine a specific intra / inter-prediction mode.

[0096] The prediction unit (320) can generate a prediction signal based on various prediction methods described below. For example, the prediction unit (320) can apply intra prediction or inter prediction for prediction of a single block, and can also apply intra prediction and inter prediction simultaneously. This can be called combined inter and intra prediction (CIIP) mode. In addition, the prediction unit can be based on an intra block copy (IBC) prediction mode or a palette mode for prediction of a block. The IBC prediction mode or palette mode can be used for content image / video coding such as games, such as screen content coding (SCC). IBC basically performs prediction within the current picture, but can be performed similarly to inter prediction in that it derives a reference block within the current picture. That is, IBC can utilize at least one of the inter prediction techniques described herein. The palette mode can be viewed as an example of intra coding or intra prediction. When palette mode is applied, information about the palette table and palette index may be signaled and included in the video / image information.

[0097] The intra prediction unit (331) can predict the current block by referring to samples within the current picture. The referenced samples may be located in the neighborhood of the current block, or may be located a certain distance away from the current block, depending on the prediction mode. In intra prediction, the prediction modes may include one or more non-directional modes and multiple directional modes. The intra prediction unit (331) may also determine the prediction mode applied to the current block by using the prediction mode applied to the neighboring blocks.

[0098] The inter prediction unit (332) can derive a prediction block for the current block based on a reference block (reference sample array) specified by a motion vector on a reference picture. At this time, in order to reduce the amount of motion information transmitted in the inter prediction mode, the motion information can be predicted in units of blocks, sub-blocks, or samples based on the correlation of the motion information between the neighboring blocks and the current block. The motion information can include a motion vector and a reference picture index. The motion information can further include inter prediction direction information (L0 prediction, L1 prediction, Bi prediction, etc.). In the case of inter prediction, the neighboring blocks can include spatial neighboring blocks existing in the current picture and temporal neighboring blocks existing in the reference picture. For example, the inter prediction unit (332) can construct a motion information candidate list based on the neighboring blocks, and derive the motion vector and / or reference picture index of the current block based on the received candidate selection information. Inter prediction can be performed based on various prediction modes, and information about the prediction can include information indicating an inter prediction mode for the current block.

[0099] The addition unit (340) can generate a restoration signal (restored picture, restoration block, restoration sample array) by adding the acquired residual signal to the prediction signal (prediction block, prediction sample array) output from the prediction unit (including the inter-prediction unit (332) and / or intra-prediction unit (331)). When there is no residual for the block to be processed, such as when skip mode is applied, the prediction block can be used as the restoration block.

[0100] The addition unit (340) may be referred to as a restoration unit or restoration block generation unit. The generated restoration signal may be used for intra prediction of the next processing target block within the current picture, may be output after filtering as described below, or may be used for inter prediction of the next picture. Meanwhile, LMCS (luma mapping with chroma scaling) may be applied during the picture decoding process.

[0101] The filtering unit (350) can improve subjective / objective image quality by applying filtering to the restored signal. For example, the filtering unit (350) can apply various filtering methods to the restored picture to generate a modified restored picture, and transmit the modified restored picture to the memory (360), specifically, to the DPB of the memory (360). The various filtering methods can include deblocking filtering, sample adaptive offset, adaptive loop filter, bilateral filter, etc.

[0102] The (modified) reconstructed picture stored in the DPB of the memory (360) can be used as a reference picture in the inter prediction unit (332). The memory (360) can store motion information of a block from which motion information in the current picture is derived (or decoded) and / or motion information of blocks in an already reconstructed picture. The stored motion information can be transmitted to the inter prediction unit (332) to be used as motion information of a spatial neighboring block or motion information of a temporal neighboring block. The memory (360) can store reconstructed samples of reconstructed blocks in the current picture and transmit them to the intra prediction unit (331).

[0103] In this specification, the embodiments described in the filtering unit (260), the inter prediction unit (221), and the intra prediction unit (222) of the encoding device (200) can be applied to the filtering unit (350), the inter prediction unit (332), and the intra prediction unit (331) of the decoding device (300) in the same or corresponding manner, respectively.

[0104] Figure 4 illustrates an example of a video / image decoding method to which one embodiment can be applied.

[0105] In image / video coding, the pictures that make up an image / video can be encoded / decoded according to a series of decoding orders. The picture order corresponding to the output order of the decoded pictures can be set differently from the decoding order, and based on this, not only forward prediction but also backward prediction can be performed during inter prediction.

[0106] In FIG. 4, S400 may be performed in the entropy decoding unit (310) of the aforementioned decoding device (300), S410 may be performed in the prediction unit (330), S420 may be performed in the residual processing unit (320), S430 may be performed in the addition unit (340), and S440 may be performed in the filtering unit (350). S400 may include a decoding procedure according to the present disclosure, S410 may include an inter / intra prediction procedure according to the present disclosure, S420 may include a residual processing procedure according to the present disclosure, S430 may include a block / picture restoration procedure according to the present disclosure, and S440 may include an in-loop filtering procedure according to the present disclosure.

[0107] Referring to FIG. 4, the decoding device obtains image / video information from a bitstream (S400), performs prediction based on the obtained image / video information (S410), and restores a picture through residual processing (S420, inverse quantization for quantized transform coefficients, inverse transformation) (S430).

[0108] A modified restored picture can be generated by applying an in-loop filtering procedure (S440) to a restored picture generated through the above restoration procedure, and the modified restored picture can be output as a decoded picture and can be stored in a buffer or memory of a decoding device to be used as a reference picture in an inter prediction procedure when decoding a next picture. In some cases, the in-loop filtering procedure can be omitted, in which case the restored picture can be output as a decoded picture and can be stored in a buffer or memory of a decoding device to be used as a reference picture in an inter prediction procedure when decoding a next picture.

[0109] The in-loop filtering procedure (S440) may include a deblocking filtering procedure, a sample adaptive offset (SAO) procedure, an adaptive loop filter (ALF) procedure, and / or a bi-lateral filter procedure, and some or all of them may be omitted. In addition, one or some of the deblocking filtering procedure, the sample adaptive offset (SAO) procedure, the adaptive loop filter (ALF) procedure, and the bi-lateral filter procedure may be sequentially applied, or all of them may be sequentially applied. For example, the SAO procedure may be performed after the deblocking filtering procedure is applied to the restored picture. Or, for example, the ALF procedure may be performed after the deblocking filtering procedure is applied to the restored picture. This may also be performed in an encoding device.

[0110] FIG. 5 illustrates an example of a video / image encoding method to which an embodiment of the present disclosure can be applied.

[0111] In FIG. 5, the prediction step (S500) may be performed in the prediction unit (220) of the encoding device (200) described above, residual processing (S510) based on the prediction result may be performed in the residual processing unit (230), and the step (S520) of encoding image information including prediction information and residual information may be performed in the entropy encoding unit (240). S500 may include an inter / intra prediction procedure according to the present disclosure, S510 may include a residual processing procedure according to the present disclosure, and S520 may include an encoding procedure according to the present disclosure.

[0112] The encoding procedure may optionally include a procedure for encoding information for picture restoration (e.g., prediction information, residual information, partitioning information, etc.) and outputting it in the form of a bitstream, as well as a procedure for generating a restored picture for the current picture and a procedure for applying in-loop filtering to the restored picture.

[0113] The encoding device (200) can derive (corrected) residual samples from the quantized transform coefficients through the inverse quantization unit (234) and the inverse transformation unit (235), and can generate a restored picture based on the prediction samples and (corrected) residual samples, which are outputs of S500. The restored picture generated in this way can be the same as the restored picture generated by the decoding device (300) described above. A modified restored picture can be generated through an in-loop filtering procedure for the restored picture, which can be stored in a buffer or memory, and, as in the case of the decoding device, can be used as a reference picture in the inter prediction procedure when encoding a subsequent picture.

[0114] As described above, some or all of the in-loop filtering procedure may be omitted in some cases. When the in-loop filtering procedure is performed, (in-loop) filtering-related information (parameters) may be encoded by the entropy encoding unit (240) and output in the form of a bitstream, and the decoding device (300) may perform the in-loop filtering procedure in the same manner as the encoding device based on the filtering-related information.

[0115] Through this in-loop filtering procedure, noise occurring during image / video coding, such as blocking artifacts and ringing artifacts, can be reduced, and subjective / objective image quality can be improved. In addition, by performing the in-loop filtering procedure in both the encoding device (200) and the decoding device (300), the same prediction results can be derived from the encoding device (200) and the decoding device (300), thereby increasing the reliability of picture coding and reducing the amount of data that must be transmitted for picture coding.

[0116] As described above, the picture restoration procedure can be performed not only in the decoding device (300) but also in the encoding device (200). A restoration block can be generated based on intra-prediction / inter-prediction for each block, and a restoration picture including the restoration blocks can be generated. If the current picture / slice / tile group is an I picture / slice / tile group, the blocks included in the current picture / slice / tile group can be restored based only on intra-prediction. On the other hand, if the current picture / slice / tile group is a P or B picture / slice / tile group, the blocks included in the current picture / slice / tile group can be restored based on intra-prediction or inter-prediction. In this case, inter-prediction may be applied to some blocks in the current picture / slice / tile group, and intra-prediction may be applied to some remaining blocks.

[0117] The color component of a picture may include a luma component and a chroma component, and embodiments according to the present disclosure may be applied to the luma component and the chroma component unless explicitly limited in the present disclosure.

[0118] Meanwhile, when intra prediction is performed, the prediction unit (220, 330) of the encoding device (200) / decoding device (300) can derive a reference sample according to the intra prediction mode of the current block among the surrounding samples of the current block, and can generate a prediction sample of the current block based on the reference sample.

[0119] For example, (i) the prediction sample can be derived based on the average or interpolation of neighboring reference samples of the current block, and (ii) the prediction sample can be derived based on a reference sample existing in a specific (prediction) direction with respect to the prediction sample among the neighboring reference samples of the current block. The case of (i) can be called a non-directional mode or a non-angular mode, and the case of (ii) can be called a directional mode or an angular mode.

[0120] Additionally, linear interpolation intra prediction (LIP) may be applied to perform intra prediction on the current block by linearly interpolating prediction sample values ​​generated based on the intra prediction mode of the current block.

[0121] Additionally, a temporary prediction sample of the current block may be derived based on filtered peripheral reference samples, and a prediction sample of the current block may be derived by weighting at least one reference sample derived according to an intra prediction mode among existing peripheral reference samples, i.e., unfiltered peripheral reference samples, and the temporary prediction sample. Such prediction may be referred to as Position Dependent Intra Prediction Combination (PDPC).

[0122] In addition, intra prediction encoding can be performed by selecting a reference sample line with the highest prediction accuracy among the surrounding multiple reference sample lines of the current block, deriving a prediction sample using the reference sample located in the prediction direction of the selected line, and then instructing (signaling) the used reference sample line to the decoding device. This case can be referred to as multi-reference line intra prediction (MRL) or MRL-based intra prediction.

[0123] Additionally, the current block can be divided into vertical or horizontal subpartitions, and intra prediction can be performed based on the same intra prediction mode, while peripheral reference samples can be derived and utilized for each subpartition. In other words, in this case, the intra prediction mode for the current block is applied equally to the subpartitions, but peripheral reference samples can be derived and utilized for each subpartition, thereby improving intra prediction performance in some cases. This prediction method can be called intra subpartitions (ISP) or ISP-based intra prediction.

[0124] Additionally, if the prediction direction based on the prediction sample points between surrounding reference samples, i.e., if the prediction direction points to a fractional sample location, the value of the prediction sample can be derived through interpolation of multiple reference samples located around the prediction direction (around the fractional sample location).

[0125] Meanwhile, when inter prediction is applied, the prediction unit of the encoding device / decoding device can perform inter prediction on a block-by-block basis to derive prediction samples. Inter prediction can refer to a prediction derived in a manner dependent on data elements (e.g., sample values, or motion information) of pictures other than the current picture. When inter prediction is applied to the current block, a predicted block (prediction sample array) for the current block can be derived based on a reference block (reference sample array) specified by a motion vector on a reference picture pointed to by a reference picture index.

[0126] At this time, in order to reduce the amount of motion information transmitted in the inter prediction mode, the motion information of the current block can be predicted in units of blocks, sub-blocks, or samples based on the correlation of the motion information between the neighboring blocks and the current block. The motion information may include a motion vector and / or a reference picture index. The motion information may further include information on the inter prediction type (L0 prediction, L1 prediction, Bi prediction, etc.). When inter prediction is applied, the neighboring blocks may include spatial neighboring blocks existing in the current picture and temporal neighboring blocks existing in the reference picture.

[0127] A reference picture including a reference block and a reference picture including a temporal neighboring block may be the same or different. The temporal neighboring blocks may be called collocated reference blocks, collocated CUs (colCUs), etc., and the reference pictures including the temporal neighboring blocks may be called collocated pictures (colPic). For example, a motion information candidate list may be constructed based on neighboring blocks of a current block, and a flag or index information indicating which candidate is selected (used) to derive a motion vector and / or reference picture index of the current block may be signaled.

[0128] Inter prediction can be performed based on various prediction modes. For example, in the case of skip mode and merge mode, the motion information of the current block may be the same as the motion information of the selected neighboring block. In the case of skip mode, unlike the merge mode, a residual signal may not be transmitted. In the case of motion vector prediction (MVP) mode, the motion vector of the selected neighboring block may be used as a motion vector predictor, and the motion vector difference may be signaled. In this case, the motion vector of the current block can be derived using the sum of the motion vector predictor and the motion vector difference.

[0129] The motion information may include L0 motion information and / or L1 motion information depending on the inter prediction type (L0 prediction, L1 prediction, Bi prediction, etc.). The L0 motion vector may be called L0 motion vector or MVL0, and the L1 motion vector may be called L1 motion vector or MVL1. Prediction based on an L0 motion vector may be called L0 prediction, prediction based on an L1 motion vector may be called L1 prediction, and prediction based on both the L0 motion vector and the L1 motion vector may be called bi(Bi) prediction. Here, the L0 motion vector may indicate a motion vector associated with an L0 reference picture list, and the L1 motion vector may indicate a motion vector associated with an L1 reference picture list. The L0 reference picture list may include pictures that are earlier in output order than the current picture as reference pictures, and the L1 reference picture list may include pictures that are later in output order than the current picture. Previous pictures may be called forward pictures, and subsequent pictures may be called backward pictures.

[0130] The L0 reference picture list may include pictures that are later in output order than the current picture as reference pictures. For example, in the L0 reference picture list, previous pictures may be indexed first and subsequent pictures may be indexed next. The L1 reference picture list may include pictures that are earlier in output order than the current picture as reference pictures. For example, in the L1 reference picture list, the subsequent pictures may be indexed first and subsequent pictures may be indexed next. Here, the output order may correspond to the POC (picture order count) order.

[0131] That is, in the present embodiment, the distinction between L0 and L1 does not necessarily limit the output order of reference pictures. As described above, the L0 reference picture list may also include backward reference pictures, the L1 reference picture list may also include forward reference pictures, and it is also possible for the L0 reference picture list and the L1 reference picture list to include the same reference picture list. L0 and L1 may simply be used to distinguish between different lists or between reference pictures.

[0132] Figures 6 and 7 illustrate examples of an inter prediction based video / image encoding method to which one embodiment can be applied.

[0133] Referring to FIG. 6, the encoding device (200) can perform inter prediction for the current block (S600). The encoding device can determine or derive the inter prediction mode and motion information of the current block, and generate prediction samples of the current block. Here, the inter prediction mode determination, motion information derivation, and prediction sample generation procedures may be performed simultaneously, or one procedure may be performed before the other. For example, as illustrated in FIG. 7, the inter prediction unit (221) of the encoding device (200) may include a prediction mode determination unit (221a), a motion information derivation unit (221b), and a prediction sample derivation unit (221c), and the prediction mode determination unit (221a) may determine the prediction mode for the current block, the motion information derivation unit (221b) may derive motion information of the current block, and the prediction sample derivation unit (221c) may derive prediction samples of the current block.

[0134] For example, the inter prediction unit of the encoding device can search for a block similar to the current block within a certain area (search area) of reference pictures through motion estimation, and derive a reference block whose difference from the current block is minimal or below a certain standard. Based on this, a reference picture index indicating a reference picture where the reference block is located can be derived, and a motion vector can be derived based on the positional difference between the reference block and the current block. The encoding device can determine a mode to be applied to the current block among various prediction modes. The encoding device can compare RD costs for the various prediction modes and determine an optimal prediction mode for the current block.

[0135] For example, when the skip mode or merge mode is applied to the current block, the encoding device (200) may configure a merge candidate list, and derive a reference block among the reference blocks indicated by the merge candidates included in the merge candidate list, in which the difference between samples with the current block, that is, the difference in SAD (Sum of Absolute Differences) or SATD (Sum of Absolute Transformed Differences) is minimum or below a certain standard. In this case, a merge candidate associated with the derived reference block is selected, and merge index information indicating the selected merge candidate may be generated and signaled to the decoding device. Motion information of the current block may be derived using motion information of the selected merge candidate.

[0136] As another example, when the (A)MVP mode is applied to the current block, the encoding device (200) may configure an (A)MVP candidate list described below, and use the motion vector of a motion vector predictor (mvp) candidate selected from among motion vector predictor (mvp) candidates included in the (A)MVP candidate list as the motion vector predictor of the current block. In this case, for example, a motion vector pointing to a reference block derived by the above-described motion estimation may be used as the motion vector of the current block, and a motion vector predictor candidate having a motion vector with the smallest difference from the motion vector of the current block among the motion vector predictor candidates may be the selected motion vector predictor candidate. A Motion Vector Difference (MVD), which is the difference obtained by subtracting the motion vector predictor from the motion vector of the current block, may be derived. In this case, information about the MVD may be signaled to the decoding device. Additionally, when the (A)MVP mode is applied, the value of the reference picture index can be configured as reference picture index information and signaled separately to the decoding device.

[0137] The encoding device (200) can derive residual samples based on predicted samples (S610). The encoding device can derive residual samples by comparing the original samples of the current block with the predicted samples.

[0138] The encoding device (200) can encode image information including prediction information and residual information (S620). The encoding device can output the encoded image information in the form of a bitstream. The prediction information is information about prediction and may include prediction mode information (e.g., skip flag, merge flag, or merge index, etc.) and / or motion information. The motion information may include candidate selection information (e.g., merge index, mvp flag, or mvp index), which is information for deriving a motion vector. In addition, the motion information may include information about the above-described MVD and / or reference picture index information. In addition, the motion information may include information indicating whether L0 prediction, L1 prediction, or bi-prediction is applied. The residual information is information about residual samples. The residual information may include information about quantized transform coefficients for the residual samples.

[0139] The output bitstream may be stored in a (digital) storage medium and transmitted to a decoding device (300), or may be transmitted to a decoding device (300) via a network. Here, the storage medium may be a computer-readable storage medium capable of storing data non-temporarily.

[0140] Meanwhile, as described above, the encoding device (200) can generate a restored picture (including restored samples and restored blocks) based on reference samples and residual samples. This is to derive the same prediction result as that performed by the decoding device (300) from the encoding device, thereby increasing coding efficiency. Accordingly, the encoding device (200) can store the restored picture (or restored samples, restored blocks) in memory and utilize it as a reference picture for inter prediction. As described above, an in-loop filtering procedure, etc. can be further applied to the restored picture.

[0141] FIG. 8 and FIG. 9 illustrate examples of inter-prediction based video / image decoding methods to which embodiments of the present disclosure can be applied.

[0142] A video / image decoding procedure based on inter prediction may roughly include, for example:

[0143] Referring to FIGS. 8 and 9, the decoding device (300) can perform an operation corresponding to the operation performed in the encoding device (200). The decoding device can perform a prediction on the current block based on the received prediction information and derive prediction samples.

[0144] Specifically, the decoding device (300) can determine a prediction mode for the current block based on the received prediction information (S700). The prediction mode determination unit (332a) of the decoding device (300) can determine or derive which inter prediction mode is applied to the current block based on the prediction mode information in the prediction information.

[0145] For example, based on the merge flag, it can be determined whether the current block is subject to merge mode or (A)MVP mode. Alternatively, one of various inter prediction mode candidates can be selected based on the mode index. The inter prediction mode candidates can include skip mode, merge mode, and / or (A)MVP mode, or can include various inter prediction modes described below.

[0146] The decoding device (300) can derive motion information of the current block based on the determined inter prediction mode (S710). For example, when skip mode or merge mode is applied to the current block, the motion information derivation unit (332b) of the decoding device (300) can construct a merge candidate list described below and select one merge candidate from among the merge candidates included in the merge candidate list. This selection can be performed based on the above-described selection information (merge index). Motion information of the current block can be derived using motion information of the selected merge candidate. Motion information of the selected merge candidate can be used as motion information of the current block.

[0147] As another example, when the (A)MVP mode is applied to the current block, the decoding device (300) may construct an (A)MVP candidate list described below, and use the motion vector of an MVP candidate selected from among the MVP (motion vector predictor) candidates included in the (A)MVP candidate list as the MVP of the current block. This selection may be performed based on the selection information (mvp flag or mvp index) described above. In this case, the MVD of the current block may be derived based on information about the MVD, and the motion vector of the current block may be derived based on the MVP and MVD of the current block. In addition, the reference picture index of the current block may be derived based on reference picture index information. A picture indicated by a reference picture index within the reference picture list for the current block may be derived as a reference picture referenced for inter prediction of the current block.

[0148] Meanwhile, the motion information of the current block may be derived without constructing a candidate list, in which case the motion information of the current block may be derived according to the procedure initiated in the prediction mode. In this case, the candidate list construction described above may be omitted.

[0149] The decoding device (300) can generate prediction samples for the current block based on the motion information of the current block (S720). In this case, the prediction sample derivation unit (332c) of the decoding device (300) can derive a reference picture based on the reference picture index of the current block, and derive prediction samples of the current block using samples of a reference block pointed to by the motion vector of the current block on the reference picture. In this case, as described below, a prediction sample filtering procedure may be further performed on all or part of the prediction samples of the current block, depending on the case.

[0150] In other words, the inter prediction unit (332) of the decoding device (300) may include a prediction mode determination unit (332a), a motion information derivation unit (332b), and a prediction sample derivation unit (332c), and may determine a prediction mode for the current block based on the prediction mode information received from the prediction mode determination unit (332a), derive motion information (motion vector and / or reference picture index, etc.) of the current block based on the information about motion information received from the motion information derivation unit (332b), and derive or generate prediction samples of the current block from the prediction sample derivation unit (332c).

[0151] The decoding device (300) derives residual samples for the current block based on the received residual information (S730). The decoding device (300) generates restoration samples for the current block based on the prediction samples and residual samples (S740), and can generate a restoration picture based on these. As described above, in-loop filtering procedures, etc. may be further applied to the restoration picture.

[0152] FIG. 10 exemplarily illustrates an inter prediction procedure to which an embodiment of the present disclosure can be applied.

[0153] As described above, the inter prediction procedure may include an inter prediction mode determination step, a motion information derivation step according to the determined prediction mode, and a prediction performance (prediction sample generation) step based on the derived motion information. The inter prediction procedure may be performed in the encoding device (200) and the decoding device (300) as described above. In this document, the coding device may include an encoding device and / or a decoding device.

[0154] Referring to FIG. 10, the coding device determines an inter prediction mode for a current block (S800). Various inter prediction modes may be used to predict the current block within a picture. For example, various modes may be used, such as merge mode, skip mode, MVP (Motion Vector Prediction) mode, affine mode, sub-block merge mode, MMVD (Merge with MVD) mode, and SMVD (Symmetric Motion Vector Difference) mode. Decoder side Motion Vector Refinement (DMVR) mode, Adaptive Motion Vector Resolution (AMVR) mode, Bi-prediction with CU-level Weight (BCW), Bi-Directional Optical Flow (BDOF), etc. may also be used as auxiliary modes.

[0155] In the present disclosure, the affine mode may be referred to as the affine motion prediction mode. In addition, the MVP mode may be referred to as the Advanced Motion Vector Prediction (AMVP) mode. In the present disclosure, motion information candidates derived from some modes and / or some modes may be included as one of the motion information-related candidates of other modes. For example, a History-based Motion Vector Predictor (HMVP) candidate may be added as a merge candidate of the merge / skip mode, or may be added as a motion vector predictor candidate of the AMVP mode. When an HMVP candidate is used as a motion information candidate of the merge mode or the skip mode, the HMVP candidate may be referred to as an HMVP merge candidate.

[0156] Prediction mode information indicating the inter-prediction mode of the current block can be signaled from the encoding device to the decoding device. The prediction mode information can be included in the bitstream and received by the decoding device. The prediction mode information can include index information indicating one of multiple candidate modes. Alternatively, the inter-prediction mode can be indicated through hierarchical signaling of flag information.

[0157] In this case, the prediction mode information may include one or more flags. For example, a skip flag may be signaled to indicate whether skip mode is applied, a merge flag may be signaled to indicate whether merge mode is applied when skip mode is not applied, and MVP mode may be indicated to be applied when merge mode is not applied, or additional flags may be signaled for additional distinction. Affine mode may be signaled as an independent mode, or as a mode dependent on merge mode or MVP mode. For example, an affine mode may include an affine merge mode and an affine MVP mode.

[0158] The coding device can derive motion information for the current block (S810). The motion information can be derived based on the inter-prediction mode determined in the aforementioned step. The coding device can perform inter-prediction using the motion information of the current block. The encoding device can derive optimal motion information for the current block through a motion estimation procedure.

[0159] For example, the encoding device (200) can search for a similar reference block with a high correlation in units of fractional pixels within a predetermined search range within the reference picture using an original block within the original picture for the current block, and thereby derive motion information. The similarity of blocks can be derived based on the difference in phase-based sample values. For example, the similarity of blocks can be calculated based on the SAD between the current block (or the template of the current block) and the reference block (or the template of the reference block). In this case, motion information can be derived based on the reference block with the smallest SAD within the search range. The derived motion information can be encoded in various ways based on an inter prediction mode and transmitted to a decoding device in the form of a bitstream.

[0160] The decoding device (300) can derive the motion information from the bitstream.

[0161] The coding device can perform inter prediction based on motion information for the current block to generate prediction samples (S820). The current block containing the prediction samples may be referred to as a prediction block.

[0162] Meanwhile, information indicating whether the above-described List0 (L0) prediction, List1 (L1) prediction, or bi-prediction is used for the current block (current coding unit) can be signaled. This information may be called motion prediction direction information, inter-prediction direction information, or inter-prediction indication information, and may be configured / encoded / signaled, for example, in the form of an inter_pred_idc syntax element. That is, the inter_pred_idc syntax element can indicate whether the above-described List0 (L0) prediction, List1 (L1) prediction, or bi-prediction is used for the current block (current coding unit). In this document, for the convenience of explanation, the inter-prediction type (L0 prediction, L1 prediction, or BI prediction) indicated by the inter_pred_idc syntax element may be indicated as motion prediction direction. L0 prediction may be represented as pred_L0, L1 prediction as pred_L1, and bi-prediction as pred_BI. For example, depending on the value of the inter_pred_idc syntax element, the prediction type can be indicated as in Table 1 below.

[0163] [Table 1]

[0164]

[0165] As described above, a picture may include one or more slices. A slice may have one of the following slice types: intra (I) slice, predictive (P) slice, and bi-predictive (B) slice. The slice type may be indicated based on slice type information. For blocks within an I slice, inter prediction is not used for prediction, and only intra prediction can be used. Of course, even in this case, the original sample values ​​can be coded and signaled without prediction. For blocks within a P slice, either intra prediction or inter prediction can be used, and when inter prediction is used, only uni prediction can be used. On the other hand, for blocks within a B slice, either intra prediction or inter prediction can be used, and when inter prediction is used, up to bi prediction can be used.

[0166] L0 and L1 may include reference pictures encoded / decoded before the current picture. For example, L0 may include reference pictures that are before and / or after the current picture in POC order, and L1 may include reference pictures that are after and / or before the current picture in POC order. For example, L0 may be assigned a relatively lower reference picture index to reference pictures that are before the current picture in POC order, and L1 may be assigned a relatively lower reference picture index to reference pictures that are after the current picture in POC order. For B slices, bi-prediction may be applied, and in this case, either uni-directional bi-prediction or bi-directional bi-prediction may be applied. Bi-directional bi-prediction may be called true bi-prediction.

[0167] For example, information about the inter-prediction mode of the current block can be coded and signaled at the CU (CU syntax) level, or implicitly determined based on conditions. In this case, some modes may be explicitly signaled, while others may be implicitly derived.

[0168] For example, the CU syntax can carry information about (inter)prediction mode, etc.

[0169] [Table 2]

[0170]

[0171]

[0172]

[0173]

[0174] Here, cu_skip_flag can indicate whether the skip mode is applied to the current block (CU). When pred_mode_flag is 0, it indicates that the current coding unit is encoded in the inter prediction mode. When pred_mode_flag is 1, it indicates that the current coding unit is encoded in the intra prediction mode. When pred_mode_ibc_flag is 1, it indicates that the current coding unit is encoded in the IBC prediction mode (Intra Block Copy prediction mode). When pred_mode_ibc_flag is 0, it indicates that the current coding unit is not encoded in the IBC prediction mode. When pcm_flag[x0][y0] is 1, it indicates that the pcm_sample( ) syntax structure exists and the transform_tree( ) syntax structure does not exist in the coding unit including the luma coding block at the position (x0, y0).

[0175] If pcm_flag[x0][y0] is 0, it indicates that the pcm_sample( ) syntax structure does not exist. That is, pcm_flag can indicate whether the pulse coding modulation (PCM) mode is applied to the current block. If the PCM mode is applied to the current block, prediction / transformation / quantization, etc. are not applied, and the values ​​of the original samples in the current block can be coded and signaled.

[0176] If intra_mip_flag[x0][y0] is 1, it indicates that the intra prediction type for the luma sample is Matrix-based Intra Prediction (MIP). If intra_mip_flag[x0][y0] is 0, it indicates that the intra prediction type for the luma sample is not MIP. In other words, intra_mip_flag can indicate whether the MIP prediction mode (type) is applied to the current block (its luma sample).

[0177] intra_chroma_pred_mode[x0][y0] indicates the intra prediction mode for the chroma samples of the current block. general_merge_flag[x0][y0] indicates whether the inter prediction parameters of the current coding unit are inferred from the surrounding inter-predicted partitions. That is, general_merge_flag can indicate that general merge is available, and when the value of general_merge_flag is 1, regular merge mode, MMVD mode, and merge subblock mode can be available. For example, when the value of general_merge_flag is 1, the merge data syntax can be parsed from the encoded video / image information (or bitstream), and the merge data syntax can be configured / coded to include information as shown in Table 3 below.

[0178] [Table 3]

[0179]

[0180] Here, if regular_merge_flag[x0][y0] is 1, it indicates that the regular merge mode is used to generate the inter prediction parameters of the current coding unit. That is, regular_merge_flag indicates whether the regular merge mode is applied to the current block.

[0181] When mmvd_merge_flag[x0][y0] is 1, it indicates that Merge Mode with Motion Vector Difference (MMVD) is used to generate inter prediction parameters of the current coding unit. That is, mmvd_merge_flag indicates whether MMVD is applied to the current block.

[0182] mmvd_cand_flag[x0][y0] indicates whether the first candidate (0) or the second candidate (1) is selected in the merge candidate list, and the selected candidate is constructed using the motion vector difference (MVD) derived from mmvd_distance_idx[x0][y0] and mmvd_direction_idx[x0][y0]. mmvd_distance_idx[x0][y0] indicates the index used to derive MmvdDistance[x0][y0]. mmvd_direction_idx[x0][y0] indicates the index used to derive MmvdSign[x0][y0].

[0183] merge_subblock_flag[x0][y0] indicates whether subblock-based inter prediction parameters are used for the current coding unit. That is, merge_subblock_flag can indicate whether subblock merge mode (or affine merge mode) is applied to the current block. merge_subblock_idx[x0][y0] indicates the index of the merge candidate selected from the subblock-based merge candidate list.

[0184] ciip_flag[x0][y0] indicates whether Combined Inter-picture and Intra-picture Prediction (CIIP) is applied to the current coding unit.

[0185] merge_triangle_idx0[x0][y0] represents the index of the first selected merge candidate in the triangle shape-based motion compensation candidate list. merge_triangle_idx1[x0][y0] represents the index of the second selected merge candidate in the triangle shape-based motion compensation candidate list.

[0186] merge_idx[x0][y0] represents the index of the merge candidate selected from the merge candidate list.

[0187] Meanwhile, referring back to the CU syntax, mvp_l0_flag can indicate a candidate to be selected from the MVP candidate list 0 for deriving the MVP of the current block when the MVP mode is applied. mvp_l1_flag[x0][y0] has the same meaning as mvp_l0_flag, but in this case, l0 and list 0 are replaced with l1 and list 1, respectively.

[0188] inter_pred_idc[x0][y0] indicates whether list 0 (list0), list 1 (list1), or bi-prediction is used for the current coding unit.

[0189] When sym_mvd_flag[x0][y0] is 1, it indicates that the syntax elements ref_idx_l0[x0][y0], ref_idx_l1[x0][y0], and the mvd_coding(x0, y0, refList, cpIdx) syntax structure when refList is 1 do not exist. That is, sym_mvd_flag indicates whether symmetric MVD is used in mvd coding.

[0190] ref_idx_l0[ x0 ][ y0 ] specifies the list 0 reference picture index for the current coding unit. ref_idx_l1[ x0 ][ y0 ] has the same meaning as ref_idx_l0, except that l0, L0, and list 0 are replaced by l1, L1, and list 1, respectively.

[0191] If inter_affine_flag[ x0 ][ y0 ] is 1, it indicates that affine model-based motion compensation is used to generate prediction samples for the current coding unit when decoding a P slice or a B slice.

[0192] If cu_affine_type_flag[ x0 ][ y0 ] is 1, it indicates that the prediction sample is generated using the 6-parameter affine model-based motion compensation for the current coding unit when decoding a P slice or a B slice.

[0193] If cu_affine_type_flag[ x0 ][ y0 ] is 0, it indicates that the prediction samples are generated using a 4-parameter affine model-based motion compensation.

[0194] amvr_flag[ x0 ][ y0 ] specifies the precision of the motion vector difference. The array indices x0, y0 represent the position (x0, y0) of the upper left luma sample of the target coding block with respect to the upper left luma sample of the picture. If amvr_flag[ x0 ][ y0 ] is 0, it indicates that the precision of the motion vector difference is 1 / 4 of a luma sample. If amvr_flag[ x0 ][ y0 ] is 1, it indicates that the precision of the motion vector difference is additionally specified by amvr_precision_flag[ x0 ][ y0 ]. If amvr_precision_flag[ x0 ][ y0 ] is 0, the precision of the motion vector difference is one integer luma sample if inter_affine_flag[ x0 ][ y0 ] is 0, otherwise it is 1 / 16 luma sample. If amvr_precision_flag[ x0 ][ y0 ] is 1, the precision of the motion vector difference is 4 luma samples if inter_affine_flag[ x0 ][ y0 ] is 0, otherwise it is one integer luma sample. The array indices x0, y0 indicate the position (x0, y0) of the upper left luma sample of the target coding block relative to the upper left luma sample of the picture.

[0195] bcw_idx[ x0 ][ y0 ] specifies the weight index in bidirectional prediction using CU weights.

[0196] A predicted block for a current block can be derived based on motion information derived according to a prediction mode. The predicted block can include prediction samples (prediction sample array) of the current block. If the motion vector of the current block points to fractional sample units, an interpolation procedure can be performed, through which prediction samples of the current block can be derived based on reference samples of fractional sample units within a reference picture.

[0197] When affine inter prediction is applied to the current block, prediction samples can be generated based on sample / subblock unit MV. When bi-prediction is applied, prediction samples derived from L0 prediction (i.e., prediction using reference pictures in the reference picture list L0 and MVL0) and L1 prediction (i.e., prediction using reference pictures in the reference picture list L1 and MVL1) can be used as prediction samples of the current block through a weighted sum or weighted average (according to phase). As described above, when the reference pictures used for L0 prediction and the reference pictures used for L1 prediction are located in different temporal directions with respect to the current picture (i.e., bi-prediction and bidirectional prediction), this can be called true bi-prediction.

[0198] As mentioned above, applying a weighted average to pairwise prediction can be referred to as BCW (Bi-prediction with CU-level weight). When pairwise prediction is applied, a pairwise prediction signal (pairwise prediction samples) can be derived through a weighted average of the L0 prediction signal and the L1 prediction signal, as shown in Equation 1 below.

[0199] [Formula 1]

[0200]

[0201] For example, when applying a weighted average to a pairwise prediction, five weights are allowed, and the available weights can be set to w ∈{-2, 3, 4, 5, 10}. For each pairwise prediction CU, the weight w can be determined in one of two ways:

[0202] -For non-merge CUs, the weight index is signaled after motion vector difference.

[0203] - For merge CUs, the weight index is inferred based on the surrounding blocks and the selected merge candidate index.

[0204] Weighted average biprediction is applied only to CUs with more than 256 luma samples (i.e., CUs with width X height greater than 256). For low-delay pictures, all five weights are used, while for non-low-delay pictures, only three weights (w ∈{3, 4, 5}) are used.

[0205] In the encoding device (200), a fast search algorithm can be applied to quickly search the weight index without increasing complexity. The related algorithm can be summarized as follows:

[0206] -When combined with AMVR (Adaptive Motion Vector Resolution), unequal weights are conditionally checked for 1-pel and 4-pel motion vector precision only when the current picture is low-delay.

[0207] - When combined with affine mode, Affine ME (motion estimation) is performed on asymmetric weights only when affine mode is selected as the current optimal mode.

[0208] -If the two reference pictures are identical, the asymmetric weights are only checked conditionally.

[0209] - Also, asymmetric weights are not explored if certain conditions are met, such as the difference in POC between the current picture and the reference picture, the coding QP, and the temporal level.

[0210] The BCW weight index is coded using a single context-coded bin followed by bypass-coded bins. The first context-coded bin indicates whether equal weights are used. If asymmetric weights are used, additional bins can be signaled using bypass to indicate the use of asymmetric weights.

[0211] Weighted Prediction (WP) is a coding tool for efficiently encoding video content that includes fading. WP can signal weights and offsets for each reference picture in each list (L0, L1), and the weights and offsets of the corresponding reference pictures are applied during motion compensation. WP and BCW are technologies designed to suit different types of video content. To reduce decoder design complexity, the BCW weight index is not signaled when a CU uses WP, in which case the weight w is assumed to be 4 (equal weight).

[0212] For merge CUs, the weight index is inferred based on the surrounding blocks and the selected merge candidate index. This approach can be applied not only to the standard merge mode but also to the inherited affine merge mode. In the constructed affine merge mode, affine motion information is constructed based on motion information obtained from up to three blocks. In this case, the procedure for deriving the BCW index of a CU is as follows:

[0213] 1) Divide the BCW index range {0,1,2,3,4} into three groups {0}, {1,2,3}, and {4}. If the BCW indices of all control points belong to the same group, proceed to step 2. Otherwise, the BCW index of the currently configured candidate is set to 2.

[0214] 2) If two or more control points have the same BCW index, the corresponding index value is assigned to the candidate. Otherwise, the BCW index of the currently configured candidate is set to 2.

[0215] As described above, restoration samples and restoration pictures can be generated based on the prediction samples derived as described above, and then procedures such as in-loop filtering can be performed.

[0216] Inter prediction can be performed using motion information of the current block. The encoding device can derive optimal motion information for the current block through a motion estimation procedure. For example, the encoding device can search for similar reference blocks with high correlation within a predetermined search range within the reference picture using the original block within the original picture for the current block, in fractional pixel units, and thereby derive motion information. The similarity between blocks can be derived based on the difference in phase-based sample values. For example, the similarity between blocks can be calculated based on the SAD between the current block (or a template of the current block) and the reference block (or a template of the reference block). In this case, motion information can be derived based on the reference block with the smallest SAD within the search range. The derived motion information can be signaled to the decoding device in various ways based on the inter prediction mode.

[0217] When merge mode is applied, the motion information of the current prediction block is not directly transmitted, but rather the motion information of the surrounding prediction blocks is used to derive the motion information of the current prediction block. Therefore, the motion information of the current prediction block can be indicated by transmitting flag information indicating that merge mode is used and a merge index indicating which surrounding prediction block was used. The above merge mode may be referred to as regular merge mode.

[0218] In order to perform merge mode, the encoder must search for merge candidate blocks used to derive motion information of the current prediction block. For example, up to five merge candidate blocks may be used, but the present invention is not limited thereto. In addition, the maximum number of merge candidate blocks may be transmitted in a slice header or a tile group header, but the present invention is not limited thereto. After finding the merge candidate blocks, the encoder can generate a merge candidate list, and select the merge candidate block with the lowest cost among them as the final merge candidate block.

[0219] The present invention provides various embodiments for merge candidate blocks constituting the merge candidate list.

[0220] FIG. 11 is a diagram showing examples of blocks used as spatial merge candidates in an encoding method or a decoding method according to one embodiment, and FIG. 12 is a flowchart showing an example of a process for constructing a merge candidate list in an encoding method or a decoding method according to one embodiment.

[0221] The above merge candidate list may utilize, for example, five merge candidate blocks. For example, four spatial merge candidates and one temporal merge candidate may be utilized. As a specific example, in the case of spatial merge candidates, the blocks illustrated in FIG. 11 may be utilized as spatial merge candidates. Hereinafter, the spatial merge candidates or the spatial MVP candidates described below may be referred to as SMVPs, and the temporal merge candidates or the temporal MVP candidates described below may be referred to as TMVPs.

[0222] The merge candidate list for the current block above can be constructed based on the procedure illustrated in FIG. 12, for example.

[0223] A coding device (encoding device or decoding device) searches the spatial neighboring blocks of the current block and inserts the derived spatial merge candidates into a merge candidate list (S910). For example, the spatial neighboring blocks may include a lower left corner neighboring block, a left neighboring block, an upper right corner neighboring block, an upper neighboring block, and an upper left corner neighboring block of the current block. However, this is merely an example, and in addition to the above-described spatial neighboring blocks, additional neighboring blocks such as a right neighboring block, a lower neighboring block, and a lower right neighboring block may also be used as spatial neighboring blocks. The coding device may search the spatial neighboring blocks based on priorities to detect available blocks, and derive motion information of the detected blocks as the spatial merge candidates. For example, the coding device may search the five blocks illustrated in FIG. 11 in the order of A1, B1, B0, A0, and B2, and sequentially index the available candidates to form a merge candidate list.

[0224] The coding device searches for temporal neighboring blocks of the current block and inserts the derived temporal merge candidates into the merge candidate list (S920). The temporal neighboring blocks may be located on a reference picture that is a different picture from the current picture where the current block is located. The reference picture where the temporal neighboring blocks are located may be called a collocated picture or col picture. The temporal neighboring blocks may be searched in the order of the lower right corner neighboring blocks and the lower right center block of the co-located block for the current block on the col picture. Meanwhile, when motion data compression is applied, specific motion information can be stored as representative motion information for each storage unit in the col picture. In this case, there is no need to store motion information for all blocks within a certain storage unit, and thus the effect of motion data compression can be achieved. In this case, the predetermined storage unit may be predetermined, for example, as a 16x16 sample unit, or an 8x8 sample unit, or size information for the predetermined storage unit may be signaled from the encoding device to the decoding device.

[0225] When motion data compression is applied, the motion information of temporal neighboring blocks can be replaced with representative motion information of a certain storage unit where the temporal neighboring blocks are located. That is, in this case, from an implementation perspective, a temporal merge candidate can be derived based on the motion information of a prediction block that covers a position that is arithmetically shifted to the right by a certain value based on the coordinates of the temporal neighboring block (upper left sample position), rather than a prediction block located at the coordinates of the temporal neighboring block.

[0226] For example, if the schedule storage unit is 2 n x2 nIn the case of sample units, if the coordinates of the temporal surrounding block are (xTnb, yTnb), then the modified position is ((xTnb>>n)<<n), (yTnb> >n)< <n))에 위치하는 예측 블록의 움직임 정보가 시간적 머지 후보를 위하여 사용될 수 있다. 구체적으로 상기 일정 저장 단위가 16x16 샘플 단위인 경우, 시간적 주변 블록의 좌표가 (xTnb, yTnb)라 하면, 수정된 위치인 ((xTnb> >4)<<4), (yTnb>>4)<<4)) motion information of the prediction block located at can be used for the temporal merge candidate. Alternatively, if the unit of storage is an 8x8 sample unit and the coordinates of the temporal surrounding block are (xTnb, yTnb), the motion information of the prediction block located at the modified position ((xTnb>>3)<<3), (yTnb>>3)<<3)) can be used for the temporal merge candidate.

[0227] The encoding device can determine whether the current number of merge candidates is less than the maximum number of merge candidates (S930). The maximum number of merge candidates may be predefined or signaled from the encoding device (200) to the decoding device (300). For example, the encoding device (200) may generate information regarding the maximum number of merge candidates, encode the information, and transmit it to the decoding device (300) in the form of a bitstream. If the maximum number of merge candidates is filled, the subsequent candidate addition process may not be performed.

[0228] If the number of current merge candidates is smaller than the maximum number of merge candidates as a result of the verification, the coding device inserts an additional merge candidate into the merge candidate list (S940). The additional merge candidate may include, for example, at least one of a history-based merge candidate(s), a pair-wise average merge candidate(s), an ATMVP, a combined bi-predictive merge candidate (when the slice / tile group type of the current slice / tile group is type B), and / or a zero-vector merge candidate.

[0229] If the number of current merge candidates is not less than the maximum number of merge candidates as a result of the verification, the coding device can terminate the construction of the merge candidate list. In this case, the encoding device (200) can select an optimal merge candidate among the merge candidates constituting the merge candidate list based on RD (rate-distortion) cost, and can signal selection information (e.g., merge index) indicating the selected merge candidate to the decoding device (300). The decoding device (300) can select an optimal merge candidate based on the merge candidate list and the selection information.

[0230] As described above, the motion information of the selected merge candidate can be used as the motion information of the current block, and prediction samples of the current block can be derived based on the motion information of the current block. The encoding device (200) can derive residual samples of the current block based on the prediction samples, and signal residual information about the residual samples to the decoding device (300). As described above, the decoding device (300) can generate restoration samples based on the residual samples and the prediction samples derived based on the residual information, and generate a restoration picture based on the same.

[0231] When skip mode is applied, the motion information of the current block can be derived in the same manner as when merge mode is applied. However, when skip mode is applied, the residual signal for the corresponding block is omitted, and the predicted samples can be directly used as restored samples.

[0232] A History-based Motion Vector Predictor (HMVP) merge candidate can be added to the merge list after spatial and temporal merge candidates. In this method, the motion information of a previously coded block is stored in a table and used as the MVP of the current CU. The table containing multiple HMVP candidates is maintained during the encoding / decoding process, and the table is initialized (emptied) whenever a new CTU row begins. If there is a CU encoded using an inter prediction method rather than a subblock prediction method, the motion information of the CU is added as the last entry in the table and registered as a new HMVP candidate.

[0233] For example, the size S of the HMVP table can be set to 5, which means that up to 5 history-based MVP (HMVP) candidates can be added to the table. When inserting a new motion candidate into the table, a constrained first-in-first-out (FIFO) method is applied. Duplicates are first checked. If an identical HMVP candidate exists in the table, the entry is removed and all subsequent HMVP candidates are moved forward.

[0234] HMVP candidates can be used in the merge candidate list construction process. The most recent HMVP candidates in the table are sequentially reviewed and inserted into the merge candidate list after the temporal TMVP candidates. A redundancy check is performed to determine whether the HMVP candidate overlaps with existing spatial or temporal merge candidates.

[0235] To reduce the number of duplicate checks, the following simplification method is introduced:

[0236] - The number of HMVP candidates used to generate the merge list is set to M when (N ≤ 4), where N is the number of candidates already existing in the merge list and M is the number of HMVP candidates available in the table. On the other hand, when N > 4, the number of HMVP candidates used is set to (8 - N).

[0237] - When the total number of available merge candidates reaches the maximum number of allowed merge candidates minus 1, the process of constructing merge candidates from HMVP ends.

[0238] In this specification, a pair-wise average merge candidate may also be referred to as a pair-wise average candidate or a pair-wise candidate.

[0239] A pairwise average candidate is generated by averaging predefined pairs within the existing merge candidate list, and these predefined pairs are defined as {(0, 1), (0, 2), (1, 2), (0, 3), (1, 3), (2, 3)}, where each number represents an index within the merge candidate list.

[0240] The average motion vector is calculated individually for each reference list. If two motion vectors exist within a single reference list, the average is used even if the two vectors point to different reference images. Alternatively, if only one motion vector exists, that vector is used as is. Alternatively, if no motion vectors exist, the reference list is left invalid.

[0241] If the merge list is not full after adding pairwise average candidates, insert a zero MVP at the end of the list until the maximum number of merge candidates is reached.

[0242] In Merge mode, implicitly derived motion information is directly used to generate prediction samples for the current CU. Additionally, Merge mode with Motion Vector Differences (MMVD) is introduced. Since Skip mode and Merge mode use similar motion information derivation procedures, MMVD can also be applied to Skip mode. After signaling the Skip flag and Merge flag for the current CU, an MMVD flag (e.g., mmvd_flag) can be signaled to indicate whether MMVD mode is used.

[0243] In MMVD, after a merge candidate is selected, it is further refined by additionally signaled MVD information. If MMVD is applied to the current block (i.e., mmvd_flag is 1), additional information about MMVD can be signaled. This additional information can include a merge candidate flag (e.g., mmvd_merge_flag) indicating whether the first candidate (0) or the second candidate (1) in the merge candidate list is used along with the motion vector difference, an index specifying the motion magnitude (e.g., mmvd_distance_idx), and an index specifying the motion direction (e.g., mmvd_direction_idx). That is, in MMVD mode, one of the first two candidates in the merge list is selected as the MV basis, and which candidate to use is specified by mmvd_merge_flag.

[0244] The distance index specifies motion size information and represents a predefined offset relative to the starting point. The offset is added to either the horizontal or vertical component of the starting motion vector (starting MV). The relationship between the distance index and the predefined offset is specified in Table 4.

[0245] [Table 4]

[0246]

[0247] Here, if slice_fpel_mmvd_enabled_flag is 1, it indicates that the merge mode using motion vector difference (MVD) in the current slice uses integer sample precision. Conversely, if slice_fpel_mmvd_enabled_flag is 0, it indicates that the merge mode in the current slice can use fractional sample precision. If the flag is not present, the value of slice_fpel_mmvd_enabled_flag is inferred to be 0. The slice_fpel_mmvd_enabled_flag syntax element can be signaled via the slice header.

[0248] The direction index indicates the direction of the MVD relative to the starting point. The direction index can indicate one of four directions as listed in Table 5 below. It should be noted that the meaning of the MVD sign may vary depending on the information about the starting motion vector. If the starting motion vector is a uni-prediction MV or a bi-prediction MV and the two reference pictures point in the same direction of the current picture (i.e., the POCs of the two reference pictures are both greater than or less than the POC of the current picture), the sign specified in Table 5 specifies the sign of the MV offset added to the starting motion vector.

[0249] On the other hand, if the starting motion vector is a bipredicted motion vector but the two lists point in different directions (i.e., the POC of one reference picture is greater than the POC of the current picture and the other is less), the signs in Table 5 specify the signs of the offsets added to the MV components of list 0 (list0), and the MVs of list 1 (list1) have the opposite signs.

[0250] [Table 5]

[0251]

[0252] Both components of the MMVD offset (MmvdOffset[x0][y0]) are derived as follows:

[0253] [Formula 2]

[0254] MmvdOffset[ x0 ][ y0 ]

[0000] = ( MmvdDistance[ x0 ][ y0 ] << 2 ) * MmvdSign[ x0 ][ y0 ][0]

[0255] MmvdOffset[ x0 ][ y0 ]

[0001] = ( MmvdDistance[ x0 ][ y0 ] << 2 ) * MmvdSign[ x0 ][ y0 ][1]

[0256] The MVP (Motion Vector Prediction) mode may also be referred to as the AMVP (advanced motion vector prediction) mode. When the MVP mode is applied, a motion vector predictor (mvp) candidate list can be generated using the motion vectors of the reconstructed spatial neighboring blocks and / or the motion vectors corresponding to the temporal neighboring blocks (or Col blocks). That is, the motion vectors of the reconstructed spatial neighboring blocks and / or the motion vectors corresponding to the temporal neighboring blocks can be used as motion vector predictor candidates. When paired prediction is applied, an mvp candidate list for deriving L0 motion information and an mvp candidate list for deriving L1 motion information can be generated and used separately.

[0257] The above-described prediction information (or information regarding prediction) may include selection information (e.g., MVP flag or MVP index) indicating an optimal motion vector predictor candidate selected from among the motion vector predictor candidates included in the list. At this time, the prediction unit (330) of the decoding device (300) may use the selection information to select a motion vector predictor of the current block from among the motion vector predictor candidates included in the motion vector candidate list.

[0258] The prediction unit (220) of the encoding device (200) can obtain a motion vector difference (MVD) between the motion vector of the current block and the motion vector predictor, and can encode and output it in the form of a bitstream. That is, the motion vector difference can be obtained as a value obtained by subtracting the motion vector predictor from the motion vector of the current block. At this time, the prediction unit (330) of the decoding device (300) can obtain the motion vector difference included in the information regarding the prediction (e.g., motion information), and derive the motion vector of the current block through the addition of the motion vector difference and the motion vector predictor. The prediction unit (330) of the decoding device (300) can obtain or derive a reference picture index indicating a reference picture, etc. from the information regarding the prediction. For example, a motion vector predictor candidate list can be configured as follows:

[0259] Search for spatial candidate blocks for motion vector prediction and insert them into the prediction candidate list.

[0260] Check if the number of spatial candidate blocks is less than 2

[0261] If the number of spatial candidate blocks is less than 2, search for temporal candidate blocks and insert them into the prediction candidate list.

[0262] If no temporal candidate block is available, use zero motion vector.

[0263] If the number of spatial candidate blocks is not less than 2, the construction of the motion vector predictor candidate list is terminated.

[0264] Meanwhile, when the MVP mode is applied, the reference picture index can be explicitly signaled. In this case, the reference picture index for L0 prediction (refidxL0) and the reference picture index for L1 prediction (refidxL1) can be signaled separately. For example, when the MVP mode is applied and bi-prediction (BI prediction) is applied, information about both refidxL0 and refidxL1 can be signaled.

[0265] When the MVP mode is applied, information about the MVD derived from the encoding device (200) as described above may be signaled or encoded and transmitted to the decoding device (300). The information about the MVD may include, for example, information indicating x and y components for the MVD absolute value and sign. In this case, information indicating whether the MVD absolute value is greater than 0 and greater than 1, and the MVD remainder may be signaled in stages. For example, information indicating whether the MVD absolute value is greater than 1 may be signaled only when the value of the flag information indicating whether the MVD absolute value is greater than 0 is 1.

[0266] For example, information about an MVD can be encoded in an encoding device and signaled to a decoding device using the following syntax:

[0267] [Table 6]

[0268]

[0269] For example, MVD[compIdx] can be derived based on abs_mvd_greater0_flag[compIdx] * ( abs_mvd_minus2[compIdx] + 2 ) * ( 1 2 * mvd_sign_flag[compIdx]). Here, compIdx (or cpIdx) represents the index of each component and can have the value 0 or 1. A compIdx value of 0 can represent the x component, and a compIdx value of 1 can represent the y component. However, this is just an example, and values ​​for each component can be represented using a coordinate system other than the x, y coordinate system.

[0270] Meanwhile, MVD for L0 prediction (MVDL0) and MVD for L1 prediction (MVDL1) may be signaled separately, and information about MVD may include information about MVDL0 and / or information about MVDL1. For example, if MVP mode is applied to the current block and BI prediction is applied, information about MVDLO and information about MVDL1 may both be signaled. In addition, information about MVDL1 may be derived from information about MVDL0, or information about MVDL0 may be derived from information about MVDL1.

[0271] Meanwhile, when BI prediction is applied, symmetric MVD mode may be used considering coding efficiency. In this case, signaling of some of the motion information may be omitted. For example, when symmetric MVD mode is applied to the current block, information about refidxL0, information about refidxL1, and information about MVDL1 may not be signaled from the encoding device to the decoding device, but may be derived internally. For example, when MVP mode and BI prediction are applied to the current block, flag information indicating whether symmetric MVD mode is applied (e.g., symmetric MVD flag information or sym_mvd_flag syntax element) may be signaled, and when the value of the flag information is 1, the decoding device may determine that symmetric MVD mode is applied to the current block.

[0272] When the symmetric MVD mode is applied (i.e., when the value of the symmetric MVD flag information is 1), information about mvp_l0_flag, mvp_l1_flag, and MVDL0 may be explicitly signaled, and signaling of information about refidxL0, information about refidxL1, and information about MVDL1 may be omitted and derived internally as described above. For example, refidxL0 may be derived as an index pointing to a previous reference picture that is closest to the current picture in POC order within reference picture list 0 (which may be referred to as list 0 or L0). refidxL1 may be derived as an index pointing to a subsequent reference picture that is closest to the current picture in POC order within reference picture list 1 (which may be referred to as list 1 or L1).

[0273] Or, for example, both refidxL0 and refidxL1 may be derived as 0. Or, for example, the refidxL0 and refidxL1 may be derived as the minimum indices having the same POC difference in relation to the current picture. As a specific example, when [POC of the current picture] - [POC of the first reference picture indicated by refidxL0] is referred to as the first POC difference, and [POC of the second reference picture indicated by refidxL1] is referred to as the second POC difference, only when the first POC difference and the second POC difference are the same, the value of refidxL0 pointing to the first reference picture may be derived as the value of refidxL0 of the current block, and the value of refidxL1 pointing to the second reference picture may be derived as the value of refidxL1 of the current block. Also, for example, if there are multiple sets in which the first POC difference and the second POC difference are the same, refidxL0 and refidxL1 of the set with the minimum difference can be derived as refidxL0 and refidxL1 of the current block.

[0274] MVDL1 can be derived as -MVDL0. For example, the final MV for the current block can be derived as shown in Equation 3 below.

[0275] [Formula 3]

[0276]

[0277] As described above, the SMVD mode can be used when there are reference pictures temporally ahead (forward direction) and backward (backward direction) of the current picture, that is, when the True-Bi prediction condition is satisfied, and the closest reference picture in each direction can be used without signaling the reference picture index.

[0278] When only one motion vector (using a translaton motion model) is used to represent the motion of an encoded block, this motion vector may not actually represent the optimal motion of each pixel. Determining the optimal motion vector at the pixel level can improve encoding efficiency. To this end, we describe an affine motion prediction method that uses an affine motion model to encode. Affine motion prediction can represent the motion vector at the pixel level of a block using two, three, or four motion vectors.

[0279] Figure 13 is a diagram showing four motions expressed in an affine motion model, and Figure 14 is a diagram showing motion vectors used for affine motion prediction.

[0280] The affine motion model can express the four motions shown in Fig. 13. The affine motion model that expresses three of the motions (translation, scale, and rotation) that the affine motion model can express is called the similarity (or simplified) affine motion model, and the following description will be based on the similarity (or simplified) affine motion model. However, the disclosed embodiment is not limited to the motion model.

[0281] As illustrated in Fig. 14, affine motion prediction can determine the motion vector of the pixel position included in the current block (Cur) using two or more control point motion vectors (CPMV). At this time, the set of motion vectors is called an affine motion vector field (MVF) and can be determined by the following equation 3.

[0282] For the 4-parameter affine motion model, the motion vector at sample location (x, y) can be derived according to the following equation.

[0283] [Formula 4]

[0284]

[0285] For the 6-parameter affine motion model, the motion vector at sample location (x, y) can be derived according to the following equation.

[0286] [Formula 5]

[0287]

[0288] Figure 15 is a diagram for a case where the affine MVF is determined in 4*4 subblock units.

[0289] Referring to Figure 15, is the CPMV of the CP at the top-left corner position of the coding block. is the CPMV of the CP at the top-right corner location, is the CPMV of the CP at the bottom-left corner location. And W corresponds to the width of the current block, H corresponds to the height of the current block, is the motion vector at position {x, y}.

[0290] During the encoding / decoding process, the affine MVF can be determined on a pixel-by-pixel basis or on a pre-defined subblock basis. When determined on a pixel-by-pixel basis, a motion vector is obtained based on each pixel value, and when determined on a subblock-by-subblock basis, the motion vector of the corresponding block is obtained based on the pixel value at the center of the subblock (the lower right side of the center, i.e., the lower right sample among the four central samples). In the above example, it is assumed and explained that the affine MVF is determined on a 4*4 subblock basis, but this is for convenience of explanation, and the size of the subblock can be varied.

[0291] That is, when affine prediction is available, the motion models applicable to the current block may include a translational motion model, a 4-parameter affine motion model, and a 6-parameter affine motion model. Here, the translational motion model may represent a model in which existing block-unit motion vectors are used, the 4-parameter affine motion model may represent a model in which two CPMVs are used, and the 6-parameter affine motion model may represent a model in which three CPMVs are used.

[0292] Affine motion prediction may include affine MVP (or affine inter) mode and affine merge. In affine motion prediction, the motion vector of the current block can be derived on a sample-by-sample or sub-block-by-subblock basis.

[0293] In affine merge mode, the control point motion vector (CPMV) can be determined based on the affine motion model of the neighboring blocks encoded using affine motion prediction. Neighboring blocks encoded using affine motion prediction in the search order can be used in affine merge mode. If one or more neighboring blocks are encoded using affine motion prediction, the current block can be encoded using AF_MERGE.

[0294] That is, when the affine merge mode is applied, the CPMVs of the current block can be derived using the CPMVs of the surrounding blocks. In this case, the CPMVs of the surrounding blocks can be used as the CPMVs of the current block as they are, or the CPMVs of the surrounding blocks can be modified based on the size of the surrounding blocks and the size of the current block and then used as the CPMVs of the current block.

[0295] Meanwhile, in the case of affine merge where MV is derived in units of subblocks, it can be called subblock merge mode, and this can be indicated based on merge_subblock_flag (value 1). In this case, the affine merging candidate list described later can also be called a subblock merging candidate list. In this case, the subblock merging candidate list can further include a candidate derived by SbTMVP described later. In this case, the candidate derived by sbTMVP can be used as a candidate for index 0 of the subblock merge candidate list. In other words, the candidate derived by sbTMVP can be positioned before the inherited affine candidates and constructed affine candidates described later in the subblock merge candidate list.

[0296] When affine merge mode is applied, an affine merge candidate list may be constructed to derive CPMVs for the current block. The affine merge candidate list may include, for example, at least one of the following candidates:

[0297] - Inherited affine candidates

[0298] - Constructed affine candidates

[0299] - Zero MV candidates

[0300] Here, the inherited affine candidates are candidates derived based on the CPMVs of the surrounding blocks when the surrounding blocks are coded in the affine mode, the constructed affine candidates are candidates derived by constructing CPMVs based on the MVs of the corresponding CP surrounding blocks for each CPMV unit, and the zero MV candidate can represent a candidate composed of CPMVs whose value is 0.

[0301] The list of affine merge candidates could be structured, for example, as follows:

[0302] - Insert inherited candidates into the candidate list

[0303] - Insert candidates into the candidate list

[0304] - Insert zero MV candidates into the candidate list

[0305] Here, a zero MV candidate can be optionally inserted, for example, into the candidate list when the number of current candidates is less than the number of maximum candidates.

[0306] There can be at most two inherited affine candidates, derived from the affine motion models of the left and upper surrounding blocks, respectively: one from the left surrounding CU, and the other from the upper surrounding CU.

[0307] The positions of the inherited affine moving predictors can be explained with reference to the aforementioned Figure 11.

[0308] Referring back to Figure 11, for the left predictor, the scan order is A0 --> A1, and for the upper predictor, the scan order is B0 --> B1 --> B2. Only the first detected inheritance candidate on each side is selected. No pruning is performed between two inheritance candidates. Once a neighboring affine CU is identified, the CPMVP candidate in the affine merge list of the current CU is derived using the control point motion vectors of the CU.

[0309] Figure 16 is a diagram showing the inheritance of control point motion vectors.

[0310] Referring to the example of Fig. 16, when block A at the lower left is encoded in the affine mode, motion vectors v2, v3, and v4 corresponding to the upper left, upper right, and lower left positions of the CU including block A are acquired. When block A is encoded with a 4-parameter affine model, two CPMVs of the current CU are calculated based on v2 and v3. When block A is encoded with a 6-parameter affine model, three CPMVs of the current CU are v 2 , It is calculated based on v₃ and v₄.

[0311] Figure 17 is a diagram showing an example of the location of a candidate for the configured affine merge mode.

[0312] Constructed affine candidate refers to a method of constructing a candidate by combining the translational motion information of each control point. The motion information for the control point is derived from specific spatial and temporal neighboring blocks as shown in Fig. 17. CPMV (k=1, 2, 3, 4) represents the kth position of the control point. For CPMV1, the order of blocks B2 --> B3 --> A2 is checked, and the MV of the first available block is used. CPMV2 is checked in the order of blocks B1 --> B0, and CPMV3 is checked in the order of blocks A1 --> A0. CPMV4 uses Temporal Motion Vector Predictor (TMVP) if available.

[0313] After acquiring the four control point motion vectors, affine merge candidates are constructed based on the motion information. The control point MVs are combined in the following order:

[0314] {CPMV1, CPMV₂, CPMV₃},

[0315] {CPMV1, CPMV₂, CPMV₄},

[0316] {CPMV1, CPMV₃, CPMV₄},

[0317] {CPMV1, CPMV₃, CPMV₄},

[0318] {CPMV1, CPMV₂},

[0319] {CPMV1, CPMV₃}

[0320] A combination using three CPMVs forms a six-parameter affine merge candidate, and a combination using two CPMVs forms a four-parameter affine merge candidate. To prevent motion scaling, if the reference indices between control points are different, the control point MV combination is discarded.

[0321] In the affine MVP mode, after determining two or more CPMVPs and CPMVs for the current block, the control point motion vector difference (CPMVD) corresponding to the difference value is transmitted from the encoding device (200) to the decoding device (300).

[0322] For example, if the value of affine merge flag or merge_subblock_flag is 0, the affine MVP mode can be applied. Or, for example, if the value of inter_affine_flg is 1, the affine MVP mode can be applied. The affine MVP mode may also be called the affine CP MVP mode. Alternatively, the affine MVP mode may be called the affine inter mode or the inter-affine mode. The affine MVP candidate list described below may be called the control point motion vector predictor candidate list (CPMVP candidate list).

[0323] When the affine MVP mode is applied, an affine MVP candidate list can be constructed to derive CPMVs for the current block. The affine MVP candidate list may include, for example, at least one of the following candidates. For example, the affine MVP candidate list may include at most n candidates (e.g., 2):

[0324] - Inherited affine MVP candidates derived by extrapolation from the control point motion vectors (CPMVs) of surrounding CUs.

[0325] - Affine MVP candidates derived using the translational motion vectors (MVs) of surrounding CUs.

[0326] - Additional candidates based on translational motion vectors from surrounding CUs

[0327] - Zero Motion Vector (Zero MV) Candidate

[0328] Here, the inherited affine MVP candidate is a candidate derived based on the CPMVs of the surrounding blocks when the surrounding blocks are coded in affine mode, the constructed affine MVP candidate is a candidate derived by constructing CPMVs based on the MVs of the CP surrounding blocks for each CPMV unit, and the zero motion vector candidate may represent a candidate composed of CPMVs whose value is 0. When the maximum number of candidates for the affine MVP candidate list is 2, the candidates below 2) above may be considered and added when the current number of candidates is less than 2. In addition, additional candidates based on the translational motion vectors from the surrounding CUs may be derived in the following order:

[0329] - If the number of candidates is less than 2 and the CPMV0 of the constructed candidate is valid, CPMV0 is used as the affine MVP candidate. That is, the candidate whose MVs of CP0, CP1, and CP2 are all equal to the CPMV0 of the constructed candidate is considered.

[0330] - If the number of candidates is less than 2 and the CPMV1 of the constructed candidate is valid, CPMV1 is used as the affine MVP candidate. That is, the candidate whose MVs of CP0, CP1, and CP2 are all equal to the CPMV1 of the constructed candidate is considered.

[0331] - If the number of candidates is less than 2 and the CPMV2 of the constructed candidate is valid, CPMV2 is used as the affine MVP candidate. That is, the candidate whose MVs of CP0, CP1, and CP2 are all equal to the CPMV2 of the constructed candidate is considered.

[0332] - If the number of candidates is less than 2, TMVP is used as the affine MVP candidate.

[0333] The above affine MVP candidates can be composed, for example, as follows:

[0334] - Insert inherited candidates into the candidate list

[0335] - Insert the configured candidates into the candidate list

[0336] - Insert additional candidates based on the translation MVs from surrounding CUs.

[0337] - Insert zero MVs candidates

[0338] However, inserting additional candidates based on the translational MVs from the surrounding CUs and inserting zero MVs candidates can be optionally performed when the current number of candidates is less than 2.

[0339] The examination order of inherited affine MVP candidates is identical to that of inherited affine merge candidates. The only difference is that for MVP candidates, only affine CUs with the same reference picture as the current block are considered. No pruning is applied when inserting an inherited affine motion predictor into the candidate list.

[0340] The constructed MVP candidates are derived only from specific spatially neighboring blocks. The same inspection order as used to construct affine merge candidates is used. Additionally, the reference picture indices of the neighboring blocks are also inspected. The block detected first in the inspection order, encoded in inter mode and having the same reference picture as the current CU, is used.

[0341] The control point motion vector (CPMV) of an affine CU can be stored in a separate buffer. The stored CPMV is only used to generate CPMVPs inherited from affine merge mode and affine AMVP mode for subsequently encoded CUs. Sub-block motion vectors (MVs) derived from the CPMV are used for motion compensation, MV derivation within the merge / AMVP list in translational mode, and deblocking.

[0342] To avoid using the picture line buffer for additional CPMVs, inheritance of affine motion data from a CU located in an above CTU is handled differently from inheritance from a normal neighboring CU. If a candidate CU for affine motion data inheritance exists in an above CTU line, the motion vectors of the lower left and lower right subblocks in that line buffer are used to derive the affine MVP instead of the CPMV. In this way, the CPMV is stored only in the local buffer. If the candidate CU is encoded with a 6-parameter affine, the affine model is degraded to a 4-parameter model.

[0343] Subblock-based Temporal Motion Vector Prediction (SbTMVP) can be used. Similar to TMVP, this method utilizes motion fields within spatially co-located pictures to improve merge mode and motion vector prediction for the current picture's CU. The co-located pictures used in TMVP are also used in SbTMVP. SbTMVP differs from TMVP in two key aspects:

[0344] -TMVP predicts motion at the CU level, whereas SbTMVP predicts motion at the sub-CU level within a CU.

[0345] -TMVP obtains temporal prediction motion vectors from a collocated block (bottom-right or bottom-center of the current CU) in the same-position picture, whereas SbTMVP applies a motion shift before obtaining temporal prediction information from the collocated picture. This motion shift is derived based on the motion vector of one of the spatial neighboring blocks of the current CU.

[0346] Figure 18 shows the procedure of SbTMVP.

[0347] Referring to Fig. 18, SbTMVP predicts the motion vectors of sub-CUs within the current CU in two steps. In the first step, the spatial neighboring block A1 in Fig. 18 (a) is first examined. If block A1 has a motion vector that uses the same location image as a reference image, this motion vector, also called temporal MV (tempMV), is selected as the motion shift to be applied. If such motion information does not exist, the motion shift is set to (0, 0).

[0348] In the second step, the motion shift derived in the first step is applied to the coordinates of the current block and used as a position to obtain motion information (motion vector and reference index) of the sub-CU level within the same-position picture. The example in (b) of Fig. 18 assumes that the motion of the A1 block is set as the motion shift. Then, for each sub-CU, the motion information of the current sub-CU is derived using the motion information of the block (the smallest motion grid including the sub-CU center pixel) within the same-position picture corresponding to the sub-CU. The center sample (below-right center sample) of the sub-CU may correspond to the below-right sample, which is one of the four center samples, when the width and height of the sub-CU are even.

[0349] After the motion information of the same-position sub-CU is identified in this way, the information is converted into the motion vector and reference index of the current sub-CU, similar to the TMVP method. At this time, temporal motion scaling is applied so that the reference picture of the temporal prediction motion vector can be aligned with the reference picture of the current CU.

[0350] For signaling affine merge mode (also known as subblock-based merge mode), a combined subblock-based merge list containing both SbTMVP candidates and affine merge candidates can be used. Whether SbTMVP mode is enabled is controlled by a flag in the Sequence Parameter Set (SPS). When SbTMVP mode is enabled, the SbTMVP predictor is added as the first entry in the subblock merge candidate list, followed by the affine merge candidates. The maximum size of the affine merge candidate list is allowed to be five.

[0351] The sub-CU size used in SbTMVP can be fixed to 8Х8, and like the affine merge mode, the SbTMVP mode can only be applied to CUs whose width and height are both 8 or greater.

[0352] The encoding logic for additional SbTMVP merge candidates is the same as for other merge candidates. That is, for each CU within a P slice or B slice, an additional Rate-Distortion (RD) check may be performed to determine whether to use the SbTMVP candidate.

[0353] GPM can be supported for inter prediction. GPM is a type of merge mode that can be signaled using a CU-level flag. Other merge modes can include regular merge mode, MMVD mode, CIIP mode, and sub-block merge mode. Each possible CU size is wxh = 2. m x 2 n A total of 64 partitions can be supported, where m,n ∈ {3 ... 6}, excluding 8x64 and 64x8.

[0354] When this mode is used, a CU can be divided into two parts by a geometrically positioned straight line. The location of the dividing line can be mathematically derived from the angle and offset parameters of a specific partition. Each part of the geometric partition of the CU can be inter-predicted using its own motion. Only a single prediction is allowed for each partition, i.e., each part has one motion vector and one reference index. A uni-prediction motion constraint can be applied to ensure that only two motion-compensated predictions are required for each CU, similar to conventional bi-prediction.

[0355] Figure 19 illustrates GPM segments grouped at the same angle as an example.

[0356] A single predicted motion for each partition can be derived using a process as illustrated in Fig. 19.

[0357] When GPM is used for the current block, a geometric partition index indicating the partition mode of the geometric partition (angle and offset) and two merge indices (one for each partition) can be additionally signaled. The maximum number of GPM candidate sizes can be explicitly signaled in the SPS, and the syntax binarization of the GPM merge indices can be specified. After predicting each part of the geometric partition, the sample values ​​along the geometric partitioning edge can be adjusted using adaptive weighted blending processing, such as blending along the geometric partitioning edge described below. This is a prediction signal for the entire CU, and the transform and quantization processes can be applied to the entire CU as in other prediction modes. Finally, the motion field of the CU predicted using GPM can be stored in the motion field storage for the GPM geometric partitioning mode.

[0358] Combined inter and intra prediction (CIIP) may be applied to the current block. An additional flag (e.g., ciip_flag) may be signaled to indicate whether the CIIP mode applies to the current CU. For example, when the CU is coded in merge mode, if the CU contains at least 64 luma samples (i.e., the product of the CU width and the CU height is greater than or equal to 64), and both the CU width and the CU height are less than 128 luma samples, the additional flag may be signaled to indicate whether the CIIP mode applies to the current CU.

[0359] CIIP prediction combines inter-prediction signals and intra-prediction signals. The inter-prediction signal P_inter in CIIP mode can be derived using the same inter-prediction process applied in regular merge mode, and the intra-prediction signal P_intra can be derived according to the regular intra-prediction process in planar mode. The intra- and inter-prediction signals can then be combined using a weighted average.

[0360] Figure 20 illustrates neighboring blocks used in CIIP weight derivation. Here, the weight values ​​can be calculated as follows based on the coding modes of the neighboring blocks on the left and above:

[0361] - If the upper neighbor is available and intra-coded, isIntraTop is set to 1, otherwise isIntraTop is set to 0.

[0362] - If the left neighbor is available and intra-coded, isIntraLeft is set to 1, otherwise isIntraLeft is set to 0.

[0363] - If (isIntraTop + isIntraLeft) is 2, wt is set to 3

[0364] - Otherwise, if (isIntraTop + isIntraLeft) is 1, wt is set to 2

[0365] - Otherwise, wt is set to 1

[0366] The CIIP prediction can be constructed as shown in Equation 6 below.

[0367] [Formula 6]

[0368]

[0369] Meanwhile, in a GPM including inter and intra prediction, final predicted samples can be generated by weighting the inter-predicted samples and intra-predicted samples for each GPM-separated region. The inter-predicted samples are derived from the inter-GPM, while the intra-predicted samples can be derived from an intra-prediction mode (IPM) candidate list and an index signaled from an encoding device. For example, the IPM candidate list size can be predefined as 3.

[0370] Figure 21 illustrates available IPM candidates for GPM including inter and intra prediction.

[0371] Available IPM candidates can be parallel angular mode (Parallel mode), perpendicular angular mode (Perpendicular mode), and planar mode with respect to the GPM block boundary, as illustrated in (a), (b), and (c) of Fig. 21. Furthermore, as illustrated in (d) of Fig. 21, GPMs including inter and intra prediction can be restricted to reduce signaling overhead for IPMs and prevent an increase in the intra prediction circuit size of the hardware decoder. In addition, direct motion vector and IPM storage in the GPM blending region can be introduced to further improve coding performance.

[0372] In DIMD and adjacent mode-based IPM derivation, parallel modes can be registered first. Therefore, if there are no identical IPM candidates in the list, up to two IPM candidates derived from the decoder-side intra-mode derivation (DIMD) method and / or neighboring blocks can be registered.

[0373] In deriving the neighbor mode, there are up to five locations for the available neighbor blocks, but this may be limited by the GPM block boundary angles already used in GPM using template matching (GPM-TM).

[0374] Table 7 shows the locations of available neighboring blocks for deriving IPM candidates according to the GPM block boundary angle. A and L can represent the upper and left sides of the predicted block.

[0375] [Table 7]

[0376]

[0377] GPM-Intra can be combined with GPM-MMVD (GPM with merge with motion vector difference). To further improve coding performance, TIMD can be used to generate IPM candidates for GPM-Intra. Parallel modes can be registered first, followed by TIMD, DIMD, and IPM candidates from neighboring blocks.

[0378] Template Matching (TM) is a decoder-side motion vector (MV) derivation method that finds the optimal match between a template of the current coding unit (CU) within the current picture (i.e., the upper and / or left adjacent blocks of the current CU) and a block within a reference picture (i.e., a block of the same size as the template) to improve the motion information of the current coding unit (CU).

[0379] Figure 22 is a diagram illustrating a method for deriving motion vectors from template matching. For example, as illustrated in Figure 22, a more appropriate motion vector is searched for around the initial motion of the current CU within the [-8, +8] pel search range. Furthermore, the search step size may be determined according to the AMVR mode, and TM may be applied continuously with the bidirectional matching process in merge mode.

[0380] In AMVP mode, the MVP candidate is determined based on the template matching error to select the candidate with the smallest difference between the template of the current block and the template of the reference block, and then TM is performed only on the MVP candidate for MV refinement. The TM refines the MVP candidate using an iterative diamond search within the pel search range [-8, +8] starting from full-pel MVD precision (or 4-pel precision in 4-pel AMVR mode). The AMVP candidate can be further refined by a cross search using full-pel MVD precision (or 4-pel precision in 4-pel AMVR mode), and then sequentially refined to half-pel and quarter-pel precision according to Table 8 specified in AMVR mode.

[0381] [Table 8]

[0382]

[0383] This search process ensures that AMVP candidates maintain the same MV precision specified by the AMVR mode even after the TM process. During the search process, if the difference between the previous minimum cost and the current minimum cost in the iteration is less than a threshold equal to the block area, the search process ends.

[0384] In Merge mode, a similar search method is applied to merge candidates specified by the merge index. As shown in Table 8 above, TM can be performed up to 1 / 8 pel MVD precision, or steps exceeding the half-pel MVD precision can be skipped. This depends on whether the alternative interpolation filter used when AMVR is in half-pel mode is used based on the merged motion information.

[0385] Additionally, when TM mode is enabled, template matching can operate as an independent process or as an additional MV refinement process between block-based and sub-block-based bidirectional matching (BM) methods. This depends on whether BM satisfies the activation conditions and can be applied.

[0386] Previously, when use_integer_mv_flag was set to 0 in the slice header, the motion vector of a CU and the predicted motion vector difference (MVD) were signaled in units of quarter-luma samples. In the disclosed embodiment, a CU-level adaptive motion vector resolution (AMVR) scheme is introduced. AMVR allows encoding the MVD of a CU with different precisions. Depending on the current CU mode (normal AMVP mode or affine AMVP mode), the MVD of the CU can be adaptively selected from the following precisions:

[0387] - Normal AMVP mode: quarter luma sample, integer luma sample, or four luma sample

[0388] - Affine AMVP mode: quarter luma samples, integer luma samples, or 1 / 16 luma samples

[0389] The CU-level MVD resolution indication is conditionally signaled only if the current CU contains at least one non-zero MVD component. If all MVD components (i.e., horizontal and vertical MVDs for reference lists L0 and L1) are zero, quarter luma sample MVD resolution is automatically inferred. If the CU has at least one non-zero MVD component, a first flag is signaled to indicate whether an MVD with quarter luma sample precision is used for the CU. If the first flag is zero, no further signaling is required, and an MVD with quarter luma sample precision is used for the current CU. Otherwise, a second flag is signaled to indicate whether an integer luma sample or a 4 luma sample MVD precision is used for regular AMVP CUs. The same second flag is also used for affine AMVP CUs to indicate either integer luma sample or 1 / 16 luma sample MVD precision.

[0390] To ensure that the reconstructed motion vectors have the intended precision (quarter luma samples, integer luma samples, or 4 luma samples), the motion vector predictor (MVP) of the corresponding CU is rounded to the same precision as the MVD before being added to the MVD. The motion vector predictor is rounded towards zero, meaning that a negative MVP is rounded towards positive infinity, and a positive MVP is rounded towards negative infinity.

[0391] The encoding device (200) determines the optimal motion vector precision for the current CU through rate-distortion (RD) cost evaluation. To avoid performing three CU-level RD checks for each MVD precision, in one example, the RD check for MVD precision excluding quarter luma samples can be conditionally performed.

[0392] In the normal AMVP mode, the RD cost of quarter luma samples and integer luma sample precision is first calculated. Then, the RD cost of integer luma sample precision is compared to whether it is smaller than that of quarter luma samples to determine whether the RD cost of 4 luma sample MVD precision needs to be further evaluated. If the RD cost of quarter luma samples is significantly smaller than that of integer luma samples, the RD evaluation for 4 luma sample MVD precision is omitted.

[0393] For affine AMVP mode, after RD cost evaluation for affine merge / skip mode, merge / skip mode, regular AMVP mode with quarter luma sample precision, and affine AMVP mode with quarter luma sample precision, if affine inter mode is not selected, affine inter mode with 1 / 16 luma sample or integer sample precision is not evaluated.

[0394] Additionally, the affine parameters obtained in the affine inter mode with quarter luma sample precision are used as a starting point for exploration in the affine inter mode with 1 / 16 luma sample and integer luma sample precision.

[0395] Meanwhile, to reduce memory load, motion information from previously decoded reference pictures can be stored in fixed-area units. This can be called temporal motion field storage, motion field compression, or motion data compression. In this case, the storage unit for motion information can be set differently depending on whether affine mode is applied.

[0396] For example, the highest precision for explicitly signaled motion vectors is quarter luma samples. In some inter-prediction modes, such as affine mode, motion vectors are derived with 1 / 16th luma sample precision, and motion-compensated prediction is performed with 1 / 16th sample precision. In terms of internal motion field storage, all motion vectors are stored with 1 / 16th luma sample precision.

[0397] For example, for storing the temporal motion field used in TMVP and ATMVP, motion field compression can be performed with 8Х8 granularity.

[0398] Decoder-side Motion Vector Refinement (DMVR) is a method for performing motion prediction by refining motion information from surrounding blocks at the decoder side. When DMVR is applied, the decoder can derive refined motion information through cost comparison based on templates generated using motion information from surrounding blocks in merge / skip mode. This improves motion prediction accuracy and compression performance without requiring additional signaling information.

[0399] Here, for convenience of explanation, the description focuses on the decoding device (300), but DMVR can also be performed in the same manner in the encoding device (200).

[0400] In pairwise prediction, refined MVs are searched for each of the reference picture list L0 and the reference picture list L1 within the search region around the initial MV.

[0401] The Block Matching (BM) method calculates the distortion between two candidate blocks in the reference picture list L0 and list L1. For each MV candidate within the search range surrounding the initial MV, the Sum of Absolute Differences (SAD) between the differential MVs is calculated.

[0402] The MV candidate with the smallest SAD becomes the refined MV, which is used to generate the biprediction signal.

[0403] The decoding device (300) may invoke the DMVR process to improve the accuracy of the initial motion compensation prediction (i.e., motion compensation prediction through the conventional merge / skip mode). For example, the decoding device (300) may perform the DMVR process when the prediction mode of the current block is the merge mode or the skip mode, and bidirectional bidirectional prediction is applied to the current block in which the bidirectional reference pictures are in opposite directions with respect to the current picture in display order.

[0404] For example, DMVR can be applied to CUs encoded with the following modes and conditions:

[0405] - Merge mode at CU level with bidirectional MV

[0406] - One reference image is in the past, and the other reference image is in the future, based on the current image.

[0407] - The distance between the two reference images and the current image (i.e., the POC difference) is the same.

[0408] - CU contains more than 64 luma samples

[0409] - The height and width of the CU are both greater than 8 luma samples.

[0410] - BCW (weighted bi-prediction) weight index indicates equal weights

[0411] - WP (weighted prediction) is not applied to the current block.

[0412] The refined MV derived from the DMVR process is used to generate inter-prediction samples and also to generate temporal MVPs for future image encoding. Meanwhile, the original MV is used for deblocking processing and also to generate spatial MVPs for future CU encoding.

[0413] Meanwhile, the IBC (Intra Block Copy) mode can apply similar concepts to the motion vector, motion vector difference, and motion vector predictor used in the inter prediction mode. This will be described in detail below.

[0414] The IBC mode is implemented as a block-level encoding mode, and the encoding device (200) performs block matching (BM) to find an optimal block vector (or motion vector) for each coding unit (CU). Here, the block vector represents the displacement from the current block to a reference block already restored within the same image.

[0415] Luma block vectors of CUs encoded with IBC have integer precision, and chroma block vectors are also rounded to integers. When combined with AMVR, IBC mode can optionally use motion vector precision of 1 pixel (pel) or 4 pixels.

[0416] CUs encoded with IBC can be treated as a third prediction mode, separate from the intra or inter prediction modes. IBC mode is applicable to CUs whose luma block width and height are each 64 pixels or less.

[0417] The encoding device (200) performs hash-based motion estimation for IBC. In particular, RD-based judgment is performed for blocks with either a width or height of 16 pixels or less. In non-merge mode, hash-based search is performed first, and if no valid candidate is found, local block matching-based search is additionally performed.

[0418] In hash-based search, the hash key (32-bit CRC) matching between the current block and the reference block is extended to all allowed block sizes. For every location in the current image, the hash key is calculated based on the 4X4 subblock. If the current block is larger, the hash key of the reference block is determined to match if the hash keys of all 4X4 subblocks constituting the current block all match the hash key of the reference location. If multiple reference blocks have the same hash key, the block vector cost is calculated for each reference block, and the reference block with the minimum cost is selected.

[0419] In block matching search, the search range is set to include both the previous and current CTUs. At the CU level, IBC mode is signaled via a flag, which can be either IBC AMVP mode or IBC Skip / Merge mode:

[0420] - IBC Skip / Merge Mode: A merge candidate index is used to indicate which block vector from the list of candidate block vectors generated from surrounding IBC-encoded blocks to use to predict the current block. The merge list consists of spatial candidates, history-based motion prediction (HMVP) candidates, and pairwise candidates.

[0421] - IBC AMVP mode: The block vector difference is encoded in the same way as the motion vector difference (MVD). The block vector prediction method uses one prediction candidate each from the left and upper neighboring blocks (if encoded with IBC). If either neighboring block is unavailable, the default block vector is used as the predictor. A flag is signaled to indicate the index of the block vector predictor candidate used.

[0422] The IBC Merge / AMVP list configuration can be modified as follows:

[0423] - Only if an IBC merge / AMVP candidate is valid, can the candidate be inserted into the IBC merge / AMVP candidate list.

[0424] - The above-right, bottom-left, and above-left spatial candidates and one pairwise average candidate can be added to the IBC merge / AMVP candidate list.

[0425] - Template-based adaptive reordering (ARMC-TM: Adaptive Reordering based on Matching Cost - Template Matching) is applied to the IBC merge list.

[0426] The size of the HMVP table for IBC can be increased to 25. After a maximum of 20 IBC merge candidates are derived through full pruning, these candidates are reordered together. After reordering, the six candidates with the lowest template matching cost are selected as candidates for the final IBC merge list.

[0427] The zero vector candidates used to populate the IBC merge / AMVP list are replaced with a set of Block Vector Predictor (BVP) candidates located within the IBC reference area. The zero vector is not considered a valid block vector in IBC merge mode and is therefore discarded as a BVP from the IBC candidate list.

[0428] Figure 23 is a diagram showing padding candidates for replacing zero vectors in the IBC list.

[0429] Referring to Figure 23, three candidates are located at the nearest corners of the reference area, and three additional candidates are determined at the centers of three sub-areas (A, B, C). The coordinates of each candidate are determined based on the width and height of the current block, and the △X and △Y parameters.

[0430] Template matching can be used in both IBC merge mode and IBC AMVP mode. The IBC-TM merge list is modified compared to the list used in regular IBC merge mode, and candidates are selected based on a pruning method based on the motion distance between candidates, similar to regular TM merge mode. The last zero motion candidate is replaced with the motion vectors left (-W, 0), up (0, -H), and up-left (-W, -H), where W represents the width of the current CU and H represents the height.

[0431] In IBC-TM merge mode, selected candidates are refined using a template matching technique prior to Rate-Distortion Optimization (RDO) or decoding processing. IBC-TM merge mode competes with the traditional IBC merge mode, and the TM-merge flag is signaled.

[0432] In IBC-TM AMVP mode, up to three candidates are selected from the IBC-TM merge list. Each of these three candidates is refined using a template matching technique and sorted based on the template matching cost of the refined results. Only the top two candidates are then considered in the motion estimation process, as in the conventional method.

[0433] Figure 24 is a diagram showing an example of another IBC reference area at the current CU location.

[0434] Template-matching refinement is straightforward in both IBC-TM merge and AMVP modes. This is because the IBC motion vectors must (i) be integers and (ii) lie within the reference region depicted in Figure 24. Therefore, in IBC-TM merge mode, all refinements are performed with integer precision, while in IBC-TM AMVP mode, they are performed with either integer or 4-pixel precision, depending on the AMVR value. This refinement is performed directly by accessing samples without interpolation. In both cases, the refined motion vectors and the templates used at each refinement step must satisfy the constraints of the reference region.

[0435] Figure 25 is a diagram illustrating an example of a reference area when a CTU is coded. The area highlighted in bold in Figure 25 is the reference area, and the white area is an invalid area.

[0436] The reference region of IBC can extend upwards by two CTU rows. Fig. 25 shows the reference region for CTU (m, n). Specifically, when coding CTU (m, n), the reference region includes CTUs with indices (m-2, n-2)...(W, n-2), (0, n-1)...(W, n-1), (0, n)...(m, n), where W represents the maximum horizontal index within the current tile, slice, or picture. When the CTU size is 256, the reference region is limited to one CTU row upwards. The search range of the pixel-wise block vector search (also called local search) is limited to [-(C << 1), C >> 2] in the horizontal direction and [-C, C >> 2] in the vertical direction to match the reference region extension, where C represents the CTU size.

[0437] As described above, HMVP can store a predefined number of decoded motion information in an independent buffer, and the stored motion information can be considered in the subsequent process of constructing the MVP candidate list. The independent buffer for HMVP is basically initialized in units of CTU Rows for parallel processing of the decoder, and therefore the first coding block (coding unit) of each CTU Row cannot use an HMVP candidate as an MVP candidate. The first coding block of each CTU Row not only has limited use of HMVP candidates, but also may not have spatially adjacent blocks available, so there is a possibility that it will have a prediction block with a large motion vector difference or low accuracy. This can occur more frequently when dividing the screen using methods such as SUB-PICTIRE / SLICE / TILE, and can cause a degradation in compression performance. According to one embodiment, when initializing the motion information stored in the HMVP buffer, compression performance can be improved by considering default motion information determined by considering the characteristics of the image as an HMVP candidate. Here, the default motion information may include various motion information such as a default motion vector and a default reference picture index. However, since the following embodiment is described with a focus on a motion vector, the default motion information and the default motion vector may be used interchangeably.

[0438] Various default motion information may be used for the above HMVP candidates. The default motion information may collectively refer to motion information that can be considered when the HMVP buffer is initialized and predefined motion information.

[0439] For example, the default motion information may have a predefined fixed value. By using the value to derive unidirectional or bidirectional motion information, the HMVP buffer can be initialized to the default value.

[0440] As another example, the default motion information may be derived by considering the characteristics of the image. The characteristics of the image may be determined by the difference in POC between the current picture and the reference picture. Alternatively, the temporal ID difference value and the difference in QP (Quantization Parameter) between the current picture and the reference picture may be determined as the characteristics of the image. For example, the default motion vector may be derived as a scaled value for the reference picture in the L0 and L1 directions by using the characteristics of the image, and the candidates for the default motion vector may vary depending on whether the scaling value changes on a picture-by-picture or slice-by-slice basis.

[0441] The above default motion vector can be derived by considering the characteristics of another image. Another characteristic of the image can be the movement of an object within a reference picture. A global motion vector can be derived by considering the motion vector within the reference picture and the directionality of the motion vector, and a default motion vector can be constructed using the global motion vector.

[0442] The above default motion vector can be considered during the initialization process of HMVP and can be applied to various prediction modes or prediction tools, such as inter mode, merge mode, affine mode, and GPM mode. Furthermore, it is not limited to the HMVP initialization process and can be utilized in various ways, such as being considered as a single MVP candidate or in place of a zero vector.

[0443] Specific examples are described in detail below.

[0444] Typically, MVP candidates can include motion information of spatially adjacent blocks of the current block (spatial candidates), motion information of temporally adjacent blocks (temporal candidates), HMVP candidates, pairwise average candidates, and zero vector candidates. Motion vectors within the HMVP buffer may be included in the process of constructing these MVP candidates. The HMVP buffer stores motion information of previously decoded blocks, and when the buffer is full, the motion vectors filled in the buffer are removed first and the motion vectors of the recently decoded blocks are stored in a FIFO manner. Furthermore, when a candidate identical to the recently decoded motion information exists within the buffer, the candidate within the buffer is removed, and the most recently decoded motion vector is stored, thereby increasing the priority of the overlapping candidate. Among the candidates filled in the HMVP buffer, candidates recently added to the buffer, i.e., candidates with a large buffer index, may be given priority in the MVP candidate construction process. To ensure candidate diversity, variations such as giving priority to candidates with a small buffer index may be added.

[0445] Fig. 26 is a flowchart illustrating an example of a decoding method according to one embodiment.

[0446] A decoding method according to one embodiment can be performed by the decoding device (300) described above. In the embodiment described below, descriptions are made focusing on contents not described above to avoid redundant descriptions, and the description described below does not solely support the embodiment of the decoding method. The description of the operation of the decoding device (300) described above, the description of the decoding method (e.g., FIGS. 1, 3, 4, 8 to 10, etc.), and the descriptions of various prediction modes or prediction types (FIGS. 11 to 25) can be equally applied to the embodiment described below as long as they do not conflict with each other.

[0447] Referring to FIG. 26, a decoding method according to one embodiment includes a step of obtaining image information from a bitstream (S1000), a step of constructing an MVP candidate list including MVPs (Motion Vector Predictors) for a current block based on the obtained image information (S1010), and a step of generating a prediction block for the current block based on at least one MVP candidate in the MVP candidate list (S1020).

[0448] Descriptions of steps S1000, S1010, and S1020 are the same as those described above in FIGS. 1, 3, 4, 8 to 10, etc., so redundant descriptions are omitted here.

[0449] In addition, a process of deriving a residual sample and generating a restored sample can be performed, and since the description thereof is the same as described above, the description thereof is omitted here.

[0450] The step of constructing the MVP candidate list (S1010) may include adding motion information stored in the HMVP buffer to the MVP candidate list. As described above, the MVP candidates may include motion information of spatially adjacent blocks of the current block (spatial candidates), motion information of temporally adjacent blocks (temporal candidates), HMVP candidates, pairwise average candidates, zero vector candidates, etc. Accordingly, in certain cases (e.g., when the MVP candidate list is not filled with only motion information of spatially adjacent blocks or motion information of temporally adjacent blocks), motion information stored in the HMVP buffer may be added to the MVP candidate list.

[0451] As explained above, the HMVP buffer stores a predetermined number of motion information of decoded blocks, and the stored motion information is deleted when a predetermined condition is satisfied. That is, the HMVP buffer can be initialized. For example, the HMVP buffer can be initialized in units of CTU Rows. That is, the HMVP buffer can be initialized when the CTU to be decoded is the first CTU of the CTU Row, that is, the CTU located in the first column.

[0452] In a decoding method according to one embodiment, when the HMVP buffer is initialized, the HMVP buffer may not be emptied, but default motion information derived or determined according to predetermined conditions may be stored. By storing predefined motion information instead of emptying the buffer during the process of initializing the motion information stored in the HMVP buffer, a wider variety of motion information can be utilized, and compression performance degradation that may occur due to the initialization of the HMVP buffer can be prevented.

[0453] The above default motion information may include multiple motion vectors having fixed values. A detailed description of the default motion vectors determined by fixed values ​​will be provided later.

[0454] The above default motion information includes a default motion vector for a first direction and a default motion vector for a second direction, and the default motion vector for the second direction can be determined based on the default motion vector for the first direction. This case is a case where pair prediction is applied to the current block, and a detailed description thereof will be provided later.

[0455] The above default motion information may be determined based on the characteristics of the image. In this case, the default motion information may be determined by applying scaling based on the distance between the current picture and the reference picture to a pre-default motion vector having a fixed value.

[0456] Alternatively, the default motion information may be determined by applying scaling based on the difference between the temporal ID of the current picture and the temporal ID of the reference picture to a free default motion vector having a fixed value.

[0457] Alternatively, the default motion information may be determined by applying scaling based on the difference between the time QP (quantization parameter) of the current picture and the QP (quantization parameter) of the reference picture to a free default motion vector having a fixed value.

[0458] Alternatively, the default motion information may be determined based on prediction mode information or motion information of the decoded pictures.

[0459] A predetermined condition for deleting motion information stored in an HMVP buffer may include completion of decoding for n (where n is an integer greater than or equal to 2) Coding Tree Unit (CTU) rows.

[0460] Fig. 27 is a flowchart illustrating an encoding method according to one embodiment.

[0461] An encoding method according to one embodiment can be performed by the encoding device (200) described above. In the embodiment described below, descriptions are made focusing on contents not described above to avoid redundant descriptions, and the description described below does not solely support the embodiment of the encoding method. The descriptions regarding the operation of the encoding device (200) described above, the descriptions regarding the decoding method (e.g., FIGS. 1, 2, 5, 6, 7, 10, etc.), and the descriptions regarding various prediction modes or prediction types (FIGS. 11 to 25) can be equally applied to the embodiment described below as long as they do not conflict with each other.

[0462] Referring to FIG. 27, an encoding method according to one embodiment includes a step of determining a prediction mode of a current block (S1100), a step of constructing an MVP candidate list including MVP candidates for the current block based on the prediction mode (S1110), and a step of generating residual samples for the current block based on at least one MVP candidate in the MVP candidate list (S1120).

[0463] Descriptions of steps S1100, S1110, and S1120 are the same as those described previously in FIGS. 2, 5, 6, 7, and 10, so redundant descriptions are omitted here.

[0464] Additionally, image information including residual information and prediction mode information can be encoded. Since the description thereof is the same as described above, a detailed description thereof is omitted here.

[0465] The step of constructing the MVP candidate list (S1110) may include adding motion information stored in the HMVP buffer to the MVP candidate list. As described above, the MVP candidates may include motion information of spatially adjacent blocks of the current block (spatial candidates), motion information of temporally adjacent blocks (temporal candidates), HMVP candidates, pairwise average candidates, zero vector candidates, etc. Accordingly, in certain cases (e.g., when the MVP candidate list is not filled with only motion information of spatially adjacent blocks or motion information of temporally adjacent blocks), motion information stored in the HMVP buffer may be added to the MVP candidate list.

[0466] As previously explained, the HMVP buffer stores a predetermined number of motion information of decoded blocks, and when a predetermined condition is satisfied, the stored motion information is deleted. That is, the HMVP buffer can be initialized. For example, the HMVP buffer can be initialized in units of CTU rows. That is, the HMVP buffer can be initialized when decoding of one CTU row is completed.

[0467] In an encoding method according to one embodiment, when the HMVP buffer is initialized, the HMVP buffer may not be emptied, but default motion information derived or determined according to predetermined conditions may be stored. By storing predefined motion information instead of emptying the buffer during the process of initializing the motion information stored in the HMVP buffer, a wider variety of motion information can be utilized, and compression performance degradation that may occur due to the initialization of the HMVP buffer can be prevented.

[0468] The above default motion information may include multiple motion vectors having fixed values. A detailed description of the default motion vectors determined by fixed values ​​will be provided later.

[0469] The above default motion information includes a default motion vector for a first direction and a default motion vector for a second direction, and the default motion vector for the second direction can be determined based on the default motion vector for the first direction. This case is a case where pair prediction is applied to the current block, and a detailed description thereof will be provided later.

[0470] The above default motion information may be determined based on the characteristics of the image. In this case, the default motion information may be determined by applying scaling based on the distance between the current picture and the reference picture to a pre-default motion vector having a fixed value.

[0471] Alternatively, the default motion information may be determined by applying scaling based on the difference between the temporal ID of the current picture and the temporal ID of the reference picture to a free default motion vector having a fixed value.

[0472] Alternatively, the default motion information may be determined by applying scaling based on the difference between the time QP (quantization parameter) of the current picture and the QP (quantization parameter) of the reference picture to a free default motion vector having a fixed value.

[0473] Alternatively, the default motion information may be determined based on prediction mode information or motion information of the decoded pictures.

[0474] A predetermined condition for deleting motion information stored in an HMVP buffer may include completion of decoding for n (where n is an integer greater than or equal to 2) Coding Tree Unit (CTU) rows.

[0475] The bitstream generated by the encoding method described above can be temporarily stored in a computer-readable storage medium. In addition, the bitstream can be generated according to the encoding method described above and then transmitted to a decoding device (300), etc. A transmission device that transmits the bitstream can include at least one processor that generates the bitstream according to the encoding method, and a transmitter that transmits the generated bitstream.

[0476] Below, a detailed description is provided of a method for deriving or determining default motion information stored in an HMVP buffer. The following description can be equally applied to the decoding and encoding methods according to one embodiment.

[0477] As described above, in order to prevent compression performance degradation that may occur due to the initialization of the HMVP buffer, predefined motion information is stored instead of emptying the buffer during the process of initializing the motion information stored in the buffer, so that more diverse motion information can be utilized. In the present embodiment, the predefined motion information can be expressed as default motion information. In the present embodiment, the default motion vector is mainly described, but motion information including information such as a prediction mode, a motion vector, a reference picture index, and a BCW index can be included in the default motion information.

[0478] For example, when the size of the HMVP buffer is N (N>0, for example, N is 5) and the x, y components of the default motion vector, D_MV, are expressed as (x, y), the default motion vector for HMVP can have a fixed value as follows:

[0479] D_MV[0] = (4, 0)

[0480] D_MV[1] = (0, 4)

[0481] D_MV[2] = (4, 8)

[0482] D_MV[3] = (8, 4)

[0483] D_MV[4] = (8, 8)

[0484] The default motion vectors listed above are just examples, and more default motion vectors can be configured as the buffer size increases. Furthermore, the HMVP buffer can be filled entirely or partially with default motion vectors.

[0485] Since the compression performance can be improved depending on the order in which the motion vectors included in the HMVP buffer are inserted into the MVP candidate list, the order of the default motion vectors or the order in which the default motion vectors are inserted into the MVP candidate list can also be changed. Accordingly, the order in which the default motion vectors are filled into the MVP candidate list can be determined differently depending on the mode using inter mode, merge mode, or other history-based motion information. The above example of the default motion vector can be applied to unidirectional prediction.

[0486] The default motion vector for bi-prediction can be defined as follows. When both a picture that is temporally preceding the current picture and a picture that is later than the current picture exist in the reference picture list of the current picture, the default motion vector for bi-prediction can be derived. As an example of configuring the default motion vector for bi-prediction, when the x, y components of the default motion vector D_MV0 in the L0 direction are (x_0, y_0), the x, y components (x_1, y_1) of the default motion vector D_MV1 in the L1 direction can be derived as (x_1, y_1) = (-x_0, -y_0). In addition, when two or more pictures that are temporally preceding the current picture exist in the reference picture list of the current picture, the default motion vector for bi-prediction can be derived. For example, when the x, y components of the default motion vector D_MV0 in the L0 direction are (x_0, y_0), the x, y components (x_1, y_1) of the default motion vector D_MV1 in the L1 direction can be derived as (x_1, y_1) = scale * (x_0, y_0). In this case, the scale can be a scale value derived based on the POC difference between the current picture and the L0 picture and the POC difference between the current picture and the L1 picture.

[0487] Additionally, when there are only pictures that are temporally ahead of the current picture in the reference picture list of the current picture, it is possible to restrict the derivation to only a unidirectional default motion vector.

[0488] Among the default motion information, the default motion vector can be derived and set according to the aforementioned method, and the reference picture index can use a predefined value. Each default motion information can have a different reference picture index value, or it can have the same reference picture index value.

[0489] For example, the predefined value may be 0. Or, it may be an index pointing to the picture closest to the current picture. Or, it may include information such as the BCW index and whether to apply a half-pel interpolation filter as default motion information, and these values ​​may also be used in advance. For example, the predefined value of the BCW index may be a default value meaning equal weight. In addition, the predefined value for whether to apply a half-pel interpolation filter may be FALSE.

[0490] In one embodiment, default motion information can be determined by considering the characteristics of the image. This method can be combined with the previously described method (setting to a predefined fixed value).

[0491] When the size of the independent buffer for HMVP is N (N>0, for example, N is 5) and the x, y components of the pre-default motion vector, D_MV, are expressed as (x, y), the pre-default motion vector for HMVP can be defined as follows. In the present embodiment, a default motion vector defined in advance before reflecting the characteristics of the image is referred to as a pre-default motion vector:

[0492] D_MV[0] = (4, 0)

[0493] D_MV[1] = (0, 4)

[0494] D_MV[2] = (4, 8)

[0495] D_MV[3] = (8, 4)

[0496] D_MV[4] = (8, 8)

[0497] The above motion vectors are just an example, and as the buffer size increases, more free default motion vectors can be defined. The free default motion vectors can be changed as follows, taking into account the characteristics of the image. Specifically, the x, y components (x_0, y_0) of the default motion vector D_MV0 in the L0 direction and the x, y components (x_1, y_1) of the default motion vector D_MV1 in the L1 direction can be configured as follows using the free default motion vectors:

[0498] (x_0, y_0) = scale0 * (x, y)

[0499] (x_1, y_1) = scale1 * (x, y)

[0500] In addition, the above-defined free default motion vector can be used as the default motion vector of L0, and the default motion vector of L1 can be derived by considering the characteristics of the image. That is, the default motion vector in the L0 direction and the default motion vector in the L1 direction can be derived as follows:

[0501] (x_0, y_0) = (x, y)

[0502] (x_1, y_1) = scale1 / scale0 * (x, y)

[0503] The form of multiplication using the above scaling value can be applied by changing it to the form of shift (shift0, shift1), and the offset (offset0, offset1) in the calculation formula can be applied.

[0504] (x_0, y_0) = ((x, y) << shift0) + offset0

[0505] (x_1, y_1) = ((x, y) << shift1) + offset1

[0506] Fig. 28 is a drawing showing the distance between the current picture and the reference picture in each direction, and Fig. 29 is a drawing showing an example of a method for applying a scaling value for each section.

[0507] The image characteristics for determining default motion information in each direction are as follows, and more than one characteristic can be considered together:

[0508] 1) The default motion vector can be determined based on the distance between the current picture and the reference picture. Whether the motion vector is unidirectional or bidirectional can be determined based on the presence or absence of available reference pictures and the POC of the reference pictures, and the motion vector can be determined based on the distance (POC Difference) between the reference picture and the current picture in each direction.

[0509] 1.1) The scale0 and scale1 values ​​can be determined based on the distance (d0) between the current picture and the reference picture in the L0 direction and the distance (d1) between the current picture and the reference picture in the L1 direction, as shown in FIG. 28. For example, when the current picture and reference pictures in each direction exist, the values ​​of d0 and d1 themselves can be used as the values ​​of scale0 and scale1 for deriving the default motion vector. The method using the POC difference is not limited to the POC difference value itself, and scaling can also be applied using the difference value between d0 and d1. That is, the default motion vector in one direction can be fixed and used using |d0 - d1|, and scaling can be applied when deriving the default motion vector in the other direction. In addition, scaling can be applied using the ratio between d0 and d1. That is, the default motion vector in one direction can be fixed and used using d1 / d0 or d0 / d1, and scaling can be applied when deriving the default motion vector in the other direction. Additionally, a (-) sign may be applied to the scaling value depending on the direction of the reference picture. For example, if the L0 reference picture is temporally ahead of the current picture and the L1 reference picture is temporally behind the current picture, a (+) sign may be applied to the scaling value for the motion vector in the L0 direction, and a (-) sign may be applied to the scaling value for the motion vector in the L1 direction.

[0510] 1.2) After dividing the distance between the current picture and the reference picture in the L0 direction (d0) and the distance between the current picture and the reference picture in the L1 direction (d1) into specific areas, different scaling values ​​can be applied to each section. CASE1 in Fig. 29 shows an example in which the distance between the current picture and the reference picture is divided into equal intervals, and CASE2 shows an example in which the distance is divided unequally. The representative values ​​for each section can be used as the values ​​of scale0 and scale1 to derive the default motion vector, and examples in which the start and midpoints of each section are used as representative values ​​are shown in CASE1 and CASE2. This is just one example, and the method for determining the representative value can be applied by changing it to a specific value within the section. If the scale value is 0, the default motion vector may not be changed (i.e., the product of 0 is not applied). The method using the POC difference is not limited to the POC difference value itself, and the difference value between d0 and d1 can be divided into sections and then scaled using the representative value.

[0511] 1.3) Whether to apply scaling can be determined based on the distance (d0) between the current picture and the reference picture in the L0 direction and the distance (d1) between the current picture and the reference picture in the L1 direction. For example, whether to apply scaling can be determined if the difference between d0 and d1 is greater than a predefined threshold. At this time, examples of predefined thresholds such as 8 and 16 can be considered.

[0512] 2) The default motion vector can be determined based on the difference between the temporal ID (time ID) of the current picture and the temporal ID of the reference picture. For HMVP, when a predefined reference picture index exists, the difference between the temporal ID of the current picture and the temporal ID of the reference picture in each direction can be considered for the unidirectional or bidirectional motion vector. The method of determining the default motion vector using the above POC difference can be applied in a similar way to the method considering the temporal ID. That is, the difference between temporal IDs can be used as a scaling value for the default motion vector, and after quantizing the difference between temporal IDs into a specific section, the representative value of each section can be used as a scaling value for the default motion vector. In addition, whether to apply scaling for the default motion vector can be determined based on the temporal ID value.

[0513] 3) The default motion vector can be determined based on the difference between the QP (quantization parameter) of the current picture and the QP of the reference picture. When unidirectional or bidirectional reference picture indices for HMVP exist in a predefined manner, the difference between the QP of the current picture and the QP of the reference picture in each direction can be considered. The method of determining the default motion vector using the above POC difference can be applied in a similar way to the method considering the QP. That is, the difference value between QPs can be used as a scaling value for the default motion vector, and after quantizing the difference value between QPs into a specific section, the representative value of each section can be used as a scaling value for the default motion vector. In addition, whether to apply scaling for the default motion vector can be determined based on the QP value.

[0514] 4) The default motion vector to be included in the HMVP buffer can be determined by considering the resolution of the current picture. For example, if the resolution is large, a motion vector with a large value can be assigned because it tends to have greater motion. Also, if the resolution is small, a motion vector with a small value can be assigned. This is just one example, and the resolution of the reference picture can be changed. In addition, it is possible to configure multiple default motion vectors according to various conditions. Specific examples are shown below.

[0515] 4.1) If the resolution of the current picture is greater than 1280x720

[0516] D_MV[0] = (16, 8)

[0517] D_MV[1] = (8, 16)

[0518] D_MV[2] = (16, 32)

[0519] D_MV[3] = (32, 16)

[0520] D_MV[4] = (32, 32)

[0521] 4.2) If the resolution of the current picture is less than or equal to 1280x720

[0522] D_MV[0] = (4, 0)

[0523] D_MV[1] = (0, 4)

[0524] D_MV[2] = (4, 8)

[0525] D_MV[3] = (8, 4)

[0526] D_MV[4] = (8, 8)

[0527] Meanwhile, motion information and mode information from previously decoded pictures can also be used as image characteristics to determine default motion information. A specific method for utilizing information from previously decoded pictures is as follows:

[0528] 1) The prediction mode information of a previously decoded picture can be used. For example, regular mode and non-regular mode can be considered. Here, regular mode refers to AMVP and regular merge mode, and non-regular mode is a mode that expresses relatively complex motion, and may include affine mode, GPM mode, CIIP mode, regular merge mode with motion vector refinement, etc. However, the above definition of prediction mode is an example, and the method of distinguishing prediction modes may be changed. The occurrence rate of the prediction mode of a decoded picture can be determined by accumulating the block size (width x height) of the block having the corresponding mode, or more simply, the occurrence rate can be determined by counting the number of occurrences of the prediction mode. Based on this, the default motion vector of the current picture can be determined as follows:

[0529] 1.1) If the occurrence rate of regular merge mode is higher than that of non-regular merge mode.

[0530] D_MV[0] = (16, 8)

[0531] D_MV[1] = (8, 16)

[0532] D_MV[2] = (16, 32)

[0533] D_MV[3] = (32, 16)

[0534] D_MV[4] = (32, 32)

[0535] 1.2) When the occurrence rate of non-regular merge mode is higher than that of regular merge mode.

[0536] D_MV[0] = (4, 0)

[0537] D_MV[1] = (0, 4)

[0538] D_MV[2] = (4, 8)

[0539] D_MV[3] = (8, 4)

[0540] D_MV[4] = (8, 8)

[0541] 2) Motion information from previously decoded pictures can be utilized. In this case, the motion vector of a previously decoded picture can be described interchangeably as a global vector, global motion vector, or global movement. The following methods can be considered to determine the characteristics of an image using a global vector.

[0542] 2.1) The motion vector of the decoded picture is quantized into predefined sections so that the degree of motion of each picture can be identified. Specifically, when the motion vector (x component, y component) of each block exists, each x, y component value can be rounded (quantized) to a predefined value and divided into sections. At this time, the rounding (quantization) step can be changed, and the representative value of the most frequently occurring motion section can be determined as the global vector. The number of global vectors is not limited to one, and multiple candidates can be configured. At this time, a fixed number of global vectors can be configured according to the occurrence frequency (number of occurrences or area of ​​the block with the corresponding value) of each x, y component section. In addition, the global vector can be modified, such as configuring multiple global vectors in a form where an offset is applied based on one representative global vector. The candidates determined in this way can be considered as default motion vectors applied at the time of initialization of the HMVP buffer.

[0543] 2.2) The direction of motion can be determined based on the sign of the motion vector of the decoded picture. Fig. 30 is a diagram showing an example of a combination of signs that the motion vector of the decoded picture can have. Specifically, when the motion vector (x component, y component) of each block exists, each component can have a positive sign and a negative sign. Therefore, the motion vector can be divided into four sign combinations, such as (+, +), (+, -), (-, +), (-, -), as shown in (a) of Fig. 30. At this time, the sign combination of the motion vector can be changed depending on whether 0 is included in each x, y component, such as (x > 0, y >= 0), (x >= 0, y < 0), (x < 0, y >= 0), (x < 0, y <= 0). Alternatively, the direction of motion can be determined excluding the value of 0. This is just one example, and it is also possible to determine the direction of the global vector in more detail by dividing it into 8 or 16 areas, as shown in (b) of Fig. 30. Through this, the motion vector for each coding block (coding unit) in the decoded picture can be determined as one of 4, 8, or 16 code combinations, and the most common code combination can be determined as the motion direction of the global vector. The directionality of the global vector is not limited to one and multiple candidates can be configured. At this time, the directionality of a fixed number of global vectors can be configured according to the occurrence frequency (number of occurrences or area of ​​the block with the corresponding value) of each section of each x, y component. The candidates determined in this way can be considered as a tool for determining the directionality of the default motion vector applied at the time of initializing the HMVP buffer. As an example, some examples are shown below, such as when the (+, +) combination is determined to be the directionality of the global vector, the default motion vector is set to include the motion vector of the (+, +) combination.

[0544] 2.2.1) When the direction of the global vector is a combination of (+,+)

[0545] D_MV[0] = (16, 8)

[0546] D_MV[1] = (8, 16)

[0547] D_MV[2] = (16, 32)

[0548] D_MV[3] = (32, 16)

[0549] D_MV[4] = (32, 32)

[0550] 2.2.2) When the direction of the global vector is a combination of (-,-)

[0551] D_MV[0] = (-16, -8)

[0552] D_MV[1] = (-8, -16)

[0553] D_MV[2] = (-16, -32)

[0554] D_MV[3] = (-32, -16)

[0555] D_MV[4] = (-32, -32)

[0556] 2.3) When considering the global vector and the directionality of the global vector, some sections within the picture may be excluded. For example, since the background within the picture has little motion information, the global vector and the directionality of the global vector may be calculated by not including the motion vector of the edge part of the picture. As another example, if the x and y components of the motion vector have a value of '0', the value may be excluded because it may overlap with the zero vector as a candidate for MVP. As another example, if the x and y components of the motion vector are smaller than a predefined threshold, the value may be excluded. In this case, the threshold may be set to a value such as 1, 2, 3, or 4.

[0557] Meanwhile, the values ​​of the aforementioned default motion vectors are merely examples, and the scope of the disclosed embodiments is not limited to these examples. Furthermore, the default motion vector candidates determined based on the prediction mode can be applied with more detailed modifications. That is, a more diverse set of candidates can be constructed by considering each prediction mode and its occurrence rate.

[0558] While the aforementioned methods have been described based on utilizing motion information from previously decoded pictures, the unit for accumulating or utilizing previously decoded motion information is not limited to pictures. That is, among previously decoded pictures, a specific range can be pre-determined and only motion information within that range can be utilized. This allows for the exclusion of motion information from pictures that are too far away from the current picture and thus have relatively different video characteristics.

[0559] As another example, considering that the existing HMVP initialization is performed in units of CTU rows, when accumulating motion information of previously decoded pictures, motion information can be accumulated independently for each CTU row, and the default motion information of the corresponding current picture can be determined based on this. By generalizing this, in addition to the above-mentioned accumulation of the entire decoded picture, accumulation of decoded pictures within a specific range, and independent accumulation of motion information for each CTU row, motion information can be accumulated in units defined in advance, and the default motion information of the current picture can be determined based on this.

[0560] The method according to one embodiment is not limited to utilizing the motion information of the picture decoded immediately before in the decoding order. Even if the picture is decoded previously, the motion information of the picture may not be utilized if the temporal ID is different from that of the picture to be currently decoded. In other words, the motion information within the same temporal ID may be utilized to determine the default vector. In this case, the motion information for each temporal ID may be replaced with the motion information of the picture with the same temporal ID that was recently decoded.

[0561] All of the methods described so far can be used alone, or two or more methods can be combined.

[0562] Meanwhile, the default motion vector can also be extended and applied when constructing the MVP candidate list. The MVP candidate list can be included in inter-mode merge mode, and can also be included in various prediction modes such as affine mode and GPM mode. When the MVP candidate configuration is limited, such as when the coding block (coding unit) exists on the boundary of a picture / slice / tile, or when the prediction mode of an adjacent block is different from that of the current block, the compression performance of the default motion vector described in the present invention can be increased.

[0563] In general, MVP candidates can be considered as motion information of spatially adjacent blocks of the current block (spatial candidates), motion information of temporally adjacent blocks (temporal candidates), HMVP candidates, pairwise average candidates, zero vector candidates, etc. However, when information of adjacent blocks is limited, zero vectors are mainly used. In a method according to one embodiment, compression performance can be improved by considering the default motion vector described above instead of the zero vector candidate. That is, the default motion vector can be considered before considering the zero vector candidate, or only the default motion vector can be considered without considering the zero vector candidate.

[0564] The maximum number of default motion vectors included in the MVP candidate list can follow a predetermined value. For example, if the maximum number of MVP candidates is M, M-1 default motion vector candidates can be included in the MVP candidate list. When applying the method according to the example, default motion vectors for HMVP candidates may not be considered.

[0565] As mentioned earlier, HMVP buffer initialization is performed in units of CTU Rows. With recent technological advancements, the processing speed of decoding devices is increasing, and thus the unit of parallel processing can also increase. Accordingly, the unit of HMVP buffer initialization can also be expanded to nCTU Rows (where n is an integer greater than or equal to 2), for example, 2CTU Rows or 3CTU Rows. This example describes an HMVP buffer initialization method that can vary for parallel processing of multiple CTU Rows.

[0566] The syntax table in Table 9 below demonstrates the process of initializing the HMVP buffer for each CTU Row within each slice. The HMVP buffer is initialized by initializing NumHmvpCand and NumHmvpIbcCand, which indicate the buffer size, to 0.

[0567] [Table 9]

[0568]

[0569] For the method of initializing the HMVP buffer in units of multiple CTU Rows described in the example, the slice_data() syntax table can be changed as shown in Table 10 below. In the syntax table below, the syntax indicated by strikethrough is a syntax element that is deleted from the existing syntax. The addition of syntax elements is not separately indicated. In (CtbAddrY % N == 0), N can have an integer greater than 1, and N can be changed to 2, 3, 4, etc. That is, by initializing CtbAddrY representing the CTU Row line in multiples of N, the initialization of the HMVP buffer can be processed in units of multiple CTU Rows. In this case, the condition for initializing the HMVP buffer can be satisfied when the CTU to be decoded is located in the multiple of N (N is an integer greater than or equal to 2) CTU Row and is located in the first column of the row.

[0570] [Table 10]

[0571]

[0572] Additionally, for IBC buffers initialized in units of CTU Rows, the syntax can be changed as shown in Table 11 below according to the initialization method of the HMVP buffer. In other words, it is also possible to initialize an IBC buffer in units of multiple CTU Rows.

[0573] [Table 11]

[0574]

[0575]

[0576] As described in the example, if the buffer initialization point changes, the default motion vector described above may be applied or omitted. Furthermore, if the buffer initialization point changes, the region within the picture considered when setting the default motion vector by considering the motion information of the previous picture may change. Alternatively, various modifications are possible, such as allowing a limited number of default motion vectors or using the default motion vector as an MVP candidate rather than the default motion vector for an HMVP candidate.

[0577] Meanwhile, as mentioned above, HMVP technology can also be applied to IBC mode. When HMVP technology is used in IBC mode, an IBC BV (block vector) is inserted into the history buffer for future reference. The default motion information described so far can also be applied to IBC mode. That is, even when HMVP candidates are used to determine block vectors in IBC mode, default motion information derived or determined according to the above-described method can be considered as a candidate when initializing the HMVP buffer. Changes (such as a change in terminology to default block vector) made by applying the above-described embodiment to IBC mode are obvious to those skilled in the art and are included in the scope of the present embodiment.

[0578] FIG. 31 is a diagram illustrating an example of a content streaming system to which an embodiment according to the present disclosure can be applied.

[0579] Referring to FIG. 31, a content streaming system to which the embodiment(s) of the present specification are applied may largely include an encoding server, a streaming server, a web server, a media storage, a user device, and a multimedia input device.

[0580] The encoding server compresses content input from multimedia input devices such as smartphones, cameras, and camcorders into digital data, generates a bitstream, and transmits it to the streaming server. Alternatively, if multimedia input devices such as smartphones, cameras, and camcorders directly generate bitstreams, the encoding server may be omitted.

[0581] The above bitstream can be generated by an encoding method or a bitstream generation method to which the embodiment(s) of the present specification are applied, and the streaming server can temporarily store the bitstream during the process of transmitting or receiving the bitstream.

[0582] The streaming server transmits multimedia data to a user device based on a user request via a web server, and the web server acts as an intermediary to inform the user of available services. When a user requests a desired service from the web server, the web server transmits the request to the streaming server, and the streaming server transmits the multimedia data to the user. At this time, the content streaming system may include a separate control server, in which case the control server controls commands / responses between each device within the content streaming system.

[0583] The streaming server can receive content from a media repository and / or an encoding server. For example, when receiving content from the encoding server, the content can be received in real time. In this case, to provide a smooth streaming service, the streaming server can store the bitstream for a certain period of time.

[0584] Examples of the user devices may include mobile phones, smart phones, laptop computers, digital broadcasting terminals, personal digital assistants (PDAs), portable multimedia players (PMPs), navigation devices, slate PCs, tablet PCs, ultrabooks, wearable devices (e.g., smartwatches, smart glasses, HMDs), digital TVs, desktop computers, digital signage, etc.

[0585] Each server within the above content streaming system can be operated as a distributed server, in which case data received from each server can be processed in a distributed manner.

[0586] The claims set forth in this specification may be combined in various ways. For example, the technical features of the method claims of this specification may be combined and implemented as a device, and the technical features of the device claims of this specification may be combined and implemented as a method. Furthermore, the technical features of the method claims and the technical features of the device claims of this specification may be combined and implemented as a device, and the technical features of the method claims and the technical features of the device claims of this specification may be combined and implemented as a method.

[0587] Embodiments according to the present disclosure can be used to encode / decode images.

Claims

1. A step of obtaining image information from a bitstream; A step of constructing a list of MVP candidates including MVPs (Motion Vector Predictors) for the current block based on the above image information; and A step of generating a prediction block for the current block based on at least one MVP candidate in the MVP candidate list; The steps for forming the above MVP candidate list are: Including adding motion information stored in the HMVP (History-based Motion Vector Predictor) buffer to the MVP candidate list, In the above HMVP buffer, The movement information of the decoded block is stored in a predetermined number, A method in which, when a predetermined condition is satisfied, the stored movement information is deleted and default movement information is stored.

2. In paragraph 1, The above default movement information is, A method comprising a plurality of motion vectors having fixed values.

3. In paragraph 1, The above default movement information is, Contains a default motion vector for the first direction and a default motion vector for the second direction, The default motion vector for the second direction is A method determined based on a default motion vector for the first direction.

4. In paragraph 3, The above default movement information is, A method determined based on the characteristics of the image.

5. In paragraph 4, The above default movement information is, A method for determining a pre-default motion vector having a fixed value by applying scaling based on the distance between the current picture and the reference picture.

6. In paragraph 4, The above default movement information is, A method determined by applying scaling based on the difference between the temporal ID of the current picture and the temporal ID of the reference picture to a pre-default motion vector having a fixed value.

7. In paragraph 4, The above default movement information is, A method determined by applying scaling based on the difference between the temporal QP (quantization parameter) of the current picture and the QP (quantization parameter) of the reference picture to a pre-default motion vector having a fixed value.

8. In paragraph 1, The above default movement information is, A method determined based on prediction mode information or motion information of decoded pictures.

9. In paragraph 1, The above pre-determined conditions are, A method that is satisfied when the CTU (Coding Tree Unit) to be decoded is located in the first column of a row that is a multiple of N (N is an integer greater than or equal to 2).

10. Step of determining the prediction mode of the current block; A step of constructing an MVP candidate list including MVP (Motion Vector Predictor) candidates for the current block based on the above prediction mode; and A step of generating residual samples for the current block based on at least one MVP candidate in the MVP candidate list; The steps for forming the above MVP candidate list are: Including adding motion information stored in the HMVP (History-based Motion Vector Predictor) buffer to the MVP candidate list, In the above HMVP buffer, The movement information of the decoded block is stored in a predetermined number, A method in which, when a predetermined condition is satisfied, the stored movement information is deleted and default movement information is stored.

11. In paragraph 10, The above default movement information is, A method comprising a plurality of motion vectors having fixed values.

12. In paragraph 10, The above default movement information is, Contains a default motion vector for the first direction and a default motion vector for the second direction, The default motion vector for the second direction is A method determined based on a default motion vector for the first direction.

13. In paragraph 12, The above default movement information is, A method determined based on the characteristics of the image.

14. In paragraph 13, The above default movement information is, A method for determining a pre-default motion vector having a fixed value by applying scaling based on the distance between the current picture and the reference picture.

15. In paragraph 13, The above default movement information is, A method determined by applying scaling based on the difference between the temporal ID of the current picture and the temporal ID of the reference picture to a pre-default motion vector having a fixed value.

16. In paragraph 13, The above default movement information is, A method determined by applying scaling based on the difference between the temporal QP (quantization parameter) of the current picture and the QP (quantization parameter) of the reference picture to a pre-default motion vector having a fixed value.

17. In paragraph 10, The above default movement information is, A method determined based on prediction mode information or motion information of decoded pictures.

18. In paragraph 10, The above pre-determined conditions are, A method that is satisfied when the CTU (Coding Tree Unit) to be decoded is located in the first column of a row that is a multiple of N (N is an integer greater than or equal to 2).

19. In a computer-readable storage medium in which a bitstream generated by an encoding method is stored, The above encoding method is, A step of determining the prediction mode of the current block; A step of constructing an MVP candidate list including MVP (Motion Vector Predictor) candidates for the current block based on the above prediction mode; and A step of generating residual samples for the current block based on at least one MVP candidate in the MVP candidate list; The steps for forming the above MVP candidate list are: Including adding motion information stored in the HMVP (History-based Motion Vector Predictor) buffer to the MVP candidate list, In the above HMVP buffer, The movement information of the decoded block is stored in a predetermined number, A computer-readable storage medium in which the stored movement information is deleted and default movement information is stored when a predetermined condition is satisfied.

20. In a method for transmitting data about an image, A step of obtaining a bitstream for the image, wherein the bitstream is generated based on the steps of: determining a prediction mode of a current block; constructing an MVP candidate list including MVP (Motion Vector Predictor) candidates for the current block based on the prediction mode; and generating residual samples for the current block based on at least one MVP candidate in the MVP candidate list; and A step of transmitting the data including the bitstream; The steps for forming the above MVP candidate list are: Including adding motion information stored in the HMVP (History-based Motion Vector Predictor) buffer to the MVP candidate list, In the above HMVP buffer, The movement information of the decoded block is stored in a predetermined number, A method in which, when a predetermined condition is satisfied, the stored movement information is deleted and default movement information is stored.

Images