HISTORY-BASED IMAGE CODING METHOD AND APPARATUS THEREOF

Abstract

To provide an efficient image decoding method.SOLUTION: The method comprises the steps of: deriving a history-based motion vector prediction (HMVP) buffer for a current block; configuring motion information candidate list on the basis of an HMVP candidate included in the HMVP buffer; deriving motion information of the current block based on the motion information candidate list; deriving a reference picture index of the current block on the basis of the motion information; deriving a motion vector of the current block on the basis of the motion information; generating a prediction sample to the current block on the basis of the reference picture index and the motion vector; and generating a reconstructed sample on the basis of the prediction sample. The current picture includes one or more tiles and a plurality of tile rows and tile lines. The tile is a rectangular area of a CTU (coding tree unit) within a specific tile row and a specific tile line in the current picture.SELECTED DRAWING: Figure 27

Description

[Technical field]

[0001] The present technology relates to image coding, and more particularly to a history-based image coding method and apparatus. [Background technology]

[0002] Recently, the demand for high-resolution, high-quality images / videos such as 4K or 8K or higher UHD (Ultra High Definition) images / videos is increasing in various fields. As the image / video data has higher resolution and quality, the amount of information or bits to be transmitted increases relatively compared to existing image / video data, so that when image data is transmitted using existing media such as wired or wireless broadband lines or image / video data is stored using existing storage media, the transmission costs and storage costs increase.

[0003] In addition, interest in and demand for immersive media such as virtual reality (VR), artificial reality (AR) content and holograms has been increasing recently, and broadcasts of images / videos that have different image characteristics from real images, such as game images, are on the rise.

[0004] Therefore, in order to effectively compress and transmit, store, and play back high-resolution, high-quality image / video information having the above-mentioned various characteristics, a highly efficient image / video compression technique is required. Summary of the Invention [Problem to be solved by the invention]

[0005] The technical problem of this document is to provide a method and apparatus for increasing image coding efficiency.

[0006] Another technical problem of this document is to provide an efficient inter-prediction method and apparatus.

[0007] Another technical problem of this document is to provide a method and apparatus for deriving a history-based motion vector.

[0008] Another technical problem of this document is to provide a method and apparatus for efficiently deriving History-based Motion Vector Prediction (HMVP) candidates.

[0009] Another technical problem of this document is to provide a method and apparatus for efficiently updating the HMVP buffer.

[0010] Another technical problem of this document is to provide a method and apparatus for efficiently initializing an HMVP buffer. [Means for solving the problem]

[0011] According to an embodiment of the present document, there is provided an image decoding method performed by a decoding device, the method comprising the steps of deriving a History-based Motion Vector Prediction (HMVP) buffer for a current block, constructing a motion information candidate list based on HMVP candidates included in the HMVP buffer, deriving motion information for the current block based on the motion information candidate list, generating a prediction sample for the current block based on the motion information, and generating a reconstructed sample based on the prediction sample, characterized in that one or more tiles exist in the current picture, and the HMVP buffer is initialized with a first CTU of a CTU row that includes the current block in the current tile.

[0012] According to another embodiment of the present document, there is provided a decoding device for performing image decoding, the decoding device includes a prediction unit for deriving an HMVP (History-based Motion Vector Prediction) buffer for a current block, constructing a motion information candidate list based on HMVP candidates included in the HMVP buffer, deriving motion information for the current block based on the motion information candidate list, and generating a predicted sample for the current block based on the motion information, and an adder unit for generating a reconstructed sample based on the predicted sample, wherein one or more tiles exist in the current picture, and the HMVP buffer is initialized with a first CTU of a CTU row having the current block in the current tile.

[0013] According to yet another embodiment of the present document, there is provided an image encoding method performed by an encoding device, the method comprising the steps of deriving a History-based Motion Vector Prediction (HMVP) buffer for a current block, constructing a motion information candidate list based on HMVP candidates included in the HMVP buffer, deriving motion information for the current block based on the motion information candidate list, generating a prediction sample for the current block based on the motion information, deriving a residual sample based on the prediction sample, and encoding image information having information on the residual sample, characterized in that one or more tiles exist in the current picture, and the HMVP buffer is initialized with a first CTU of a CTU row including the current block in the current tile.

[0014] According to yet another embodiment of the present document, there is provided an encoding device for performing image encoding, comprising: a prediction unit for deriving an HMVP (History-based Motion Vector Prediction) buffer for a current block, constructing a motion information candidate list based on HMVP candidates included in the HMVP buffer, deriving motion information for the current block based on the motion information candidate list, and generating a prediction sample for the current block based on the motion information, a residual processing unit for deriving a residual sample based on the prediction sample, and an entropy encoding unit for encoding image information having information related to the residual sample, wherein one or more tiles exist in the current picture, and the HMVP buffer is initialized with a first CTU of a CTU row having the current block in the current tile.

[0015] According to yet another embodiment of the present document, there is provided a digital storage medium having stored thereon image data having encoded image information produced by an image encoding method performed by an encoding device.

[0016] According to yet another embodiment of the present document, there is provided a digital storage medium having stored thereon image data having encoded image information which causes a decoding device to perform an image decoding method. Effect of the Invention

[0017] According to one embodiment of this document, the overall image / video compression efficiency can be increased.

[0018] According to one embodiment of this document, the amount of transmitted data required for residual processing can be reduced through efficient inter prediction.

[0019] According to one embodiment of this document, the HMVP buffer can be managed efficiently.

[0020] According to one embodiment of this document, parallel processing can be supported through efficient HMVP buffer management.

[0021] According to one embodiment of this document, motion vectors for inter prediction can be derived efficiently. [Brief description of the drawings]

[0022] [Figure 1] FIG. 1 illustrates a schematic diagram of an example of a video / image coding system in which embodiments of the present document may be applied. [Diagram 2] FIG. 1 is a diagram illustrating the configuration of a video / image encoding device to which the embodiments of this document can be applied. [Diagram 3] FIG. 1 is a diagram illustrating the configuration of a video / image decoding device to which an embodiment of the present document can be applied. [Figure 4] FIG. 1 illustrates an example of an inter-prediction based video / image encoding method. [Diagram 5] FIG. 2 illustrates an example of an inter-prediction based video / image decoding method. [Figure 6] FIG. 13 is a diagram exemplarily illustrating an inter prediction procedure. [Figure 7] 1 is a diagram illustrating an example of spatially adjacent blocks used for motion information candidate derivation in conventional merge or AMVP mode. [Figure 8] FIG. 2 illustrates a schematic diagram of an example of an HMVP candidate-based decoding procedure. [Figure 9] FIG. 13 is a diagram illustrating an example of an HMVP table update according to the FIFO rule. [Figure 10] FIG. 13 is a diagram illustrating an example of an HMVP table update according to the restricted FIFO rule. [Figure 11] FIG. 1 is a diagram illustrating an example of WPP (Wavefront Parallel Processing), which is one of the techniques for parallel processing. [Figure 12] FIG. 1 is a diagram illustrating an example of a problem that occurs when applying a general HMVP method in consideration of parallel processing. [Figure 13]FIG. 13 is an exemplary diagram illustrating a method for initializing a history management buffer (HMVP buffer) according to one embodiment of the present document. [Figure 14] 1 is a diagram illustrating an exemplary HMVP buffer management method according to an embodiment. [Figure 15] 13 is a diagram illustrating an exemplary HMVP buffer management method according to another embodiment. [Figure 16] 13 is a diagram illustrating an example of an HMVP buffer initialization method in a tile structure. [Figure 17] FIG. 13 is a diagram illustrating an example of an HMVP buffer initialization method for the first CTU of a tile according to another embodiment. [Figure 18] FIG. 13 is a diagram showing an example of an HMVP management buffer initialization method for the first CTU in a CTU row in each tile according to yet another embodiment. [Figure 19] FIG. 13 is a diagram showing an example of a structure in which tiles and slices coexist. [Figure 20] A diagram showing an example of a method for initializing an HMVP buffer for the first CTU in each tile. [Figure 21] A diagram showing an example of a method for initializing an HMVP buffer for each slice in a tile. [Figure 22] A figure showing an example of initializing an HMVP buffer for the first CTU of the first tile in a tile group. [Diagram 23] A figure showing an example of initializing an HMVP buffer for the first CTU of each tile in a tile group. [Figure 24] A figure showing an example of initializing an HMVP buffer for the CTU rows of each tile in a tile group. [Diagram 25] FIG. 1 illustrates a schematic diagram of an example of a video / image encoding method and associated components including an inter-prediction method according to an embodiment(s) of the present document. [Figure 26]FIG. 1 illustrates a schematic diagram of an example of a video / image encoding method including an inter-prediction method and associated components according to an embodiment or embodiments of the present document. [Figure 27] FIG. 1 illustrates a schematic diagram of an example of an image decoding method including an inter-prediction method and associated components according to an embodiment of the present document. [Figure 28] FIG. 1 illustrates a schematic diagram of an example of an image decoding method including an inter-prediction method and associated components according to an embodiment of the present document. [Figure 29] FIG. 1 illustrates an example of a content streaming system in which embodiments such as those disclosed in this document may be applied. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0023] The method presented in this document may be modified in various ways and may have various embodiments, and a specific embodiment will be illustrated in the drawings and described in detail. The terms used in this specification are merely used to describe a specific embodiment and are not intended to limit the technical idea of ​​the method presented in this document. The singular expression includes the expression "at least one" unless the context clearly indicates otherwise. In this specification, the terms "include" or "have" are intended to specify the presence of features, numbers, steps, operations, components, parts, or combinations thereof described in the specification, and should be understood not to preclude the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

[0024] Meanwhile, each configuration in the drawings described in this document is illustrated independently for the convenience of explaining the different characteristic functions, etc., and does not mean that each configuration is realized by separate hardware or software. For example, two or more of each configuration may be combined to form one configuration, and one configuration may be divided into multiple configurations. An embodiment in which each configuration is integrated and / or separated is also included in the disclosure scope of this document as long as it does not deviate from the essence of the method disclosed in this document.

[0025] This document relates to video / image coding. For example, the methods / embodiments disclosed in this document can be applied to methods disclosed in the Versatile Video Coding (VVC) standard. Also, the methods / embodiments disclosed in this document can be applied to methods disclosed in the Essential Video Coding (EVC) standard, the AOMedia Video 1 (AV1) standard, the 2nd Generation Of Audio Video Coding Standard (AVS2), or next-generation video / image coding standards (e.g., H.267 or H.268).

[0026] In this document, various embodiments for video / image coding are presented, and unless otherwise stated, the embodiments can also be implemented in combination with each other.

[0027] In this document, a video may mean a collection of a series of images over time. A picture generally means a unit showing one image at a specific time, and a slice / tile is a unit constituting a part of a picture in coding. A slice / tile may include one or more coding tree units (CTUs). A picture may consist of one or more slices / tiles. A picture may consist of one or more tile groups. A tile group may include one or more tiles. A brick may represent a rectangular region of CTU rows within a tile in a picture. A tile may be partitioned into multiple bricks, each of which consisting of one or more CTU rows within the tile. A tile that is not partitioned into multiple bricks may be also referred to as a brick.A brick scan can refer to a specific sequential ordering of CTUs partitioning a picture in which the CTUs are ordered consecutively in CTU raster scan in a brick, bricks within a tile are ordered consecutively in a raster scan of the bricks of the tile, and tiles in a picture are ordered consecutively in a raster scan of the tiles of the picture. A tile is a rectangular region of CTUs within a particular tile column and a particular tile row in a picture. The tile column is a rectangular region of CTUs having a height equal to the height of the picture and a width specified by syntax elements in the picture parameter set.The tile row is a rectangular region of CTUs having a height specified by syntax elements in the picture parameter set and a width equal to the width of the picture. A tile scan is a specific sequential ordering of CTUs partitioning a picture in which the CTUs are ordered consecutively in CTU raster scan in a tile whereas tiles in a picture are ordered consecutively in a raster scan of the tiles of the picture. A slice includes an integer number of bricks of a picture that may be exclusively contained in a single NAL unit. A slice may consist of either a number of complete tiles or only a consecutive sequence of complete bricks of one tile.In this document, the terms tile group and slice can be used interchangeably. For example, in this document, tile group / tile group header is also called slice / slice header.

[0028] A pixel or a pel may mean the smallest unit that constitutes one picture (or image). A term corresponding to a pixel may be "sample." A sample generally indicates a pixel or a pixel value, and may indicate only the value of a pixel / pixel of a luma component, or may indicate only the value of a pixel / pixel of a chroma component.

[0029] A unit may indicate a basic unit of image processing. A unit may include at least one of a specific region of a picture and information related to the region. A unit may include one luma block and two chroma (e.g., cb, cr) blocks. A unit may be mixed with terms such as a block or an area in some cases. In a general case, an M×N block may include a set (or array) of samples or transform coefficients consisting of M columns and N rows.

[0030] In this document, the terms " / " and "," should be interpreted to indicate "and / or." For instance, the expression "A / B" may mean "A and / or B," and "A, B" may mean "A and / or B." Also, "A / B / C" may mean "at least one of A, B, and / or C." (In this document, the term " / " and "," should be interpreted to indicate "and / or." For instance, the expression "A / B" may mean "A and / or B." Further, "A, B" may mean "A and / or B." Further, "A / B / C" may mean "at least one of A, B, and / or C." Also, "A / B / C" may mean "at least one of A, B, and / or C.")

[0031] Additionally, in this document, the term “or” should be interpreted to indicate “and / or.” For example, “A or B” may comprise 1) only A, 2) only B, and / or 3) both A and B. In other words, the term “or” in this document should be interpreted to indicate “additionally or alternatively.”

[0032] Hereinafter, the embodiments of the present disclosure will be described in more detail with reference to the accompanying drawings. Hereinafter, the same reference numerals will be used for the same components in the drawings, and duplicated descriptions of the same components may be omitted.

[0033] FIG. 1 illustrates a schematic diagram of an example of a video / image coding system in which embodiments of the present document may be applied.

[0034] As shown in Figure 1, a video / image coding system can include a first device (source device) and a second device (receiving device). The source device can transmit encoded video / image information or data to the receiving device in file or streaming form via a digital storage medium or network.

[0035] The source device may include a video source, an encoding device, and a sending unit. The receiving device may include a receiving unit, a decoding device, and a renderer. The encoding device may also be called a video / image encoding device, and the decoding device may also be called a video / image decoding device. The transmitter may be included in the encoding device. The receiver may be included in the decoding device. The renderer may also include a display unit, which may be a separate device or an external component.

[0036] A video source can acquire video / images via a video / image capture, synthesis or generation process, etc. A video source can include a video / image capture device and / or a video / image generation device. A video / image capture device can include, for example, one or more cameras, a video / image archive containing previously captured video / images, etc. A video / image generation device can include, for example, a computer, a tablet, a smartphone, etc., and can (electronically) generate video / images. For example, a virtual video / image can be generated via a computer, etc., in which case the video / image capture process can be substituted as the process by which the associated data is generated.

[0037] An encoding device can encode an input video / image. The encoding device can perform a series of steps such as prediction, transformation, quantization, etc. for compression and coding efficiency. The encoded data (encoded video / image information) can be output in the form of a bitstream.

[0038] The transmitting unit can transmit the encoded video / image information or data output in the form of a bitstream to a receiving unit of a receiving device via a digital storage medium or a network in the form of a file or streaming. The digital storage medium can include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. The transmitting unit can include elements for generating a media file through a predetermined file format and can include elements for transmission via a broadcasting / communication network. The receiving unit can receive / extract the bitstream and transmit it to a decoding device.

[0039] The decoding device can decode the video / image by performing a series of steps such as inverse quantization, inverse transformation, prediction, etc., which correspond to the operations of the encoding device.

[0040] The renderer can render the decoded video / images. The rendered video / images can be displayed via a display unit.

[0041] 2 is a diagram for explaining the configuration of a video / image encoding device to which the embodiments of this document can be applied. Hereinafter, the video encoding device may include an image encoding device.

[0042] As shown in FIG. 2, the encoding device 200 may include an image partitioner 210, a predictor 220, a residual processor 230, an entropy encoder 240, an adder 250, a filter 260, and a memory 270. The predictor 220 may include an inter predictor 221 and an intra predictor 222. The residual processor 230 may include a transformer 232, a quantizer 233, a dequantizer 234, and an inverse transformer 235. The residual processor 230 may further include a subtractor 231. The adder 250 may also be referred to as a reconstructor or a reconstructed block generator. The image division unit 210, the prediction unit 220, the residual processing unit 230, the entropy encoding unit 240, the addition unit 250, and the filtering unit 260 may be configured by one or more hardware components (e.g., an encoder chip set or a processor) depending on the embodiment. In addition, the memory 270 may include a decoded picture buffer (DPB) and may be configured by a digital storage medium. The above hardware components may further include the memory 270 as an internal / external component.

[0043] The image division unit 210 may divide an input image (or picture, frame) input to the encoding device 200 into one or more processing units. As an example, the processing unit may be called a coding unit (CU). In this case, the coding unit may be recursively divided from a coding tree unit (CTU) or a largest coding unit (LCU) according to a quad-tree binary-tree ternary-tree (QTBTTT) structure. For example, one coding unit may be divided into a plurality of coding units of a deeper depth based on a quad-tree structure, a binary tree structure, and / or a ternary tree. In this case, for example, the quad-tree structure may be applied first, and then the binary tree structure and / or the ternary tree may be applied. Alternatively, the binary tree structure may be applied first. Based on the final coding unit that is not further divided, the coding procedure according to this document may be performed. In this case, based on coding efficiency according to image characteristics, the largest coding unit may be used as the final coding unit, or if necessary, the coding unit may be recursively divided into coding units of lower depths, and a coding unit of an optimal size may be used as the final coding unit. Here, the coding procedure may include procedures such as prediction, transformation, and restoration, which will be described later. As another example, the processing unit may further include a prediction unit (PU) or a transform unit (TU). In this case, the prediction unit and the transform unit may each be divided or partitioned from the final coding unit described above.The prediction unit is a unit of sample prediction, or the transform unit is a unit for deriving transform coefficients and / or a unit for deriving a residual signal from the transform coefficients.

[0044] The unit may be mixed with terms such as block or area in some cases. In the general case, an M×N block may refer to a set of samples or transform coefficients consisting of M columns and N rows. A sample may generally refer to a pixel or pixel value, may refer to only a pixel / pixel value of a luma component, or may refer to only a pixel / pixel value of a chroma component. A sample may be used as a term corresponding to one picture (or image) for a pixel or a pel.

[0045] The encoding device 200 may generate a residual signal (residual block, residual sample array) by subtracting a prediction signal (predicted block, prediction sample array) output from the inter prediction unit 221 or the intra prediction unit 222 from an input image signal (original block, original sample array), and the generated residual signal is transmitted to the conversion unit 232. In this case, as illustrated, a unit that subtracts a prediction signal (prediction block, prediction sample array) from an input image signal (original block, original sample array) in the encoder 200 is also called a subtraction unit 231. The prediction unit may perform prediction on a block to be processed (hereinafter, referred to as a current block) and generate a predicted block including a prediction sample for the current block. The prediction unit may determine whether intra prediction or inter prediction is applied in units of the current block or CU. The prediction unit may generate various information related to prediction, such as prediction mode information, and transmit the information to the entropy encoding unit 240, as described later in the description of each prediction mode. The prediction information can be encoded by the entropy encoding unit 240 and output in the form of a bitstream.

[0046] The intra prediction unit 222 may predict the current block by referring to samples in the current picture. The referenced samples may be located in the neighborhood of the current block or may be located away from the current block depending on the prediction mode. In the intra prediction, the prediction mode may include a plurality of non-directional modes and a plurality of directional modes. The non-directional mode may include, for example, a DC mode and a planar mode. The directional mode may include, for example, 33 directional prediction modes or 65 directional prediction modes depending on the degree of granularity of the prediction direction. However, this is merely an example, and more or less directional prediction modes may be used depending on the setting. The intra prediction unit 222 may also determine the prediction mode to be applied to the current block by using the prediction mode applied to the neighboring block.

[0047] The inter prediction unit 221 may derive a predicted block for the current block based on a reference block (reference sample array) identified by a motion vector on a reference picture. In this case, in order to reduce the amount of motion information transmitted in the inter prediction mode, the motion information may be predicted in units of blocks, sub-blocks, or samples based on the correlation of the motion information between the neighboring blocks and the current block. The motion information may include a motion vector and a reference picture index. The motion information may further include inter prediction direction (L0 prediction, L1 prediction, Bi prediction, etc.) information. In the case of inter prediction, the neighboring blocks may include a spatial neighboring block present in the current picture and a temporal neighboring block present in the reference picture. The reference picture including the reference block and the reference picture including the temporal neighboring block may be the same or different. The temporal neighboring block may be called a collocated reference block, a collocated CU (colCU), etc., and the reference picture including the temporal neighboring block is also called a collocated picture (colPic). For example, the inter prediction unit 221 may generate information indicating which candidate is used to construct a motion information candidate list based on neighboring blocks and derive a motion vector and / or a reference picture index of the current block. Inter prediction may be performed based on various prediction modes, and for example, in the case of skip mode and merge mode, the inter prediction unit 221 may use motion information of neighboring blocks as motion information of the current block. In the case of skip mode, unlike the merge mode, a residual signal is not transmitted.In the case of a Motion Vector Prediction (MVP) mode, the motion vector of a neighboring block can be used as a motion vector predictor, and the motion vector of the current block can be indicated by signaling the motion vector difference.

[0048] The prediction unit 220 may generate a prediction signal based on various prediction methods described below. For example, the prediction unit may apply intra prediction or inter prediction for prediction of one block, as well as intra prediction and inter prediction simultaneously. This is also called Combined Inter and Intra Prediction (CIIP). The prediction unit may also be based on an Intra Block Copy (IBC) prediction mode or a palette mode for prediction of a block. The IBC prediction mode or palette mode may be used for content image / video coding such as games, for example, such as Screen Content Coding (SCC). IBC basically performs prediction within a current picture, but may be performed similarly to inter prediction in deriving a reference block within the current picture. That is, IBC may utilize at least one of the inter prediction techniques described in this document. The palette mode may be considered as an example of intra coding or intra prediction. When the palette mode is applied, a sample value within a picture may be signaled based on information regarding a palette table and a palette index.

[0049] The prediction signal generated through the prediction unit (including the inter prediction unit 221 and / or the intra prediction unit 222) may be used to generate a reconstruction signal or may be used to generate a residual signal. The transform unit 232 may generate transform coefficients by applying a transform technique to the residual signal. For example, the transform technique may include at least one of a discrete cosine transform (DCT), a discrete sine transform (DST), a graph-based transform (GBT), or a conditionally non-linear transform (CNT). Here, the GBT refers to a transform obtained from a graph when the relationship information between pixels is expressed as a graph. The CNT refers to a transform obtained based on a prediction signal generated using all previously reconstructed pixels. In addition, the transform process may be applied to pixel blocks having the same square size, or may be applied to blocks of variable sizes that are not square.

[0050] The quantizer 233 quantizes the transform coefficients and transmits them to the entropy encoder 240, which may encode the quantized signal (information on the quantized transform coefficients) and output the quantized signal as a bitstream. The information on the quantized transform coefficients may also be referred to as residual information. The quantizer 233 may rearrange the quantized transform coefficients in a block form into a one-dimensional vector form based on a coefficient scan order, and may generate information on the quantized transform coefficients based on the quantized transform coefficients in the one-dimensional vector form. The entropy encoder 240 may perform various encoding methods, such as exponential Golomb, Context-Adaptive Variable Length Coding (CAVLC), and Context-Adaptive Binary Arithmetic Coding (CABAC). The entropy encoder 240 may also encode information required for video / image restoration (e.g., values ​​of syntax elements, etc.) together with or separately from the quantized transform coefficients. The encoded information (e.g., encoded video / image information) may be transmitted or stored in the form of a bitstream in Network Abstraction Layer (NAL) unit units. The video / image information may further include information on various parameter sets, such as an adaptation parameter set (APS), a picture parameter set (PPS), a sequence parameter set (SPS) or a video parameter set (VPS). The video / image information may further include general constraint information. In this document, information and / or syntax elements transmitted / signaled from an encoding device to a decoding device may be included in the video / image information. The video / image information may be encoded through the above-mentioned encoding procedure and included in the bitstream.The bitstream may be transmitted over a network or stored in a digital storage medium. Here, the network may include a broadcast network and / or a communication network, and the digital storage medium may include various storage media such as a USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. A transmitting unit (not shown) that transmits the signal output from the entropy encoding unit 240 and / or a storage unit (not shown) that stores the signal may be configured as an internal / external element of the encoding device 200, or the transmitting unit may be included in the entropy encoding unit 240.

[0051] The quantized transform coefficients output from the quantizer 233 may be used to generate a prediction signal. For example, a residual signal (residual block or residual sample) may be restored by applying inverse quantization and inverse transform to the quantized transform coefficients via the inverse quantizer 234 and the inverse transformer 235. The adder 250 may generate a reconstructed signal (reconstructed picture, reconstructed block, reconstructed sample array) by adding a prediction signal output from the inter predictor 221 or the intra predictor 222 to the reconstructed residual signal. When there is no residual for the current block, such as when a skip mode is applied, a predicted block may be used as a reconstructed block. The adder 250 may also be referred to as a reconstruction unit or a reconstructed block generator. The generated reconstructed signal may be used for intra prediction of the next current block in the current picture, or may be used for inter prediction of the next picture after filtering as described below.

[0052] Meanwhile, Luma Mapping with Chroma Scaling (LMCS) may be applied during picture encoding and / or restoration.

[0053] The filtering unit 260 may apply filtering to the reconstructed signal to improve subjective / objective image quality. For example, the filtering unit 260 may apply various filtering methods to the reconstructed picture to generate a modified reconstructed picture, and may store the modified reconstructed picture in the memory 270, specifically, in the DPB of the memory 270. The various filtering methods may include, for example, deblock filtering, sample adaptive offset, an adaptive loop filter, a bilateral filter, etc. The filtering unit 260 may generate various information regarding filtering and transmit the information to the entropy encoding unit 240, as will be described later in relation to each filtering method. The information regarding filtering may be encoded by the entropy encoding unit 240 and output in the form of a bitstream.

[0054] The modified reconstructed picture transmitted to the memory 270 can be used as a reference picture in the inter prediction unit 221. When inter prediction is applied through this, the encoding apparatus can avoid prediction mismatch in the encoding apparatus 200 and the decoding apparatus, and can also improve encoding efficiency.

[0055] The memory 270DPB may store the modified reconstructed picture for use as a reference picture in the inter prediction unit 221. The memory 270 may store motion information of a block from which motion information in the current picture is derived (or encoded) and / or motion information of a block in an already reconstructed picture. The stored motion information may be transmitted to the inter prediction unit 221 to be utilized as motion information of a spatial neighboring block or motion information of a temporal neighboring block. The memory 270 may store reconstructed samples of reconstructed blocks in the current picture and may transmit them to the intra prediction unit 222.

[0056] FIG. 3 is a diagram illustrating a schematic configuration of a video / image decoding device to which the embodiments of this document can be applied.

[0057] As shown in FIG. 3, the decoding device 300 may include an entropy decoder 310, a residual processor 320, a predictor 330, an adder 340, a filter 350, and a memory 360. The predictor 330 may include an inter predictor 331 and an intra predictor 332. The residual processor 320 may include a dequantizer 321 and an inverse transformer 321. The entropy decoder 310, the residual processor 320, the predictor 330, the adder 340, and the filter 350 may be configured as one hardware component (e.g., a decoder chipset or processor) according to an embodiment. In addition, the memory 360 may include a decoded picture buffer (DPB) and may be configured as a digital storage medium. The above hardware components may further include a memory 360 as an internal / external component.

[0058] When a bitstream including video / image information is input, the decoding device 300 can restore an image corresponding to a process in which the video / image information is processed by the encoding device of FIG. 2. For example, the decoding device 300 can derive a unit / block based on block division related information obtained from the bitstream. The decoding device 300 can perform decoding using a processing unit applied in the encoding device. Thus, the processing unit of the decoding is, for example, a coding unit, and the coding unit can be divided from a coding tree unit or a maximal coding unit according to a quadtree structure, a binary tree structure, and / or a ternary tree structure. One or more transform units can be derived from the coding unit. Then, the restored image signal decoded and output by the decoding device 300 can be reproduced by a reproduction device.

[0059] The decoding device 300 may receive a signal output from the encoding device of FIG. 2 in the form of a bitstream, and the received signal may be decoded via the entropy decoding unit 310. For example, the entropy decoding unit 310 may derive information (e.g., video / image information) required for image restoration (or picture restoration) by parsing the bitstream. The video / image information may further include information on various parameter sets, such as an adaptation parameter set (APS), a picture parameter set (PPS), a sequence parameter set (SPS), or a video parameter set (VPS). In addition, the video / image information may further include general constraint information. In addition, the decoding device may decode pictures based on information on the parameter set and / or the general constraint information. Signaling / received information and / or syntax elements described later in this document may be decoded via the decoding procedure and obtained from the bitstream. For example, the entropy decoding unit 310 may decode information in a bitstream based on a coding method such as exponential Golomb coding, CAVLC, or CABAC, and output values ​​of syntax elements required for image restoration and quantized values ​​of transform coefficients for residuals. More specifically, the CABAC entropy decoding method may receive BINs corresponding to each syntax element in a bitstream, determine a context model using syntax element information to be decoded and decode information of adjacent and to-be-decoded blocks or information of symbols / BINs decoded in a previous step, predict the occurrence probability of BINs according to the determined context model, and perform arithmetic decoding of BINs to generate symbols corresponding to the values ​​of each syntax element.In this case, the CABAC entropy decoding method may update the context model using information of the decoded symbol / BIN for the context model of the next symbol / BIN after determining the context model. Prediction information from the information decoded by the entropy decoding unit 310 is provided to a prediction unit (inter prediction unit 332 and intra prediction unit 331), and residual values ​​entropy-decoded by the entropy decoding unit 310, i.e., quantized transform coefficients and related parameter information, may be input to the residual processing unit 320. The residual processing unit 320 may derive a residual signal (residual block, residual sample, residual sample array). In addition, filtering information from the information decoded by the entropy decoding unit 310 may be provided to the filtering unit 350. Meanwhile, a receiving unit (not shown) that receives a signal output from the encoding device may be further configured as an internal / external element of the decoding device 300, or the receiving unit may be a component of the entropy decoding unit 310. Meanwhile, the decoding device according to this document is also called a video / image / picture decoding device, and the decoding device may be divided into an information decoder (video / image / picture information decoder) and a sample decoder (video / image / picture sample decoder). The information decoder may include the entropy decoding unit 310, and the sample decoder may include at least one of the inverse quantization unit 321, the inverse transform unit 322, the addition unit 340, the filtering unit 350, the memory 360, the inter prediction unit 332, and the intra prediction unit 331.

[0060] The inverse quantization unit 321 may inverse quantize the quantized transform coefficients to output transform coefficients. The inverse quantization unit 321 may rearrange the quantized transform coefficients in a two-dimensional block form. In this case, the rearrangement may be performed based on a coefficient scan order performed in the encoding device. The inverse quantization unit 321 may perform inverse quantization on the quantized transform coefficients using a quantization parameter (e.g., quantization step size information) to obtain transform coefficients.

[0061] The inverse transform unit 322 inversely transforms the transform coefficients to obtain a residual signal (residual block, residual sample array).

[0062] The prediction unit may perform prediction on a current block and generate a predicted block including prediction samples for the current block. The prediction unit may determine whether intra prediction or inter prediction is applied to the current block based on information on the prediction output from the entropy decoding unit 310, and may determine a specific intra / inter prediction mode.

[0063] The prediction unit 330 may generate a prediction signal based on various prediction methods described below. For example, the prediction unit may apply intra prediction or inter prediction for prediction of one block, and may simultaneously apply intra prediction and inter prediction. This is also called Combined Inter and intra Prediction (CIIP). The prediction unit may also be based on an Intra Block Copy (IBC) prediction mode or a palette mode for prediction of a block. The IBC prediction mode or palette mode may be used for content image / video coding such as games, for example, Screen Content Coding (SCC). IBC basically performs prediction within a current picture, but may be performed similarly to inter prediction in deriving a reference block within the current picture. That is, IBC may utilize at least one of the inter prediction techniques described in this document. The palette mode may be considered as an example of intra coding or intra prediction. When the palette mode is applied, information regarding the palette table and palette index can be included in the video / image information and signaled.

[0064] The intra prediction unit 331 may predict the current block by referring to samples in the current picture. The referenced samples may be located in the neighborhood of the current block or may be located far away depending on the prediction mode. In intra prediction, the prediction mode may include a plurality of non-directional modes and a plurality of directional modes. The intra prediction unit 331 may also determine the prediction mode to be applied to the current block by using the prediction mode applied to the neighboring block.

[0065] The inter prediction unit 332 may derive a predicted block for the current block based on a reference block (reference sample array) identified by a motion vector on a reference picture. In this case, in order to reduce the amount of motion information transmitted in the inter prediction mode, the motion information may be predicted in units of blocks, sub-blocks, or samples based on the correlation of motion information between neighboring blocks and the current block. The motion information may include a motion vector and a reference picture index. The motion information may further include inter prediction direction (L0 prediction, L1 prediction, Bi prediction, etc.) information. In the case of inter prediction, the neighboring blocks may include spatial neighboring blocks present in the current picture and temporal neighboring blocks present in the reference picture. For example, the inter prediction unit 332 may configure a motion information candidate list based on the neighboring blocks and derive a motion vector and / or a reference picture index for the current block based on the received candidate selection information. Inter prediction may be performed based on various prediction modes, and the information regarding the prediction may include information indicating a mode of inter prediction for the current block.

[0066] The adder 340 may generate a reconstructed signal (reconstructed picture, reconstructed block, reconstructed sample array) by adding a prediction signal (predicted block, predicted sample array) output from a prediction unit (including the inter prediction unit 332 and / or the intra prediction unit 331) to the obtained residual signal. When there is no residual for a current block, such as when a skip mode is applied, the predicted block may be used as a reconstructed block.

[0067] The adder 340 is also called a reconstruction unit or a reconstruction block generator. The reconstruction signal generated may be used for intra prediction of a next block to be processed in the current picture, may be output after filtering as described below, or may be used for inter prediction of a next picture.

[0068] Meanwhile, Luma Mapping with Chroma Scaling (LMCS) can be applied during the picture decoding process.

[0069] The filtering unit 350 may apply filtering to the reconstructed signal to improve subjective / objective image quality. For example, the filtering unit 350 may apply various filtering methods to the reconstructed picture to generate a modified reconstructed picture, and may transmit the modified reconstructed picture to the memory 360, specifically, to the DPB of the memory 360. The various filtering methods may include, for example, deblock filtering, sample adaptive offset, an adaptive loop filter, a bilateral filter, etc.

[0070] The (modified) reconstructed picture stored in the DPB of the memory 360 can be used as a reference picture in the inter prediction unit 332. The memory 360 can store motion information of a block from which motion information in the current picture is derived (or decoded) and / or motion information of a block in an already reconstructed picture. The stored motion information can be transmitted to the inter prediction unit 332 to be used as motion information of a spatial neighboring block or motion information of a temporal neighboring block. The memory 360 can store reconstructed samples of reconstructed blocks in the current picture and transmit them to the intra prediction unit 331.

[0071] In this specification, the embodiments described for the filtering unit 260, inter prediction unit 221, and intra prediction unit 222 of the encoding device 200 can also be applied identically or correspondingly to the filtering unit 350, inter prediction unit 332, and intra prediction unit 331 of the decoding device 300, respectively.

[0072] As described above, prediction is performed to improve compression efficiency when performing video coding. As a result, a predicted block including a predicted sample for a current block, which is a block to be coded, can be generated. Here, the predicted block includes a predicted sample in a spatial domain (or a pixel domain). The predicted block is derived in the same way by an encoding device and a decoding device, and the encoding device can improve image coding efficiency by signaling information (residual information) on a residual between the original block and the predicted block, which is not an original sample value of the original block, to a decoding device. The decoding device can derive a residual block including residual samples based on the residual information, combine the residual block and the predicted block to generate a reconstructed block including reconstructed samples, and generate a reconstructed picture including the reconstructed block.

[0073] The residual information may be generated through a transform and quantization procedure. For example, an encoding apparatus may derive a residual block between the original block and the predicted block, perform a transform procedure on residual samples (residual sample array) included in the residual block to derive transform coefficients, and perform a quantization procedure on the transform coefficients to derive quantized transform coefficients, and then signal the associated residual information to a decoding apparatus (via a bitstream). Here, the residual information may include information such as value information, position information, transform technique, transform kernel, and quantization parameter of the quantized transform coefficients. The decoding apparatus may derive a residual sample (or a residual block) by performing an inverse quantization / inverse transform procedure based on the residual information. The decoding apparatus may generate a reconstructed picture based on the predicted block and the residual block. In addition, the encoding apparatus may derive a residual block by inverse quantizing / inverse transforming the quantized transform coefficients for reference for inter-prediction of a future picture, and generate a reconstructed picture based on the residual block.

[0074] When inter prediction is applied, the prediction unit of the encoding device / decoding device can perform inter prediction on a block-by-block basis to derive a prediction sample. Inter prediction can indicate a prediction derived in a manner that is dependent on data elements (e.g., sample values ​​or motion information) of picture(s) other than the current picture. When inter prediction is applied to the current block, a predicted block (prediction sample array) for the current block can be derived based on a reference block (reference sample array) identified by a motion vector on a reference picture pointed to by a reference picture index. At this time, in order to reduce the amount of motion information transmitted in the inter prediction mode, the motion information of the current block can be predicted on a block, sub-block, or sample basis based on the correlation of motion information between adjacent blocks and the current block. The motion information can include a motion vector and a reference picture index. The motion information can further include inter prediction type (L0 prediction, L1 prediction, Bi prediction, etc.) information. When inter prediction is applied, the neighboring blocks may include spatial neighboring blocks existing in the current picture and temporal neighboring blocks existing in the reference picture. The reference picture including the reference block and the reference picture including the temporal neighboring block may be the same or different. The temporal neighboring block may be called a collocated reference block, a collocated CU (colCU), etc., and the reference picture including the temporal neighboring block is also called a collocated picture (colPic).For example, a motion information candidate list may be constructed based on neighboring blocks of the current block, and flag or index information indicating which candidate is selected (used) to derive the motion vector and / or reference picture index of the current block may be signaled. Inter prediction may be performed based on various prediction modes, for example, in skip mode and (normal) merge mode, the motion information of the current block is the same as the motion information of the selected neighboring block. In the skip mode, unlike the merge mode, no residual signal is transmitted. In the motion information prediction (Motion Vector Prediction, MVP) mode, the motion vector of the selected neighboring block may be used as a motion vector predictor, and a motion vector difference may be signaled. In this case, the motion vector of the current block may be derived using the sum of the motion vector predictor and the motion vector difference.

[0075] A video / image encoding procedure based on inter prediction may generally include, for example, the following.

[0076] FIG. 4 illustrates an example of an inter-prediction based video / image encoding method.

[0077] The encoding apparatus performs inter prediction on a current block (S400). The encoding apparatus may derive an inter prediction mode and motion information of the current block and generate a prediction sample of the current block. Here, the inter prediction mode determination, motion information derivation, and prediction sample generation procedures may be performed simultaneously, or one procedure may be performed before the other procedures. For example, the inter prediction unit of the encoding apparatus may include a prediction mode determination unit, a motion information derivation unit, and a prediction sample derivation unit, and the prediction mode determination unit may determine a prediction mode for the current block, the motion information derivation unit may derive motion information of the current block, and the prediction sample derivation unit may derive a prediction sample of the current block. For example, the inter prediction unit of the encoding apparatus may search for a block similar to the current block within a certain area (search area) of a reference picture through motion estimation, and derive a reference block whose difference with the current block is minimum or equal to a certain criterion. Based on this, a reference picture index indicating a reference picture in which the reference block is located may be derived, and a motion vector may be derived based on a position difference between the reference block and the current block. The encoding apparatus may determine a mode to be applied to the current block from among various prediction modes. The encoding apparatus may compare RD costs for the various prediction modes to determine an optimal prediction mode for the current block.

[0078] For example, when a skip mode or a merge mode is applied to the current block, the encoding apparatus may construct a merge candidate list (to be described later) and derive a reference block, among reference blocks indicated by merge candidates included in the merge candidate list, in which a difference between the current block and the current block is minimum or equal to or less than a certain criterion. In this case, a merge candidate associated with the derived reference block may be selected, and merge index information indicating the selected merge candidate may be generated and signaled to a decoding apparatus. Motion information of the current block may be derived using motion information of the selected merge candidate.

[0079] As another example, when the (A)MVP mode is applied to the current block, the encoding apparatus may construct an (A)MVP candidate list, which will be described later, and may use a motion vector of an MVP candidate selected from the MVP (motion vector predictor) candidates included in the (A)MVP candidate list as the MVP of the current block. In this case, for example, a motion vector indicating a reference block derived by the above-mentioned motion estimation may be used as the motion vector of the current block, and an MVP candidate having a motion vector with the smallest difference from the motion vector of the current block among the MVP candidates becomes the selected MVP candidate. A motion vector difference (MVD), which is a difference obtained by subtracting the MVP from the motion vector of the current block, may be derived. In this case, information regarding the MVD may be signaled to the decoding apparatus. In addition, when the (A)MVP mode is applied, the value of the reference picture index may be configured as reference picture index information and separately signaled to the decoding apparatus.

[0080] The encoding apparatus may derive residual samples based on the predicted samples (S410). The encoding apparatus may derive the residual samples through a comparison between original samples of the current block and the predicted samples.

[0081] The encoding apparatus encodes image information including prediction information and residual information (S420). The encoding apparatus can output the encoded image information in the form of a bitstream. The prediction information is information related to the prediction procedure and can include prediction mode information (e.g., skip flag, merge flag, or mode index) and information on motion information. The information on the motion information can include candidate selection information (e.g., merge index, e.g., mvp flag or mvp index) which is information for deriving a motion vector. In addition, the information on the motion information can include the above-mentioned information on the MVD and / or reference picture index information. In addition, the information on the motion information can include information indicating whether L0 prediction, L1 prediction, or bi-prediction is applied. The residual information is information on the residual sample. The residual information can include information on quantized transform coefficients for the residual sample.

[0082] The output bitstream may be stored on a (digital) storage medium and transmitted to the decoding device, or may be transmitted to the decoding device via a network.

[0083] Meanwhile, as described above, the encoding apparatus can generate a reconstructed picture (including reconstructed samples and reconstructed blocks) based on the reference samples and the residual samples. This is to derive the same prediction result as that performed by the decoding apparatus in the encoding apparatus, thereby improving coding efficiency. Therefore, the encoding apparatus can store the reconstructed picture (or reconstructed samples, reconstructed blocks) in a memory and use it as a reference picture for inter prediction. As described above, an in-loop filtering procedure, etc. can be further applied to the reconstructed picture.

[0084] A video / image decoding procedure based on inter prediction may generally include, for example, the following.

[0085] FIG. 5 shows an example of an inter-prediction based video / image decoding method.

[0086] As shown in Figure 5, the decoding device may perform operations corresponding to those performed by the encoding device. The decoding device may perform prediction on the current block based on the received prediction information to derive a prediction sample.

[0087] Specifically, the decoding device may determine a prediction mode for the current block based on received prediction information (S500). The decoding device may determine which inter prediction mode is applied to the current block based on prediction mode information in the prediction information.

[0088] For example, it may be determined whether the merge mode or the (A)MVP mode is applied to the current block based on the merge flag. Alternatively, one of various inter prediction mode candidates may be selected based on the mode index. The inter prediction mode candidates may include a skip mode, a merge mode, and / or an (A)MVP mode, or may include various inter prediction modes described below.

[0089] The decoding apparatus derives motion information of the current block based on the determined inter prediction mode (S510). For example, when a skip mode or a merge mode is applied to the current block, the decoding apparatus may form a merge candidate list (described later) and select one merge candidate from among the merge candidates included in the merge candidate list. The selection may be performed based on the above-mentioned selection information (merge index). Motion information of the selected merge candidate may be used to derive motion information of the current block. The motion information of the selected merge candidate may be used as motion information of the current block.

[0090] As another example, when the (A)MVP mode is applied to the current block, the decoding apparatus may construct an (A)MVP candidate list described later, and may use a motion vector of an MVP candidate selected from among the MVP (motion vector predictor) candidates included in the (A)MVP candidate list as the MVP of the current block. The selection may be performed based on the selection information (mvp flag or mvp index) described above. In this case, the MVD of the current block may be derived based on information on the MVD, and the motion vector of the current block may be derived based on the mvp of the current block and the MVD. Also, the reference picture index of the current block may be derived based on the reference picture index information. A picture pointed to by the reference picture index in a reference picture list for the current block may be derived as a reference picture referenced for inter prediction of the current block.

[0091] On the other hand, as described below, the motion information of the current block may be derived without constructing a candidate list, in which case the motion information of the current block may be derived according to a procedure disclosed in a prediction mode described below. In this case, the candidate list construction as described above may be omitted.

[0092] The decoding apparatus may generate a prediction sample for the current block based on the motion information of the current block (S520). In this case, the reference picture may be derived based on the reference picture index of the current block, and the prediction sample of the current block may be derived using a sample of a reference block to which the motion vector of the current block points on the reference picture. In this case, as described below, a prediction sample filtering procedure may be further performed on all or a part of the prediction samples of the current block, depending on the case.

[0093] For example, the inter prediction unit of the decoding device may include a prediction mode determination unit, a motion information derivation unit, and a prediction sample derivation unit, and may determine a prediction mode for the current block based on prediction mode information received in the prediction mode determination unit, derive motion information (such as a motion vector and / or a reference picture index) of the current block based on information regarding the motion information received in the motion information derivation unit, and derive a prediction sample of the current block in the prediction sample derivation unit.

[0094] The decoding apparatus generates a residual sample for the current block based on the received residual information (S530). The decoding apparatus generates a reconstructed sample for the current block based on the prediction sample and the residual sample, and can generate a reconstructed picture based on the reconstructed sample (S540). Thereafter, as described above, an in-loop filtering procedure can be further applied to the reconstructed picture.

[0095] FIG. 6 exemplarily illustrates an inter prediction procedure.

[0096] As shown in FIG. 6, as described above, the inter prediction procedure may include an inter prediction mode determination step, a motion information derivation step according to the determined prediction mode, and a prediction execution (prediction sample generation) step based on the derived motion information. The inter prediction procedure may be performed in an encoding device and a decoding device as described above. In this document, a coding device may include an encoding device and / or a decoding device.

[0097] As shown in FIG. 6, the coding apparatus determines an inter prediction mode for a current block (S600). Various inter prediction modes can be used for prediction of a current block in a picture. For example, various modes such as merge mode, skip mode, Motion Vector Prediction (MVP) mode, Affine mode, sub-block merge mode, MMVD (Merge with MVD) mode, etc. can be used. Decoder side Motion Vector Refinement (DMVR) mode, Adaptive Motion Vector Resolution (AMVR) mode, Bi-prediction with CU-level weight (BCW), Bi-Directional Optical Flow (BDOF), etc. can be used as additional modes or instead. Affine mode is also called affine motion prediction mode. MVP mode is also called Advanced Motion Vector Prediction (AMVP) mode. In this document, some modes and / or motion information candidates derived by some modes may be included as one of the motion information related candidates of other modes. For example, an HMVP candidate can be added as a merge candidate of the merge / skip mode, or can be added as an MVP candidate of the MVP mode. When the HMVP candidate is used as a motion information candidate of the merge mode or skip mode, the HMVP candidate is also called an HMVP merge candidate.

[0098] Prediction mode information indicating an inter prediction mode of a current block may be signaled from an encoding device to a decoding device. The prediction mode information may be included in a bitstream and received by the decoding device. The prediction mode information may include index information indicating one of a plurality of candidate modes. Alternatively, the inter prediction mode may be indicated through hierarchical signaling of flag information. In this case, the prediction mode information may include one or more flags. For example, a skip flag may be signaled to indicate whether or not a skip mode can be applied, a merge flag may be signaled to indicate whether or not a merge mode can be applied when the skip mode is not applied, an MVP mode may be applied when the merge mode is not applied, or a flag for additional division may be further signaled. The affine mode may be signaled in an independent mode, or may be signaled in a mode dependent on the merge mode or MVP mode. For example, the affine mode may include an affine merge mode and an affine MVP mode.

[0099] The coding apparatus derives motion information for the current block (S610). The motion information may be derived based on the inter prediction mode.

[0100] The coding apparatus may perform inter prediction using motion information of the current block. The encoding apparatus may derive optimal motion information for the current block through a motion estimation procedure. For example, the encoding apparatus may search for a similar reference block having a high correlation in a fractional pixel unit within a determined search range in the reference picture using an original block in an original picture for the current block, thereby deriving motion information. The similarity of the block may be derived based on a difference in phase-based sample values. For example, the similarity of the block may be calculated based on the SAD between the current block (or the template of the current block) and the reference block (or the template of the reference block). In this case, the motion information may be derived based on the reference block having the smallest SAD in the search area. The derived motion information may be signaled to the decoding apparatus in various ways based on the inter prediction mode.

[0101] The coding apparatus performs inter prediction based on motion information for the current block (S620). The coding apparatus may derive predictive sample(s) for the current block based on the motion information. The current block including the predictive samples is also referred to as a predicted block.

[0102] Meanwhile, in inter prediction, according to the conventional merge or AMVP mode, a method of reducing the amount of motion information is used by using the motion vectors of spatially / temporally adjacent blocks of the current block as motion information candidates. For example, the adjacent blocks used to derive the motion information candidates of the current block may include the lower left corner adjacent block, the left adjacent block, the upper right corner adjacent block, the upper adjacent block, and the upper left corner adjacent block of the current block.

[0103] FIG. 7 exemplarily illustrates spatially neighboring blocks used for motion information candidate derivation in conventional merge or AMVP mode.

[0104] Basically, the spatial neighboring blocks are limited to blocks adjacent to the current block. This is to improve hardware feasibility, and problems such as an increase in line buffers occur in order to derive information on blocks far from the current block. However, using motion information on non-adjacent blocks to derive motion information candidates for the current block improves performance because various candidates can be configured. A History based Motion Vector Prediction (HMVP) method can be used to use motion information on non-adjacent blocks without increasing the line buffer. In this document, HMVP can refer to History based Motion Vector Prediction or History based Motion Vector Predictor. According to this document, inter prediction can be efficiently performed using HMVP, and parallel processing can be supported. For example, in the embodiment of this document, various methods of managing a history buffer for parallel processing are proposed, and parallel processing can be supported based on this. However, supporting parallel processing does not mean that parallel processing should be performed as a must, and a coding device may or may not perform parallel processing in consideration of hardware performance and service form. For example, if the coding device includes a multi-core processor, the coding device may process some of the slices, bricks, and / or tiles in parallel, whereas if the coding device includes a single-core processor or if the coding device includes a multi-core processor, the coding device may perform sequential processing while reducing computational and memory burdens.

[0105] The HMVP candidate according to the above-mentioned HMVP method may include motion information of a previously coded block. For example, the motion information of a previously coded block according to a block coding order in a current picture is not considered as the motion information of the current block if the previously coded block is not adjacent to the current block. However, the HMVP candidate may be considered as a motion information candidate (e.g., a merge candidate or an MVP candidate) of the current block without considering whether the previously coded block is adjacent to the current block. In this case, multiple HMVP candidates may be stored in a buffer. For example, when a merge mode is applied to the current block, an HMVP candidate (HMVP merge candidate) may be added to a merge candidate list. In this case, the HMVP candidate may be added next to the spatial merge candidate and the temporal merge candidate included in the merge candidate list.

[0106] According to the HMVP method, motion information of a previously coded block may be stored in the form of a table and may be used as a motion information candidate (e.g., merge candidate) of a current block. A table (or buffer, list) including a plurality of HMVP candidates may be maintained during the encoding / decoding procedure. The table (or buffer, list) may also be referred to as an HMVP table (or buffer, list). According to an embodiment of this document, the table (or buffer, list) may be initialized when encountering a new slice. Alternatively, according to an embodiment of this document, the table (or buffer, list) may be initialized when encountering a new CTU row. When the table is initialized, the number of HMVP candidates included in the table may be set to 0. The size of the table (or buffer, list) may be fixed to a specific value (e.g., 5). For example, if there is an inter-coded block, the associated motion information may be added as a new HMVP candidate in the last entry of the table. The above (HMVP) table is also called an (HMVP) buffer or an (HMVP) list.

[0107] 8 illustrates an example of an HMVP candidate-based decoding procedure, where the HMVP candidate-based decoding procedure may include an HMVP candidate-based inter-prediction procedure.

[0108] As shown in FIG. 8, the decoding apparatus loads an HMVP table including HMVP candidate(s) and decodes a block based on at least one of the HMVP candidate(s). Specifically, for example, the decoding apparatus may derive motion information of a current block based on at least one of the HMVP candidate(s), and may perform inter prediction on the current block based on the motion information to derive a predicted block (including a predicted sample). As described above, a reconstructed block may be generated based on the predicted block. The derived motion information of the current block may be updated in the table. In this case, the motion information may be added as a new HMVP candidate as the last entry of the table. If the number of HMVP candidates already included in the table is equal to the size of the table, the candidate that was first entered in the table may be deleted, and the derived motion information may be added as a new HMVP candidate to the last entry of the table.

[0109] FIG. 9 exemplarily shows an HMVP table update according to the FIFO rule, and FIG. 10 exemplarily shows an HMVP table update according to the limited FIFO rule.

[0110] A first-in-first-out (FIFO) rule may be applied to the table. For example, if the table size S is 16, this indicates that 16 HMVP candidates may be included in the table. If more than 16 HMVP candidates are generated from previously coded blocks, a first-in-first-out rule may be applied, so that the table may include up to 16 most recently coded motion information candidates. In this case, as shown in FIG. 9, the first-in-first-out rule may be applied to remove the oldest HMVP candidate and add a new HMVP candidate.

[0111] Meanwhile, in order to further improve coding efficiency, a limited FIFO rule may be applied as shown in FIG 10. As shown in FIG 10, when an HMVP candidate is inserted into a table, a redundancy check may be applied first. Thereby, it may be determined whether an HMVP candidate having the same motion information is already present in the table. If an HMVP candidate having the same motion information is present in the table, the HMVP candidate having the same motion information is removed from the table, and the HMVP candidates following the removed HMVP candidate are moved by one interval (i.e., each index -1) before a new HMVP candidate is inserted.

[0112] As mentioned above, the HMVP candidates can be used in the merge candidate list construction procedure. In this case, for example, all HMVP candidates that can be inserted from the last entry to the first entry in the table can be inserted next to the spatial merge candidate and the temporal merge candidate. In this case, a pruning check can be applied to the HMVP candidates. The maximum number of merge candidates allowed can be signaled, and the merge candidate list construction procedure can be terminated if the total number of available merge candidates reaches the maximum number of merge candidates.

[0113] Similarly, the HMVP candidate may also be used in the (A) MVP candidate list construction procedure. In this case, the motion vectors of the last k HMVP candidates in the HMVP table may be added next to the TMVP candidates that constitute the MVP candidate list. In this case, for example, an HMVP candidate having the same reference picture as the MVP target reference picture may be used for the MVP candidate list construction. Here, the MVP target reference picture may indicate a reference picture for inter prediction of the current block to which the MVP mode is applied. In this case, a pruning check may be applied to the HMVP candidate. The k may be, for example, 4. However, this is merely an example, and the k may have various values ​​such as 1, 2, 3, and 4.

[0114] On the other hand, if the total number of merge candidates is equal to or greater than 15, a truncated unary plus fixed length (with 3 bits) binarization method as shown in Table 1 below can be applied for merge index coding.

[0115] [Table 1]

[0116] The above table assumes that Nmrg=15, where Nmrg indicates the total number of merging candidates.

[0117] Meanwhile, when developing a solution that applies a video codec, parallel processing can be supported in image / video coding for implementation optimization.

[0118] FIG. 11 exemplarily illustrates WPP (Wavefront Parallel Processing), which is one of the techniques for parallel processing.

[0119] As shown in FIG. 11, when WPP is applied, parallel processing can be performed in units of CTU rows. In this case, when coding (encoding / decoding) a block indicated by X, there is a dependency with the position indicated by the arrow. Therefore, it is necessary to wait for the coding of the CTU on the upper right side of the block to be currently coded to be completed. In addition, when WPP is applied, the initialization of the CABAC probability table (or context information) can be performed in units of slices, and in order to perform parallel processing including entropy encoding / decoding, the CABAC probability table (or context information) must be initialized in units of CTU rows. WPP can be considered as a technology proposed to determine an efficient initialization position. When WPP is applied, each LCT row can be called a substream, and parallel processing can be supported when the coding device has multiple processing cores. For example, when WPP is applied, if three processing cores process decoding in parallel, the first processing core may decode substream 0, the second processing core may decode substream 1, and the third processing core may decode substream 2. When WPP is applied, after coding is performed (progressed) for the nth (n is an integer) substream, coding for the second CTU or LCU of the nth substream may be completed, and then coding for the n+1th substream may be performed. For example, in the case of entropy coding, if entropy coding for the second LCU of the nth substream is completed, the first LCU of the n+1th substream may be entropy coded based on context information for the second LCU of the nth substream. In this case, the number of substreams in a slice may be the same as the number of LCU rows. In addition, the number of substreams in a slice may be the same as the number of entry points. In this case, the number of entry points may be determined by the number of entry point offsets.For example, the number of entry points may have a value greater than the number of entry point offsets by 1. Information regarding the number of entry point offsets and / or information regarding the offset values ​​may be included in the above-mentioned video / image information and encoded, and may be signaled to a decoding device via a bitstream. Meanwhile, when a coding device has one processing core, it may perform coding processing in units of one sub-stream, thereby reducing memory load and coding dependency.

[0120] The above-mentioned HMVP method stores motion information derived from the coding procedure of each block as candidates by the size of a predetermined buffer (HMVP table). In this case, as many as the number of buffers can be filled as disclosed without additional conditions in FIG. 9, or candidates can be filled without overlapping by checking overlap between newly added candidates and candidates existing in the buffer (HMVP table). As a result, various candidates can be configured. However, when developing a solution using a video codec, it is generally not possible to know when HMVP candidates are filled in the buffer, so it is not possible to realize parallel processing with or without applying WPP.

[0121] FIG. 12 exemplarily illustrates the problems encountered when applying the general HMVP method while taking parallel processing into consideration.

[0122] As shown in FIG. 12, when parallelization is performed for each CTU row as in WPP, a dependency problem of the HMVP buffer may occur. For example, the HMVP buffer for the first CTU in the Nth (N>=1)th CTU row is not filled until the coding (encoding / decoding) of a block in the N-1th CTU row, for example, a block in the last CTU in the N-1th CTU row, is completed. That is, when parallel processing is applied under the current structure, the decoding device cannot know whether the HMVP candidate stored in the current HMVP buffer matches the HMVP buffer used for decoding the current (target) block. This is because a difference may occur between the HMVP buffer derived at the time of coding the current block when sequential processing is applied and the HMVP buffer derived at the time of coding the current block when parallel processing is applied.

[0123] In one embodiment of this document, in order to solve the above problems, when applying HMVP, parallel processing is supported by initializing a history management buffer (HMVP buffer).

[0124] FIG. 13 exemplarily illustrates a method for initializing a history management buffer (HMVP buffer) according to one embodiment of this document.

[0125] As shown in FIG. 13, the HMVP buffer may be initialized for each first CTU in a CTU row. That is, when coding the first CTU in a CTU row, the number of HMVP candidates included in the HMVP buffer may be set to 0 by initializing the HMVP buffer. As described above, by initializing the HMVP buffer for each CTU row, HMVP candidates derived in the coding process of a CTU located to the left of the current block may be used without restriction even when parallel processing is supported. In this case, for example, when a current CU, which is a current block, is located in the first CTU in a CTU row and the current CU corresponds to the first CU of the first CTU, the number of HMVP candidates included in the HMVP buffer is 0. Also, for example, when a CU coded before the current CU in the CTU row is coded in inter mode, an HMVP candidate may be derived based on motion information of the previously coded CU and included in the HMVP buffer.

[0126] FIG. 14 illustrates an exemplary HMVP buffer management method according to one embodiment.

[0127] As shown in FIG. 14, the HMVP buffer can be initialized in units of slices, and it can be determined whether the coding target CTU (current CTU) is the first CTU in each CTU row for the CTUs in the slice. In FIG. 14, it is described that it is determined to be the first CTU when (ctu_idx % Num) is 0. In this case, Num means the number of CTUs in each CTU row. As another example, when the above-mentioned brick concept is used, it can be determined that it is the first CTU in the CTU row (in the corresponding brick) when (ctu_idx_in_brick % BrickWidth) is 0. Here, ctu_idx_in_brick indicates the index of the corresponding CTU in the brick, and BrickWidth indicates the width of the corresponding brick in CTU units. In other words, BrickWidth can indicate the number of CTU columns in the corresponding brick. If the current CTU is the first CTU in the CTU row, the HMVP buffer is initialized (i.e., the number of candidates in the HMVP buffer is set to 0); otherwise, the HMVP buffer is maintained. Then, a prediction process for each CU in the CTU (e.g., based on merge or MVP mode) is performed, and the candidates stored in the HMVP buffer at this time can be included as motion information candidates for merge mode or MVP mode (e.g., merge candidates or MVP candidates). The motion information of the target block (current block) derived in the inter prediction process based on merge mode or MVP mode is stored (updated) in the HMVP buffer as a new HMVP candidate. In this case, the above-mentioned overlap check process may be further performed. Then, the above-mentioned procedure may be repeated for the CU and CTU.

[0128] As another example, when applying HMVP, it is possible to remove dependencies on a per-CTU basis by initializing the HMVP buffer for each CTU.

[0129] FIG. 15 illustrates an exemplary HMVP buffer management method according to another embodiment.

[0130] As shown in Figure 15, HMVP buffer initialization can be performed for each CTU without determining whether the current CTU is the first CTU in each CTU row. In this case, the HMVP buffer is initialized on a CTU-by-CTU basis, and motion information of blocks present in the CTU is stored in the HMVP table. In this case, HMVP candidates can be derived based on motion information of blocks (e.g., CUs) in the same CTU, and HMVP buffer initialization is possible without determining whether the current CTU is the first CTU in each CTU row, as described below.

[0131] As described above, the HMVP buffer can be initialized in slice units, and thus it is possible to use the motion vector of a block spatially separated from the current block. However, in this case, since parallel processing support is not possible within a slice, the above-mentioned embodiment and the like propose a method of initializing the buffer in units of CTU rows or CTUs. That is, according to the embodiment and the like of this document, the HMVP buffer can be initialized in units of slices, and within a slice, it can be initialized in units of CTU rows.

[0132] Meanwhile, when coding (encoding / decoding) one picture, the picture may be divided into slices and / or tiles. For example, the picture may be divided into slices in consideration of error resilience, or the picture may be divided into tiles in order to encode / decode a part of the picture. When one picture is divided into multiple tiles, if the HMVP management buffer is applied, initialization is performed on a CTU row basis in the picture, i.e., initializing the HMVP buffer with the first CTU of each CTU row in the picture is not suitable for the tile structure for encoding / decoding a part of the picture.

[0133] FIG. 16 exemplarily illustrates a method for initializing an HMVP buffer in a tile structure.

[0134] As shown in Figure 16, in the case of tiles 1 and 3, the HMVP management buffer is not initialized for each tile, so there is an (HMVP) dependency with tiles 0 and 2. Therefore, when tiles exist, it is possible to initialize the HMVP buffer in the following way.

[0135] As an example, the HMVP buffer can be initialized in units of CTUs. This can of course be applied without dividing into tiles, slices, etc.

[0136] As another example, the HMVP buffer can be initialized for the first CTU of each tile.

[0137] FIG. 17 illustrates an example of an HMVP buffer initialization method for the first CTU of a tile according to another embodiment.

[0138] 17, when coding the first CTU of each tile, the HMVP buffer is initialized. That is, when coding tile 0, HMVP buffer 0 may be initialized and used, and when coding tile 1, HMVP buffer 1 may be initialized and used.

[0139] As yet another example, the HMVP buffer can be initialized for the first CTU in a row of CTUs in each tile.

[0140] FIG. 18 illustrates an example of a method for initializing an HMVP management buffer for the first CTU in a row of CTUs in each tile according to yet another embodiment.

[0141] As shown in FIG. 18, the HMVP buffer may be initialized for each CTU row in each tile. For example, the HMVP buffer may be initialized with the first CTU of the first CTU row in tile n, the HMVP buffer may be initialized with the first CTU of the second CTU row in tile n, and the HVMP buffer may be initialized with the first CTU of the third CTU row in tile n. In this case, if the coding device has a multi-core processor, the coding device may initialize and use HMVP buffer 0 for the first CTU row in tile n, initialize and use HVMP buffer 1 for the second CTU row in tile n, and initialize and use HMVP buffer 2 for the third CTU row in tile n, thereby supporting parallel processing. On the other hand, if the coding device has a single-core processor, the coding device may initialize and reuse the HVMP buffer with the first CTU of each CTU row in each tile according to the coding order.

[0142] On the other hand, the tile division structure and slice division structure also allow tiles and slices to exist simultaneously within one picture.

[0143] FIG. 19 shows an example of a structure in which tiles and slices coexist.

[0144] Figure 19 exemplarily illustrates a case where one picture is divided into four tiles and each tile has two slices. As shown in Figure 19, there may be cases where both slices and tiles exist in one picture, and the HMVP buffer can be initialized as follows:

[0145] For example, the HMVP buffer can be initialized on a CTU basis. This method can be applied regardless of whether the CTU is located in a tile or a slice.

[0146] As another example, the HMVP buffer can be initialized for the first CTU in each tile.

[0147] FIG. 20 shows an example of how to initialize the HMVP buffer for the first CTU in each tile.

[0148] As shown in Figure 20, the HMVP buffer may be initialized in the first CTU of each tile. Even if there are multiple slices in one tile, HMVP buffer initialization may be performed in the first CTU in the tile.

[0149] As yet another example, HMVP buffer initialization can be performed for each slice in a tile.

[0150] FIG. 21 shows an example of how to initialize the HMVP buffer for each slice in a tile.

[0151] As shown in Figure 21, the HMVP buffer can be initialized in the first CTU of each slice in a tile. Therefore, when there are multiple slices in one tile, the HMVP buffer initialization can be performed on each of the multiple slices. In this case, the HMVP buffer initialization can be performed when processing the first CTU of each slice.

[0152] On the other hand, there can be multiple tiles without slices in one picture, or multiple tiles in one slice. In such cases, you can perform HMVP buffer initialization as follows:

[0153] As an example, the HMVP buffer may be initialized for each tile group.

[0154] FIG. 22 shows an example of initializing an HMVP buffer for the first CTU of the first tile in a tile group.

[0155] As shown in Figure 22, one picture can be divided into two tile groups, and each tile group (TileGroup0, TileGroup1) can be divided into multiple tiles. In this case, the HMVP buffer can be initialized for the first CTU of the first tile in one tile group.

[0156] As another example, the HMVP buffer may be initialized on a tile-by-tile basis within a tile group.

[0157] FIG. 23 shows an example of initializing an HMVP buffer for the first CTU of each tile in a tile group.

[0158] As shown in Figure 23, one picture can be divided into two tile groups, and each tile group (TileGroup0, TileGroup1) can be divided into multiple tiles. In this case, the HMVP buffer can be initialized for the first CTU of each tile in one tile group.

[0159] As yet another example, an HMVP buffer can be initialized for the CTU rows of each tile in the tile group.

[0160] FIG. 24 shows an example of initializing an HMVP buffer for the CTU rows of each tile in a tile group.

[0161] As shown in Figure 24, one picture can be divided into two tile groups, and each tile group (TileGroup0, TileGroup1) can be divided into multiple tiles. In this case, the HMVP buffer can be initialized with the first CTU in the CTU row of each tile in one tile group.

[0162] Alternatively, in this case, the HMVP management buffer can be initialized in units of CTUs. Of course, this can be applied without dividing tiles, slices, tile groups, etc.

[0163] Figures 25 and 26 illustrate an example of a video / image encoding method including an inter-prediction method and related components according to an embodiment or embodiments of the present document. The method disclosed in Figure 25 may be performed by the encoding device disclosed in Figure 2. Specifically, for example, S2500 to S2530 in Figure 25 may be performed by the prediction unit 220 of the encoding device, S2540 in Figure 25 may be performed by the residual processing unit 230 of the encoding device, and S2550 in Figure 25 may be performed by the entropy encoding unit 240 of the encoding device. The method disclosed in Figure 25 may include the embodiments described above in the present document.

[0164] As shown in FIG. 25, the encoding apparatus derives an HMVP buffer for a current block (S2500). The encoding apparatus may perform the HMVP buffer management method described above in the embodiments of this document. For example, the HMVP buffer may be initialized per slice, tile, or tile group. And / or, the HMVP buffer may be initialized per CTU row. In this case, the HMVP buffer may be initialized per CTU row in the slice, tile, or tile group. Here, a tile may indicate a rectangular region such as a CTU in a picture. A tile may be specified based on a specific tile row and a specific tile column in the picture. For example, there may be one or more tiles in the current picture. In this case, the HMVP buffer may be initialized with a first CTU of a CTU row including the current block in the current tile. Alternatively, there may be one or more slices in the current picture. In this case, the HMVP buffer may be initialized with a first CTU of a CTU row including the current block in the current slice. Alternatively, there may be one or more tile groups in the current picture, in which case the HMVP buffer may be initialized with the first CTU of the CTU row that contains the current block in the current tile group.

[0165] The encoding apparatus may determine whether the current CTU is the first CTU in the CTU row. In this case, the HMVP buffer may be initialized with the first CTU in the CTU row in which the current CTU including the current block is located. In other words, the HMVP buffer may be initialized when processing the first CTU in the CTU row in which the current CTU including the current block is located. If it is determined that the current CTU including the current block is the first CTU in the CTU row in the current tile, the HMVP buffer may include HMVP candidates derived based on motion information of blocks processed prior to the current block in the current CTU, and if it is determined that the current CTU is not the first CTU in the CTU row in the current tile, the HMVP buffer may include HMVP candidates derived based on motion information of blocks processed prior to the current block in the CTU row in the current tile. Also, for example, if the current CU, which is the current block, is located in the first CTU of the CTU row in the current tile and the current CU corresponds to the first CU of the first CTU, the number of HMVP candidates included in the HMVP buffer is 0. Also, for example, if a CU coded before the current CU in the CTU row in the current tile (e.g., a CU coded before the current CU in the current CTU and / or a CU in a CTU coded before the current CTU in the current CTU row) is coded in inter mode, an HMVP candidate may be derived based on motion information of the previously coded CU and included in the HMVP buffer.

[0166] If the current picture is divided into multiple tiles, the HMVP buffer may be initialized on a CTU row basis within each tile.

[0167] The HMVP buffer may be initialized on a CTU row basis within a tile or slice. For example, if a specific CTU of the CTU row is not the first CTU of the CTU row within the current picture, but is the first CTU of the CTU row within the current tile or slice, the HMVP buffer may be initialized with the specific CTU.

[0168] When the HMVP buffer is initialized, the number of HMVP candidates included in the HMVP buffer may be set to zero.

[0169] The encoding apparatus constructs a motion information candidate list based on the HMVP buffer (S2510). The HMVP buffer may include HMVP candidates, and the motion information candidate list may include the HMVP candidates.

[0170] As an example, when a merge mode is applied to the current block, the motion information candidate list may be a merge candidate list. As another example, when an (A)MVP mode is applied to the current block, the motion information candidate list may be an MVP candidate list. When a merge mode is applied to the current block, the HMVP candidate may be added to the merge candidate list if the number of available merge candidates (e.g., including spatial merge candidates and temporal merge candidates) in the merge candidate list for the current block is less than a predetermined maximum number of merge candidates. In this case, the HMVP candidate may be inserted after the spatial candidate and the temporal candidate in the merge candidate list. In other words, the HMVP candidate may be assigned an index value that is greater than the indexes assigned to the spatial candidate and the temporal candidate in the merge candidate list. When an (A)MVP mode is applied to the current block, the HMVP candidate may be added to the MVP candidate list if the number of available MVP candidates (derived based on spatial and temporal neighboring blocks) in the MVP candidate list for the current block is less than two.

[0171] The encoding apparatus may derive motion information of the current block based on the motion information candidate list (S2520).

[0172] The encoding apparatus may derive motion information of the current block based on the motion information candidate list. For example, when a merge mode or an MVP mode is applied to the current block, the HMVP candidate included in the HMVP buffer may be used as a merge candidate or an MVP candidate. For example, when a merge mode is applied to the current block, the HMVP candidate included in the HMVP buffer may be included as a candidate in a merge candidate list, and the HMVP candidate may be indicated among the candidates included in the merge candidate list based on a merge index. The merge index is prediction-related information and may be included in image / video information, which will be described later. In this case, the HMVP candidate may be assigned an index in the merge candidate list with a lower priority than the spatial merge candidates and the temporal merge candidates included in the merge candidate list. That is, the index value assigned to the HMVP candidate may be assigned a higher value than the index values ​​of the spatial merge candidates and the temporal merge candidates. As another example, when the MVP mode is applied to the current block, the HMVP candidates included in the HMVP buffer are included as candidates in a merge candidate list, and the HMVP candidates among the candidates included in the MVP candidate list can be indicated based on an MVP flag (or an MVP index). The MVP flag (or MVP index) is prediction-related information and can be included in image / video information, which will be described later.

[0173] The encoding apparatus generates a prediction sample for the current block based on the derived motion information (S2530). The encoding apparatus performs inter prediction (motion compensation) based on the motion information, and can derive the prediction sample using a reference sample that the motion information points to on a reference picture.

[0174] The encoding apparatus generates residual samples based on the predicted samples (S2540). The encoding apparatus may generate residual samples based on the original samples for the current block and the predicted samples for the current block.

[0175] The encoding apparatus derives information about the residual samples based on the residual samples, and encodes image / video information including the information about the residual samples (S2550). The information about the residual samples may be referred to as residual information and may include information about quantized transform coefficients. The encoding apparatus may perform a transform / quantization procedure on the residual samples to derive the quantized transform coefficients.

[0176] The encoded image / video information may be output in the form of a bitstream. The bitstream may be transmitted to a decoding device via a network or a storage medium. The image / video information may further include prediction-related information, which may further include information on various prediction modes (e.g., merge mode, MVP mode, etc.), MVD information, etc.

[0177] Figures 27 and 28 illustrate an example of an image decoding method including an inter prediction method and related components according to an embodiment of the present document. The method disclosed in Figure 27 may be performed by the decoding device disclosed in Figure 3. Specifically, for example, S2700 to S2730 in Figure 27 may be performed by the prediction unit 330 of the decoding device, and S2740 may be performed by the addition unit 340 of the decoding device. The method disclosed in Figure 27 may include the embodiments described above in this document.

[0178] As shown in FIG. 27, the decoding device derives an HMVP buffer for a current block (S2700). The decoding device may perform the HMVP buffer management method described above in the embodiments of this document. For example, the HMVP buffer may be initialized in units of slices, tiles, or tile groups. And / or the HMVP buffer may be initialized in units of CTU rows. In this case, the HMVP buffer may be initialized in units of CTU rows in the slice, tile, or tile group. Here, a tile may indicate a rectangular region such as a CTU in a picture. A tile may be specified based on a specific tile row and a specific tile column in the picture. For example, there may be one or more tiles in the current picture. In this case, the HMVP buffer may be initialized with a first CTU of a CTU row including the current block in the current tile. Alternatively, there may be one or more slices in the current picture. In this case, the HMVP buffer may be initialized with a first CTU of a CTU row including the current block in the current slice. Alternatively, there may be one or more tile groups in the current picture, in which case the HMVP buffer may be initialized with the first CTU of the CTU row that contains the current block in the current tile group.

[0179] The decoding device may determine whether the current CTU is the first CTU in the CTU row. In this case, the HMVP buffer may be initialized with the first CTU in the CTU row in which the current CTU including the current block is located. In other words, the HMVP buffer may be initialized when processing the first CTU in the CTU row in which the current CTU including the current block is located. If it is determined that the current CTU including the current block is the first CTU in the CTU row in the current tile, the HMVP buffer may include HMVP candidates derived based on motion information of blocks processed prior to the current block in the current CTU, and if it is determined that the current CTU is not the first CTU in the CTU row in the current tile, the HMVP buffer may include HMVP candidates derived based on motion information of blocks processed prior to the current block in the CTU row in the current tile. Also, for example, when the current CU, which is the current block, is located in the first CTU of the CTU row in the current tile and the current CU corresponds to the first CU of the first CTU, the number of HMVP candidates included in the HMVP buffer is 0. Also, for example, if a CU coded before the current CU in the CTU row in the current tile (e.g., a CU coded before the current CU in the current CTU and / or a CU in a CTU coded before the current CTU in the current CTU row) is coded in inter mode, an HMVP candidate may be derived based on motion information of the previously coded CU and included in the HMVP buffer.

[0180] If the current picture is divided into multiple tiles, the HMVP buffer may be initialized on a CTU row basis within each tile.

[0181] The HMVP buffer may be initialized on a CTU row basis within a tile or slice. For example, if a specific CTU of the CTU row is not the first CTU of the CTU row within the current picture, but is the first CTU of the CTU row within the current tile or slice, the HMVP buffer may be initialized with the specific CTU.

[0182] When the HMVP buffer is initialized, the number of HMVP candidates included in the HMVP buffer may be set to zero.

[0183] The decoding apparatus constructs a motion information candidate list based on the HMVP buffer (S2710). The HMVP buffer may include HMVP candidates, and the motion information candidate list may be constructed including the HMVP candidates.

[0184] As an example, when a merge mode is applied to the current block, the motion information candidate list may be a merge candidate list. As another example, when an (A)MVP mode is applied to the current block, the motion information candidate list may be an MVP candidate list. When a merge mode is applied to the current block, the HMVP candidate may be added to the merge candidate list if the number of available merge candidates (e.g., including spatial merge candidates and temporal merge candidates) in the merge candidate list for the current block is less than a predetermined maximum number of merge candidates. In this case, the HMVP candidate may be inserted after the spatial candidate and the temporal candidate in the merge candidate list. In other words, the HMVP candidate may be assigned an index value that is greater than the indexes assigned to the spatial candidate and the temporal candidate in the merge candidate list. When an (A)MVP mode is applied to the current block, the HMVP candidate may be added to the MVP candidate list if the number of available MVP candidates (derived based on spatial and temporal neighboring blocks) in the MVP candidate list for the current block is less than two.

[0185] The decoding apparatus may derive motion information of the current block based on the motion information candidate list (S2720).

[0186] The encoding apparatus may derive motion information of the current block based on the motion information candidate list. For example, when a merge mode or an MVP mode is applied to the current block, the HMVP candidate included in the HMVP buffer may be used as a merge candidate or an MVP candidate. For example, when a merge mode is applied to the current block, the HMVP candidate included in the HMVP buffer may be included as a candidate in a merge candidate list, and the HMVP candidate among the candidates included in the merge candidate list may be indicated based on a merge index obtained from a bitstream. In this case, the HMVP candidate may be assigned an index in the merge candidate list with a lower priority than the spatial merge candidate and the temporal merge candidate included in the merge candidate list. That is, the index value assigned to the HMVP candidate may be assigned a higher value than the index values ​​of the spatial merge candidate and the temporal merge candidate. As another example, when MVP mode is applied to the current block, the HMVP candidate included in the HMVP buffer may be included as a candidate in a merge candidate list, and the HMVP candidate may be indicated among the candidates included in the MVP candidate list based on an MVP flag (or an MVP index) obtained from the bitstream.

[0187] The decoding device generates a prediction sample for the current block based on the derived motion information (S2730). The decoding device can derive the prediction sample by using a reference sample that the motion information points to on a reference picture by performing inter prediction (motion compensation) based on the motion information. The current block including the prediction sample is also called a predicted block.

[0188] The decoding apparatus generates a reconstructed sample based on the predicted sample (S2740). As described above, a reconstructed block / picture can be generated based on the reconstructed sample. As described above, the decoding apparatus can obtain residual information (including information on quantized transform coefficients) from the bitstream, derive a residual sample based on the residual information, and generate the reconstructed sample based on the predicted sample and the residual sample. As described above, hereafter, in-loop filtering procedures such as deblock filtering, SAO and / or ALF procedures can be applied to the reconstructed picture as necessary to improve subjective / objective image quality.

[0189] In the above-described embodiments, the method is described based on a flow chart with a series of steps or blocks, but the embodiment is not limited to the order of steps, and a step may occur in a different order or simultaneously with a different step than described above. Also, a person skilled in the art can understand that the steps shown in the flow chart are not exclusive, and other steps may be included, or one or more steps of the flow chart may be deleted without affecting the scope of the embodiments of this document.

[0190] The methods according to the embodiments of the present document described above may be implemented in software form, and the encoding device and / or decoding device according to the present document may be included in devices that perform image processing, such as, for example, TVs, computers, smartphones, set-top boxes, display devices, etc.

[0191] In this document, when the embodiments are implemented in software, the methods described above can be implemented with modules (processes, functions, etc.) performing the functions described above. The modules can be stored in a memory and executed by a processor. The memory can be inside or outside the processor and can be coupled to the processor in various well-known ways. The processor can include an Application-Specific Integrated Circuit (ASIC), other chipsets, logic circuits, and / or data processing devices. The memory can include a Read-Only Memory (ROM), a Random Access Memory (RAM), a flash memory, a memory card, a storage medium, and / or other storage devices. That is, the embodiments described in this document can be implemented and executed on a processor, microprocessor, controller, or chip. For example, the functional units shown in each drawing can be implemented and executed on a computer, processor, microprocessor, controller, or chip. In this case, information (e.g., information on instructions) or algorithms for implementation can be stored in a digital storage medium.

[0192] In addition, the decoding device and encoding device to which the embodiment(s) of this document are applied may be included in a multimedia broadcast transmitting / receiving device, a mobile communication terminal, a home cinema video device, a digital cinema video device, a surveillance camera, a video interactive device, a real-time communication device such as video communication, a mobile streaming device, a storage medium, a camcorder, a video on demand (VoD) service providing device, an OTT video (Over The Top video) device, an Internet streaming service providing device, a three-dimensional (3D) video device, a VR (Virtual Reality) device, an AR (Augmented Reality) device, an image telephone video device, a transportation terminal (e.g., a vehicle (including an autonomous vehicle) terminal, an airplane terminal, a ship terminal, etc.), and a medical video device, etc., and may be used to process a video signal or a data signal. For example, an OTT video (Over The Top video) device may include a game console, a Blu-ray player, an Internet-connected TV, a home theater system, a smartphone, a tablet PC, a DVR (Digital Video Recorder), etc.

[0193] In addition, the processing method to which the embodiment(s) of this document are applied can be produced in the form of a program executed by a computer and can be stored in a computer-readable recording medium. Multimedia data having a data structure according to the embodiment(s) of this document can also be stored in a computer-readable recording medium. The computer-readable recording medium includes any kind of storage device and distributed storage device in which computer-readable data is stored. The computer-readable recording medium can include, for example, a Blu-ray disc (BD), a universal serial bus (USB), a ROM, a PROM, an EPROM, an EEPROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, and an optical data storage device. The computer-readable recording medium also includes a medium realized in the form of a carrier wave (e.g., transmission via the Internet). Also, a bit stream generated by the encoding method can be stored in a computer-readable recording medium or transmitted via a wired or wireless communication network.

[0194] Also, the embodiment(s) of this document may be implemented in a computer program product by program code, which may be executed by a computer in accordance with the embodiment(s) of this document. The program code may be stored on a computer readable carrier.

[0195] FIG. 29 illustrates an example of a content streaming system in which the embodiments disclosed herein may be applied.

[0196] As shown in FIG. 29, a content streaming system to which the embodiments of this document are applied may generally include an encoding server, a streaming server, a web server, a media storage, a user device, and a multimedia input device.

[0197] The encoding server compresses content input from a multimedia input device such as a smartphone, camera, camcorder, etc., into digital data to generate a bitstream and transmits the bitstream to the streaming server. As another example, if a multimedia input device such as a smartphone, camera, camcorder, etc. directly generates a bitstream, the encoding server may be omitted.

[0198] The bitstream may be generated by an encoding method or a bitstream generation method to which an embodiment of this document is applied, and the streaming server may temporarily store the bitstream in the process of transmitting or receiving the bitstream.

[0199] The streaming server transmits multimedia data to a user device based on a user request via a web server, and the web server serves as a medium for informing the user of available services. When a user requests a desired service from the web server, the web server transmits the request to the streaming server, and the streaming server transmits multimedia data to the user. In this case, the content streaming system may include a separate control server, and in this case, the control server serves to control commands / responses between devices in the content streaming system.

[0200] The streaming server may receive the content from a media storage device and / or an encoding server. For example, when receiving the content from the encoding server, the content may be received in real time. In this case, the streaming server may store the bitstream for a certain period of time in order to provide a smooth streaming service.

[0201] Examples of the above user devices include mobile phones, smartphones, laptop computers, digital broadcasting terminals, Personal Digital Assistants (PDAs), Portable Multimedia Players (PMPs), navigation systems, slate PCs, tablet PCs, ultrabooks (ULTRABOOK (registered trademark)), wearable devices (e.g., smartwatches, smart glasses (smart glass), and Head Mounted Displays (HMDs)), digital TVs, desktop computers, and digital signatures.

[0202] Each server in the content streaming system can be operated as a distributed server, in which case data received by each server can be processed in a distributed manner.

Claims

1. An image decoding method performed by a decoding device, comprising: deriving a History-based Motion Vector Prediction (HMVP) candidate list for the current block; constructing a motion information candidate list based on the HMVP candidates included in the HMVP candidate list, where whether to add the HMVP candidate to the motion information candidate list is checked after a time candidate in the motion information candidate list; deriving motion information of the current block based on the motion information candidate list; deriving a reference picture index for the current block based on the motion information; deriving a motion vector for the current block based on the motion information; generating a prediction sample for the current block based on the reference picture index and the motion vector; generating reconstructed samples based on the predicted samples; the current picture includes one or more tiles; the current picture includes a plurality of tile columns and tile rows; A tile is a rectangular region of a coding tree unit (CTU) within a specific tile column and a specific tile row in the current picture, The HMVP candidate list is updated based on the motion information of the previous block; The HMVP candidate list is initialized with a first CTU for each CTU row of each tile; A method, wherein upon initialization of the HMVP candidate list, a number of HMVP candidates included in the HMVP candidate list is set to zero.

2. An image encoding method performed by an encoding device, comprising: deriving a History-based Motion Vector Prediction (HMVP) candidate list for the current block; constructing a motion information candidate list based on the HMVP candidates included in the HMVP candidate list, where whether to add the HMVP candidate to the motion information candidate list is checked after a time candidate in the motion information candidate list; deriving motion information of the current block based on the motion information candidate list; deriving a reference picture index for the current block based on the motion information; deriving a motion vector for the current block based on the motion information; generating a prediction sample for the current block based on the reference picture index and the motion vector; deriving residual samples based on the prediction samples; and encoding image information including information about the residual samples; One or more tiles are in the current picture, the current picture includes a plurality of tile columns and tile rows; A tile is a rectangular region of a coding tree unit (CTU) within a specific tile column and a specific tile row in the current picture, The HMVP candidate list is updated based on the motion information of the previous block; The HMVP candidate list is initialized with a first CTU for each CTU row of each tile; A method, wherein upon initialization of the HMVP candidate list, a number of HMVP candidates included in the HMVP candidate list is set to zero.

3. A method for transmitting data relating to an image, comprising the steps of: Obtaining a bitstream generated by a method, the method comprising: deriving a History-based Motion Vector Prediction (HMVP) candidate list for the current block; constructing a motion information candidate list based on the HMVP candidates included in the HMVP candidate list, where whether to add the HMVP candidate to the motion information candidate list is checked after a time candidate in the motion information candidate list; deriving motion information of the current block based on the motion information candidate list; deriving a reference picture index for the current block based on the motion information; deriving a motion vector for the current block based on the motion information; generating a prediction sample for the current block based on the reference picture index and the motion vector; deriving residual samples based on the prediction samples; generating the bitstream by encoding image information including information about the residual samples; transmitting the data including the bitstream; One or more tiles are in the current picture, the current picture includes a plurality of tile columns and tile rows; A tile is a rectangular region of a coding tree unit (CTU) within a specific tile column and a specific tile row in the current picture, The HMVP candidate list is updated based on the motion information of the previous block; The HMVP candidate list is initialized with a first CTU for each CTU row of each tile; A transmission method, wherein the number of HMVP candidates included in the HMVP candidate list is set to zero based on the HMVP candidate list being initialized.

Images