History-based image coding method and apparatus

Abstract

To provide a method and apparatus for improving image coding efficiency.SOLUTION: The method includes deriving a history-basedMotionVectorPrediction (HMVP) buffer for a current block, constructing a motion information candidate list based on HMVP candidates included in the HMVP buffer, deriving motion information of the current block based on the motion information candidate list, deriving a reference picture index of the current block based on the motion information, deriving a motion vector of the current block based on the motion information, generating prediction samples for the current block based on the reference picture index and the motion vector, and generating reconstructed samples based on the prediction samples. The current picture includes one or more tiles and includes a plurality of tile columns and tile rows, and a tile is a rectangular region of coding tree units (CTUs) within a specific tile column and a specific tile row in the current picture.SELECTED DRAWING: Figure 27

Description

Technical Field

[0001] This technology relates to image coding, and more particularly, to a history-based image coding method and apparatus therefor.

Background Art

[0002] Recently, the demand for high-resolution and high-quality images / videos such as 4K or UHD (Ultra High Definition) images / videos of 8K or higher has been increasing in various fields. As the image / video data becomes higher in resolution and quality, the amount of information or bits to be transmitted relatively increases compared to the existing image / video data. Therefore, when transmitting image data using a medium such as an existing wired or wireless broadband line, or storing image / video data using an existing storage medium, the transmission cost and storage cost increase.

[0003] In addition, recently, the interest and demand for immersive media such as VR (Virtual Reality), AR (Artificial Reality) contents, and holograms have been increasing, and the broadcasting of images / videos having image characteristics different from real images, such as game images, has been increasing.

[0004] Therefore, in order to effectively compress, transmit, store, and reproduce the information of high-resolution and high-quality images / videos having various characteristics as described above, a highly efficient image / video compression technology is required.

Summary of the Invention

Problems to be Solved by the Invention

[0005] The technical problem of this document is to provide a method and apparatus for increasing image coding efficiency.

[0006] Another technical problem of this document is to provide an efficient inter-prediction method and apparatus.

[0007] Another technical objective of this paper is to provide a method and apparatus for deriving history-based motion vectors.

[0008] Another technical objective of this paper is to provide a method and apparatus for efficiently deriving HMVP (History-based Motion Vector Prediction) candidates.

[0009] Another technical objective of this paper is to provide a method and apparatus for efficiently updating HMVP buffers.

[0010] Another technical objective of this paper is to provide a method and apparatus for efficiently initializing an HMVP buffer. [Means for solving the problem]

[0011] According to one embodiment of this document, an image decoding method performed by a decoding device is provided. The method comprises the steps of: deriving an HMVP (History-based Motion Vector Prediction) buffer for the current block; constructing a motion information candidate list based on the HMVP candidates in the HMVP buffer; deriving motion information for the current block based on the motion information candidate list; generating a prediction sample for the current block based on the motion information; and generating a reconstructed sample based on the prediction sample. The current picture contains one or more tiles, and the HMVP buffer is initialized with the first CTU of the CTU row containing the current block within the current tile.

[0012] According to other embodiments of this document, a decoding device for image decoding is provided. The decoding device comprises a prediction unit that derives an HMVP (History-based Motion Vector Prediction) buffer for the current block, constructs a motion information candidate list based on the HMVP candidates in the HMVP buffer, derives motion information for the current block based on the motion information candidate list, and generates a prediction sample for the current block based on the motion information, and an addition unit that generates a restoration sample based on the prediction sample, wherein one or more tiles exist in the current picture, and the HMVP buffer is initialized with the first CTU of the CTU row having the current block in the current tile.

[0013] In yet another embodiment of this document, an image encoding method performed by an encoding device is provided. The method comprises the steps of: deriving an HMVP (History-based Motion Vector Prediction) buffer for the current block; constructing a motion information candidate list based on the HMVP candidates in the HMVP buffer; deriving motion information for the current block based on the motion information candidate list; generating predicted samples for the current block based on the motion information; deriving residual samples based on the predicted samples; and encoding image information having information about the residual samples, wherein one or more tiles exist in the current picture, and the HMVP buffer is initialized with the first CTU of the CTU row having the current block in the current tile.

[0014] According to yet another embodiment of this document, an encoding device for performing image encoding is provided. The encoding device comprises: a prediction unit that derives an HMVP (History-based Motion Vector Prediction) buffer for the current block, constructs a motion information candidate list based on the HMVP candidates in the HMVP buffer, derives motion information for the current block based on the motion information candidate list, and generates predicted samples for the current block based on the motion information; a residual processing unit that derives residual samples based on the predicted samples; and an entropy encoding unit that encodes image information having information about the residual samples. The current picture contains one or more tiles, and the HMVP buffer is initialized with the first CTU of the CTU row containing the current block within the current tile.

[0015] According to yet another embodiment of this document, a digital storage medium is provided which stores image data having encoded image information generated by an image encoding method performed by an encoding device.

[0016] According to yet another embodiment of this document, a digital storage medium is provided which stores image data having encoded image information that triggers an image decoding method to be performed by a decoding device. [Effects of the Invention]

[0017] According to one embodiment of this document, the overall image / video compression efficiency can be improved.

[0018] According to one embodiment of this document, the amount of data transmitted required for residual processing can be reduced through efficient interpretation.

[0019] According to one embodiment of this document, the HMVP buffer can be managed efficiently.

[0020] According to one embodiment of this document, parallel processing can be supported through efficient HMVP buffer management.

[0021] According to one embodiment of this document, a motion vector for inter prediction can be efficiently derived.

Brief Description of Drawings

[0022] [Figure 1] It is a diagram schematically showing an example of a video / image coding system to which an embodiment of this document can be applied. [Figure 2] It is a diagram schematically explaining the configuration of a video / image encoding device to which an embodiment of this document can be applied. [Figure 3] It is a diagram schematically explaining the configuration of a video / image decoding device to which an embodiment of this document can be applied. [Figure 4] It is a diagram showing an example of an inter prediction-based video / image encoding method. [Figure 5] It is a diagram showing an example of an inter prediction-based video / image decoding method. [Figure 6] It is a diagram illustratively showing an inter prediction procedure. [Figure 7] It is a diagram illustratively showing spatial adjacent blocks used for deriving motion information candidates in a conventional merge or AMVP mode. [Figure 8] It is a diagram schematically showing an example of an HMVP candidate-based decoding procedure. [Figure 9] It is a diagram illustratively showing an HMVP table update according to the FIFO rule. [Figure 10] It is a diagram illustratively showing an HMVP table update according to a restricted FIFO rule. [Figure 11] It is a diagram illustratively showing WPP (Wavefront Parallel Processing), which is one of the techniques for parallel processing. [Figure 12] It is a diagram illustratively showing problems when applying a general HMVP method considering parallel processing. [Figure 13]This figure illustrates an example of an initialization method for a history management buffer (HMVP buffer) according to one embodiment of this document. [Figure 14] This figure illustrates an HMVP buffer management method according to one embodiment. [Figure 15] This figure illustrates an HMVP buffer management method according to another embodiment. [Figure 16] This diagram illustrates an example of how to initialize an HMVP buffer in a tile structure. [Figure 17] This figure shows an example of an HMVP buffer initialization method targeting the first CTU of a tile according to another embodiment. [Figure 18] This figure shows an example of an HMVP management buffer initialization method targeting the first CTU in each CTU row within each tile according to another embodiment. [Figure 19] This diagram shows an example of a structure in which tiles and slices coexist. [Figure 20] This figure shows an example of how to initialize the HMVP buffer targeting the first CTU within each tile. [Figure 21] This figure shows an example of how to initialize an HMVP buffer for each slice within a tile. [Figure 22] This figure shows an example of initializing the HMVP buffer targeting the first CTU of the first tile in a tile group. [Figure 23] This figure shows an example of initializing the HMVP buffer targeting the first CTU of each tile in a tile group. [Figure 24] This figure shows an example of initializing an HMVP buffer for the CTU row of each tile within a tile group. [Figure 25] This figure schematically illustrates an example of a video / image encoding method and related components, including an interpretation method, according to one or more embodiments of this document. [Figure 26]This figure schematically illustrates an example of a video / image encoding method and related components, including an interpretation method, according to one or more embodiments of this document. [Figure 27] This figure schematically shows an example of an image decoding method including an interpretation method and related components according to the embodiment of this document. [Figure 28] This figure schematically shows an example of an image decoding method including an interpretation method and related components according to the embodiment of this document. [Figure 29] This figure shows an example of a content streaming system to which the embodiments disclosed in this document may be applied. [Modes for carrying out the invention]

[0023] The methods presented herein can be modified in various ways and have various embodiments. Specific embodiments are illustrated in the drawings and described in detail. Terms used herein are used solely to describe specific embodiments and are not intended to limit the technical ideas of the methods presented herein. Singular expressions include expressions such as “at least one” unless the context clearly indicates otherwise. Terms such as “includes” or “has” herein are intended to specify the existence of features, figures, steps, actions, components, parts, or combinations thereof described herein, and should be understood not to preemptively exclude the possibility of the existence or addition of one or more other features, figures, steps, actions, components, parts, or combinations thereof.

[0024] On the other hand, each configuration shown in the drawings described in this document is illustrated independently for the purpose of explaining its distinct characteristic functions, etc., and does not mean that each configuration is implemented with separate hardware or separate software. For example, two or more of the configurations can be combined to form one configuration, and one configuration can be divided into multiple configurations. Embodiments in which each configuration is integrated and / or separated are also included in the scope of disclosure in this document, as long as they do not deviate from the essence of the methods disclosed in this document.

[0025] This document relates to video / image coding. For example, the methods / embodiments disclosed in this document can be applied to methods disclosed in the VVC (Versatile Video Coding) standard. Furthermore, the methods / embodiments disclosed in this document can be applied to methods disclosed in the EVC (Essential Video Coding) standard, AV1 (AOMedia Video 1) standard, AVS2 (2nd Generation Of Audio Video Coding Standard), or next-generation video / image coding standards (e.g., H.267 or H.268).

[0026] This document presents various embodiments for video / image coding, and unless otherwise noted, these embodiments can be implemented in combination with each other.

[0027] In this document, "video" can mean a collection of images over time. "Picture" generally refers to a unit representing a single image at a specific time point in time, while "slice" or "tile" is a unit that constitutes part of a picture in coding. A slice or tile can contain one or more Coding Tree Units (CTUs). A single picture can consist of one or more slices or tiles. A single picture can consist of one or more tile groups. A tile group can contain one or more tiles. A brick can represent a rectangular region of CTU rows within a tile in a picture. A tile can be partitioned into multiple bricks, each of which consists of one or more CTU rows within the tile. Additionally, a tile that is not partitioned into multiple bricks may also be referred to as a brick.A brick scan can represent a specific sequential ordering of CTUs partitioning a picture in which the CTUs are ordered consecutively in a CTU raster scan in a brick, bricks within a tile are ordered consecutively in a raster scan of the bricks of the tile, and tiles in a picture are ordered consecutively in a raster scan of the tiles of the picture. A tile is a rectangular region of CTUs within a particular tile column and a particular tile row in a picture. The tile column is a rectangular region of CTUs having a height equal to the height of the picture and a width specified by syntax elements in the picture parameter set.The tile row is a rectangular region of CTUs having a height specified by syntax elements in the picture parameter set and a height equal to the width of the picture. A tile scan can demonstrate a specific sequential ordering of CTUs partitioning a picture in which the CTUs are ordered consecutively in CTU raster scan in a tile, whereas tiles in a picture are ordered consecutively in a raster scan of the tiles of the picture. A slice may contain an integer number of bricks of a picture that may be exclusively contained in a single NAL unit. A slice may consist of either a number of complete tiles or only a consecutive sequence of complete bricks of one tile.In this document, tile groups and slices may be used interchangeably. For example, in this document, tile group / tile group header is also referred to as slice / slice header.

[0028] A pixel or pel can refer to the smallest unit that makes up a picture (or image). Alternatively, the term "sample" can be used as a counterpart to pixel. A sample generally refers to a pixel or a pixel value, and may refer only to the pixel / pixel value of the luma component, or only to the pixel / pixel value of the chroma component.

[0029] A unit can represent a basic unit of image processing. A unit can contain at least one of a specific region of a picture and information associated with that region. A unit can contain one luma block and two chroma (e.g., cb, cr) blocks. The term unit can sometimes be used interchangeably with terms such as block or area. In general, an M×N block can contain a set (or array) of samples (or sample arrays) or transform coefficients consisting of M columns and N rows.

[0030] In this document, the terms " / " and "," are interpreted as "and / or." For example, "A / B" is interpreted as "A and / or B," and "A, B" is interpreted as "A and / or B." Additionally, "A / B / C" means "at least one of A, B, and / or C." Similarly, "A, B, C" also means "at least one of A, B, and / or C."

[0031] In addition, in this document, "or" should be interpreted as "and / or." For example, "A or B" can mean 1) only "A," 2) only "B," or 3) both "A and B." Furthermore, "or" in this document can mean "additionally or alternatively."

[0032] The embodiments of this document will be described in more detail below with reference to the attached drawings. Hereafter, the same reference numerals will be used for the same components in the drawings, and redundant descriptions of the same components may be omitted.

[0033] Figure 1 schematically shows an example of a video / image coding system to which the embodiments described in this document may be applied.

[0034] As shown in Figure 1, a video / image coding system may include a first device (source device) and a second device (receiving device). The source device can transmit encoded video / image information or data to the receiving device in file or streaming form via a digital storage medium or network.

[0035] The source device may include a video source, an encoding device, and a transmitter. The receiving device may include a receiver, a decoder, and a renderer. The encoding device is also called a video / image encoder, and the decoder is also called a video / image decoder. The transmitter may be included in the encoding device. The receiver may be included in the decoder. The renderer may include a display unit, which may consist of a separate device or external component.

[0036] A video source can acquire video / images through processes such as video / image capture, synthesis, or generation. A video source may include video / image capture devices and / or video / image generation devices. Video / image capture devices may include, for example, one or more cameras, or a video / image archive containing previously captured video / images. Video / image generation devices may include, for example, computers, tablets, and smartphones, and can generate video / images (electronically). For example, virtual video / images can be generated via a computer, in which case the video / image capture process can be substituted as the process by which the associated data is generated.

[0037] An encoding device can encode input video / images. For compression and coding efficiency, the encoding device can perform a series of steps, including prediction, transformation, and quantization. The encoded data (encoded video / image information) can be output in bitstream format.

[0038] The transmitting unit can transmit encoded video / image information or data output in bitstream format to the receiving unit of a receiving device via a digital storage medium or network in file or streaming format. The digital storage medium can include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, and SSD. The transmitting unit may include elements for generating media files via a predetermined file format and elements for transmission via a broadcast / communication network. The receiving unit can receive / extract the bitstream and transmit it to a decoding device.

[0039] A decoding device can decode video / images by performing a series of steps, such as inverse quantization, inverse transformation, and prediction, corresponding to the operation of the encoding device.

[0040] The renderer can render the decoded video / image. The rendered video / image can be displayed via the display unit.

[0041] Figure 2 is a schematic diagram illustrating the configuration of a video / image encoding device to which the embodiments described in this document may be applied. Hereinafter, the term "video encoding device" may include an image encoding device.

[0042] As shown in Figure 2, the encoding device 200 may be configured to include an image partitioner 210, a predictor 220, a residual processor 230, an entropy encoder 240, an adder 250, a filter 260, and a memory 270. The predictor 220 may include an inter-predictor 221 and an intra-predictor 222. The residual processor 230 may include a transformer 232, a quantizer 233, a dequantizer 234, and an inverse transformer 235. The residual processor 230 may further include a subtractor 231. The adder 250 is also called a reconstructor or a recontructed block generator. The aforementioned image segmentation unit 210, prediction unit 220, residual processing unit 230, entropy encoding unit 240, addition unit 250, and filtering unit 260 can be configured by one or more hardware components (e.g., an encoder chipset or processor) depending on the embodiment. Furthermore, the memory 270 may include a DPB (Decoded Picture Buffer) and may be configured by a digital storage medium. The hardware components may also further include the memory 270 as an internal / external component.

[0043] The image splitting unit 210 can split an input image (or picture, frame) input to the encoding device 200 into one or more processing units. For example, the processing unit is also called a coding unit (CU). In this case, the coding unit can be recursively split from a coding tree unit (CTU) or a largeest coding unit (LCU) using a QTBTTT (Quad-Tree Binary-Tree Ternary-Tree) structure. For example, one coding unit can be split into multiple coding units of deeper depth based on a quad-tree structure, a binary-tree structure, and / or a ternary tree. In this case, for example, the quad-tree structure may be applied first, followed by the binary-tree structure and / or the ternary tree. Alternatively, the binary-tree structure may be applied first. The coding procedure described in this document can be performed based on a final coding unit that cannot be further subdivided. In this case, the largest coding unit can be used as the final coding unit based on coding efficiency based on image characteristics, or, if necessary, the coding unit can be recursively subdivided into lower-depth coding units so that the optimally sized coding unit is used as the final coding unit. Here, the coding procedure may include procedures such as prediction, transformation, and reconstruction, which are described later. As another example, the above processing unit may further include a prediction unit (PU) or a transformation unit (TU). In this case, the prediction unit and the transformation unit can each be subdivided or partitioned from the final coding unit described above.The above prediction unit is the unit of sample prediction, or the above conversion unit is the unit for deriving (inducing) conversion coefficients and / or the unit for deriving residual signals from conversion coefficients.

[0044] The term "unit" can sometimes be confused with terms such as "block" or "area." Generally, an M×N block can represent a set of samples or transform coefficients consisting of M columns and N rows. A sample can generally represent a pixel or a pixel value, or it can represent only the pixel / pixel value of the lumen component, or only the pixel / pixel value of the chroma component. A sample can be used as a term corresponding to one picture (or image) for a pixel or a pel.

[0045] The encoding device 200 can generate a residual signal (residual block, residual sample array) by subtracting the predicted signal (predicted block, predicted sample array) output from the inter-prediction unit 221 or intra-prediction unit 222 from the input image signal (original block, original sample array), and the generated residual signal is transmitted to the conversion unit 232. In this case, as shown in the figure, the unit that subtracts the predicted signal (predicted block, predicted sample array) from the input image signal (original block, original sample array) within the encoder 200 is also called the subtraction unit 231. The prediction unit can perform predictions on the block to be processed (hereinafter referred to as the current block) and generate a predicted block that includes predicted samples for the current block. The prediction unit can determine whether intra-prediction or inter-prediction is applied on a current block or CU basis. The prediction unit can generate various prediction-related information, such as prediction mode information, and transmit it to the entropy encoding unit 240, as will be described later in the explanation of each prediction mode. The prediction information can be encoded by the entropy encoding unit 240 and output in bitstream format.

[0046] The intra-prediction unit 222 can predict the current block by referring to a sample in the current picture. The referenced sample can be located adjacent to or far from the current block, depending on the prediction mode. In intra-prediction, the prediction mode can include multiple non-directional modes and multiple directional modes. Non-directional modes can include, for example, DC mode and planar mode. Directional modes can include, for example, 33 or 65 directional prediction modes, depending on the degree of fineness of the prediction direction. However, this is merely an example, and more or fewer directional prediction modes can be used depending on the settings. The intra-prediction unit 222 can also use the prediction mode applied to the adjacent block to determine the prediction mode to be applied to the current block.

[0047] The interprediction unit 221 can derive predicted blocks relative to the current block based on reference blocks (reference sample arrays) identified by motion vectors on the reference picture. In this case, in order to reduce the amount of motion information transmitted in interprediction mode, motion information can be predicted in units of blocks, subblocks, or samples based on the correlation of motion information between adjacent blocks and the current block. The motion information may include motion vectors and reference picture indices. The motion information may further include interprediction direction information (L0 prediction, L1 prediction, Bi prediction, etc.). In the case of interprediction, adjacent blocks may include spatial neighboring blocks that exist in the current picture and temporal neighboring blocks that exist in the reference picture. The reference picture containing the above reference blocks and the reference picture containing the above temporal neighboring blocks may be the same or different. The above temporal neighboring blocks may also be called collocated reference blocks or collocated CUs (colCUs), and the reference picture containing the above temporal neighboring blocks is also called a collocated picture (colPic). For example, the interpretation unit 221 can construct a motion information candidate list based on adjacent blocks and generate information indicating which candidate is used to derive the motion vector and / or reference picture index of the current block. Interpretation can be performed based on various prediction modes; for example, in skip mode and merge mode, the interpretation unit 221 can use the motion information of adjacent blocks as the motion information of the current block. In skip mode, unlike merge mode, a residual signal is not transmitted.In Motion Vector Prediction (MVP) mode, the motion vector of an adjacent block is used as a motion vector predictor, and the motion vector difference is signaled to indicate the motion vector of the current block.

[0048] The prediction unit 220 can generate prediction signals based on various prediction methods described later. For example, the prediction unit can apply intra-prediction or inter-prediction for predictions on a single block, and can also apply intra-prediction and inter-prediction simultaneously. This is also called Combined Inter and Intra Prediction (CIIP). The prediction unit may also be based on an intra-block copy (IBC) prediction mode or a palette mode for predictions on blocks. The above IBC prediction mode or palette mode can be used for content image / video coding such as games, for example, in SCC (Screen Content Coding). IBC basically performs predictions within the current picture, but can be performed in a manner similar to inter-prediction in that it derives reference blocks within the current picture. That is, IBC can utilize at least one of the inter-prediction techniques described in this document. Palette mode can also be considered an example of intra-coding or intra-prediction. When palette mode is applied, sample values ​​within the picture can be signaled based on information about the palette table and palette index.

[0049] The prediction signal generated via the above prediction unit (including the inter-prediction unit 221 and / or the intra-prediction unit 222) can be used to generate a reconstructed signal or a residual signal. The transformation unit 232 can generate transformation coefficients by applying a transformation technique to the residual signal. For example, the transformation technique may include at least one of DCT (Discrete Cosine Transform), DST (Discrete Sine Transform), GBT (Graph-based Transform), or CNT (Conditionally Non-linear Transform). Here, GBT refers to a transformation obtained from a graph, where the relationship information between pixels is represented by a graph. CNT refers to a transformation obtained by generating a prediction signal using all previously reconstructed pixels and based on that. The transformation process can also be applied to pixel blocks of the same size that are square, or to blocks of variable size that are not square.

[0050] The quantization unit 233 quantizes the conversion coefficients and transmits them to the entropy encoding unit 240, which can encode the quantized signal (information about the quantized conversion coefficients) and output it as a bitstream. The information about the quantized conversion coefficients is also called residual information. The quantization unit 233 can rearrange the block-type quantized conversion coefficients into a one-dimensional vector form based on the coefficient scan order, and can also generate information about the quantized conversion coefficients based on the one-dimensional vector form of the quantized conversion coefficients. The entropy encoding unit 240 can perform various encoding methods, such as exponential Golomb, CAVLC (Context-Adaptive Variable Length Coding), and CABAC (Context-Adaptive Binary Arithmetic Coding). In addition to the quantized conversion coefficients, the entropy encoding unit 240 can also encode information necessary for video / image restoration (e.g., the values ​​of syntax elements) together with or separately from the quantized conversion coefficients. Encoded information (e.g., encoded video / image information) can be transmitted or stored in bitstream form in units of Network Abstraction Layer (NAL) units. The video / image information may further include information about various parameter sets, such as the Adaptation Parameter Set (APS), Picture Parameter Set (PPS), Sequence Parameter Set (SPS), or Video Parameter Set (VPS). The video / image information may also further include general constraint information. In this document, information and / or syntax elements transmitted / signaled from the encoding device to the decoding device may be included in the video / image information. The video / image information may be encoded via the encoding procedure described above and included in the bitstream.The bitstream described above can be transmitted over a network or stored on a digital storage medium. Here, the network may include broadcast networks and / or communication networks, and the digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. A transmitting unit (not shown) that transmits the signal output from the entropy encoding unit 240 and / or a storage unit (not shown) that stores the signal can be configured as an internal / external element of the encoding device 200, or the transmitting unit may be included in the entropy encoding unit 240.

[0051] The quantized conversion coefficients output from the quantization unit 233 can be used to generate a prediction signal. For example, the residual signal (residual block or residual sample) can be reconstructed by applying inverse quantization and inverse transformation to the quantized conversion coefficients via the inverse quantization unit 234 and the inverse transformation unit 235. The adder unit 250 can generate a reconstructed signal (reconstructed picture, reconstructed block, reconstructed sample array) by adding the prediction signal output from the inter-prediction unit 221 or the intra-prediction unit 222 to the reconstructed residual signal. If there is no residual for the block to be processed, such as when skip mode is applied, the predicted block can be used as the reconstructed block. The adder unit 250 is also called the reconstruction unit or reconstructed block generation unit. The generated reconstructed signal can be used for intra-prediction of the next block to be processed in the current picture, or, as described later, it can be filtered and used for inter-prediction of the next picture.

[0052] On the other hand, LMCS (Luma Mapping with Chroma Scaling) can also be applied during the picture encoding and / or restoration process.

[0053] The filtering unit 260 can improve subjective / objective image quality by applying filtering to the restored signal. For example, the filtering unit 260 can apply various filtering methods to the restored picture to generate a modified restored picture, and the modified restored picture can be stored in the memory 270, specifically in the DPB of the memory 270. The various filtering methods can include, for example, deblocking filtering, sample adaptive offset, adaptive loop filter, and bilateral filter. The filtering unit 260 can generate various filtering information and transmit it to the entropy encoding unit 240, as will be described later in the explanation of each filtering method. The filtering information can be encoded by the entropy encoding unit 240 and output in bitstream format.

[0054] The corrected restored picture sent to memory 270 can be used as a reference picture in the interpretation unit 221. When interpretation is applied via this, the encoding device can avoid prediction mismatches in the encoding device 200 and the decoding device, and can also improve encoding efficiency.

[0055] Memory 270DPB can store the corrected restored picture for use as a reference picture in the inter-prediction unit 221. Memory 270 can store motion information of blocks from which motion information in the current picture has been derived (or encoded) and / or motion information of blocks in already restored pictures. The stored motion information can be transmitted to the inter-prediction unit 221 for use as motion information of spatially adjacent blocks or motion information of temporally adjacent blocks. Memory 270 can store restored samples of restored blocks in the current picture and transmit them to the intra-prediction unit 222.

[0056] Figure 3 is a schematic diagram illustrating the configuration of a video / image decoding device to which the embodiments described in this document may be applied.

[0057] As shown in Figure 3, the decoding device 300 may be configured to include an entropy decoder 310, a residual processor 320, a predictor 330, an adder 340, a filter 350, and a memory 360. The predictor 330 may include an inter-predictor 331 and an intra-predictor 332. The residual processor 320 may include a dequantizer 321 and an inverse transformer 321. The aforementioned entropy decoder 310, residual processor 320, predictor 330, adder 340, and filtering device 350 may be configured by a single hardware component (e.g., a decoder chipset or processor) depending on the embodiment. The memory 360 may also include a Decoded Picture Buffer (DPB) and may be configured by a digital storage medium. The above hardware components may also include Memory 360 as an internal / external component.

[0058] When a bitstream containing video / image information is input, the decoding device 300 can reconstruct the image corresponding to the process by which the video / image information was processed in the encoding device shown in Figure 2. For example, the decoding device 300 can derive units / blocks based on block division-related information obtained from the bitstream. The decoding device 300 can perform decoding using the processing units applied in the encoding device. Thus, the decoding processing unit is, for example, a coding unit, which can be divided from a coding tree unit or a maximum coding unit according to a quadtree structure, a binary tree structure, and / or a ternary tree structure. One or more transformation units can be derived from the coding unit. The reconstructed image signal decoded and output via the decoding device 300 can then be reproduced via a playback device.

[0059] The decoding device 300 can receive the signal output from the encoding device shown in Figure 2 in bitstream form, and the received signal can be decoded via the entropy decoding unit 310. For example, the entropy decoding unit 310 can parse the bitstream to derive information necessary for image restoration (or picture restoration) (e.g., video / image information). The video / image information may further include information about various parameter sets, such as the adaptation parameter set (APS), picture parameter set (PPS), sequence parameter set (SPS), or video parameter set (VPS). The video / image information may also further include general constraint information. The decoding device can decode the picture based on the parameter set information and / or the general constraint information. The signaling / received information and / or syntax elements described later in this document can be decoded via the decoding procedure and obtained from the bitstream. For example, the entropy decoding unit 310 can decode information in the bitstream based on a coding method such as exponential Golomb coding, CAVLC, or CABAC, and output the values ​​of the syntax elements necessary for image reconstruction and the quantized values ​​of the conversion coefficients for the residuals. More specifically, the CABAC entropy decoding method receives a BIN corresponding to each syntax element in the bitstream, determines a context model using the syntax element information to be decoded and the decoded information of adjacent and decoded blocks or the symbol / BIN information decoded in a previous step, predicts the probability of BIN occurrence based on the determined context model, and performs arithmetic decoding of the BIN to generate a symbol corresponding to the value of each syntax element.In this case, the CABAC entropi decoding method can update the context model after determining the context model by utilizing the decoded symbol / BIN information for the context model of the next symbol / BIN. Information related to prediction from the information decoded by the entropi decoding unit 310 is provided to the prediction unit (inter-prediction unit 332 and intra-prediction unit 331), and residual values ​​from entropi decoding performed by the entropi decoding unit 310, i.e., quantized conversion coefficients and related parameter information, can be input to the residual processing unit 320. The residual processing unit 320 can derive residual signals (residual blocks, residual samples, residual sample arrays). In addition, information related to filtering from the information decoded by the entropi decoding unit 310 can be provided to the filtering unit 350. On the other hand, a receiving unit (not shown) that receives the signal output from the encoding device can be further configured as an internal / external element of the decoding device 300, or the receiving unit may be a component of the entropi decoding unit 310. On the other hand, the decoding device described in this document is also called a video / image / picture decoding device, and the decoding device can also be divided into an information decoder (video / image / picture information decoder) and a sample decoder (video / image / picture sample decoder). The information decoder may include the entropy decoding unit 310, and the sample decoder may include at least one of the inverse quantization unit 321, inverse transformation unit 322, addition unit 340, filtering unit 350, memory 360, inter-prediction unit 332, and intra-prediction unit 331.

[0060] The inverse quantization unit 321 can inverse quantize the quantized transformation coefficients and output the transformation coefficients. The inverse quantization unit 321 can rearrange the quantized transformation coefficients in a two-dimensional block form. In this case, the rearrangement can be performed based on the coefficient scan order performed by the encoding device. The inverse quantization unit 321 can perform inverse quantization on the quantized transformation coefficients using quantization parameters (e.g., quantization step size information) and obtain the transformation coefficients.

[0061] In the inverse transformation unit 322, the transformation coefficients are inversely transformed to obtain the residual signal (residual block, residual sample array).

[0062] The prediction unit can perform a prediction on the current block and generate a predicted block containing prediction samples for the current block. Based on the prediction information output from the entropy decode unit 310, the prediction unit can determine whether intra-prediction or inter-prediction is applied to the current block and can determine a specific intra / inter-prediction mode.

[0063] The prediction unit 330 can generate prediction signals based on various prediction methods described later. For example, the prediction unit can apply intra-prediction or inter-prediction for predictions on a single block, and can also apply intra-prediction and inter-prediction simultaneously. This is also called Combined Inter and intra Prediction (CIIP). The prediction unit may also be based on an intra-block copy (IBC) prediction mode or a palette mode for predictions on blocks. The above IBC prediction mode or palette mode can be used for content image / video coding such as games, for example, in SCC (Screen Content Coding). IBC basically performs predictions within the current picture, but can be performed in a manner similar to inter-prediction in that it derives reference blocks within the current picture. That is, IBC can utilize at least one of the inter-prediction techniques described in this document. Palette mode can also be considered an example of intra-coding or intra-prediction. When palette mode is applied, information regarding the palette table and palette index can be included in the video / image information and signaled.

[0064] The intra-prediction unit 331 can predict the current block by referring to a sample within the current picture. The referenced sample may be located adjacent to or far from the current block, depending on the prediction mode. In intra-prediction, the prediction mode may include multiple non-directional modes and multiple directional modes. The intra-prediction unit 331 can also determine the prediction mode to be applied to the current block by utilizing the prediction modes applied to adjacent blocks.

[0065] The interprediction unit 332 can derive a predicted block relative to the current block based on a reference block (reference sample array) identified by motion vectors on the reference picture. In this case, in order to reduce the amount of motion information transmitted in interprediction mode, motion information can be predicted in units of blocks, subblocks, or samples based on the correlation of motion information between adjacent blocks and the current block. The motion information may include motion vectors and reference picture indices. The motion information may further include interprediction direction information (L0 prediction, L1 prediction, Bi prediction, etc.). In the case of interprediction, adjacent blocks may include spatial neighboring blocks present in the current picture and temporal neighboring blocks present in the reference picture. For example, the interprediction unit 332 can construct a motion information candidate list based on adjacent blocks and derive the motion vector and / or reference picture index of the current block based on the received candidate selection information. Interprediction can be performed based on various prediction modes, and the prediction information may include information indicating the mode of interprediction for the current block.

[0066] The summing unit 340 can generate a reconstructed signal (reconstructed picture, reconstructed block, reconstructed sample array) by adding the predicted signal (predicted block, predicted sample array) output from the prediction unit (including the inter-prediction unit 332 and / or intra-prediction unit 331) to the acquired residual signal. If there is no residual for the block to be processed, such as when skip mode is applied, the predicted block can be used as the reconstructed block.

[0067] The addition unit 340 is also called the restoration unit or restoration block generation unit. The generated restoration signal can be used for intra-prediction of the next block to be processed in the current picture, or it can be output after filtering as described later, or it can be used for intra-prediction of the next picture.

[0068] On the other hand, LMCS (Luma Mapping with Chroma Scaling) can also be applied during the picture decoding process.

[0069] The filtering unit 350 can improve subjective / objective image quality by applying filtering to the restored signal. For example, the filtering unit 350 can apply various filtering methods to the restored picture to generate a modified restored picture, and can transmit the modified restored picture to the memory 360, specifically to the DPB of the memory 360. The various filtering methods can include, for example, deblocking filtering, sample adaptive offset, adaptive loop filter, and bilateral filter.

[0070] The restored picture stored (modified) in the DPB of memory 360 can be used as a reference picture by the inter-prediction unit 332. Memory 360 can store motion information of blocks from which motion information in the current picture has been derived (or decoded) and / or motion information of blocks in already restored pictures. The stored motion information can be transmitted to the inter-prediction unit 332 for use as motion information of spatially adjacent blocks or motion information of temporally adjacent blocks. Memory 360 can store restored samples of restored blocks in the current picture and transmit them to the intra-prediction unit 331.

[0071] In this specification, the embodiments described for the filtering unit 260, inter-prediction unit 221, and intra-prediction unit 222 of the encoding device 200 can be applied identically or correspondingly to the filtering unit 350, inter-prediction unit 332, and intra-prediction unit 331 of the decoding device 300, respectively.

[0072] As mentioned above, prediction is performed to improve compression efficiency when performing video coding. This allows for the generation of a predicted block containing predicted samples for the current block, which is the block to be coded. Here, the predicted block contains predicted samples in the spatial domain (or pixel domain). The predicted block is derived identically by the encoding and decoding devices, and the encoding device can improve image coding efficiency by signaling the decoding device information about the residuals between the original block and the predicted block (residual information), which is not the original sample value of the original block itself. The decoding device can derive a residual block containing residual samples based on the residual information, and can generate a restored block containing restored samples by combining the residual block and the predicted block, thereby generating a restored picture containing the restored block.

[0073] The residual information described above can be generated through transformation and quantization procedures. For example, an encoding device can signal the relevant residual information to a decoding device (via a bitstream) by deriving a residual block between the original block and the predicted block, performing a transformation procedure on the residual samples (residual sample array) contained in the residual block to derive transformation coefficients, and then performing a quantization procedure on the transformation coefficients to derive quantized transformation coefficients. Here, the residual information may include information such as the value information, position information, transformation technique, transformation kernel, and quantization parameters of the quantized transformation coefficients. Based on the residual information, the decoding device can perform an inverse quantization / inverse transformation procedure to derive residual samples (or residual blocks). The decoding device can generate a reconstructed picture based on the predicted block and the residual block. The encoding device can also derive a residual block by inverse quantization / inverse transformation of the quantized transformation coefficients for reference for subsequent interpretation of the picture, and generate a reconstructed picture based on this.

[0074] When inter-prediction is applied, the prediction unit of the encoding / decoding device can perform inter-prediction on a block-by-block basis to derive predicted samples. Inter-prediction can be a prediction derived in a manner that is dependent on data elements (e.g., sample values ​​or motion information) of picture(s) other than the current picture. When inter-prediction is applied to the current block, a predicted block (predicted sample array) for the current block can be derived based on the reference block (reference sample array) identified by the motion vector on the reference picture pointed to by the reference picture index. In this case, in order to reduce the amount of motion information transmitted in inter-prediction mode, the motion information of the current block can be predicted on a block, subblock, or sample basis based on the correlation of motion information between adjacent blocks and the current block. The motion information may include motion vectors and reference picture indexes. The motion information may further include inter-prediction type information (e.g., L0 prediction, L1 prediction, Bi prediction). When inter-prediction is applied, adjacent blocks can include spatial neighboring blocks that exist within the current picture and temporal neighboring blocks that exist in the reference picture. The reference picture containing the above reference block and the reference picture containing the above temporal neighboring block may be the same or different. The above temporal neighboring block is sometimes called a collocated reference block or collocated CU (colCU), and the reference picture containing the above temporal neighboring block is also called a collocated picture (colPic).For example, a list of motion information candidates can be constructed based on the adjacent blocks of the current block, and flags or index information can be signaled indicating which candidate is selected (used) to derive the motion vector and / or reference picture index of the current block. Interpretation can be performed based on various prediction modes; for example, in skip mode and (normal) merge mode, the motion information of the current block is the same as the motion information of the selected adjacent block. In skip mode, unlike merge mode, a residual signal is not transmitted. In Motion Vector Prediction (MVP) mode, the motion vector of the selected adjacent block can be used as a motion vector predictor, and a motion vector difference can be signaled. In this case, the motion vector of the current block can be derived using the sum of the motion vector predictor and the motion vector difference.

[0075] A video / image encoding procedure based on interpretation may, in general, include the following, for example:

[0076] Figure 4 shows an example of an interpretation-based video / image encoding method.

[0077] The encoding device performs interpretation for the current block (S400). The encoding device can derive the interpretation mode and motion information of the current block and generate prediction samples for the current block. Here, the interpretation mode determination, motion information derivation, and prediction sample generation procedures may be performed simultaneously, or one procedure may be performed before the others. For example, the interpretation unit of the encoding device may include a prediction mode determination unit, a motion information derivation unit, and a prediction sample derivation unit, where the prediction mode determination unit determines the prediction mode for the current block, the motion information derivation unit derives the motion information of the current block, and the prediction sample derivation unit derives prediction samples for the current block. For example, the interpretation unit of the encoding device can search for blocks similar to the current block within a certain area (search area) of the reference picture via motion estimation and derive reference blocks whose difference from the current block is the minimum or below a certain standard. Based on this, a reference picture index pointing to the reference picture in which the reference block is located can be derived, and a motion vector can be derived based on the positional difference between the reference block and the current block. The encoding device can determine which mode to apply to the current block from among various prediction modes. The encoding device can determine the optimal prediction mode for the current block by comparing the RD costs for the various prediction modes.

[0078] For example, when skip mode or merge mode is applied to the current block, the encoding device can configure a merge candidate list, as described later, and derive a reference block from among the reference blocks pointed to by the merge candidates included in the merge candidate list whose difference from the current block is the minimum or below a certain standard. In this case, a merge candidate associated with the derived reference block is selected, and merge index information pointing to the selected merge candidate is generated and signaled to the decoding device. The movement information of the current block can be derived using the movement information of the selected merge candidate.

[0079] As another example, when the (A)MVP mode is applied to the current block, the encoding device can configure the (A)MVP candidate list described later, and use the motion vector of the mvp (motion vector predictor) candidate selected from the mvp candidates included in the (A)MVP candidate list as the mvp of the current block. In this case, for example, the motion vector pointing to the reference block derived by the motion estimation described above can be used as the motion vector of the current block, and the mvp candidate with the smallest difference from the motion vector of the current block among the mvp candidates becomes the selected mvp candidate. The MVD (Motion Vector Difference), which is the difference obtained by subtracting the mvp from the motion vector of the current block, can be derived. In this case, information regarding the MVD can be signaled to the decoding device. Also, when the (A)MVP mode is applied, the value of the reference picture index can be configured with reference picture index information and separately signaled to the decoding device.

[0080] The encoding device can derive the residual sample based on the predicted sample (S410). The encoding device can derive the residual sample by comparing the original sample of the current block with the predicted sample.

[0081] The encoding device encodes image information including prediction information and residual information (S420). The encoding device can output the encoded image information in bitstream format. The prediction information is information related to the prediction procedure and may include prediction mode information (e.g., skip flag, merge flag, or mode index) and motion information. The motion information may include candidate selection information (e.g., merge index, mvp flag, or mvp index) which is information for deriving a motion vector. The motion information may also include the aforementioned MVD information and / or reference picture index information. Furthermore, the motion information may include information indicating whether L0 prediction, L1 prediction, or bi(paired) prediction is applied. The residual information is information about the residual sample. The residual information may include information about the quantized transformation coefficients for the residual sample.

[0082] The output bitstream can be stored in a (digital) storage medium and transmitted to a decoding device, or it can be transmitted to a decoding device via a network.

[0083] On the other hand, as mentioned above, the encoding device can generate a reconstructed picture (including the reconstructed sample and reconstructed block) based on the reference sample and the residual sample. This is so that the encoding device can derive the same prediction results as the decoding device, thereby increasing coding efficiency. Therefore, the encoding device can store the reconstructed picture (or reconstructed sample, reconstructed block) in memory and use it as a reference picture for interpretation. As mentioned above, in-loop filtering procedures and the like can be further applied to the reconstructed picture.

[0084] A video / image decoding procedure based on interpretation may, in general, include the following:

[0085] Figure 5 shows an example of an interpretation-based video / image decoding method.

[0086] As shown in Figure 5, the decoding device can perform operations corresponding to those performed by the encoding device. The decoding device can perform predictions on the current block based on the received prediction information and derive prediction samples.

[0087] Specifically, the decoding device can determine the prediction mode for the current block based on the received prediction information (S500). Based on the prediction mode information in the prediction information, the decoding device can determine which inter-prediction mode is applied to the current block.

[0088] For example, based on the merge flag, it can be determined whether the merge mode is applied to the current block or whether the (A)MVP mode is determined. Alternatively, one can be selected from a variety of inter-prediction mode candidates based on the mode index. The inter-prediction mode candidates may include skip mode, merge mode and / or (A)MVP mode, or may include the various inter-prediction modes described later.

[0089] The decoding device derives motion information for the current block based on the inter prediction mode determined above (S510). For example, if a skip mode or merge mode is applied to the current block, the decoding device can configure a merge candidate list, as described later, and select one merge candidate from among the merge candidates included in the merge candidate list. This selection can be performed based on the selection information (merge index) described above. The motion information for the current block can be derived using the motion information for the selected merge candidate. The motion information for the selected merge candidate can be used as the motion information for the current block.

[0090] As another example, when the (A)MVP mode is applied to the current block, the decoding device can configure the (A)MVP candidate list described later, and use the motion vector of the selected mvp candidate from among the mvp (motion vector predictor) candidates included in the (A)MVP candidate list as the mvp of the current block. The above selection can be performed based on the selection information (mvp flag or mvp index) described above. In this case, the MVD of the current block can be derived based on the information about the MVD, and the motion vector of the current block can be derived based on the mvp of the current block and the MVD. In addition, the reference picture index of the current block can be derived based on the reference picture index information. The picture pointed to by the reference picture index in the reference picture list for the current block can be derived as the reference picture referenced for inter prediction of the current block.

[0091] On the other hand, as will be described later, the movement information of the current block can be derived without constructing a candidate list, in which case the movement information of the current block can be derived by the procedure disclosed in the prediction mode described later. In this case, the candidate list construction described above can be omitted.

[0092] The decoding device can generate predicted samples for the current block based on the motion information of the current block (S520). In this case, the reference picture can be derived based on the reference picture index of the current block, and the predicted samples for the current block can be derived using the sample of the reference block pointed to by the motion vector of the current block on the reference picture. In this case, as will be described later, a prediction sample filtering procedure may be further performed on all or some of the predicted samples for the current block.

[0093] For example, the interpretation unit of the decoding device may include a prediction mode determination unit, a motion information derivation unit, and a prediction sample derivation unit. The prediction mode determination unit determines the prediction mode for the current block based on the prediction mode information received, the motion information derivation unit derives motion information (such as motion vectors and / or reference picture indices) for the current block based on the motion information received, and the prediction sample derivation unit derives prediction samples for the current block.

[0094] The decoding device generates a residual sample for the current block based on the received residual information (S530). The decoding device generates a reconstructed sample for the current block based on the predicted sample and the residual sample, and can generate a reconstructed picture based on this (S540). As mentioned above, further procedures such as in-loop filtering can be applied to the reconstructed picture thereafter.

[0095] Figure 6 illustrates the interpretation prediction procedure.

[0096] As shown in Figure 6, the interpretation procedure, as described above, may include an interpretation mode determination step, a motion information derivation step based on the determined prediction mode, and a prediction execution (prediction sample generation) step based on the derived motion information. The above interpretation procedure can be performed in an encoding device and a decoding device, as described above. In this document, the coding device may include an encoding device and / or a decoding device.

[0097] As shown in Figure 6, the coding device determines the inter-prediction mode for the current block (S600). A variety of inter-prediction modes can be used to predict the current block in the picture. For example, various modes such as merge mode, skip mode, MVP (Motion Vector Prediction) mode, affine mode, sub-block merge mode, and MMVD (Merge with MVD) mode can be used. DMVR (Decoder side Motion Vector Refinement) mode, AMVR (Adaptive Motion Vector Resolution) mode, Bi-prediction with CU-level weight (BCW), and Bi-Directional Optical Flow (BDOF) can be used as supplementary modes or as alternatives. The affine mode is also called affine motion prediction mode. The MVP mode is also called AMVP (Advanced Motion Vector Prediction) mode. In this document, motion information candidates derived by some modes and / or some modes may also be included as one of the motion information related candidates of other modes. For example, an HMVP candidate can be added as a merge candidate in the merge / skip mode described above, or as an MVP candidate in the MVP mode described above. When the HMVP candidate is used as a motion information candidate in the merge mode or skip mode described above, the HMVP candidate is also called an HMVP merge candidate.

[0098] Prediction mode information, which points to the inter-prediction mode of the current block, can be signaled from the encoding device to the decoding device. The prediction mode information can be included in the bitstream and received by the decoding device. The prediction mode information may include index information indicating one of several candidate modes. Alternatively, the inter-prediction mode can be indicated via hierarchical signaling of flag information. In this case, the prediction mode information may include one or more flags. For example, a skip flag may be signaled to indicate whether a skip mode can be applied, a merge flag may be signaled if a skip mode is not applied to indicate whether a merge mode can be applied, an MVP mode may be applied if a merge mode is not applied, or further flags for additional distinctions may be signaled. Affine modes may be signaled as independent modes or as modes dependent on merge modes or MVP modes, etc. For example, affine modes may include affine merge mode and affine MVP mode.

[0099] The coding device derives motion information for the current block (S610). The motion information can be derived based on the inter-prediction mode.

[0100] The coding device can perform interpretation using motion information of the current block. The encoding device can derive optimal motion information for the current block through a motion estimation procedure. For example, the encoding device can use the original block in the original picture for the current block to search for highly correlated similar reference blocks in fractional pixel units within a defined search range in the reference picture, thereby deriving motion information. Block similarity can be derived based on the difference in phase-based sample values. For example, block similarity can be calculated based on the SAD between the current block (or the template of the current block) and the reference block (or the template of the reference block). In this case, motion information can be derived based on the reference block with the smallest SAD within the search area. The derived motion information can be signaled to the decoding device in various ways based on the interpretation mode.

[0101] The coding device performs interpretation based on motion information for the current block (S620). The coding device can derive one or more predicted samples for the current block based on the motion information. The current block containing the predicted samples is also called the predicted block.

[0102] On the other hand, in interpretation, conventional merge or AMVP modes employ methods to reduce the amount of motion information by using the motion vectors of spatially / temporally adjacent blocks of the current block as motion information candidates. For example, the adjacent blocks used to derive motion information candidates for the current block could include the block adjacent to the lower left corner of the current block, the block adjacent to the left side, the block adjacent to the upper right corner, the block adjacent to the top side, and the block adjacent to the upper left corner.

[0103] Figure 7 illustrates the spatially adjacent blocks used for deriving motion information candidates in conventional merge or AMVP modes.

[0104] Basically, the spatially adjacent blocks described above were limited to blocks adjacent to the current block. This was to improve hardware feasibility, as deriving information from blocks far from the current block would cause problems such as increased line buffer size. However, using motion information from non-adjacent blocks to derive motion information candidates for the current block allows for the construction of a wider variety of candidates, thus improving performance. The HMVP (History-based Motion Vector Prediction) method can be used to use motion information from non-adjacent blocks without increasing the line buffer size. In this document, HMVP can refer to History-based Motion Vector Prediction or History-based Motion Vector Predictor. According to this document, inter-prediction can be efficiently performed using HMVP, and parallel processing can be supported. For example, the embodiments in this document propose various methods for managing the history buffer for parallel processing, and parallel processing can be supported based on these methods. However, supporting parallel processing does not mean that parallel processing must be performed; the coding device may or may not perform parallel processing depending on hardware performance and service configuration. For example, if the coding device has a multi-core processor, it can process some of the slices, bricks, and / or tiles in parallel. On the other hand, if the coding device has a single-core processor or a multi-core processor, it can also perform sequential processing while reducing the computational and memory burden.

[0105] HMVP candidates using the aforementioned HMVP method may include motion information for previously coded blocks. For example, motion information for previously coded blocks based on the block coding order within the current picture was not considered as motion information for the current block if the previously coded blocks were not adjacent to the current block. However, HMVP candidates can be considered as motion information candidates for the current block (e.g., merge candidates or MVP candidates) without considering whether the previously coded blocks are adjacent to the current block. In this case, multiple HMVP candidates can be stored in a buffer. For example, if a merge mode is applied to the current block, an HMVP candidate (HMVP merge candidate) can be added to the merge candidate list. In this case, the HMVP candidate can be added after the spatial merge candidates and temporal merge candidates included in the merge candidate list.

[0106] According to the HMVP method, motion information of previously coded blocks can be stored in a table format and used as a candidate for motion information of the current block (e.g., a merge candidate). A table (or buffer, list) containing multiple HMVP candidates can be maintained during the encoding / decoding procedure. The above table (or buffer, list) is also called the HMVP table (or buffer, list). According to one embodiment of this document, the above table (or buffer, list) can be initialized when encountering a new slice. Alternatively, according to one embodiment of this document, the above table (or buffer, list) can be initialized when encountering a new CTU row. When the above table is initialized, the number of HMVP candidates contained in the above table can be set to 0. The size of the above table (or buffer, list) can be fixed to a specific value (e.g., 5). For example, if there is an interconnected block, the relevant motion information can be added as a new HMVP candidate in the last entry of the above table. The above (HMVP) table is also called an (HMVP) buffer or (HMVP) list.

[0107] Figure 8 schematically shows an example of an HMVP candidate-based decoding procedure, which includes an HMVP candidate-based interpretation procedure.

[0108] As shown in Figure 8, the decoding device loads an HMVP table containing one or more HMVP candidates and decodes a block based on at least one of the HMVP candidates. Specifically, for example, the decoding device can derive motion information for the current block based on at least one of the HMVP candidates, perform interpretation on the current block based on the motion information, and derive a predicted block (including predicted samples). As previously mentioned, a restored block can be generated based on the predicted block. The derived motion information for the current block can be updated in the table. In this case, the motion information can be added as a new HMVP candidate as the last entry in the table. If the number of HMVP candidates already included in the table is equal to the size of the table, the first candidate to be added to the table is deleted, and the derived motion information can be added as a new HMVP candidate as the last entry in the table.

[0109] Figure 9 illustrates an HMVP table update using the FIFO rule, and Figure 10 illustrates an HMVP table update using a restricted FIFO rule.

[0110] The FIFO (First-In-First-Out) rule can be applied to the table above. For example, if the table size S is 16, this indicates that 16 HMVP candidates can be included in the table. If more than 16 HMVP candidates arise from previously coded blocks, the FIFO rule can be applied so that the table can include up to 16 most recently coded motion information candidates. In this case, as shown in Figure 9 above, the FIFO rule can be applied to remove the oldest HMVP candidate and add a new HMVP candidate.

[0111] On the other hand, to further improve coding efficiency, a restricted FIFO rule can also be applied, as shown in Figure 10. As shown in Figure 10, when inserting an HMVP candidate into the table, a redundancy check can be applied first. This allows determining whether an HMVP candidate with the same motion information already exists in the table. If an HMVP candidate with the same motion information exists in the table, that HMVP candidate with the same motion information is removed from the table, and subsequent HMVP candidates move one space (i.e., each index - 1) before a new HMVP candidate can be inserted.

[0112] As mentioned above, HMVP candidates can be used in the merge candidate list construction procedure. In this case, for example, all HMVP candidates that can be inserted from the last entry to the first entry in the table above can be inserted after the spatial and temporal merge candidates. In this case, pruning checks can be applied to the HMVP candidates. The maximum number of allowed merge candidates can be signaled, and the merge candidate list construction procedure can be terminated when the total number of available merge candidates reaches the maximum number of merge candidates.

[0113] Similarly, HMVP candidates can also be used in (A) the MVP candidate list construction procedure. In this case, the motion vectors of the last k HMVP candidates in the HMVP table can be added after the TMVP candidates that make up the MVP candidate list. In this case, for example, an HMVP candidate having the same reference picture as the MVP target reference picture can be used for the above MVP candidate list construction. Here, the MVP target reference picture can indicate the reference picture for inter prediction of the current block to which the above MVP mode is applied. In this case, a pruning check can be applied to the HMVP candidate. The above k is, for example, 4. However, this is merely an example, and the above k can have a variety of values ​​such as 1, 2, 3, 4, etc.

[0114] On the other hand, if the total number of merge candidates is equal to or greater than 15, the truncated unary plus fixed length (with 3 bits) binary method can be applied for merge index coding, as shown in Table 1 below.

[0115] [Table 1]

[0116] The table above assumes that Nmrg = 15, where Nmrg represents the total number of merge candidates.

[0117] On the other hand, when developing solutions that utilize video codecs, parallel processing can be supported in image / video coding to optimize implementation.

[0118] Figure 11 illustrates Wavefront Parallel Processing (WPP), one of the techniques for parallel processing.

[0119] As shown in Figure 11, when WPP is applied, parallel processing can be performed on a CTU row-by-row basis. In this case, when coding (encoding / decoding) the block indicated by X, a dependency (dependency) exists with the position indicated by the arrow. Therefore, it is necessary to wait for the coding of the upper right CTU of the block currently to be coded to be completed. Also, when WPP is applied, the initialization of the CABAC probability table (or context information) can be done on a slice-by-slice basis, and in order to perform parallel processing including entropy encoding / decoding, the CABAC probability table (or context information) must be initialized on a CTU row-by-row basis. WPP can be considered a technique proposed to determine an efficient initialization position. When WPP is applied, each LCT row can be called a substream, and if the coding device has multiple processing cores, parallel processing can be supported. For example, if WPP is applied and three processing cores process decoding in parallel, the first processing core can decode substream 0, the second processing core can decode substream 1, and the third processing core can decode substream 2. When WPP is applied, coding may be performed (advanced) for the nth (where n is an integer) substream, and after coding for the second CTU or LCU of the nth substream is completed, coding for the (n+1)th substream may be performed. For example, in the case of entropy coding, once entropy coding for the second LCU of the nth substream is completed, the first LCU of the (n+1)th substream can be entropy coded based on the context information for the second LCU of the nth substream. In this case, the number of substreams in a slice can be the same as the number of LCU rows. Also, the number of substreams in a slice can be the same as the number of entry points. In this case, the number of entry points can be determined by the number of entry point offsets.For example, the number of entry points can be one greater than the number of entry point offsets. Information regarding the number of entry point offsets and / or the offset values ​​can be included in the video / image information described above and encoded, and signaled to the decoding device via the bitstream. On the other hand, if the coding device has a single processing core, coding can be performed on a substream-by-substream basis, thereby reducing memory load and coding dependency.

[0120] The aforementioned HMVP method stores motion information derived from the coding procedure of each block as candidates, up to the size of a predetermined buffer (HMVP table). In this case, as disclosed in Figure 9 without additional conditions, it is possible to fill the candidates as many as the number of buffers, or to fill the candidates without duplication by checking for duplication between newly added candidates and candidates already in the buffer (HMVP table). This allows for the configuration of a diverse range of candidates. However, when developing a solution that applies a video codec, it is generally impossible to know when the HMVP candidates will fill the buffer, making it impossible to implement parallel processing with or without WPP.

[0121] Figure 12 illustrates the problems that arise when applying a general HMVP method while considering parallel processing.

[0122] As shown in Figure 12, when parallelizing on a per-CTU line basis, as in WPP, dependency issues with HMVP buffers can occur. For example, the HMVP buffer for the first CTU in the N (N>=1)th CTU line will not be filled until the coding (encoding / decoding) of the block in the (N-1)th CTU line, for example, the block in the last CTU of the (N-1)th CTU line, is completed. In other words, when parallel processing is applied under the current structure, the decoder cannot know whether the HMVP candidate stored in the current HMVP buffer matches the HMVP buffer used for decoding the current (target) block. This is because there is a risk of a difference between the HMVP buffer derived at the time of coding the current block when sequential processing is applied and the HMVP buffer derived at the time of coding the current block when parallel processing is applied.

[0123] In one embodiment of this document, in order to solve the above-mentioned problems, parallel processing is supported when applying HMVP by initializing the history management buffer (HMVP buffer).

[0124] Figure 13 illustrates an example of how to initialize a history management buffer (HMVP buffer) according to one embodiment of this document.

[0125] As shown in Figure 13, the HMVP buffer can be initialized for each first CTU in a CTU row. That is, when coding the first CTU in a CTU row, initializing the HMVP buffer ensures that the number of HMVP candidates in the HMVP buffer becomes 0. As described above, initializing the HMVP buffer for each CTU row allows the use of HMVP candidates derived during the coding process of CTUs located to the left of the current block without constraints, even when parallel processing is supported. In this case, for example, if the current CU, which is the current block, is located at the first CTU in a CTU row, and the current CU corresponds to the first CU of the first CTU, then the number of HMVP candidates in the HMVP buffer is 0. Also, for example, if a CU coded before the current CU in the above CTU row is coded in intermode, HMVP candidates can be derived based on the motion information of the previously coded CU and included in the HMVP buffer.

[0126] Figure 14 illustrates an HMVP buffer management method according to one embodiment.

[0127] As shown in Figure 14, the HMVP buffer can be initialized on a slice-by-slice basis, and it is possible to determine whether the CTU to be coded (the current CTU) is the first CTU in each CTU row, even for CTUs within a slice. In Figure 14, as an example, it is written that if (ctu_idx % Num) is 0, it is determined to be the first CTU. In this case, Num represents the number of CTUs in each CTU row. As another example, if the brick concept mentioned above is used, if (ctu_idx_in_brick % BrickWidth) is 0, it can be determined that it is the first CTU in the CTU row (within the relevant brick). Here, ctu_idx_in_brick indicates the index of the relevant CTU within the brick, and BrickWidth represents the width of the relevant brick in CTU units. In other words, BrickWidth can indicate the number of CTU columns within the relevant brick. If the current CTU is the first CTU in the CTU row, the HMVP buffer is initialized (i.e., the number of candidates in the HMVP buffer is set to 0); otherwise, the HMVP buffer is maintained. Subsequently, each CU in the relevant CTU undergoes a prediction process (e.g., merge or MVP mode based), at which point the candidates stored in the HMVP buffer may be included as merge mode or MVP mode motion information candidates (e.g., merge candidates or MVP candidates). The motion information of the target block (current block) derived in the interpretation process based on merge mode or MVP mode is stored (updated) in the HMVP buffer as a new HMVP candidate. In this case, the aforementioned duplicate check process may be performed again. Subsequently, the above procedure can be repeated for CUs and CTUs.

[0128] As another example, when applying HMVP, you can eliminate CTU-level dependencies by initializing the HMVP buffer for each CTU.

[0129] Figure 15 illustrates an HMVP buffer management method according to another embodiment.

[0130] As shown in Figure 15, HMVP buffer initialization can be performed for each CTU without determining whether the current CTU is the first CTU in each CTU row. In this case, since the HMVP buffer is initialized on a per-CTU basis, the movement information of the blocks present in the CTU is stored in the HMVP table. In this case, HMVP candidates can be derived based on the movement information of blocks (e.g., CUs) within the same CTU, and HMVP buffer initialization becomes possible without determining whether the current CTU is the first CTU in each CTU row, as shown below.

[0131] As described above, HMVP buffers can be initialized on a slice-by-slice basis, allowing the use of motion vectors of blocks spatially separated from the current block. However, in this case, parallel processing support is not possible within a slice. Therefore, in the embodiments described above, a method for initializing the buffer on a CTU row or CTU basis was proposed. That is, according to the embodiments in this document, HMVP buffers can be initialized on a slice-by-slice basis, and within a slice, they can be initialized on a CTU row basis.

[0132] On the other hand, when coding (encoding / decoding) a single picture, the picture can be divided into slices and / or tiles. For example, the picture can be divided into slices to consider error resilience, or it can be divided into tiles to encode / decode a portion of the picture. When a single picture is divided into multiple tiles, applying an HMVP managed buffer is not suitable for a tiled structure used to encode / decode a portion of a picture, as initialization is performed on a per-CTU row basis within the picture, i.e., initializing the HMVP buffer at the first CTU of each CTU row within the picture.

[0133] Figure 16 illustrates an example of how to initialize an HMVP buffer in a tile structure.

[0134] As shown in Figure 16, in the case of tile 1 and tile 3, the HMVP management buffer is not initialized for each tile, so an HMVP dependency arises with tile 0 and tile 2, respectively. Therefore, when tiles exist, it is possible to initialize the HMVP buffer in the following way.

[0135] As an example, HMVP buffers can be initialized on a CTU basis. This can, of course, be applied without distinguishing between tiles, slices, etc.

[0136] As another example, the HMVP buffer can be initialized targeting the first CTU of each tile.

[0137] Figure 17 shows an example of an HMVP buffer initialization method for the first CTU of a tile according to another embodiment.

[0138] As shown in Figure 17, the HMVP buffer is initialized when coding the first CTU of each tile. That is, when coding tile 0, HMVP buffer 0 is initialized and used, and when coding tile 1, HMVP buffer 1 is initialized and used.

[0139] As yet another example, the HMVP buffer can be initialized targeting the first CTU in the CTU row within each tile.

[0140] Figure 18 shows an example of an HMVP management buffer initialization method targeting the first CTU in each CTU row within each tile according to yet another embodiment.

[0141] As shown in Figure 18, an HVMP buffer may be initialized for each CTU row in each tile. For example, an HVMP buffer may be initialized at the first CTU of the first CTU row of tile n, an HVMP buffer may be initialized at the first CTU of the second CTU row of tile n, and an HVMP buffer may be initialized at the first CTU of the third CTU row of tile n. In this case, if the coding device has a multi-core processor, the coding device can initialize and use HVMP buffer 0 for the first CTU row of tile n, HVMP buffer 1 for the second CTU row of tile n, and HVMP buffer 2 for the third CTU row of tile n, thereby supporting parallel processing. On the other hand, if the coding device has a single-core processor, the coding device can initialize and reuse the HVMP buffer at the first CTU of each CTU row in each tile according to the coding order.

[0142] On the other hand, tile and slice division structures allow tiles and slices to coexist simultaneously within a single picture.

[0143] Figure 19 shows an example of a structure in which tiles and slices coexist.

[0144] Figure 19 illustrates a case where one picture is divided into four tiles, with two slices within each tile. As shown in Figure 19, both slices and tiles may exist within a single picture, and the HMVP buffer can be initialized as follows.

[0145] For example, HMVP buffers can be initialized on a per-CTU basis. This method can be applied regardless of whether the CTU is located on a tile or a slice.

[0146] As another example, the HMVP buffer can be initialized targeting the first CTU within each tile.

[0147] Figure 20 shows an example of how to initialize the HMVP buffer targeting the first CTU within each tile.

[0148] As shown in Figure 20, the HMVP buffer can be initialized at the first CTU of each tile. Even if there are multiple slices within a single tile, the HMVP buffer initialization can still be performed at the first CTU within that tile.

[0149] As another example, you can also initialize the HMVP buffer for each slice present within a tile.

[0150] Figure 21 shows an example of how to initialize an HMVP buffer for each slice within a tile.

[0151] As shown in Figure 21, the HMVP buffer can be initialized at the first CTU of each slice within a tile. Therefore, if there are multiple slices within a single tile, the HMVP buffer initialization can be performed for each of the multiple slices. In this case, the HMVP buffer initialization can be performed when processing the first CTU of each slice.

[0152] On the other hand, a single picture can have multiple tiles without any slices. Alternatively, a single slice can have multiple tiles. In such cases, HMVP buffer initialization can be performed as follows.

[0153] For example, an HMVP buffer can be initialized for each tile group.

[0154] Figure 22 shows an example of initializing the HMVP buffer targeting the first CTU of the first tile in the tile group.

[0155] As shown in Figure 22, one picture can be divided into two tile groups, and each tile group (TileGroup0, TileGroup1) can be further divided into multiple tiles. In this case, the HMVP buffer can be initialized targeting the first CTU of the first tile within one tile group.

[0156] As another example, an HMVP buffer can be initialized on a tile-by-tile basis within a tile group.

[0157] Figure 23 shows an example of initializing the HMVP buffer for the first CTU of each tile in a tile group.

[0158] As shown in Figure 23, one picture can be divided into two tile groups, and each tile group (TileGroup0, TileGroup1) can be further divided into multiple tiles. In this case, the HMVP buffer can be initialized targeting the first CTU of each tile within a single tile group.

[0159] As yet another example, an HMVP buffer can be initialized targeting the CTU row of each tile within a tile group.

[0160] Figure 24 shows an example of initializing the HMVP buffer for the CTU row of each tile within a tile group.

[0161] As shown in Figure 24, one picture can be divided into two tile groups, and each tile group (TileGroup0, TileGroup1) can be further divided into multiple tiles. In this case, the HMVP buffer can be initialized with the first CTU of the CTU row of each tile within one tile group.

[0162] Alternatively, in this case as well, the HMVP management buffer can be initialized on a CTU basis. This can, of course, be applied without distinguishing between tiles, slices, tile groups, etc.

[0163] Figures 25 and 26 schematically illustrate an example of a video / image encoding method and related components, including an interpretation method, according to one or more embodiments of this document. The method disclosed in Figure 25 can be performed by the encoding apparatus disclosed in Figure 2. Specifically, for example, steps S2500 to S2530 in Figure 25 can be performed by the prediction unit 220 of the encoding apparatus, step S2540 in Figure 25 can be performed by the residual processing unit 230 of the encoding apparatus, and step S2550 in Figure 25 can be performed by the entropy encoding unit 240 of the encoding apparatus. The method disclosed in Figure 25 may include embodiments such as those described above in this document.

[0164] As shown in Figure 25, the encoding device derives an HMVP buffer for the current block (S2500). The encoding device can perform the HMVP buffer management method described above in the embodiments of this document. For example, the HMVP buffer can be initialized in units of slices, tiles, or tile groups. And / or, the HMVP buffer can be initialized in units of CTU rows. In this case, the HMVP buffer can be initialized in units of CTU rows within the slice, tile, or tile group. Here, a tile can represent a rectangular region such as a CTU within a picture. Tiles can be specified based on a specific tile row and a specific tile column within the picture. For example, there may be one or more tiles in the current picture. In this case, the HMVP buffer can be initialized with the first CTU of the CTU row containing the current block within the current tile. Alternatively, there may be one or more slices in the current picture. In this case, the HMVP buffer can be initialized with the first CTU of the CTU row containing the current block within the current slice. Alternatively, one or more tile groups may exist within the current picture. In this case, the HMVP buffer can be initialized with the first CTU of the CTU row containing the current block within the current tile group.

[0165] The encoding device can determine whether the current CTU is the first CTU in the CTU row. In this case, the HMVP buffer can be initialized with the first CTU in the CTU row where the current CTU containing the current block is located. In other words, the HMVP buffer can be initialized when processing the first CTU in the CTU row where the current CTU containing the current block is located. If it is determined that the current CTU containing the current block is the first CTU in the CTU row within the current tile, the HMVP buffer may include HMVP candidates derived based on the motion information of blocks processed before the current block within the current CTU. If it is determined that the current CTU is not the first CTU in the CTU row within the current tile, the HMVP buffer may include HMVP candidates derived based on the motion information of blocks processed before the current block within the CTU row within the current tile. Furthermore, for example, if the current CU, which is the current block, is located in the first CTU of the CTU row within the current tile, and the current CU corresponds to the first CU of the first CTU, then the number of HMVP candidates included in the HMVP buffer is 0. Also, for example, if a CU coded before the current CU in the CTU row within the current tile (for example, a CU coded before the current CU in the current CTU and / or a CU within a CTU coded before the current CTU in the current CTU row) is coded in intermode, then an HMVP candidate can be derived based on the movement information of the previously coded CU and included in the HMVP buffer.

[0166] If the above picture is divided into multiple tiles, the above HMVP buffer can be initialized on a CTU row-by-row basis within each tile.

[0167] The above HMVP buffer can be initialized on a per-CTU row basis within a tile or slice. For example, if a specific CTU in the above CTU row is not the first CTU of the above CTU row in the above picture, but rather the first CTU of the above CTU row in the current tile or current slice, the above HMVP buffer can be initialized with the above specific CTU.

[0168] When the above HMVP buffer is initialized, the number of HMVP candidates contained in the above HMVP buffer can be set to 0.

[0169] The encoding device configures a motion information candidate list based on the HMVP buffer (S2510). The HMVP buffer may contain HMVP candidates, and the motion information candidate list containing the HMVP candidates may be configured.

[0170] As an example, when merge mode is applied to the current block, the motion information candidate list can be a merge candidate list. As another example, when (A)MVP mode is applied to the current block, the motion information candidate list can be an MVP candidate list. When merge mode is applied to the current block, the HMVP candidate can be added to the merge candidate list if the number of available merge candidates (e.g., including spatial merge candidates and temporal merge candidates) in the merge candidate list for the current block is less than a predetermined maximum number of merge candidates. In this case, the HMVP candidate can be inserted after the spatial and temporal candidates in the merge candidate list. In other words, the HMVP candidate may be assigned an index value greater than the index assigned to the spatial and temporal candidates in the merge candidate list. When (A)MVP mode is applied to the current block, the HMVP candidate can be added to the MVP candidate list if the number of available MVP candidates (derived based on spatially adjacent blocks and temporally adjacent blocks) in the MVP candidate list for the current block is less than two.

[0171] The encoding device can derive motion information for the current block based on the above motion information candidate list (S2520).

[0172] The encoding device can derive motion information for the current block based on the motion information candidate list. For example, if merge mode or MVP mode is applied to the current block, the HMVP candidates included in the HMVP buffer can be used as merge candidates or MVP candidates. For example, if merge mode is applied to the current block, the HMVP candidates included in the HMVP buffer are included as candidates in the merge candidate list, and the HMVP candidate can be specified from among the candidates included in the merge candidate list based on the merge index. The merge index is prediction-related information and can be included in the image / video information described later. In this case, the HMVP candidate may be assigned an index in the merge candidate list with an even lower priority than the spatial merge candidates and temporal merge candidates included in the merge candidate list. That is, the index value assigned to the HMVP candidate may be assigned a higher value than the index values ​​of the spatial merge candidates and temporal merge candidates. As another example, if MVP mode is applied to the current block, the HMVP candidates included in the HMVP buffer are included as candidates in the merge candidate list, and the HMVP candidates among the candidates included in the MVP candidate list can be indicated based on the MVP flag (or MVP index). The MVP flag (or MVP index) is prediction-related information and may be included in the image / video information described later.

[0173] The encoding device generates a predicted sample for the current block based on the derived motion information (S2530). The encoding device performs interpretation (motion compensation) based on the motion information and can derive a predicted sample using the reference sample pointed to by the motion information on the reference picture.

[0174] The encoding device generates residual samples based on the predicted samples (S2540). The encoding device can generate residual samples based on the original samples for the current block and the predicted samples for the current block.

[0175] The encoding device derives information about the residual sample based on the residual sample and encodes image / video information including the information about the residual sample (S2550). The information about the residual sample can be called residual information and may include information about quantized conversion coefficients. The encoding device can perform a conversion / quantization procedure on the residual sample to derive quantized conversion coefficients.

[0176] The encoded image / video information can be output in bitstream format. The bitstream can be transmitted to a decoding device via a network or storage medium. The image / video information may further include prediction-related information, which may further include information on various prediction modes (e.g., merge mode, MVP mode, etc.), MVD information, and so on.

[0177] Figures 27 and 28 schematically show an example of an image decoding method and related components, including an interpretation method according to an embodiment of this document. The method disclosed in Figure 27 can be performed by the decoding apparatus disclosed in Figure 3. Specifically, for example, steps S2700 to S2730 in Figure 27 can be performed by the prediction unit 330 of the decoding apparatus, and step S2740 can be performed by the addition unit 340 of the decoding apparatus. The method disclosed in Figure 27 may include embodiments described in this document.

[0178] As shown in Figure 27, the decoding device derives an HMVP buffer for the current block (S2700). The decoding device can perform the HMVP buffer management method described above in the embodiments of this document. For example, the HMVP buffer can be initialized on a slice, tile, or tile group basis. And / or, the HMVP buffer can be initialized on a CTU row basis. In this case, the HMVP buffer can be initialized on a CTU row basis within the slice, tile, or tile group. Here, a tile can represent a rectangular region such as a CTU within a picture. A tile can be specified based on a specific tile row and a specific tile column within the picture. For example, there may be one or more tiles in the current picture. In this case, the HMVP buffer can be initialized with the first CTU of the CTU row containing the current block within the current tile. Alternatively, there may be one or more slices in the current picture. In this case, the HMVP buffer can be initialized with the first CTU of the CTU row containing the current block within the current slice. Alternatively, one or more tile groups may exist within the current picture. In this case, the HMVP buffer can be initialized with the first CTU of the CTU row containing the current block within the current tile group.

[0179] The decoding device can determine whether the current CTU is the first CTU in the CTU row. In this case, the HMVP buffer can be initialized with the first CTU in the CTU row where the current CTU containing the current block is located. In other words, the HMVP buffer can be initialized when processing the first CTU in the CTU row where the current CTU containing the current block is located. If it is determined that the current CTU containing the current block is the first CTU in the CTU row within the current tile, the HMVP buffer may include HMVP candidates derived based on the movement information of blocks processed before the current block within the current CTU. If it is determined that the current CTU is not the first CTU in the CTU row within the current tile, the HMVP buffer may include HMVP candidates derived based on the movement information of blocks processed before the current block within the CTU row within the current tile. Furthermore, for example, if the current CU, which is the current block, is located in the first CTU of the CTU row within the current tile, and the current CU corresponds to the first CU of the first CTU, then the number of HMVP candidates included in the HMVP buffer is 0. Also, for example, if a CU coded before the current CU in the CTU row within the current tile (for example, a CU coded before the current CU in the current CTU and / or a CU within a CTU coded before the current CTU in the current CTU row) is coded in intermode, then an HMVP candidate can be derived based on the movement information of the previously coded CU and included in the HMVP buffer.

[0180] If the above picture is divided into multiple tiles, the above HMVP buffer can be initialized on a CTU row-by-row basis within each tile.

[0181] The above HMVP buffer can be initialized on a per-CTU row basis within a tile or slice. For example, if a specific CTU in the above CTU row is not the first CTU of the above CTU row in the above picture, but rather the first CTU of the above CTU row in the current tile or current slice, the above HMVP buffer can be initialized with the above specific CTU.

[0182] When the above HMVP buffer is initialized, the number of HMVP candidates contained in the above HMVP buffer can be set to 0.

[0183] The decoding device constructs a motion information candidate list based on the HMVP buffer (S2710). The HMVP buffer may contain HMVP candidates, and the motion information candidate list containing the HMVP candidates may be constructed.

[0184] As an example, when merge mode is applied to the current block, the motion information candidate list can be a merge candidate list. As another example, when (A)MVP mode is applied to the current block, the motion information candidate list can be an MVP candidate list. When merge mode is applied to the current block, the HMVP candidate can be added to the merge candidate list if the number of available merge candidates (e.g., including spatial merge candidates and temporal merge candidates) in the merge candidate list for the current block is less than a predetermined maximum number of merge candidates. In this case, the HMVP candidate can be inserted after the spatial and temporal candidates in the merge candidate list. In other words, the HMVP candidate may be assigned an index value greater than the index assigned to the spatial and temporal candidates in the merge candidate list. When (A)MVP mode is applied to the current block, the HMVP candidate can be added to the MVP candidate list if the number of available MVP candidates (derived based on spatially adjacent blocks and temporally adjacent blocks) in the MVP candidate list for the current block is less than two.

[0185] The decoding device can derive motion information for the current block based on the above motion information candidate list (S2720).

[0186] The encoding device can derive motion information for the current block based on the motion information candidate list. For example, if merge mode or MVP mode is applied to the current block, the HMVP candidates included in the HMVP buffer can be used as merge candidates or MVP candidates. For example, if merge mode is applied to the current block, the HMVP candidates included in the HMVP buffer are included as candidates in the merge candidate list, and the HMVP candidate among the candidates included in the merge candidate list can be indicated based on the merge index obtained from the bitstream. In this case, the HMVP candidate may be assigned an index in the merge candidate list with an even lower priority than the spatial merge candidates and temporal merge candidates included in the merge candidate list. That is, the index value assigned to the HMVP candidate may be higher than the index values ​​of the spatial merge candidates and temporal merge candidates. As another example, if MVP mode is applied to the current block, the HMVP candidate contained in the HMVP buffer is included as a candidate in the merge candidate list, and the HMVP candidate may be indicated among the candidates in the MVP candidate list based on the MVP flag (or MVP index) obtained from the bitstream.

[0187] The decoding device generates predicted samples for the current block based on the derived motion information (S2730). By performing interpretation (motion compensation) based on the motion information, the decoding device can derive predicted samples using the reference samples that the motion information points to on the reference picture. The current block containing the predicted samples is also called the predicted block.

[0188] The decoding device generates a reconstructed sample based on the predicted sample (S2740). As previously stated, a reconstructed block / picture can be generated based on the reconstructed sample. As previously stated, the decoding device can obtain residual information (including information on quantized conversion coefficients) from the bitstream, derive a residual sample based on the residual information, and generate the reconstructed sample based on the predicted sample and the residual sample. Thereafter, as necessary, in-loop filtering procedures such as deblock filtering, SAO and / or ALF procedures can be applied to the reconstructed picture to improve subjective / objective image quality, as previously stated.

[0189] In the embodiments described above, the method is explained based on a flowchart in a series of steps or blocks, but the embodiments are not limited to the order of the steps, and some steps may occur in a different order or simultaneously with other steps than those described above. Furthermore, those skilled in the art will understand that the steps shown in the flowchart are not exclusive, other steps may be included, or one or more steps in the flowchart may be deleted without affecting the scope of the embodiments described herein.

[0190] The methods according to the embodiments of this document described above can be implemented in software form, and the encoding and / or decoding devices according to this document can be included in, for example, image processing devices such as TVs, computers, smartphones, set-top boxes, and display devices.

[0191] In this document, when embodiments are implemented in software, the methods described above can be implemented by modules (processes, functions, etc.) that perform the functions described above. These modules are stored in memory and can be executed by a processor. The memory may be internal or external to the processor and may be connected to the processor by a variety of well-known means. The processor may include an ASIC (Application-Specific Integrated Circuit), other chipsets, logic circuits, and / or data processing devices. The memory may include ROM (Read-Only Memory), RAM (Random Access Memory), flash memory, memory cards, storage media, and / or other storage devices. In other words, the embodiments described in this document can be implemented and executed on a processor, microprocessor, controller, or chip. For example, the functional units shown in each drawing can be implemented and executed on a computer, processor, microprocessor, controller, or chip. In this case, information on instructions or algorithms for implementation may be stored in a digital storage medium.

[0192] Furthermore, the decoding and encoding devices to which one or more embodiments of this document apply can include multimedia broadcasting transceivers, mobile communication terminals, home cinema video equipment, digital cinema video equipment, surveillance cameras, video interaction devices, real-time communication devices such as video communications, mobile streaming devices, storage media, camcorders, video-on-demand (VoD) service providers, OTT video (Over The Top video) devices, internet streaming service providers, 3D video devices, VR (Virtual Reality) devices, AR (Augmented Reality) devices, image-phone video devices, transportation terminals (e.g., vehicle terminals (including autonomous vehicles), airplane terminals, ship terminals, etc.), and medical video equipment, and can be used to process video signals or data signals. For example, OTT video (Over The Top video) devices can include game consoles, Blu-ray players, internet-connected TVs, home theater systems, smartphones, tablet PCs, DVRs (Digital Video Recorders), etc.

[0193] Furthermore, the processing methods to which one or more embodiments of this document apply can be produced in the form of a program executed on a computer and stored on a computer-readable recording medium. Multimedia data having the data structure relating to one or more embodiments of this document can also be stored on a computer-readable recording medium. The computer-readable recording medium includes all kinds of storage and distributed storage devices that store data that can be read by a computer. The computer-readable recording medium may include, for example, Blu-ray discs (BDs), Universal Serial Bus (USB), ROMs, PROMs, EPROMs, EEPROMs, RAMs, CD-ROMs, magnetic tapes, floppy disks, and optical data storage devices. The computer-readable recording medium may also include media implemented in the form of a carrier wave (e.g., transmission over the Internet). In addition, a bitstream generated by an encoding method may be stored on a computer-readable recording medium or transmitted over a wireless network.

[0194] Furthermore, one or more embodiments of this document can be implemented as a computer program product using program code, and such program code can be executed on a computer according to one or more embodiments of this document. The program code can be stored on a computer-readable carrier.

[0195] Figure 29 shows an example of a content streaming system to which the embodiments disclosed in this document may be applied.

[0196] As shown in Figure 29, the content streaming system to which the embodiments of this document apply may generally include an encoding server, a streaming server, a web server, media storage, user equipment, and multimedia input devices.

[0197] The above-mentioned encoding server is responsible for compressing content input from multimedia input devices such as smartphones, cameras, and camcorders into digital data to generate a bitstream, and then transmitting this bitstream to the above-mentioned streaming server. In other cases, if the multimedia input device such as a smartphone, camera, or camcorder directly generates the bitstream, the above-mentioned encoding server can be omitted.

[0198] The bitstream described above can be generated by an encoding method or bitstream generation method to which an embodiment of this document applies, and the streaming server can temporarily store the bitstream in the process of transmitting or receiving the bitstream.

[0199] The streaming server transmits multimedia data to user devices based on user requests via a web server, and the web server acts as an intermediary to inform users about available services. When a user requests a desired service from the web server, the web server transmits this to the streaming server, which then transmits the multimedia data to the user. In this case, the content streaming system may include a separate control server, in which case the control server controls the commands and responses between the devices within the content streaming system.

[0200] The above-mentioned streaming server can receive content from a media storage device and / or an encoding server. For example, when receiving content from the above-mentioned encoding server, the content can be received in real time. In this case, in order to provide a smooth streaming service, the above-mentioned streaming server can store the above-mentioned bitstream for a certain period of time.

[0201] Examples of user devices mentioned above include mobile phones, smartphones, laptop computers, digital broadcasting terminals, PDAs (Personal Digital Assistants), PMPs (Portable Multimedia Players), navigation systems, slate PCs, tablet PCs, ultrabooks (registered trademark), wearable devices (such as smartwatches, smart glasses, and HMDs (Head Mounted Displays)), digital TVs, desktop computers, and digital signatures (Signature).

[0202] Each server within the above content streaming system can be operated as a distributed server, in which case the data received by each server can be processed in a distributed manner.

Claims

1. An image decoding method performed by a decoding device, The steps include: deriving an HMVP (History-based Motion Vector Prediction) buffer for the current block, A step of constructing a motion information candidate list based on the HMVP candidates contained in the HMVP buffer, wherein whether to add the HMVP candidates to the motion information candidate list is checked after the time candidates in the motion information candidate list. The steps include: deriving motion information for the current block based on the aforementioned list of motion information candidates; The steps include: deriving the reference picture index of the current block based on the motion information; The steps include: deriving the motion vector of the current block based on the motion information; The steps include generating a predicted sample for the current block based on the reference picture index and the motion vector, The step of generating a reconstructed sample based on the predicted sample includes, The current picture contains one or more tiles. The aforementioned picture includes multiple tile columns and tile rows, A tile is a rectangular area of ​​a coding tree unit (CTU) within a specific tile column and a specific tile row in the current picture. The aforementioned block corresponds to one of the coding units (CUs) separated from the CTU, The aforementioned HMVP buffer is updated based on the movement information of the previous block. The HMVP buffer is initialized with the first CTU for each CTU row of each tile, Based on the initialization of the HMVP buffer, the number of HMVP candidates contained in the HMVP buffer is set to zero. The method further includes the step of determining whether the CTU is the first CTU in the CTU row of the current tile, The HMVP buffer for the current block is derived based on the result of the decision, A method in which the HMVP buffer is initialized for the CTU based on the result of the determination that the CTU is the first CTU in the CTU row of the current tile.

2. An image encoding method performed by an encoding device, The steps include: deriving an HMVP (History-based Motion Vector Prediction) buffer for the current block, A step of constructing a motion information candidate list based on the HMVP candidates contained in the HMVP buffer, wherein whether to add the HMVP candidates to the motion information candidate list is checked after the time candidates in the motion information candidate list. The steps include: deriving motion information for the current block based on the aforementioned list of motion information candidates; The steps include: deriving the reference picture index of the current block based on the motion information; The steps include: deriving the motion vector of the current block based on the motion information; The steps include generating a predicted sample for the current block based on the reference picture index and the motion vector, The steps include: deriving residual samples based on the aforementioned predicted samples; The step includes encoding image information that includes information about the residual sample, If one or more tiles exist in the current picture, The aforementioned picture includes multiple tile columns and tile rows, A tile is a rectangular area of ​​a coding tree unit (CTU) within a specific tile column and a specific tile row in the current picture. The aforementioned block corresponds to one of the coding units (CUs) separated from the CTU, The aforementioned HMVP buffer is updated based on the movement information of the previous block. The HMVP buffer is initialized with the first CTU for each CTU row of each tile, Based on the initialization of the HMVP buffer, the number of HMVP candidates contained in the HMVP buffer is set to zero. The method further includes the step of determining whether the CTU is the first CTU in the CTU row of the current tile, The HMVP buffer for the current block is derived based on the result of the decision, A method in which the HMVP buffer is initialized for the CTU based on the result of the determination that the CTU is the first CTU in the CTU row of the current tile.

3. A method for transmitting image-related data, A step of obtaining a bitstream generated by a method, wherein the method is The steps include: deriving an HMVP (History-based Motion Vector Prediction) buffer for the current block, A step of constructing a motion information candidate list based on the HMVP candidates contained in the HMVP buffer, wherein whether to add the HMVP candidates to the motion information candidate list is checked after the time candidates in the motion information candidate list. The steps include: deriving motion information for the current block based on the aforementioned list of motion information candidates; The steps include: deriving the reference picture index of the current block based on the motion information; The steps include: deriving the motion vector of the current block based on the motion information; The steps include generating a predicted sample for the current block based on the reference picture index and the motion vector, The steps include: deriving residual samples based on the aforementioned predicted samples; A step of generating the bitstream by encoding image information including information about the residual sample, and a step of generating based on, The step of transmitting the data, which includes the bitstream, If one or more tiles exist in the current picture, The aforementioned picture includes multiple tile columns and tile rows, A tile is a rectangular area of ​​a coding tree unit (CTU) within a specific tile column and a specific tile row in the current picture. The aforementioned block corresponds to one of the coding units (CUs) separated from the CTU, The aforementioned HMVP buffer is updated based on the movement information of the previous block. The HMVP buffer is initialized with the first CTU for each CTU row of each tile, Based on the initialization of the HMVP buffer, the number of HMVP candidates contained in the HMVP buffer is set to zero. The method further includes the step of determining whether the CTU is the first CTU in the CTU row of the current tile, The HMVP buffer for the current block is derived based on the result of the decision, A transmission method in which, based on the result of the determination that the CTU is the first CTU in the CTU row of the current tile, the HMVP buffer is initialized for the CTU.

Images