Method and apparatus for constructing prediction candidates based on HMVP

By configuring prediction candidates using HMVP without pruning, the method optimizes image coding efficiency for high-resolution and immersive media, addressing complexity issues in existing technologies.

JP7588125B2Active Publication Date: 2024-11-21TCL KING ELECTRICAL APPLIANCES HUIZHOU
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
JP2022196205
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Priority Date
2018-10-02
Filing Date
2022-12-08
Publication Date
2024-11-21
Estimated Expiration
2039-10-02

AI Technical Summary

Technical Problem

Existing image coding technologies face challenges in increasing efficiency, particularly in handling high-resolution and immersive media, and inter-screen prediction methods, while pruning processes in configuring prediction candidates lead to increased complexity.

Method used

The method and apparatus configure prediction candidates based on history-based motion vector prediction (HMVP) without pruning, optimizing the AMVP candidate list by adding HMVP candidates, deriving motion information, and generating prediction samples for improved coding efficiency.

Benefits of technology

This approach enhances overall image/video compression efficiency by preventing complexity increases due to pruning, enabling efficient coding for high-resolution and immersive media.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007588125000002
    Figure 0007588125000002
  • Figure 0007588125000003
    Figure 0007588125000003
  • Figure 0007588125000004
    Figure 0007588125000004
Patent Text Reader

Abstract

To provide a method for improving image coding efficiency. [Solution] An image decoding method includes the steps of: constructing an AMVP candidate list including at least one AMVP candidate for a current block; deriving an HMVP candidate list for a current block including an HMVP candidate for the current block; selecting at least one HMVP candidate from the HMVP candidates in the HMVP candidate list; adding the at least one HMVP candidate to the AMVP candidate list to derive an updated AMVP candidate list; deriving motion information for the current block based on the updated AMVP candidate list; deriving a predicted sample for the current block based on the motion information for the current block; and generating a reconstructed sample for the current block based on the predicted sample for the current block.
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present disclosure relates to image coding techniques, and more particularly to a method and apparatus for constructing prediction candidates based on history-based Motion vector Prediction (HMVP) in an image coding system. [Background technology]

[0002] In recent years, the demand for high-resolution, high-quality images / videos such as 4K or 8K or higher UHD (Ultra High Definition) images / videos is increasing in various fields. As the image / video data has higher resolution and quality, the amount of information or bits to be transmitted increases relatively compared to existing image / video data, so that when image data is transmitted using a medium such as an existing wired or wireless broadband line or image / video data is stored using an existing storage medium, the transmission cost and storage cost increase.

[0003] In addition, in recent years, interest in and demand for immersive media such as virtual reality (VR), artificial reality (AR) content, and holograms has been increasing, and the broadcast of images / videos having different image characteristics from real images, such as game images, has been increasing.

[0004] Therefore, there is a demand for highly efficient image / video compression technology to effectively compress and transmit, store, and play back high-resolution, high-quality image / video information having the various characteristics described above. Summary of the Invention [Problem to be solved by the invention]

[0005] A technical problem of the present disclosure is to provide a method and apparatus for improving image coding efficiency.

[0006] Another technical object of the present disclosure is to provide a method and apparatus for performing coding using an inter-picture prediction method.

[0007] Yet another technical object of the present disclosure is to provide a method and apparatus for configuring prediction candidates based on HMVP in inter-picture prediction.

[0008] Yet another technical object of the present disclosure is to provide a method and apparatus for omitting a pruning process in order to prevent an increase in complexity due to the pruning process when constructing prediction candidates based on HMVP in inter-picture prediction. [Means for solving the problem]

[0009] According to an embodiment of the present disclosure, there is provided an image decoding method performed by a decoding device, comprising the steps of: constructing an AMVP candidate list including at least one AMVP candidate for a current block; deriving a history-based motion vector prediction (HMVP) candidate list for the current block including HMVP candidates for the current block; selecting at least one HMVP candidate from the HMVP candidates in the HMVP candidate list; adding the at least one HMVP candidate to the AMVP candidate list to derive an updated AMVP candidate list; deriving motion information for the current block based on the updated AMVP candidate list; deriving a prediction sample for the current block based on the motion information for the current block; and generating a reconstructed sample for the current block based on the prediction sample for the current block.

[0010] According to another embodiment of the present disclosure, there is provided a decoding device for performing image decoding, comprising: a prediction unit for forming an AMVP candidate list including at least one AMVP candidate for a current block; deriving an HMVP candidate list for the current block including an HMVP candidate for the current block; selecting at least one HMVP candidate from the HMVP candidates in the HMVP candidate list; adding the at least one HMVP candidate to the AMVP candidate list to derive an updated AMVP candidate list; deriving motion information for the current block based on the updated AMVP candidate list; deriving a predicted sample for the current block based on the motion information for the current block; and an adder for generating a reconstructed sample for the current block based on the predicted sample for the current block.

[0011] According to yet another embodiment of the present disclosure, there is provided an image encoding method performed by an encoding device, comprising the steps of: constructing an AMVP candidate list including at least one AMVP candidate for a current block; deriving an HMVP candidate list for the current block including an HMVP candidate for the current block; selecting at least one HMVP candidate from the HMVP candidates in the HMVP candidate list; adding the at least one HMVP candidate to the AMVP candidate list to derive an updated AMVP candidate list; deriving motion information for the current block based on the updated AMVP candidate list; deriving a predictive sample for the current block based on the motion information for the current block; deriving a residual sample for the current block based on the predictive sample for the current block; and encoding image information including information on the residual sample.

[0012] According to yet another embodiment of the present disclosure, there is provided an encoding device for performing image encoding, comprising: an AMVP candidate list including at least one AMVP candidate for a current block; an HMVP candidate list for the current block including an HMVP candidate for the current block; a prediction unit for deriving an updated AMVP candidate list by selecting at least one HMVP candidate from the HMVP candidates in the HMVP candidate list and adding the at least one HMVP candidate to the AMVP candidate list; a prediction unit for deriving a prediction sample for the current block based on the motion information for the current block, a residual processing unit for deriving a residual sample for the current block based on the prediction sample for the current block, and an entropy encoding unit for encoding image information including information on the residual sample.

[0013] According to yet another embodiment of the present disclosure, there is provided a decoder-readable storage medium storing information regarding instructions for causing a video decoding device to perform a decoding method according to some embodiments.

[0014] According to yet another embodiment of the present disclosure, a decoder-readable storage medium is provided that stores information regarding instructions that cause a video decoding device to perform a decoding method according to an embodiment. The decoding method according to an embodiment includes the steps of: constructing an AMVP candidate list including at least one AMVP candidate for a current block; deriving a history-based motion vector prediction (HMVP) candidate list for the current block including HMVP candidates for the current block; selecting at least one HMVP candidate from the HMVP candidates in the HMVP candidate list; adding the at least one HMVP candidate to the AMVP candidate list to derive an updated AMVP candidate list; deriving motion information for the current block based on the updated AMVP candidate list; deriving a prediction sample for the current block based on the motion information for the current block; and generating a reconstruction sample for the current block based on the prediction sample for the current block. Effect of the Invention

[0015] According to the present disclosure, the overall image / video compression efficiency can be improved.

[0016] According to the present disclosure, image coding efficiency can be improved by utilizing an inter-prediction method.

[0017] According to the present disclosure, by configuring prediction candidates based on HMVP in inter prediction, it is possible to improve image coding efficiency.

[0018] According to the present disclosure, when prediction candidates are configured based on HMVP in inter prediction, the pruning process can be omitted, thereby preventing an increase in complexity due to the pruning process. [Brief description of the drawings]

[0019] [Figure 1] 1 illustrates generally an example of a video / image coding system to which the present disclosure may be applied; [Diagram 2] A diagram that illustrates the configuration of a video / image encoding device to which the present disclosure can be applied. [Diagram 3] 1 is a diagram illustrating the schematic configuration of a video / image decoding device to which the present disclosure can be applied. [Figure 4] 1 illustrates a schematic diagram of an example of a decoding procedure based on HMVP candidates. [Figure 5a] 13 is a diagram illustrating a process of updating an HMVP buffer according to one embodiment. [Figure 5b] 13 is a diagram illustrating a process of updating an HMVP buffer according to one embodiment. [Figure 6] FIG. 1 is a diagram for explaining an HMVP prediction method according to some embodiments. [Figure 7] FIG. 1 is a diagram for explaining an HMVP prediction method according to some embodiments. [Figure 8] FIG. 1 is a diagram for explaining an HMVP prediction method according to some embodiments. [Figure 9] FIG. 1 is a diagram for explaining an HMVP prediction method according to some embodiments. [Figure 10] FIG. 1 is a diagram for explaining an HMVP prediction method according to some embodiments. [Figure 11] FIG. 1 is a diagram for explaining an HMVP prediction method according to some embodiments. [Figure 12] FIG. 1 is a diagram for explaining an HMVP prediction method according to some embodiments. [Figure 13] FIG. 1 is a diagram for explaining an HMVP prediction method according to some embodiments. [Figure 14] 1 is a flowchart illustrating an operation of an encoding device according to an embodiment. [Figure 15] FIG. 1 is a block diagram showing a configuration of an encoding device according to an embodiment. [Figure 16] 10 is a flowchart illustrating an operation of a decoding device according to an embodiment. [Figure 17] 1 is a block diagram showing a configuration of a decoding device according to an embodiment; [Figure 18] 1 illustrates an example of a content streaming system to which the disclosure of this document may be applied. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0020] According to an embodiment of the present disclosure, an image decoding method performed by a decoding device is provided. The method includes the steps of: constructing an AMVP candidate list including at least one AMVP candidate for a current block; deriving a history-based motion vector prediction (HMVP) candidate list for the current block including HMVP candidates for the current block; selecting at least one HMVP candidate from the HMVP candidates in the HMVP candidate list; adding the at least one HMVP candidate to the AMVP candidate list to derive an updated AMVP candidate list; deriving motion information for the current block based on the updated AMVP candidate list; deriving a prediction sample for the current block based on the motion information for the current block; and generating a reconstruction sample for the current block based on the prediction sample for the current block.

[0021] The present disclosure may be modified in various ways and may have various embodiments, and a specific embodiment will be illustrated in the drawings and described in detail. However, this is not intended to limit the present disclosure to a specific embodiment. Terms commonly used in this specification are used merely to describe a specific embodiment, and are not intended to limit the technical idea of ​​the present disclosure. A singular expression includes a plural expression unless the context clearly indicates otherwise. In this specification, the terms "include" or "have" are intended to specify the presence of features, numbers, steps, operations, components, parts, or combinations thereof described in the specification, and should be understood not to preclude the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

[0022] Meanwhile, each configuration in the drawings described in this disclosure is illustrated independently for the convenience of explaining the different characteristic functions, etc., and does not mean that each configuration is realized by separate hardware or software. For example, two or more of each configuration may be combined to form one configuration, and one configuration may be divided into multiple configurations. An embodiment in which each configuration is integrated and / or separated is also included in the scope of the present disclosure as long as it does not deviate from the essence of the present disclosure.

[0023] Hereinafter, preferred embodiments of the present disclosure will be described in more detail with reference to the accompanying drawings. Hereinafter, the same reference numerals are used for the same components in the drawings, and duplicated descriptions of the same components may be omitted.

[0024] FIG. 1 illustrates generally an example of a video / image coding system to which the present disclosure may be applied.

[0025] As shown in Figure 1, a video / image coding system may include a first device (source device) and a second device (receiving device). The source device may transmit encoded video / image information or data to the receiving device in a file or streaming form via a digital storage medium or a network.

[0026] The source device may comprise a video source, an encoding device, and a sending unit. The receiving device may comprise a receiving unit, a decoding device, and a renderer. The encoding device may be referred to as a video / image encoding device, and the decoding device may be referred to as a video / image decoding device. The transmitter may be comprised in the encoding device. The receiver may be comprised in the decoding device. The renderer may comprise a display unit, which may be a separate device or an external component.

[0027] A video source may obtain video / images through a video / image capture, synthesis, or generation process. A video source may include a video / image capture device and / or a video / image generation device. A video / image capture device may include, for example, one or more cameras, a video / image archive containing previously captured video / images, etc. A video / image generation device may include, for example, a computer, a tablet, a smartphone, etc., and may (electronically) generate video / images. For example, a virtual video / image may be generated via a computer, etc., in which case the video / image capture process may be replaced by a process in which related data is generated.

[0028] An encoding device can encode an input video / image. The encoding device can perform a series of steps such as prediction, transformation, quantization, etc. for compression and coding efficiency. The encoded data (encoded video / image information) can be output in the form of a bitstream.

[0029] The transmitting unit may transmit the encoded video / image information or data output in the form of a bitstream to a receiving unit of a receiving device via a digital storage medium or a network in the form of a file or streaming. The digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. The transmitting unit may include elements for generating a media file through a predetermined file format and may include elements for transmission via a broadcasting / communication network. The receiving unit may receive / extract the bitstream and transmit it to a decoding device.

[0030] The decoding device can decode the video / image by performing a series of steps such as inverse quantization, inverse transformation, prediction, etc., which correspond to the operations of the encoding device.

[0031] The renderer can render the decoded video / images. The rendered video / images can be displayed via a display unit.

[0032] This document relates to video / image coding. For example, the methods / embodiments disclosed in this document can be applied to methods disclosed in the versatile video coding (VVC) standard, the essential video coding (EVC) standard, the AOMedia Video 1 (AV1) standard, the 2nd generation of audio video coding standard (AVS2), or the next generation video / image coding standard (e.g., H.267 or H.268, etc.).

[0033] In this document, various embodiments relating to video / image coding are presented, which may be performed in combination with each other, unless otherwise stated.

[0034] In this document, a video may mean a collection of a series of images over time. A picture generally means a unit showing one image at a specific time, and a slice / tile is a unit constituting a part of a picture in coding. A slice / tile may include one or more coding tree units (CTUs). A picture may be composed of one or more slices / tiles. A picture may be composed of one or more tile groups. A tile group may include one or more tiles. A brick may represent a rectangular region of CTU rows within a tile in a picture. A tile may be partitioned into multiple bricks, each of which consisting of one or more CTU rows within the tile. A tile that is not partitioned into multiple bricks may be also referred to as a brick.A brick scan may refer to a specific sequential ordering of CTUs partitioning a picture in which the CTUs are ordered consecutively in CTU raster scan in a brick, bricks within a tile are ordered consecutively in a raster scan of the bricks of the tile, and tiles in a picture are ordered consecutively in a raster scan of the tiles of the picture. A tile is a rectangular region of CTUs within a particular tile column and a particular tile row in a picture. The tile column is a rectangular region of CTUs having a height equal to the height of the picture and a width specified by syntax elements in the picture parameter set.The tile row is a rectangular region of CTUs having a height specified by syntax elements in the picture parameter set and a width equal to the width of the picture. A tile scan is a specific sequential ordering of CTUs partitioning a picture in which the CTUs are ordered consecutively in CTU raster scan in a tile whereas tiles in a picture are ordered consecutively in a raster scan of the tiles of the picture. A slice includes an integer number of bricks of a picture that may be exclusively contained in a single NAL unit.A slice may consist of either a number of complete tiles or only a consecutive sequence of complete bricks of one tile. In this document, tile group and slice may be used interchangeably. For example, tile group / tile group header may be referred to as slice / slice header in this document.

[0035] A pixel or a pel may refer to the smallest unit that constitutes one picture (or image). A term corresponding to a pixel may be "sample." A sample may generally indicate a pixel or a pixel value, may indicate only a pixel / pixel value of a luma component, or may indicate only a pixel / pixel value of a chroma component.

[0036] A unit may indicate a basic unit of image processing. A unit may include at least one of a specific region of a picture and information related to the region. One unit may include one luma block and two chroma (e.g., cb, cr) blocks. A unit may be used in combination with terms such as block or area, depending on the case. In a general case, an M×N block may include a set (or array) of samples or transform coefficients consisting of M columns and N rows.

[0037] In this document, the term “ / ” and “,” should be interpreted to indicate “and / or.” For instance, the expression “A / B” may mean “A and / or B.” Further, “A,B” may mean “A and / or B.” Further, “A / B / C” may mean “at least one of A,B, and / or C.” Also, “A / B / C” may mean “at least one of A,B, and / or C.”

[0038] Further, in the document, the term “or” should be interpreted to indicate “and / or.” For instance, the expression “A or B” may comprise 1) only A, 2) only B, and / or 3) both A and B. In other words, the term “or” in this document should be interpreted to indicate “additionally or alternatively.”

[0039] 2 is a diagram for explaining the schematic configuration of a video / image encoding device to which the present disclosure can be applied. Hereinafter, the video encoding device may include an image encoding device.

[0040] As shown in FIG. 2, the encoding device 200 may be configured to include an image partitioner 210, a predictor 220, a residual processor 230, an entropy encoder 240, an adder 250, a filter 260, and a memory 270. The predictor 220 may include an inter prediction unit 221 and an intra prediction unit 222. The residual processor 230 may include a transformer 232, a quantizer 233, a dequantizer 234, and an inverse transformer 235. The residual processor 230 may further include a subtractor 231. The adder 250 may be referred to as a reconstructor or a reconstructed block generator. The image division unit 210, the prediction unit 220, the residual processing unit 230, the entropy encoding unit 240, the addition unit 250, and the filtering unit 260 may be configured by one or more hardware components (e.g., an encoder chip set or a processor) according to an embodiment. In addition, the memory 270 may include a decoded picture buffer (DPB) and may be configured by a digital storage medium. The hardware components may further include the memory 270 as an internal / external component.

[0041] The image division unit 210 may divide an input image (or picture, frame) input to the encoding device 200 into one or more processing units. As an example, the processing units may be called coding units (CUs). In this case, the coding units may be recursively divided from a coding tree unit (CTU) or a largest coding unit (LCU) according to a quad-tree binary-tree ternary-tree (QTBTTT) structure. For example, one coding unit may be divided into a plurality of coding units of a deeper depth based on a quad-tree structure, a binary tree structure, and / or a ternary structure. In this case, for example, the quad-tree structure may be applied first, and then the binary tree structure and / or the ternary structure may be applied. Alternatively, the binary tree structure may be applied first. The coding procedure according to the present disclosure may be performed based on a final coding unit that is not further divided. In this case, the maximum coding unit may be used as the final coding unit based on coding efficiency according to image characteristics, or the coding unit may be recursively divided into coding units of lower depths as necessary, and a coding unit of an optimal size may be used as the final coding unit. Here, the coding procedure may include procedures such as prediction, transformation, and restoration, which will be described later. As another example, the processing unit may further include a prediction unit (PU: Prediction Unit) or a transform unit (TU: Transform Unit). In this case, the prediction unit and the transform unit may each be divided or partitioned from the final coding unit described above.The prediction unit may be a unit of sample prediction, and the transform unit may be a unit for deriving transform coefficients and / or a unit for deriving a residual signal from the transform coefficients.

[0042] The unit may be used interchangeably with terms such as block or area. In general, an M×N block may refer to a collection of samples or transform coefficients consisting of M columns and N rows. A sample may generally refer to a pixel or pixel value, may refer to only a pixel / pixel value of a luma component, or may refer to only a pixel / pixel value of a chroma component. A sample may be used as a term that corresponds to a pixel or pel of one picture (or image).

[0043] The encoding device 200 may generate a residual signal (residual block, residual sample array) by subtracting a prediction signal (predicted block, prediction sample array) output from the inter prediction unit 221 or the intra prediction unit 222 from an input image signal (original block, original sample array), and the generated residual signal is transmitted to the conversion unit 232. In this case, as illustrated, a unit in the encoder 200 that subtracts a prediction signal (predicted block, prediction sample array) from an input image signal (original block, original sample array) may be called a subtraction unit 231. The prediction unit may perform prediction on a block to be processed (hereinafter, referred to as a current block) and generate a predicted block including a prediction sample for the current block. The prediction unit may determine whether intra prediction or inter prediction is applied in units of the current block or CU. The prediction unit may generate various information related to prediction, such as prediction mode information, and transmit the information to the entropy encoding unit 240, as will be described later in the description of each prediction mode. The prediction information can be encoded by the entropy encoding unit 240 and output in the form of a bitstream.

[0044] The intra prediction unit 222 may predict the current block by referring to samples in the current picture. The referenced samples may be located in the neighborhood of the current block or may be located away from the current block depending on the prediction mode. In the intra prediction, the prediction mode may include a plurality of non-directional modes and a plurality of directional modes. The non-directional modes may include, for example, a DC mode and a planar mode. The directional modes may include, for example, 33 directional prediction modes or 65 directional prediction modes depending on the fineness of the prediction direction. However, this is merely an example, and more or less directional prediction modes may be used depending on the settings. The intra prediction unit 222 may also determine the prediction mode to be applied to the current block using the prediction modes applied to the neighboring blocks.

[0045] The inter prediction unit 221 may derive a predicted block for a current block based on a reference block (reference sample array) identified by a motion vector on a reference picture. In this case, in order to reduce the amount of motion information transmitted in the inter prediction mode, the motion information may be predicted in units of blocks, sub-blocks, or samples based on the correlation of motion information between a neighboring block and a current block. The motion information may include a motion vector and a reference picture index. The motion information may further include inter prediction direction (L0 prediction, L1 prediction, Bi prediction, etc.) information. In the case of inter prediction, the neighboring block may include a spatial neighboring block present in the current picture and a temporal neighboring block present in the reference picture. The reference picture including the reference block and the reference picture including the temporal neighboring block may be the same or different. The temporal neighboring block may be called a collocated reference block, a collocated CU (col CU), etc., and the reference picture including the temporal neighboring block may be called a collocated picture (colPic). For example, the inter prediction unit 221 may generate information indicating which candidate is used to construct a motion information candidate list based on neighboring blocks and derive a motion vector and / or a reference picture index of the current block. Inter prediction may be performed based on various prediction modes, and for example, in the case of a skip mode and a merge mode, the inter prediction unit 221 may use motion information of neighboring blocks as motion information of the current block. In the case of the skip mode, unlike the merge mode, a residual signal may not be transmitted.In the case of motion vector prediction (MVP) mode, the motion vector of the current block can be indicated by using the motion vector of a neighboring block as a motion vector predictor and signaling the motion vector difference.

[0046] The prediction unit 220 may generate a prediction signal based on various prediction methods, which will be described later. For example, the prediction unit may apply not only intra prediction or inter prediction for prediction of one block, but also intra prediction and inter prediction simultaneously. This may be called combined inter and intra prediction (CIIP). In addition, the prediction unit may be based on an intra block copy (IBC) prediction mode or a palette mode for prediction of a block. The IBC prediction mode or palette mode may be used for content image / video coding such as games, for example, screen content coding (SCC). IBC basically performs prediction within a current picture, but may be performed similarly to inter prediction in deriving a reference block within the current picture. That is, IBC may utilize at least one of the inter prediction techniques described in this document. The palette mode may be considered as an example of intra coding or intra prediction. When the palette mode is applied, a sample value within a picture may be signaled based on information regarding a palette table and a palette index.

[0047] The prediction signal generated through the prediction unit (including the inter prediction unit 221 and / or the intra prediction unit 222) may be used to generate a restored signal or may be used to generate a residual signal. The transform unit 232 may generate transform coefficients by applying a transform technique to the residual signal. For example, the transform technique may include at least one of a discrete cosine transform (DCT), a discrete sine transform (DST), a Karhunen-Loeve transform (KLT), a graph-based transform (GBT), or a conditionally non-linear transform (CNT). Here, the GBT refers to a transform obtained from a graph when the relationship information between pixels is to be expressed as a graph. The CNT refers to a transform obtained based on a prediction signal generated using all previously reconstructed pixels. In addition, the transform process may be applied to pixel blocks having the same square size, and may also be applied to non-square variable-size blocks.

[0048] The quantizer 233 quantizes the transform coefficients and transmits the quantized transform coefficients to the entropy encoder 240, which may encode the quantized signal (information on the quantized transform coefficients) and output the quantized signal to a bitstream. The information on the quantized transform coefficients may be referred to as residual information. The quantizer 233 may rearrange the quantized transform coefficients in a block form into a one-dimensional vector form based on a coefficient scan order, and may generate information on the quantized transform coefficients based on the quantized transform coefficients in the one-dimensional vector form. The entropy encoder 240 may perform various encoding methods, such as exponential Golomb, context-adaptive variable length coding (CAVLC), and context-adaptive binary arithmetic coding (CABAC). The entropy encoder 240 may encode information required for video / image restoration (e.g., values ​​of syntax elements, etc.) together with or separately from the quantized transform coefficients. The encoded information (e.g., encoded video / image information) may be transmitted or stored in a network abstraction layer (NAL) unit unit in the form of a bitstream. The video / image information may further include information on various parameter sets, such as an adaptation parameter set (APS), a picture parameter set (PPS), a sequence parameter set (SPS), or a video parameter set (VPS). The video / image information may further include general constraint information. In this document, information and / or syntax elements transmitted / signaled from an encoding device to a decoding device may be included in the video / image information. The video / image information may be encoded through the above-mentioned encoding procedure and included in the bitstream.The bitstream may be transmitted via a network or may be stored in a digital storage medium, where the network may include a broadcasting network and / or a communication network, and the digital storage medium may include various storage media such as a USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. A transmitter (not shown) for transmitting the signal output from the entropy encoding unit 240 and / or a storage unit (not shown) for storing the signal may be configured as an internal / external element of the encoding device 200, or the transmitter may be included in the entropy encoding unit 240.

[0049] The quantized transform coefficients output from the quantizer 233 may be used to generate a prediction signal. For example, a residual signal (residual block or residual sample) may be restored by applying inverse quantization and inverse transform to the quantized transform coefficients via the inverse quantizer 234 and the inverse transformer 235. The adder 155 may generate a reconstructed signal (reconstructed picture, reconstructed block, reconstructed sample array) by adding the reconstructed residual signal to the prediction signal output from the inter prediction unit 221 or the intra prediction unit 222. When there is no residual for the current block, such as when a skip mode is applied, a predicted block may be used as a reconstructed block. The adder 250 may be referred to as a reconstruction unit or a reconstructed block generator. The generated reconstructed signal may be used for intra prediction of the next current block in the current picture, and may also be used for inter prediction of the next picture after filtering, as described below.

[0050] Meanwhile, luma mapping with chroma scaling (LMCS) may be applied during picture encoding and / or reconstruction.

[0051] The filtering unit 260 may apply filtering to the reconstructed signal to improve subjective / objective image quality. For example, the filtering unit 260 may apply various filtering methods to the reconstructed picture to generate a modified reconstructed picture, and may store the modified reconstructed picture in the memory 270, specifically, in the DPB of the memory 270. The various filtering methods may include, for example, deblocking filtering, sample adaptive offset, an adaptive loop filter, a bilateral filter, etc. The filtering unit 260 may generate various information related to filtering and transmit it to the entropy encoding unit 240, as will be described later in relation to each filtering method. The information related to filtering may be encoded by the entropy encoding unit 240 and output in the form of a bitstream.

[0052] The modified reconstructed picture transmitted to the memory 270 can be used as a reference picture in the inter prediction unit 221. When inter prediction is applied through this, the encoding apparatus can avoid prediction mismatch between the encoding apparatus 100 and the decoding apparatus, and can also improve encoding efficiency.

[0053] The DPB of the memory 270 may store the modified reconstructed picture to be used as a reference picture in the inter prediction unit 221. The memory 270 may store motion information of a block from which motion information in the current picture is derived (or encoded) and / or motion information of a block in an already reconstructed picture. The stored motion information may be transmitted to the inter prediction unit 221 to be used as motion information of a spatial neighboring block or motion information of a temporal neighboring block. The memory 270 may store reconstructed samples of reconstructed blocks in the current picture and transmit them to the intra prediction unit 222.

[0054] FIG. 3 is a diagram illustrating a schematic configuration of a video / image decoding device to which the present disclosure can be applied.

[0055] As shown in FIG. 3, the decoding device 300 may be configured to include an entropy decoder 310, a residual processor 320, a predictor 330, an adder 340, a filter 350, and a memory 360. The predictor 330 may include an inter predictor 331 and an intra predictor 332. The residual processor 320 may include a dequantizer 321 and an inverse transformer 321. The entropy decoder 310, the residual processor 320, the predictor 330, the adder 340, and the filter 350 may be configured as one hardware component (e.g., a decoder chipset or processor) according to an embodiment. In addition, the memory 360 may include a decoded picture buffer (DPB) and may be configured as a digital storage medium. The hardware components may further include a memory 360 as an internal / external component.

[0056] If a bitstream including video / image information is input, the decoding device 300 can restore an image corresponding to a process in which the video / image information is processed by the encoding device of FIG. 3. For example, the decoding device 300 can derive a unit / block based on block division related information obtained from the bitstream. The decoding device 300 can perform decoding using a processing unit applied in the encoding device. Thus, the processing unit for decoding can be, for example, a coding unit, and the coding unit can be divided from a coding tree unit or a maximum coding unit according to a quad tree structure, a binary tree structure, and / or a ternary tree structure. One or more transform units can be derived from the coding unit. Then, the restored image signal decoded and output by the decoding device 300 can be reproduced by a reproduction device.

[0057] The decoding device 300 may receive a signal output from the encoding device of FIG. 3 in the form of a bitstream, and the received signal may be decoded via the entropy decoding unit 310. For example, the entropy decoding unit 310 may derive information (e.g., video / image information) required for image restoration (or picture restoration) by parsing the bitstream. The video / image information may further include information on various parameter sets, such as an adaptation parameter set (APS), a picture parameter set (PPS), a sequence parameter set (SPS), or a video parameter set (VPS). In addition, the video / image information may further include general constraint information. The decoding device may further decode pictures based on information on the parameter set and / or the general constraint information. Signaling / received information and / or syntax elements described later in this document may be decoded via the decoding procedure and obtained from the bitstream. For example, the entropy decoding unit 310 may decode information in a bitstream based on a coding method such as exponential Golomb coding, CAVLC, or CABAC, and output values ​​of syntax elements required for image restoration and quantized values ​​of transform coefficients related to residuals. More specifically, the CABAC entropy decoding method may receive bins corresponding to each syntax element in the bitstream, determine a context model using syntax element information to be decoded and decode information of neighboring and to-be-decoded blocks or information of symbols / bins decoded in a previous step, predict the occurrence probability of bins according to the determined context model, and perform arithmetic decoding of the bins to generate symbols corresponding to the values ​​of each syntax element. In this case, the CABAC entropy decoding method may update the context model using information of the decoded symbol / bin for the context model of the next symbol / bin after determining the context model.Among the information decoded by the entropy decoding unit 310, information related to prediction is provided to a prediction unit (inter prediction unit 332 and intra prediction unit 331), and residual values ​​entropy decoded by the entropy decoding unit 310, i.e., quantized transform coefficients and related parameter information, may be input to the residual processing unit 320. The residual processing unit 320 may derive a residual signal (residual block, residual sample, residual sample array). Also, among the information decoded by the entropy decoding unit 310, information related to filtering may be provided to the filtering unit 350. Meanwhile, a receiving unit (not shown) for receiving a signal output from the encoding device may be further configured as an internal / external element of the decoding device 300, or the receiving unit may be a component of the entropy decoding unit 310. Meanwhile, the decoding device according to this document may be called a video / image / picture decoding device, and the decoding device may be divided into an information decoder (video / image / picture information decoder) and a sample decoder (video / image / picture sample decoder). The information decoder may include the entropy decoding unit 310, and the sample decoder may include at least one of the inverse quantization unit 321, the inverse transform unit 322, the addition unit 340, the filtering unit 350, the memory 360, the inter prediction unit 332, and the intra prediction unit 331.

[0058] The inverse quantization unit 321 may inverse quantize the quantized transform coefficients and output the transform coefficients. The inverse quantization unit 321 may rearrange the quantized transform coefficients in a two-dimensional block form. In this case, the rearrangement may be performed based on a coefficient scan order performed in the encoding device. The inverse quantization unit 321 may perform inverse quantization on the quantized transform coefficients using a quantization parameter (e.g., quantization step size information) to obtain transform coefficients.

[0059] The inverse transform unit 322 inversely transforms the transform coefficients to obtain a residual signal (residual block, residual sample array).

[0060] The prediction unit may perform prediction on a current block and generate a predicted block including prediction samples for the current block. The prediction unit may determine whether intra prediction or inter prediction is applied to the current block based on the prediction information output from the entropy decoding unit 310, and may determine a specific intra / inter prediction mode.

[0061] The prediction unit 320 may generate a prediction signal based on various prediction methods described later. For example, the prediction unit may apply intra prediction or inter prediction for prediction of one block, and may simultaneously apply intra prediction and inter prediction. This may be called combined inter and intra prediction (CIIP). In addition, the prediction unit may be based on an intra block copy (IBC) prediction mode or a palette mode for prediction of a block. The IBC prediction mode or palette mode may be used for content image / video coding such as games, for example, screen content coding (SCC). IBC basically performs prediction within a current picture, but may be performed similarly to inter prediction in deriving a reference block within the current picture. That is, IBC may utilize at least one of the inter prediction techniques described in this document. The palette mode may be considered as an example of intra coding or intra prediction. When the palette mode is applied, information regarding a palette table and a palette index may be included in the video / image information and signaled.

[0062] The intra prediction unit 331 may predict a current block by referring to samples in a current picture. The referenced samples may be located in the neighborhood of the current block or may be located away from the current block depending on a prediction mode. In intra prediction, prediction modes may include a plurality of non-directional modes and a plurality of directional modes. The intra prediction unit 331 may also determine a prediction mode to be applied to the current block using prediction modes applied to neighboring blocks.

[0063] The inter prediction unit 332 may derive a predicted block for the current block based on a reference block (reference sample array) identified by a motion vector on a reference picture. In this case, in order to reduce the amount of motion information transmitted in the inter prediction mode, the motion information may be predicted in units of blocks, sub-blocks, or samples based on correlation of motion information between neighboring blocks and the current block. The motion information may include a motion vector and a reference picture index. The motion information may further include inter prediction direction (L0 prediction, L1 prediction, Bi prediction, etc.) information. In the case of inter prediction, the neighboring blocks may include spatial neighboring blocks present in the current picture and temporal neighboring blocks present in the reference picture. For example, the inter prediction unit 332 may configure a motion information candidate list based on the neighboring blocks and derive a motion vector and / or a reference picture index for the current block based on the received candidate selection information. Inter prediction may be performed based on various prediction modes, and the information on the prediction may include information indicating a mode of inter prediction for the current block.

[0064] The adder 340 can generate a reconstructed signal (reconstructed picture, reconstructed block, reconstructed sample array) by adding the acquired residual signal to a prediction signal (predicted block, predicted sample array) output from a prediction unit (including an inter prediction unit 332 and / or an intra prediction unit 331). When there is no residual for a current block, such as when a skip mode is applied, the predicted block can be used as a reconstructed block.

[0065] The adder 340 may be referred to as a reconstruction unit or a reconstruction block generator. The generated reconstruction signal may be used for intra prediction of a next block to be processed in a current picture, and may be output after filtering as described below, or may be used for inter prediction of a next picture.

[0066] Meanwhile, LMCS (luma mapping with chroma scaling) can be applied during the picture decoding process.

[0067] The filtering unit 350 may apply filtering to the reconstructed signal to improve subjective / objective image quality. For example, the filtering unit 350 may apply various filtering methods to the reconstructed picture to generate a modified reconstructed picture, and may transmit the modified reconstructed picture to the memory 360, specifically, to the DPB of the memory 360. The various filtering methods may include, for example, deblocking filtering, sample adaptive offset, an adaptive loop filter, a bilateral filter, etc.

[0068] The (modified) reconstructed picture stored in the DPB of the memory 360 may be used as a reference picture in the inter prediction unit 332. The memory 360 may store motion information of a block from which motion information in the current picture is derived (or decoded) and / or motion information of a block in an already reconstructed picture. The stored motion information may be transmitted to the inter prediction unit 260 to be used as motion information of a spatially neighboring block or motion information of a temporally neighboring block. The memory 360 may store reconstructed samples of reconstructed blocks in the current picture and transmit them to the intra prediction unit 331.

[0069] In this specification, the embodiments described for the filtering unit 260, inter prediction unit 221, and intra prediction unit 222 of the encoding device 100 can also be applied identically or correspondingly to the filtering unit 350, inter prediction unit 332, and intra prediction unit 331 of the decoding device 300, respectively.

[0070] Meanwhile, as described above, prediction is performed to improve compression efficiency when performing video coding. As a result, a predicted block including predicted samples for a current block, which is a block to be coded, can be generated. Here, the predicted block includes predicted samples in a spatial domain (or a pixel domain). The predicted block is derived in the same way in an encoding device and a decoding device, and the encoding device can improve image coding efficiency by signaling information (residual information) about the residual between the original block and the predicted block, which is not an original sample value of the original block, to the decoding device. The decoding device can derive a residual block including residual samples based on the residual information, and can generate a reconstructed block including reconstructed samples by adding the residual block and the predicted block, and can generate a reconstructed picture including the reconstructed block.

[0071] The residual information may be generated through a transform and quantization procedure. For example, the encoding apparatus may derive a residual block between the original block and the predicted block, perform a transform procedure on the residual samples (residual sample array) included in the residual block to derive transform coefficients, and perform a quantization procedure on the transform coefficients to derive quantized transform coefficients, and then signal the related residual information (through a bitstream) to the decoding apparatus. Here, the residual information may include information such as value information, position information, transform technique, transform kernel, and quantization parameter of the quantized transform coefficients. The decoding apparatus may derive a residual sample (or a residual block) by performing an inverse quantization / inverse transform procedure based on the residual information. The decoding apparatus may generate a reconstructed picture based on the predicted block and the residual block. Also, the encoding apparatus may derive a residual block by inverse quantizing / inverse transforming the quantized transform coefficients for reference for inter-prediction of a future picture, and generate a reconstructed picture based on the residual block.

[0072] 4 illustrates an example of a decoding procedure based on an HMVP candidate, where the decoding procedure based on an HMVP candidate may include an inter-prediction procedure based on an HMVP candidate.

[0073] As shown in FIG. 4, the decoding apparatus loads an HMVP table including HMVP candidates (etc.) and decodes a block based on at least one of the HMVP candidates (etc.). Specifically, for example, the decoding apparatus may derive motion information of a current block based on at least one of the HMVP candidates (etc.), and may derive a predicted block (including a predicted sample) by performing inter prediction on the current block based on the motion information. As described above, a reconstructed block may be generated based on the predicted block. The derived motion information of the current block may be updated to the table. In this case, the motion information may be added as a new HMVP candidate to the last entry of the table. If the number of HMVP candidates already included in the table is equal to the size of the table, a candidate that was entered for the first time in the table may be deleted, and the derived motion information may be added as a new HMVP candidate to the last entry of the table.

[0074] 5A and 5B are diagrams illustrating a process of updating an HMVP buffer according to an embodiment of the present invention, more specifically, FIG 5A illustrates an example of an HMVP table update according to a FIFO rule, and FIG 5B illustrates an example of an HMVP table update according to a limited FIFO rule.

[0075] A first-in-first-out (FIFO) rule may be applied to the table shown in FIG. 5a. For example, if the table size S is 16, this indicates that 16 HMVP candidates may be included in the table. If more than 16 HMVP candidates are generated from previously coded blocks, a FIFO rule may be applied, whereby the table may include up to 16 most recently coded motion information candidates. In this case, as shown in FIG. 5a, the FIFO rule may be applied to remove the oldest HMVP candidate and add a new HMVP candidate.

[0076] Meanwhile, in order to further improve coding efficiency, a limited FIFO rule may be applied as shown in Fig. 5b. As shown in Fig. 5b, when an HMVP candidate is inserted into a table, a redundancy check may be applied first. Through this, it may be determined whether an HMVP candidate having the same motion information is already present in the table. If an HMVP candidate having the same motion information is present in the table, the HMVP candidate having the same motion information may be removed from the table, and the HMVP candidates following the removed HMVP candidate may be moved one by one (i.e., each index -1), and then a new HMVP candidate may be inserted.

[0077] As mentioned above, HMVP candidates can be used in the merge candidate list construction procedure. In this case, all HMVP candidates that can be inserted, for example from the last entry to the first entry in the table, can be inserted next to the spatial merge candidate and the temporal merge candidate. In this case, a pruning check can be applied to the HMVP candidates. The maximum number of merge candidates allowed can be signaled, and the merge candidate list construction procedure can be terminated if the total number of available merge candidates reaches the maximum number of merge candidates.

[0078] Similarly, the HMVP candidate may also be used in the (A) MVP candidate list construction procedure. In this case, the motion vectors of the last k HMVP candidates in the HMVP table may be added next to the TMVP candidates that constitute the MVP candidate list. In this case, for example, the HMVP candidate having the same reference picture as the MVP target reference picture may be used for the MVP candidate list construction. Here, the MVP target reference picture may indicate a reference picture for inter prediction of the current block to which the MVP mode is applied. In this case, a pruning check may be applied to the HMVP candidate. The k may be, for example, 4. However, this is an example, and the k may have various values ​​such as 1, 2, 3, 4, etc.

[0079] On the other hand, if the total number of merge candidates is equal to or greater than 15, a truncated unary plus fixed length (with 3 bits) binarization method may be applied for merge index coding, as shown in Table 1 below.

[0080] [Table 1]

[0081] The above table assumes that Nmrg=15, where Nmrg indicates the total number of merging candidates.

[0082] 6 to 13 are diagrams for explaining an HMVP prediction method according to some embodiments.

[0083] In some embodiments, a method for deriving motion information in a buffer for a candidate may be provided in the process of using motion information of the HMVP buffer as a prediction candidate. HMVP may perform a process for deriving a prediction candidate for the current block by popping the most recent motion information from the motion buffer in the process of pushing the motion information of the current block to the buffer and configuring a motion candidate for the next block. At this time, a pruning process may be performed to check the similarity between the popped prediction candidate and existing prediction candidates (e.g., motion information of neighboring blocks, temporal motion information, etc.).

[0084] If the pruning process is not performed, the motion information most recently pushed and stored in the HMVP buffer is highly likely to be the same as the motion information of the neighboring blocks already configured, so the same motion information may configure the prediction candidate list. However, since this pruning process requires a comparison operation with a previously configured prediction candidate every time, it not only increases coding complexity, but also has a low compression efficiency when compared with other prediction candidate configuration methods that do not perform the pruning process but have the same / similar complexity operation. Therefore, in one embodiment, a method of handling popped prediction candidates may be proposed to eliminate the pruning process but improve prediction performance.

[0085] In one embodiment according to FIG. 6, we propose a method to pop the oldest prediction candidate from the HMVP buffer, which is not the most recent prediction candidate.

[0086] When the buffer is configured as shown in Fig. 6, the motion information of the current block is decoded and stored based on the last available buffer index of the buffer, but in the process of decoding the motion information of the block next to the current block, the motion information based on the most prioritized buffer index (i.e., the smallest buffer index) can be used first. This method not only has the effect of efficiently deriving neighboring motion information and other motion information in the example shown in Fig. 7, but also has the effect of decoding the current block using distant motion information.

[0087] In one embodiment according to Fig. 8, we propose a method for determining prediction candidates to pop in HMVP based on sub-sampling. More specifically, as shown in Fig. 8, the motion information stored in the buffer can be sub-sampled and used as prediction candidates.

[0088] In one example, the oldest (or smallest HMVP buffer index) motion information can be used after sub-sampling. As shown in FIG. 9, the 0th, 2nd, and 4th motion information (or motion information based on the 0th, 2nd, and 4th HMVP buffer indexes) sub-sampled during the process of encoding the current block can be used as prediction candidates.

[0089] In another example, the most recent motion information can be used in the subsampling process. As shown in FIG. 10, the most recent subsampled motion information can be used as a prediction candidate in the subsampling process.

[0090] In yet another example, when prediction candidates for a current block are not all configured even if the prediction candidates are popped in order when subsampling to configure prediction candidates, prediction candidates can be configured by selecting candidates that were not selected after subsampling again in order. As shown in Figure 11, when motion information is configured by subsampling but prediction candidates for a current block are not all configured, it can be seen that after further subsampling, the motion information derived through the first subsampling and other motion information are used as prediction candidates.

[0091] In one embodiment according to FIG. 12, instead of using the popped prediction candidates directly in the HMVP buffer, average motion information of the popped prediction candidates is calculated, and the calculated motion information can be used as the prediction candidate.

[0092] As shown in Fig. 12, when using motion information, the motion information can be obtained by calculating the average with other motion information, rather than using the stored motion information as is. Although Fig. 12 illustrates an example of popping the oldest motion information from the HMVP buffer, a method of popping from the most recent motion information is also possible. In this case, since the motion information used as a prediction candidate in a state where the pruning process has not been performed is not the most recently stored motion information, there is a low probability that the same motion information as the surrounding motion information will occur.

[0093] As shown in Figure 13, instead of using the prediction candidate popped from the HMVP buffer as it is, an offset can be applied to the motion vector, and then the motion information to which the offset is applied can be used as a prediction candidate. In Figure 13, it can be seen that the popped motion information MV_O and the offset are added and used as a prediction candidate.

[0094] FIG. 14 is a flowchart showing the operation of an encoding device according to an embodiment, and FIG. 15 is a block diagram showing the configuration of an encoding device according to an embodiment.

[0095] The encoding apparatus according to Figures 14 and 15 may perform operations corresponding to those of the decoding apparatus according to Figures 16 and 17. Therefore, the operations of the decoding apparatus described later in Figures 16 and 17 may be similarly applied to the encoding apparatus according to Figures 14 and 15.

[0096] Each step disclosed in FIG. 14 may be performed by the encoding device 200 disclosed in FIG. 2. More specifically, S1400 to S1450 may be performed by the prediction unit 220 disclosed in FIG. 2, S1460 may be performed by the residual processing unit 230 disclosed in FIG. 2, and S1470 may be performed by the entropy encoding unit 240 disclosed in FIG. 2. Furthermore, the operations of S1400 to S1420 are based on a part of the contents described above in FIG. 3. Therefore, the detailed contents that overlap with the contents described above in FIG. 2 and FIG. 3 will be omitted or simplified.

[0097] As shown in Fig. 15, an encoding device according to an embodiment may include a prediction unit 220, a residual processing unit 230, and an entropy encoding unit 240. However, in some cases, not all of the components shown in Fig. 15 may be essential components of an encoding device, and the encoding device may be realized with more or less components than those shown in Fig. 15.

[0098] In an encoding device according to one embodiment, the prediction unit 220, the residual processing unit 230, and the entropy encoding unit 240 may each be implemented on a separate chip, or at least two or more components may be implemented on a single chip.

[0099] An encoding device according to an embodiment may construct an AMVP candidate list including at least one AMVP candidate for a current block (S1400). More specifically, a prediction unit 220 of the encoding device may construct an AMVP candidate list including at least one AMVP candidate for a current block.

[0100] An encoding device according to an embodiment may derive an HMVP candidate list for the current block including HMVP candidates for the current block (S1410). More specifically, a prediction unit 220 of the encoding device may derive an HMVP candidate list for the current block including HMVP candidates for the current block.

[0101] An encoding device according to an embodiment may select at least one HMVP candidate from the HMVP candidates in the HMVP candidate list (S1420). More specifically, a prediction unit 220 of the encoding device may select at least one HMVP candidate from the HMVP candidates in the HMVP candidate list. In one example, selecting at least one HMVP candidate may be interpreted as popping at least one HMVP candidate.

[0102] The encoding device according to an embodiment may add the at least one HMVP candidate to the AMVP candidate list to derive an updated AMVP candidate list (S1430). More specifically, the prediction unit 220 of the encoding device may add the at least one HMVP candidate to the AMVP candidate list to derive an updated AMVP candidate list.

[0103] An encoding device according to an embodiment may derive motion information for the current block based on the updated AMVP candidate list (S1440). More specifically, a prediction unit 220 of the encoding device may derive motion information for the current block based on the updated AMVP candidate list.

[0104] According to an embodiment, the encoding device may derive a prediction sample for the current block based on the motion information for the current block (S1450). More specifically, the prediction unit 220 of the encoding device may derive a prediction sample for the current block based on the motion information for the current block.

[0105] According to an embodiment, the encoding device may derive a residual sample for the current block based on the predicted sample for the current block (S1460). More specifically, the residual processor 230 of the encoding device may derive a residual sample for the current block based on the predicted sample for the current block.

[0106] The encoding device according to an embodiment may encode image information including information about the residual samples (S1470). More specifically, the entropy encoding unit 240 of the encoding device may encode image information including information about the residual samples.

[0107] In one embodiment, the step of selecting at least one HMVP candidate may include the steps of applying a pruning process to one of the HMVP candidates based on the at least one AMVP candidate in the AMVP candidate list, and determining whether to add the HMVP candidate to the AMVP candidate list based on the pruning process.

[0108] In one embodiment, selecting the at least one HMVP candidate may include selecting the at least one HMVP candidate from among the HMVP candidates based on an HMVP candidate list index of the HMVP candidate.

[0109] In one embodiment, the method may be characterized in that a pruning process based on the at least one AMVP candidate is not applied to each of the at least one HMVP candidate.

[0110] In one embodiment, the at least one HMVP candidate may include an HMVP candidate having the smallest HMVP candidate list index among the HMVP candidates.

[0111] According to an embodiment, an encoding device may derive sampled HMVP candidates by applying sampling to HMVP candidates, and the at least one HMVP candidate may be selected from the sampled HMVP candidates based on an HMVP candidate list index of the sampled HMVP candidate.

[0112] According to an embodiment, the encoding apparatus may update the HMVP candidate list based on the motion information for the current block, and the updated HMVP candidate list may be used when deriving motion information for a block to be encoded next to the current block.

[0113] According to the encoding device and operating method of the encoding device of Figures 14 and 15, the encoding device can construct an AMVP candidate list including at least one AMVP candidate for a current block (S1400), derive an HMVP candidate list for the current block including an HMVP candidate for the current block (S1410), select at least one HMVP candidate from the HMVP candidates in the HMVP candidate list (S1420), add the at least one HMVP candidate to the AMVP candidate list to derive an updated AMVP candidate list (S1430), derive motion information for the current block based on the updated AMVP candidate list (S1440), derive a predictive sample for the current block based on the motion information for the current block (S1450), derive a residual sample for the current block based on the predictive sample for the current block (S1460), and encode image information including information about the residual sample (S1470). That is, when prediction candidates are constructed based on HMVP in inter prediction, the pruning process can be omitted, thereby preventing an increase in complexity due to the pruning process.

[0114] FIG. 16 is a flowchart showing the operation of a decoding device according to an embodiment, and FIG. 17 is a block diagram showing the configuration of a decoding device according to an embodiment.

[0115] Each step disclosed in Fig. 16 may be performed by the decoding device 300 disclosed in Fig. 3. More specifically, S1600 to S1650 may be performed by the prediction unit 330 disclosed in Fig. 3, and S1660 may be performed by the addition unit 340 disclosed in Fig. 3. Furthermore, the operations of S1600 to S1660 are based on a part of the contents described above in Figs. 4 to 13. Therefore, the description of specific contents that overlap with the contents described above in Figs. 3 to 13 will be omitted or simplified.

[0116] As shown in Fig. 17, a decoding device according to an embodiment may include an entropy decoding unit 310, a prediction unit 330, and an addition unit 340. However, in some cases, all of the components shown in Fig. 17 may not be essential components of the decoding device, and the decoding device may be realized with more or less components than those shown in Fig. 17.

[0117] In the decoding device according to an embodiment, the prediction unit 330 and the addition unit 340 may be implemented on separate chips, or at least two of the components may be implemented on a single chip.

[0118] A decoding device according to an embodiment may generate an AMVP candidate list including at least one AMVP candidate for a current block (S1600). More specifically, a prediction unit 330 of the decoding device may generate an AMVP candidate list including at least one AMVP candidate for a current block.

[0119] A decoding device according to an embodiment may derive a history-based motion vector prediction (HMVP) candidate list for the current block including HMVP candidates for the current block (S1610). More specifically, a prediction unit 330 of the decoding device may derive a HMVP candidate list for the current block including HMVP candidates for the current block.

[0120] A decoding device according to an embodiment may select at least one HMVP candidate from the HMVP candidates in the HMVP candidate list (S1620). More specifically, a prediction unit 330 of the decoding device may select at least one HMVP candidate from the HMVP candidates in the HMVP candidate list. In one example, selecting at least one HMVP candidate may be interpreted as popping at least one HMVP candidate.

[0121] In one embodiment, the decoding device may add the at least one HMVP candidate to the AMVP candidate list to derive an updated AMVP candidate list (S1630). More specifically, the prediction unit 330 of the decoding device may add the at least one HMVP candidate to the AMVP candidate list to derive an updated AMVP candidate list.

[0122] According to an embodiment, the decoding device may derive motion information for the current block based on the updated AMVP candidate list (S1640). More specifically, the prediction unit 330 of the decoding device may derive motion information for the current block based on the updated AMVP candidate list.

[0123] According to an embodiment, the decoding device may derive a prediction sample for the current block based on motion information for the current block (S1650). More specifically, the prediction unit 330 of the decoding device may derive a prediction sample for the current block based on the motion information for the current block.

[0124] According to an embodiment, the decoding device may generate reconstructed samples for the current block based on the predicted samples for the current block (S1660). More specifically, the adder 340 of the decoding device may generate reconstructed samples for the current block based on the predicted samples for the current block.

[0125] In one embodiment, the step of selecting at least one HMVP candidate may include the steps of applying a pruning process to one of the HMVP candidates based on the at least one AMVP candidate in the AMVP candidate list, and determining whether to add the HMVP candidate to the AMVP candidate list based on the pruning process.

[0126] In one embodiment, selecting the at least one HMVP candidate may include selecting the at least one HMVP candidate from among the HMVP candidates based on an HMVP candidate list index of the HMVP candidate.

[0127] In one embodiment, the method may be characterized in that a pruning process based on the at least one AMVP candidate is not applied to each of the at least one HMVP candidate.

[0128] In one embodiment, the at least one HMVP candidate may include an HMVP candidate having the smallest HMVP candidate list index among the HMVP candidates.

[0129] According to an embodiment, a decoding device may derive sampled HMVP candidates by applying sampling to the HMVP candidates, and the at least one HMVP candidate may be selected from the sampled HMVP candidates based on an HMVP candidate list index of the sampled HMVP candidate.

[0130] According to an embodiment, the decoding device may update the HMVP candidate list based on the motion information for the current block, and the updated HMVP candidate list may be used when deriving motion information for a block to be decoded next to the current block.

[0131] According to the decoding device and the operating method of the decoding device disclosed in Figures 16 and 17, the decoding device constructs an AMVP candidate list including at least one AMVP candidate for a current block (S1600), derives an HMVP (history-based motion vector prediction) candidate list for the current block including HMVP candidates for the current block (S1610), selects at least one HMVP candidate from the HMVP candidates in the HMVP candidate list (S1620), adds the at least one HMVP candidate to the AMVP candidate list to derive an updated AMVP candidate list (S1630), derives motion information for the current block based on the updated AMVP candidate list (S1640), derives a predicted sample for the current block based on the motion information for the current block (S1650), and generates a reconstructed sample for the current block based on the predicted sample for the current block (S1660). That is, when prediction candidates are constructed based on HMVP in inter prediction, the pruning process can be omitted, thereby preventing an increase in complexity due to the pruning process.

[0132] In the above-described embodiments, the method is described based on a flowchart as a series of steps or blocks, but the present disclosure is not limited to the order of steps, and certain steps may occur in a different order or simultaneously with different steps than those described above. Also, those skilled in the art will understand that the steps shown in the flowchart are not exclusive, and other steps may be included, or one or more steps of the flowchart may be deleted without affecting the scope of the present disclosure.

[0133] The above-described method according to the present disclosure can be realized in software form, and the encoding device and / or decoding device according to the present disclosure can be included in an image processing device such as a TV, a computer, a smartphone, a set-top box, or a display device.

[0134] When the embodiments and the like in the present disclosure are implemented in software, the above-described methods can be implemented in modules (processes, functions, etc.) that perform the above-described functions. The modules can be stored in a memory and executed by a processor. The memory can be inside or outside the processor and can be coupled to the processor in various well-known ways. The processor can include an application-specific integrated circuit (ASIC), other chipsets, logic circuits, and / or data processing devices. The memory can include a read-only memory (ROM), a random access memory (RAM), a flash memory, a memory card, a storage medium, and / or other storage devices. That is, the embodiments and the like described in the present disclosure can be implemented and performed on a processor, microprocessor, controller, or chip. For example, the functional units illustrated in each drawing can be implemented and performed on a computer, processor, microprocessor, controller, or chip. In this case, information (e.g., information on instructions) or algorithms for implementation can be stored in a digital storage medium.

[0135] In addition, the decoding device and the encoding device to which the present disclosure is applied may be included in a multimedia broadcast transmitting / receiving device, a mobile communication terminal, a home cinema video device, a digital cinema video device, a surveillance camera, a video interactive device, a real-time communication device such as a video communication, a mobile streaming device, a storage medium, a camcorder, a custom video (VoD) service providing device, an over-the-top video (OTT) device, an internet streaming service providing device, a three-dimensional (3D) video device, a virtual reality (VR) device, an argument reality (AR) device, an image telephone video device, a transportation means terminal (e.g., a vehicle (including an autonomous vehicle) terminal, an airplane terminal, a ship terminal, etc.), and a medical video device, and may be used to process a video signal or a data signal. For example, an over-the-top video (OTT) device may include a game console, a Blu-ray player, an internet-connected TV, a home theater system, a smartphone, a tablet PC, a digital video recorder (DVR), etc.

[0136] In addition, the processing method to which the present disclosure is applied may be produced in the form of a program executed by a computer and may be stored in a computer-readable recording medium. Multimedia data having a data structure according to the present disclosure may also be stored in a computer-readable recording medium. The computer-readable recording medium includes any kind of storage device and distributed storage device in which computer-readable data is stored. The computer-readable recording medium may include, for example, a Blu-ray Disc (BD), a Universal Serial Bus (USB), a ROM, a PROM, an EPROM, an EEPROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, and an optical data storage device. The computer-readable recording medium may also include a medium realized in the form of a carrier wave (e.g., transmission via the Internet). Also, a bit stream generated by the encoding method may be stored in a computer-readable recording medium or transmitted via a wired or wireless communication network.

[0137] Furthermore, the embodiments of the present disclosure may be implemented in a computer program product by program code, which may be executed by a computer in accordance with the embodiments of the present disclosure. The program code may be stored on a computer readable carrier.

[0138] FIG. 18 illustrates an example of a content streaming system to which the disclosure of this document may be applied.

[0139] As shown in FIG. 18, a content streaming system to which the present disclosure is applied can broadly include an encoding server, a streaming server, a web server, a media storage, a user device, and a multimedia input device.

[0140] The encoding server compresses content input from a multimedia input device such as a smartphone, camera, camcorder, etc. into digital data to generate a bitstream and transmits the bitstream to the streaming server. As another example, if a multimedia input device such as a smartphone, camera, camcorder, etc. directly generates a bitstream, the encoding server may be omitted.

[0141] The bitstream may be generated by an encoding method or a bitstream generation method to which the present disclosure is applied, and the streaming server may temporarily store the bitstream in the process of transmitting or receiving the bitstream.

[0142] The streaming server transmits multimedia data to a user device based on a user request via a web server, and the web server acts as a medium to inform the user of available services. When a user requests a desired service from the web server, the web server transmits the request to the streaming server, and the streaming server transmits multimedia data to the user. In this case, the content streaming system may include a separate control server, and in this case, the control server controls commands / responses between devices in the content streaming system.

[0143] The streaming server may receive content from a media repository and / or an encoding server. For example, when the content is received from the encoding server, the content may be received in real time. In this case, the streaming server may store the bitstream for a certain period of time in order to provide a smooth streaming service.

[0144] Examples of the user devices include mobile phones, smart phones, laptop computers, digital broadcasting terminals, personal digital assistants (PDAs), portable multimedia players (PMPs), navigation systems, slate PCs, tablet PCs, ultrabooks, wearable devices (e.g., smartwatches, smart glass, head mounted displays (HMDs)), digital TVs, desktop computers, digital signage, etc.

[0145] Each server in the content streaming system can be operated as a distributed server, in which case data received by each server can be processed in a distributed manner.

Claims

1. An image decoding method performed by a decoding device, receiving residual information from a bitstream; constructing an AMVP candidate list including at least one AMVP candidate for a current block in a current picture; deriving a history-based motion vector prediction (HMVP) buffer for the current block, the history-based motion vector prediction (HMVP) buffer including HMVP candidates for the current block; deriving an updated AMVP candidate list by adding at least one additional candidate, the at least one additional candidate being derived using at least one HMVP candidate selected from among the HMVP candidates in the HMVP buffer; deriving motion information for the current block based on the updated AMVP candidate list; deriving a prediction sample for the current block based on the motion information for the current block; deriving a residual sample for the current block based on the residual information; generating a reconstructed sample for the current block based on the predicted sample for the current block and the residual sample for the current block; the at least one HMVP candidate is selected from among the HMVP candidates based on an HMVP buffer index; The at least one HMVP candidate is selected based on a priority, the priority being based on the HMVP buffer index; An HMVP candidate with a relatively low HMVP buffer index has a higher priority than an HMVP candidate with a relatively high HMVP buffer index; The at least one HMVP candidate is used to derive the at least one additional candidate based on a reference picture of the at least one HMVP candidate being equal to a reference picture of the current block.

2. An image encoding method performed by an encoding device, comprising: constructing an AMVP candidate list including at least one AMVP candidate for a current block in a current picture; deriving a history-based motion vector prediction (HMVP) buffer for the current block, the history-based motion vector prediction (HMVP) buffer including HMVP candidates for the current block; deriving an updated AMVP candidate list by adding at least one additional candidate, the at least one additional candidate being derived using at least one HMVP candidate selected from among the HMVP candidates in the HMVP buffer; deriving motion information for the current block based on the updated AMVP candidate list; deriving a prediction sample for the current block based on the motion information for the current block; deriving a residual sample for the current block based on the predicted sample for the current block; generating residual information based on the residual samples for the current block; encoding image information including the residual information; the at least one HMVP candidate is selected from among the HMVP candidates based on an HMVP buffer index; the at least one HMVP candidate is selected based on a priority order; The priority is based on the HMVP buffer index, An HMVP candidate with a relatively low HMVP buffer index has a higher priority than an HMVP candidate with a relatively high HMVP buffer index; 11. A method for encoding an image, comprising: based on a reference picture of the at least one HMVP candidate being equal to a reference picture of the current block, the at least one HMVP candidate being used to derive the at least one additional candidate.

3. A method for transmitting data for an image, comprising: obtaining a bitstream for the image; transmitting the data including the bitstream; the bitstream is generated based on: constructing an AMVP candidate list including at least one AMVP candidate for a current block in a current picture; deriving a history-based motion vector prediction (HMVP) buffer for the current block including HMVP candidates for the current block; deriving an updated AMVP candidate list by adding at least one additional candidate, the at least one additional candidate being derived using at least one HMVP candidate selected from among the HMVP candidates in the HMVP buffer; deriving motion information for the current block based on the updated AMVP candidate list; deriving predictive samples for the current block based on the motion information for the current block; deriving residual samples for the current block based on the predictive samples for the current block; generating residual information based on the residual samples for the current block; and encoding image information including the residual information; the at least one HMVP candidate is selected from among the HMVP candidates based on an HMVP buffer index; the at least one HMVP candidate is selected based on a priority order; The priority is based on the HMVP buffer index, An HMVP candidate with a relatively low HMVP buffer index has a higher priority than an HMVP candidate with a relatively high HMVP buffer index; The at least one HMVP candidate is used to derive the at least one additional candidate based on a reference picture of the at least one HMVP candidate being equal to a reference picture of the current block.

4. An image decoding method performed by a decoding device, comprising: receiving residual information from a bitstream; constructing an AMVP candidate list including at least one AMVP candidate for a current block in a current picture; deriving a history-based motion vector prediction (HMVP) buffer for the current block, the history-based motion vector prediction (HMVP) buffer including HMVP candidates for the current block; deriving an updated AMVP candidate list by adding at least one additional candidate, the at least one additional candidate being derived using at least one HMVP candidate selected from among the HMVP candidates in the HMVP buffer; deriving motion information for the current block based on the updated AMVP candidate list; deriving a prediction sample for the current block based on the motion information for the current block; deriving a residual sample for the current block based on the residual information; generating a reconstructed sample for the current block based on the predicted sample for the current block and the residual sample for the current block; the at least one HMVP candidate is selected from among the HMVP candidates based on an HMVP buffer index; The at least one HMVP candidate is selected based on a priority, the priority being based on the HMVP buffer index; An HMVP candidate with a relatively low HMVP buffer index has a higher priority than an HMVP candidate with a relatively high HMVP buffer index; The at least one HMVP candidate is used to derive the at least one additional candidate without checking whether motion information of the at least one HMVP candidate is equal to motion information of the at least one AMVP candidate included in the AMVP candidate list, and the checking is based on a reference picture of the at least one HMVP candidate being equal to a reference picture of the current block.

5. An image encoding method performed by an encoding device, comprising: constructing an AMVP candidate list including at least one AMVP candidate for a current block in a current picture; deriving a history-based motion vector prediction (HMVP) buffer for the current block, the history-based motion vector prediction (HMVP) buffer including HMVP candidates for the current block; deriving an updated AMVP candidate list by adding at least one additional candidate, the at least one additional candidate being derived using at least one HMVP candidate selected from among the HMVP candidates in the HMVP buffer; deriving motion information for the current block based on the updated AMVP candidate list; deriving a prediction sample for the current block based on the motion information for the current block; deriving a residual sample for the current block based on the predicted sample for the current block; generating residual information based on the residual samples for the current block; encoding image information including the residual information; the at least one HMVP candidate is selected from among the HMVP candidates based on an HMVP buffer index; the at least one HMVP candidate is selected based on a priority order; The priority is based on the HMVP buffer index, An HMVP candidate with a relatively low HMVP buffer index has a higher priority than an HMVP candidate with a relatively high HMVP buffer index; The at least one HMVP candidate is used to derive the at least one additional candidate without checking whether motion information of the at least one HMVP candidate is equal to motion information of the at least one AMVP candidate included in the AMVP candidate list, the checking being based on a reference picture of the at least one HMVP candidate being equal to a reference picture of the current block.

6. A method for transmitting data for an image, comprising: obtaining a bitstream for the image; transmitting the data including the bitstream; the bitstream is generated based on: constructing an AMVP candidate list including at least one AMVP candidate for a current block in a current picture; deriving a history-based motion vector prediction (HMVP) buffer for the current block including HMVP candidates for the current block; deriving an updated AMVP candidate list by adding at least one additional candidate, the at least one additional candidate being derived using at least one HMVP candidate selected from among the HMVP candidates in the HMVP buffer; deriving motion information for the current block based on the updated AMVP candidate list; deriving predictive samples for the current block based on the motion information for the current block; deriving residual samples for the current block based on the predictive samples for the current block; generating residual information based on the residual samples for the current block; and encoding image information including the residual information; the at least one HMVP candidate is selected from among the HMVP candidates based on an HMVP buffer index; the at least one HMVP candidate is selected based on a priority order; The priority is based on the HMVP buffer index, An HMVP candidate with a relatively low HMVP buffer index has a higher priority than an HMVP candidate with a relatively high HMVP buffer index; The at least one HMVP candidate is used to derive the at least one additional candidate without checking whether motion information of the at least one HMVP candidate is equal to motion information of the at least one AMVP candidate included in the AMVP candidate list, and the checking is based on a reference picture of the at least one HMVP candidate being equal to a reference picture of the current block.