Image encoding / decoding method, bitstream transmission method, and recording medium storing bitstream.

Abstract

The present invention provides an image encoding / decoding method, a bitstream transmission method, and a computer-readable recording medium for storing bitstreams. [Solution] The image decoding method according to the present disclosure is an image decoding method performed by an image decoding device, and includes the steps of: configuring a list of prediction candidates for the current block; inducing an error value for the current block based on whether or not the reference picture referenced by the prediction candidates in the list of prediction candidates is a resampled reference picture; and reordering the order of the prediction candidates in the list of prediction candidates based on the error value, wherein the error value is inducing a predetermined value based on whether or not the reference picture is a resampled reference picture.

Description

Technical Field

[0001] The present disclosure relates to an image encoding / decoding method, a method for transmitting a bitstream, and a recording medium storing a bitstream, and more particularly, to a method for setting a template matching error value according to the application of reference picture sampling.

Background Art

[0002] Recently, the demand for high-resolution, high-quality images, such as HD (High Definition) images and UHD (Ultra High Definition) images, has been increasing in various fields. As image data becomes higher in resolution and quality, the amount of information or bits to be transmitted relatively increases compared to conventional image data. The increase in the amount of information or bits to be transmitted results in an increase in transmission costs and storage costs.

[0003] Thus, there is a need for a highly efficient image compression technique for effectively transmitting, storing, and reproducing information of high-resolution, high-quality images.

Summary of the Invention

Problems to be Solved by the Invention

[0004] An object of the present disclosure is to provide an image encoding / decoding method and apparatus with improved encoding / decoding efficiency.

[0005] Another object of the present disclosure is to provide a method for performing inter prediction using template matching.

[0006] Another object of the present disclosure is to provide a method for setting a template matching error value according to the application of reference picture sampling.

[0007] Furthermore, this disclosure aims to set a relatively lower priority for a prediction candidate without calculating its template matching error value if the reference picture of the prediction candidate is a reference picture to which reference picture resampling has been applied.

[0008] Furthermore, this disclosure aims to provide a non-temporary computer-readable recording medium for storing a bitstream generated by the image encoding method provided herein.

[0009] Furthermore, this disclosure aims to provide a non-temporary computer-readable recording medium for storing a bitstream that is received by the image decoding device provided herein, decoded, and used to restore an image.

[0010] Furthermore, this disclosure aims to provide a method for transmitting a bitstream generated by the image encoding method according to this disclosure.

[0011] The technical problems that this disclosure seeks to solve are not limited to those described above, and other technical problems not mentioned above will be clearly understood by a person with ordinary skill in the art to which this disclosure pertains from the following description. [Means for solving the problem]

[0012] An image decoding method according to one aspect of the present disclosure is an image decoding method performed by an image decoding device, comprising the steps of: configuring a list of prediction candidates for a current block; inducing an error value for the current block based on whether or not the reference picture referenced by the prediction candidates in the list of prediction candidates is a resampled reference picture; and reordering the order of the prediction candidates in the list of prediction candidates based on the error value, wherein the error value is inducing a predetermined value based on whether or not the reference picture is a resampled reference picture.

[0013] An image coding method according to another aspect of the present disclosure is an image coding method performed by an image coding device, comprising the steps of: configuring a list of prediction candidates for a current block; inducing an error value for the current block based on whether or not the reference picture referenced by the prediction candidates in the list of prediction candidates is a resampled reference picture; and reordering the order of the prediction candidates in the list of prediction candidates based on the error value, wherein the error value is induced to a predetermined value based on whether or not the reference picture is a resampled reference picture.

[0014] A computer-readable recording medium according to another aspect of the present disclosure can store a bitstream generated by an image encoding method or apparatus of the present disclosure.

[0015] Another aspect of the transmission method of the present disclosure can transmit a bitstream generated by the image encoding method or apparatus of the present disclosure.

[0016] The features described above, which are a brief summary of this disclosure, are merely illustrative examples of the detailed description of this disclosure described below and do not limit the scope of this disclosure. [Effects of the Invention]

[0017] According to this disclosure, an image encoding / decoding method and apparatus with improved encoding / decoding efficiency can be provided.

[0018] Furthermore, this disclosure provides a method for performing inter prediction using template matching.

[0019] Furthermore, according to this disclosure, since a lower rank is assigned to the prediction candidate of the reference picture to which reference picture resampling is applied, errors that occur during the reference picture resampling process are not reflected in the prediction process of the current block, and the prediction of the current block can be achieved more accurately.

[0020] Also, according to the present disclosure, a non-temporary computer-readable recording medium for storing a bitstream generated by the image encoding method according to the present disclosure can be provided.

[0021] Also, according to the present disclosure, a non-temporary computer-readable recording medium for storing a bitstream that is received by the image decoding apparatus according to the present disclosure, decoded, and used for restoring an image can be provided.

[0022] Also, according to the present disclosure, a method for transmitting a bitstream generated by an image encoding method can be provided.

[0023] The effects obtained in the present disclosure are not limited to the above-described effects, and other effects not described above will be clearly understood by those having ordinary knowledge in the technical field to which the present disclosure pertains from the following description.

Brief Description of Drawings

[0024] [Figure 1] It is a diagram schematically showing a video coding system to which an embodiment according to the present disclosure can be applied. [Figure 2] It is a diagram schematically showing an image encoding apparatus to which an embodiment according to the present disclosure can be applied. [Figure 3] It is a diagram schematically showing an image decoding apparatus to which an embodiment according to the present disclosure can be applied. [Figure 4] It is a diagram schematically showing an inter prediction unit of an image encoding apparatus. [Figure 5] It is a flowchart showing a method for encoding an image based on inter prediction. [Figure 6] It is a diagram schematically showing an inter prediction unit of an image decoding apparatus. [Figure 7] It is a flowchart showing a method for decoding an image based on inter prediction. [Figure 8] It is a flowchart showing an inter prediction method. [Figure 9]This is a diagram to explain the MMVD mode. [Figure 10] This is a diagram to explain affine mode. [Figure 11] This is a diagram to explain affine mode. [Figure 12] This is a diagram to explain affine mode. [Figure 13] This is a diagram illustrating template matching and rearrangement based on template matching. [Figure 14] This is a diagram illustrating template matching and rearrangement based on template matching. [Figure 15] This is a diagram illustrating template matching and rearrangement based on template matching. [Figure 16] This is a flowchart showing an image encoding / decoding method according to one embodiment of the present invention. [Figure 17] This is a flowchart showing an image encoding / decoding method according to another embodiment of the present disclosure. [Figure 18] This is a flowchart showing an image encoding / decoding method according to another embodiment of the present disclosure. [Figure 19] This is a flowchart showing an image encoding / decoding method according to another embodiment of the present disclosure. [Figure 20] This is a flowchart showing an image encoding / decoding method according to another embodiment of the present disclosure. [Figure 21] This figure illustrates a content streaming system to which the embodiments described herein can be applied. [Modes for carrying out the invention]

[0025] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the attached drawings, so that they can be easily implemented by a person with ordinary skill in the art to which the present disclosure pertains. However, the present disclosure can be implemented in a variety of different forms and is not limited to the embodiments described herein.

[0026] In describing embodiments of this disclosure, if it is determined that a specific description of a known configuration or function would obscure the gist of this disclosure, such detailed description will be omitted. In the drawings, parts unrelated to the description of this disclosure will be omitted, and similar parts will be denoted by the same reference numerals.

[0027] In this disclosure, when one component is described as being “connected,” “joined,” or “linked” to another component, this can include not only direct connections but also indirect connections where another component exists between them. Furthermore, when one component is described as “containing” or “having” another component, this means, unless otherwise stated to the contrary, that it may include another component rather than excluding it.

[0028] In this disclosure, terms such as "first," "second," etc., are used solely for the purpose of distinguishing one component from another, and do not limit the order or importance of the components unless otherwise specified. Therefore, within the scope of this disclosure, a first component in one embodiment may be called a second component in another embodiment, and similarly, a second component in one embodiment may be called a first component in another embodiment.

[0029] In this disclosure, components that are distinguished from each other are used to clearly describe their respective characteristics and do not necessarily mean that the components are separate. In other words, multiple components may be integrated to constitute a single hardware or software unit, or a single component may be distributed to constitute multiple hardware or software units. Therefore, such integrated or distributed embodiments are also included in the scope of this disclosure, without needing to be specifically mentioned.

[0030] In this disclosure, the components described in various embodiments are not necessarily essential components, and some may be optional components. Therefore, embodiments consisting of a subset of the components described in one embodiment are also included in the scope of this disclosure. Furthermore, embodiments that include additional components in addition to the components described in various embodiments are also included in the scope of this disclosure.

[0031] This disclosure relates to the encoding and decoding of images, and the terms used in this disclosure may have their ordinary meanings in the art to which this disclosure pertains, unless otherwise defined herein.

[0032] In this disclosure, "picture" generally means a unit representing any one image within a specific time period, and "slice / tile" is an encoding unit that constitutes part of a picture, and a single picture can consist of one or more slices / tiles. Furthermore, a slice / tile may contain one or more CTUs (coding tree units).

[0033] In this disclosure, “pixel” or “pel” may mean the smallest unit that constitutes a picture (or image). The term “sample” may also be used as a counterpart to pixel. A sample may generally represent a pixel or a pixel value, or it may represent only the pixel / pixel value of the luma component, or only the pixel / pixel value of the chroma component.

[0034] In this disclosure, “unit” can refer to a basic unit of image processing. A unit may include at least one of a specific region of a picture and information associated with that region. A unit may be used interchangeably with terms such as “sample array,” “block,” or “area,” as it may be used. Generally, an M×N block may include a set (or array) of samples (or sample arrays) or transform coefficients consisting of M columns and N rows.

[0035] In this disclosure, “current block” can mean any one of the following: “current coding block,” “current coding unit,” “block to encode,” “block to decode,” or “block to process.” If prediction is performed, “current block” can mean “current prediction block” or “block to predict.” If transformation (inverse transformation) / quantization (inverse quantization) is performed, “current block” can mean “current transformation block” or “block to transform.” If filtering is performed, “current block” can mean “block to filter.”

[0036] Furthermore, in this disclosure, "current block" may mean the block containing all of the rumor component blocks and chroma component blocks, or the "rumor block of the current block," unless there is an explicit mention of a chroma block. The rumor component block of the current block may be expressed with an explicit mention of a rumor component block, such as "rumor block" or "current rumor block." Similarly, the chroma component block of the current block may be expressed with an explicit mention of a chroma component block, such as "chroma block" or "current chroma block."

[0037] In this disclosure, " / " and "," may be interpreted as "and / or." For example, "A / B" and "A, B" may be interpreted as "A and / or B." Also, "A / B / C" and "A, B, C" may mean "at least one of A, B and / or C."

[0038] In this disclosure, “or” may be interpreted as “and / or.” For example, “A or B” may mean 1) “A” only, 2) “B” only, or 3) “A and B.” Alternatively, in this disclosure, “or” may mean “additionally or alternatively.”

[0039] Overview of the video coding system

[0040] Figure 1 is a schematic diagram showing a video coding system to which the embodiments of this disclosure can be applied.

[0041] A video coding system according to one embodiment may include an encoding device 10 and a decoding device 20. The encoding device 10 can transmit encoded video and / or image information or data to the decoding device 20 via a digital storage medium or network in file or streaming format.

[0042] An encoding device 10 according to one embodiment may include a video source generation unit 11, an encoding unit 12, and a transmission unit 13. A decoding device 20 according to one embodiment may include a receiving unit 21, a decoding unit 22, and a rendering unit 23. The encoding unit 12 may be called a video / image encoding unit, and the decoding unit 22 may be called a video / image decoding unit. The transmission unit 13 may be included in the encoding unit 12. The receiving unit 21 may be included in the decoding unit 22. The rendering unit 23 may also include a display unit, which may be configured as a separate device or external component.

[0043] The video source generation unit 11 can acquire video / images through processes such as video / image capture, synthesis, or generation. The video source generation unit 11 may include a video / image capture device and / or a video / image generation device. The video / image capture device may include, for example, one or more cameras, or a video / image archive containing previously captured video / images. The video / image generation device may include, for example, a computer, tablet, and smartphone, and may generate video / images (electronically). For example, virtual video / images may be generated via a computer, in which case the video / image capture process may be replaced by a process in which the relevant data is generated.

[0044] The encoding unit 12 can encode the input video / image. The encoding unit 12 can perform a series of steps such as prediction, transformation, and quantization for compression and encoding efficiency. The encoding unit 12 can output the encoded data (encoded video / image information) in bitstream format.

[0045] The transmission unit 13 can acquire encoded video / image information or data output in bitstream format and transmit it in file or streaming format to the receiving unit 21 of the decoding device 20 or other external object via a digital storage medium or network. The digital storage medium can include various storage media such as USB, SD, CD, DVD, Blu-ray®, HDD, and SSD. The transmission unit 13 can include elements for generating media files via a predetermined file format and elements for transmission via a broadcast / communication network. The transmission unit 13 can be provided as a transmission device separate from the encoding device 12, in which case the transmission device can include at least one processor that acquires encoded video / image information or data output in bitstream format, and a transmission unit that transmits it in file or streaming format. The receiving unit 21 can extract / receive the bitstream from the storage medium or network and transmit it to the decoding unit 22.

[0046] The decoding unit 22 can decode the video / image by performing a series of steps such as inverse quantization, inverse transform, and prediction, corresponding to the operation of the encoding unit 12.

[0047] The rendering unit 23 can render the decoded video / image. The rendered video / image can be displayed via the display unit.

[0048] Overview of Image Encoding Devices

[0049] Figure 2 is a schematic diagram showing an image encoding device to which the embodiments of this disclosure can be applied.

[0050] As shown in Figure 2, the image coding device 100 may include an image splitting unit 110, a subtraction unit 115, a transformation unit 120, a quantization unit 130, an inverse quantization unit 140, an inverse transformation unit 150, an addition unit 155, a filtering unit 160, a memory 170, an inter-prediction unit 180, an intra-prediction unit 185, and an entropy coding unit 190. The inter-prediction unit 180 and the intra-prediction unit 185 can together be called the "prediction unit". The transformation unit 120, the quantization unit 130, the inverse quantization unit 140, and the inverse transformation unit 150 may be included in a residual processing unit. The residual processing unit may further include a subtraction unit 115.

[0051] All or at least some of the multiple components constituting the image encoding device 100 can be implemented by a single hardware component (e.g., an encoder or processor) depending on the embodiment. Furthermore, the memory 170 may include a DPB (decoded picture buffer) and can be implemented by a digital storage medium.

[0052] The image splitting unit 110 can split an input image (or picture, frame) input to the image encoding device 100 into one or more processing units. For example, the processing units may be called coding units (CUs). Coding units can be obtained by recursively splitting a coding tree unit (CTU) or the largest coding unit (LCU) using a QT / BT / TT (Quad-tree / binary-tree / ternary-tree) structure. For example, a single coding unit can be split into multiple coding units of deeper depth based on a quad-tree structure, a binary-tree structure and / or a ternary-tree structure. For the splitting of coding units, a quad-tree structure may be applied first, followed by a binary-tree structure and / or a ternary-tree structure. Based on the final coding unit that cannot be further split, the coding procedure according to this disclosure can be performed. The largest coding unit can be used as the final coding unit, or a lower-depth coding unit obtained by dividing the largest coding unit can be used as the final coding unit. Here, the coding procedure may include procedures such as prediction, transformation, and / or restoration, as described later. As another example, the processing units of the coding procedure may be prediction units (PU) or transformation units (TU). The prediction unit and the transformation unit may be divided or partitioned from the final coding unit, respectively. The prediction unit may be a unit of sample prediction, and the transformation unit may be a unit that derives transformation coefficients and / or a unit that derives a residual signal from transformation coefficients.

[0053] The prediction unit (inter-prediction unit 180 or intra-prediction unit 185) can make predictions for the block to be processed (current block) and generate a predicted block that includes prediction samples for the current block. The prediction unit can determine whether intra-prediction or inter-prediction is applied to the current block or on a CU basis. The prediction unit can generate various information regarding the prediction of the current block and transmit it to the entropy coding unit 190. The prediction information can be encoded by the entropy coding unit 190 and output in bitstream format.

[0054] The intra-prediction unit 185 can predict the current block by referring to a sample in the current picture. The referenced sample may be located in the vicinity (neighbor) or at a distance from the current block, according to the intra-prediction mode and / or intra-prediction technique. The intra-prediction mode may include a plurality of non-directional modes and a plurality of directional modes. The non-directional modes may include, for example, a DC mode and a Planar mode. The directional modes may include, for example, 33 directional prediction modes or 65 directional prediction modes, depending on the degree of fineness of the prediction direction. However, this is merely an example, and more or fewer directional prediction modes may be used depending on the settings. The intra-prediction unit 185 may also determine the prediction mode to be applied to the current block using the prediction modes applied to the surrounding blocks.

[0055] The interprediction unit 180 can derive a predicted block relative to the current block based on a reference block (reference sample array) identified by motion vectors on the reference picture. In this case, in order to reduce the amount of motion information transmitted in interprediction mode, motion information can be predicted in units of blocks, subblocks, or samples based on the correlation of motion information between the surrounding blocks and the current block. The motion information may include motion vectors and reference picture indices. The motion information may further include interprediction direction information (L0 prediction, L1 prediction, Bi prediction, etc.). In the case of interprediction, the surrounding blocks may include spatial neighboring blocks present in the current picture and temporal neighboring blocks present in the reference picture. The reference picture containing the reference block and the reference picture containing the temporal neighboring block may be the same or different from each other. The temporal neighboring block may be called a collocated reference block, collocated CU (colCU), etc. The reference picture containing the temporal neighboring block may be called a collocated picture (colPic). For example, the interpretation unit 180 can construct a motion information candidate list based on surrounding blocks and generate information indicating which candidate is used to derive the motion vector and / or reference picture index of the current block. Interpretation can be performed based on various prediction modes; for example, in skip mode and merge mode, the interpretation unit 180 can use the motion information of surrounding blocks as the motion information of the current block. In skip mode, unlike merge mode, the residual signal may not be transmitted.In motion vector prediction (MVP) mode, the motion vector of the surrounding block is used as the motion vector predictor, and the motion vector of the current block can be signaled by encoding the motion vector difference and an indicator for the motion vector predictor. The motion vector difference can represent the difference between the motion vector of the current block and the motion vector predictor.

[0056] The prediction unit can generate a prediction signal based on various prediction methods and / or techniques described later. For example, the prediction unit can apply intra-prediction or inter-prediction to predict the current block, and can also apply intra-prediction and inter-prediction simultaneously. A prediction method that applies intra-prediction and inter-prediction simultaneously to predict the current block can be called CIIP (combined inter and intra prediction). The prediction unit can also perform intra-block copy (IBC) to predict the current block. Intra-block copy can be used for content image / video coding such as in games, for example, in SCC (screen content coding). IBC is a method of predicting the current block using a reference block that has already been restored in the current picture at a predetermined distance from the current block. When IBC is applied, the position of the reference block in the current picture can be encoded as a vector (block vector) corresponding to the predetermined distance. IBC basically performs prediction within the current picture, but it can be performed similarly to inter-prediction in that it derives the reference block within the current picture. In other words, IBC can use at least one of the interpretation techniques described in this disclosure.

[0057] The predicted signal generated by the prediction unit can be used to generate a reconstructed signal or a residual signal. The subtraction unit 115 can generate a residual signal (residual block, residual sample array) by subtracting the predicted signal output from the prediction unit (predicted block, predicted sample array) from the input image signal (original block, original sample array). The generated residual signal can be transmitted to the conversion unit 120.

[0058] The transformation unit 120 can generate transformation coefficients by applying transformation techniques to the residual signal. For example, the transformation techniques may include at least one of the following: DCT (Discrete Cosine Transform), DST (Discrete Sine Transform), KLT (Karhunen-Loeve Transform), GBT (Graph-Based Transform), or CNT (Conditionally Non-linear Transform). Here, GBT refers to a transformation obtained from a graph, where the relationship information between pixels is represented by a graph. CNT refers to a transformation obtained by generating a prediction signal using all previously reconstructed pixels. The transformation process can be applied to pixel blocks of the same size and square shape, or to non-square, variable-sized blocks.

[0059] The quantization unit 130 can quantize the conversion coefficients and transmit them to the entropy coding unit 190. The entropy coding unit 190 can encode the quantized signal (information about the quantized conversion coefficients) and output it in bitstream format. The information about the quantized conversion coefficients can be called residual information. The quantization unit 130 can rearrange the block-form quantized conversion coefficients into a one-dimensional vector format based on the coefficient scan order, and can also generate information about the quantized conversion coefficients based on the one-dimensional vector format of the quantized conversion coefficients.

[0060] The entropy coding unit 190 can perform various coding methods, such as exponential Golomb, CAVLC (context-adaptive variable length coding), and CABAC (context-adaptive binary arithmetic coding). In addition to the quantized conversion coefficients, the entropy coding unit 190 can also encode information necessary for video / image restoration (e.g., the values ​​of syntax elements) together or separately. The encoded information (e.g., encoded video / image information) can be transmitted or stored in bitstream format in units of NAL (network abstraction layer) units. The video / image information may further include information about various parameter sets, such as adaptive parameter sets (APS), picture parameter sets (PPS), sequence parameter sets (SPS), or video parameter sets (VPS). The video / image information may also further include general constraint information. The signaling information, transmitted information and / or syntax elements referred to in this disclosure may be encoded via the encoding procedure described above and included in the bitstream.

[0061] The bitstream can be transmitted over a network or stored on a digital storage medium. Here, the network may include broadcast networks and / or communication networks, and the digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, and SSD. A transmission unit (not shown) for transmitting the signal output from the entropy encoding unit 190 and / or a storage unit (not shown) for storing it may be provided as an internal / external element of the image encoding device 100, or the transmission unit may be provided as a component of the entropy encoding unit 190.

[0062] The quantized conversion coefficients output from the quantization unit 130 can be used to generate a residual signal. For example, by applying inverse quantization and inverse transformation to the quantized conversion coefficients via the inverse quantization unit 140 and the inverse transformation unit 150, a residual signal (residual block or residual sample) can be reconstructed.

[0063] The adder 155 can generate a reconstructed signal (reconstructed picture, reconstructed block, reconstructed sample array) by adding the reconstructed residual signal to the prediction signal output from the inter-prediction unit 180 or the intra-prediction unit 185. If there is no residual for the block to be processed, such as when skip mode is applied, the predicted block can be used as the reconstructed block. The adder 155 may be called the reconstruction unit or the reconstructed block generation unit. The generated reconstructed signal can be used for intra-prediction of the next block to be processed in the current picture, or, as described later, for inter-prediction of the next picture after filtering.

[0064] The filtering unit 160 can improve subjective / objective image quality by applying filtering to the restored signal. For example, the filtering unit 160 can apply various filtering methods to the restored picture to generate a modified restored picture, and the modified restored picture can be stored in the memory 170, specifically in the DPB of the memory 170. The various filtering methods can include, for example, deblocking filtering, sample adaptive offset, adaptive loop filter, and bilateral filter. The filtering unit 160 can generate various filtering-related information, as will be described later in the explanation of each filtering method, and transmit it to the entropy coding unit 190. The filtering-related information can be encoded by the entropy coding unit 190 and output in bitstream format.

[0065] The corrected restored picture transmitted to memory 170 can be used as a reference picture in the interpretation unit 180. When interpretation is applied via this, the image encoding device 100 can avoid prediction mismatches between the image encoding device 100 and the image decoding device, and can also improve encoding efficiency.

[0066] The DPB in memory 170 can store the modified restored picture for use as a reference picture in the inter-prediction unit 180. Memory 170 can store motion information of blocks from which motion information in the current picture has been derived (or encoded) and / or motion information of blocks in the picture that have already been restored. The stored motion information can be transmitted to the inter-prediction unit 180 for use as motion information of spatially surrounding blocks or motion information of temporally surrounding blocks. Memory 170 can store restored samples of restored blocks in the current picture and transmit them to the intra-prediction unit 185.

[0067] Overview of the image decoding device

[0068] Figure 3 is a schematic diagram showing an image decoding apparatus to which the embodiments of this disclosure can be applied.

[0069] As shown in Figure 3, the image decoding device 200 can be configured to include an entropy decoding unit 210, an inverse quantization unit 220, an inverse transform unit 230, an additive unit 235, a filtering unit 240, a memory 250, an inter-prediction unit 260, and an intra-prediction unit 265. The inter-prediction unit 260 and the intra-prediction unit 265 can together be called the "prediction unit". The inverse quantization unit 220 and the inverse transform unit 230 can be included in the residual processing unit.

[0070] All or at least some of the multiple components constituting the image decoding device 200 can be implemented by a single hardware component (e.g., a decoder or processor) according to the embodiment. Furthermore, the memory 170 may include a DPB and can be implemented by a digital storage medium.

[0071] An image decoding device 200, having received a bitstream containing video / image information, can restore the image by executing a process corresponding to the process performed in the image encoding device 100 in Figure 2. For example, the image decoding device 200 can perform decoding using the processing unit applied in the image encoding device. Therefore, the decoding processing unit can be, for example, a coding unit. The coding unit can be obtained by dividing a coding tree unit or a maximum coding unit. The restored image signal decoded and output via the image decoding device 200 can then be reproduced via a playback device (not shown).

[0072] The image decoding device 200 can receive the signal output from the image encoding device 2 in bitstream format. The received signal can be decoded via the entropy decoding unit 210. For example, the entropy decoding unit 210 can parse the bitstream to derive information necessary for image restoration (or picture restoration) (e.g., video / image information). The video / image information may further include information about various parameter sets, such as adaptive parameter set (APS), picture parameter set (PPS), sequence parameter set (SPS), or video parameter set (VPS). The video / image information may also further include general constraint information. The image decoding device may further use the parameter set information and / or the general constraint information to decode the image. The signaling information, received information, and / or syntax elements referred to in this disclosure can be obtained from the bitstream by decoding via the decoding procedure. For example, the entropy decoding unit 210 can decode information in the bitstream based on a coding method such as exponential Golomb coding, CAVLC, or CABAC, and output the values ​​of syntax elements necessary for image reconstruction and the quantized values ​​of conversion coefficients related to the residual. More specifically, the CABAC entropy decoding method receives bins corresponding to each syntax element from the bitstream, determines a context model using the syntax element information to be decoded, the decoding information of the surrounding blocks and the blocks to be decoded, or the symbol / bin information decoded in a previous step, predicts the probability of bin occurrence based on the determined context model, and performs arithmetic decoding of the bins to generate symbols corresponding to the values ​​of each syntax element. At this time, after determining the context model, the CABAC entropy decoding method can update the context model using the decoded symbol / bin information for the context model of the next symbol / bin.Of the information decoded by the entropy decoding unit 210, information related to prediction is provided to the prediction unit (inter-prediction unit 260 and intra-prediction unit 265), and the residual values ​​that have undergone entropy decoding in the entropy decoding unit 210, i.e., quantized conversion coefficients and related parameter information, can be input to the inverse quantization unit 220. In addition, of the information decoded by the entropy decoding unit 210, information related to filtering can be provided to the filtering unit 240. On the other hand, a receiving unit (not shown) that receives signals output from the image coding device may be further provided as an internal / external element of the image decoding device 200, or the receiving unit may be provided as a component of the entropy decoding unit 210.

[0073] On the other hand, the image decoding device according to this disclosure may be called a video / image / picture decoding device. The image decoding device may also include an information decoder (video / image / picture information decoder) and / or a sample decoder (video / image / picture sample decoder). The information decoder may include an entropy decoding unit 210, and the sample decoder may include at least one of an inverse quantization unit 220, an inverse transform unit 230, an adder unit 235, a filtering unit 240, a memory 250, an inter-prediction unit 260, and an intra-prediction unit 265.

[0074] The inverse quantization unit 220 can inverse quantize the quantized transformation coefficients and output the transformation coefficients. The inverse quantization unit 220 can rearrange the quantized transformation coefficients in a two-dimensional block format. In this case, the rearrangement can be performed based on the coefficient scan order performed by the image encoding device. The inverse quantization unit 220 can perform inverse quantization on the quantized transformation coefficients using quantization parameters (e.g., quantization step size information) to obtain the transformation coefficients.

[0075] The inverse conversion unit 230 can inversely convert the conversion coefficients to obtain residual signals (residual blocks, residual sample arrays).

[0076] The prediction unit can make predictions for the current block and generate a predicted block containing prediction samples for the current block. Based on the prediction information output from the entropy decoding unit 210, the prediction unit can determine whether intra-prediction or inter-prediction is applied to the current block and can determine a specific intra / inter-prediction mode (prediction technique).

[0077] As described in the explanation of the prediction unit of the image coding device 100, the prediction unit can generate prediction signals based on various prediction methods (techniques) described later.

[0078] The intra-prediction unit 265 can predict the current block by referring to the samples in the current picture. The description of the intra-prediction unit 185 can also be applied to the intra-prediction unit 265.

[0079] The interprediction unit 260 can derive a predicted block relative to the current block based on a reference block (reference sample array) identified by motion vectors on a reference picture. In this case, to reduce the amount of motion information transmitted in interprediction mode, motion information can be predicted in block, sub-block, or sample units based on the correlation of motion information between surrounding blocks and the current block. The motion information may include motion vectors and reference picture indices. The motion information may further include interprediction direction information (L0 prediction, L1 prediction, Bi prediction, etc.). In interprediction, surrounding blocks may include spatial neighboring blocks present in the current picture and temporal neighboring blocks present in the reference picture. For example, the interprediction unit 260 can construct a motion information candidate list based on surrounding blocks and derive the motion vector and / or reference picture index of the current block based on the received candidate selection information. Interprediction can be performed based on various prediction modes (techniques), and the prediction information may include information indicating the mode (technique) of interprediction for the current block.

[0080] The adder 235 can generate a reconstructed signal (reconstructed picture, reconstructed block, reconstructed sample array) by adding the acquired residual signal to the predicted signal (predicted block, predicted sample array) output from the prediction unit (including the inter-prediction unit 260 and / or intra-prediction unit 265). If there is no residual for the block to be processed, such as when skip mode is applied, the predicted block can be used as the reconstructed block. The description of the adder 155 also applies to the adder 235. The adder 235 is sometimes called the reconstruction unit or reconstructed block generation unit. The generated reconstructed signal can be used for intra-prediction of the next block to be processed in the current picture, or for inter-prediction of the next picture via filtering, as described later.

[0081] The filtering unit 240 can improve subjective / objective image quality by applying filtering to the restored signal. For example, the filtering unit 240 can apply various filtering methods to the restored picture to generate a modified restored picture, and the modified restored picture can be stored in the memory 250, specifically in the DPB of the memory 250. The various filtering methods can include, for example, deblocking filtering, sample adaptive offset, adaptive loop filter, and bilateral filter.

[0082] The restored picture stored (modified) in the DPB of memory 250 can be used as a reference picture in the inter-prediction unit 260. Memory 250 can store motion information of blocks from which motion information in the current picture has been derived (or decoded) and / or motion information of blocks in the picture that have already been restored. The stored motion information can be transmitted to the inter-prediction unit 260 for use as motion information of spatially surrounding blocks or motion information of temporally surrounding blocks. Memory 250 can store restored samples of restored blocks in the current picture and transmit them to the intra-prediction unit 265.

[0083] In this specification, the embodiments described for the filtering unit 160, inter-prediction unit 180, and intra-prediction unit 185 of the image coding device 100 can be applied similarly or in a corresponding manner to the filtering unit 240, inter-prediction unit 260, and intra-prediction unit 265 of the image decoding device 200, respectively.

[0084] Interpretation

[0085] The prediction units of the image encoding device 100 and the image decoding device 200 can perform interpretation on a block-by-block basis to derive predicted samples. Interpretation may be a prediction derived in a manner dependent on data elements of pictures other than the current picture (e.g., sample values ​​or motion information). When interpretation is applied to the current block, a predicted block (predicted sample array) for the current block can be derived based on a reference block (reference sample array) identified by a motion vector on the reference picture pointed to by the reference picture index. In this case, in order to reduce the amount of motion information transmitted in interpretation mode, the motion information of the current block can be predicted on a block, subblock, or sample basis based on the correlation of motion information between the surrounding blocks and the current block. The motion information may include a motion vector and a reference picture index. The motion information may further include interpretation type information (L0 prediction, L1 prediction, Bi prediction, etc.). When interpretation is applied, the surrounding blocks may include spatial neighboring blocks present in the current picture and temporal neighboring blocks present in the reference picture. The reference picture containing the reference block and the reference picture containing the time-peripheral block may be identical or different from each other. The time-peripheral block may be called a collocated reference block (colCU), and the reference picture containing the time-peripheral block may be called a collocated picture (colPic). For example, a list of motion information candidates may be constructed based on the surrounding blocks of the current block, and flags or index information may be signaled to indicate which candidate is selected (used) to derive the motion vector and / or reference picture index of the current block.Interpretation can be performed based on various prediction modes. For example, in skip mode and merge mode, the motion information of the current block may be the same as the motion information of the selected surrounding block. In skip mode, unlike merge mode, the residual signal may not be transmitted. In motion vector prediction (MVP) mode, the motion vector of the selected surrounding block can be used as a motion vector predictor, and the motion vector difference can be signaled. In this case, the motion vector of the current block can be derived using the sum of the motion vector predictor and the motion vector difference.

[0086] The motion information may include L0 motion information and / or L1 motion information depending on the interpretation type (L0 prediction, L1 prediction, Bi prediction, etc.). A motion vector in the L0 direction may be called an L0 motion vector or MVL0, and a motion vector in the L1 direction may be called an L1 motion vector or MVL1. A prediction based on an L0 motion vector may be called an L0 prediction, a prediction based on an L1 motion vector may be called an L1 prediction, and a prediction based on both the L0 motion vector and the L1 motion vector may be called a bi (Bi) prediction. Here, an L0 motion vector may represent a motion vector associated with a reference picture list L0 (L0), and an L1 motion vector may represent a motion vector associated with a reference picture list L1 (L1). The reference picture list L0 may include earlier pictures as reference pictures in output order from the current picture, and the reference picture list L1 may include subsequent pictures in output order from the current picture. The earlier pictures may be called forward (reference) pictures, and the subsequent pictures may be called backward (reference) pictures. The aforementioned reference picture list L0 may further include subsequent pictures as reference pictures in output order from the current picture. In this case, the previous picture may be indexed first in the reference picture list L0, and the subsequent picture may be indexed next. The aforementioned reference picture list L1 may further include previous pictures as reference pictures in output order from the current picture. In this case, the subsequent picture may be indexed first in the reference picture list L1, and the previous picture may be indexed next. Here, the output order may correspond to the POC (picture order count) order.

[0087] Figure 4 is a schematic diagram of the interpretation unit (180) of the image decoding device 100, and Figure 5 is a flowchart showing a method for encoding an image based on interpretation.

[0088] The image coding device 100 can perform inter prediction for the current block (S510). The image coding device 100 can derive the inter prediction mode and motion information of the current block and generate prediction samples for the current block. Here, the procedures for determining the inter prediction mode, deriving motion information, and generating prediction samples may be performed simultaneously, or any one of the procedures may be performed before the others. For example, the inter prediction unit 180 of the image coding device 100 may include a prediction mode determination unit 181, a motion information derivation unit 182, and a prediction sample derivation unit 183, where the prediction mode determination unit 181 determines the prediction mode for the current block, the motion information derivation unit 182 derives the motion information for the current block, and the prediction sample derivation unit 183 derives prediction samples for the current block. For example, the interpretation unit 180 of the image coding device 100 can search for blocks similar to the current block within a certain area (search area) of the reference picture via motion estimation, and derive a reference block whose difference from the current block is the minimum or below a certain standard. Based on this, it can derive a reference picture index that points to the reference picture where the reference block is located, and derive a motion vector based on the positional difference between the reference block and the current block. The image coding device 100 can determine which of the various prediction modes is applied to the current block. The image coding device 100 can compare the RD costs for the various prediction modes and determine the optimal prediction mode for the current block.

[0089] For example, when skip mode or merge mode is applied to the current block, the image encoding device 100 can configure a merge candidate list, as described later, and derive a reference block from among the reference blocks pointed to by the merge candidates included in the merge candidate list whose difference from the current block is the minimum or below a certain standard. In this case, a merge candidate associated with the derived reference block is selected, and merge index information pointing to the selected merge candidate is generated and signaled to the decoding device. The motion information of the current block can be derived using the motion information of the selected merge candidate.

[0090] As another example, when the (A)MVP mode is applied to the current block, the image encoding device 100 can configure the (A)MVP candidate list described later, and use the motion vector of the selected mvp candidate from among the mvp (motion vector predictor) candidates included in the (A)MVP candidate list as the mvp of the current block. In this case, for example, the motion vector pointing to the reference block derived by the motion estimation described above can be used as the motion vector of the current block, and the mvp candidate having the motion vector with the smallest difference from the motion vector of the current block can become the selected mvp candidate. The MVD (motion vector difference), which is the difference obtained by subtracting the mvp from the motion vector of the current block, can be derived. In this case, information regarding the MVD can be signaled to the image decoding device 200. Also, when the (A)MVP mode is applied, the value of the reference picture index can be configured with reference picture index information and separately signaled to the image decoding device 200.

[0091] The image coding device 100 can derive a residual sample based on the predicted sample (S520). The image coding device 100 can derive the residual sample by comparing the original sample of the current block with the predicted sample.

[0092] The image encoding device 100 can encode image information including prediction information and residual information (S530). The image encoding device 100 can output the encoded image information in bitstream format. The prediction information may include information related to the prediction procedure, such as prediction mode information (e.g., skip flag, merge flag, or mode index) and motion information. The motion information may include candidate selection information (e.g., merge index, mvp flag, or mvp index) which is information for deriving a motion vector. The motion information may also include the above-mentioned MVD information and / or reference picture index information. Furthermore, the motion information may include information indicating whether L0 prediction, L1 prediction, or bi prediction is applied. The residual information is information about the residual sample. The residual information may include information about the quantized transformation coefficients for the residual sample.

[0093] The output bitstream may be stored on a (digital) storage medium and transmitted to the image decoding device 200, or it may be transmitted to the image decoding device 200 via a network.

[0094] On the other hand, as described above, the image coding device 100 can generate a reconstructed picture (including a reconstructed sample and a reconstructed block) based on the reference sample and the residual sample. This is because the image coding device 100 derives the same prediction results as the image decoding device 200, thereby improving coding efficiency. Therefore, the image coding device 100 can store the reconstructed picture (or reconstructed sample, reconstructed block) in memory and use it as a reference picture for interpretation. As described above, in-loop filtering procedures and the like can be further applied to the reconstructed picture.

[0095] Figure 6 is a schematic diagram of the interpretation unit 260 of the image decoding device 200, and Figure 7 is a flowchart showing a method for decoding an image based on interpretation.

[0096] The image decoding device 200 can perform operations corresponding to those performed by the image encoding device 100. The image decoding device 200 can make predictions for the current block based on the received prediction information and derive prediction samples.

[0097] Specifically, the image decoding device 200 can determine a prediction mode for the current block based on the received prediction information (S710). Based on the prediction mode information in the prediction information, the image decoding device 200 can determine which interpretation mode is applied to the current block.

[0098] For example, based on the merge flag, it can be determined whether the merge mode is applied to the current block or whether the (A)MVP mode is determined. Alternatively, one of several inter-prediction mode candidates can be selected based on the mode index. The inter-prediction mode candidates may include skip mode, merge mode, and / or (A)MVP mode, or may include various inter-prediction modes as described later.

[0099] The image decoding device 200 can derive motion information for the current block based on the determined interprediction mode (S720). For example, if a skip mode or merge mode is applied to the current block, the image decoding device 200 can configure a merge candidate list, which will be described later, and select one merge candidate from among the merge candidates included in the merge candidate list. The selection can be made based on the selection information (merge index) described above. The motion information for the current block can be derived using the motion information for the selected merge candidate. The motion information for the selected merge candidate can be used as the motion information for the current block.

[0100] As another example, when the (A)MVP mode is applied to the current block, the image decoding device 200 configures the (A)MVP candidate list described later, and can use the motion vector of the selected mvp candidate from among the mvp (motion vector predictor) candidates included in the (A)MVP candidate list as the mvp of the current block. The selection can be made based on the selection information (mvp flag or mvp index) described above. In this case, the MVD of the current block can be derived based on the information regarding the MVD, and the motion vector of the current block can be derived based on the mvp of the current block and the MVD. In addition, the reference picture index of the current block can be derived based on the reference picture index information. The picture pointed to by the reference picture index in the reference picture list for the current block can be derived as the reference picture referenced for interpretation of the current block.

[0101] On the other hand, as will be described later, the movement information of the current block can be derived without constructing a candidate list, in which case the movement information of the current block can be derived according to the procedure disclosed in the prediction mode described later. In this case, the candidate list configuration described above can be omitted.

[0102] The image decoding device 200 can generate predicted samples for the current block based on the motion information of the current block (S730). In this case, the reference picture is derived based on the reference picture index of the current block, and the predicted samples for the current block can be derived using the sample of the reference block pointed to on the reference picture by the motion vector of the current block. In this case, as will be described later, a further predicted sample filtering procedure may be performed on all or part of the predicted samples for the current block.

[0103] For example, the interpretation unit 260 of the image decoding device 200 may include a prediction mode determination unit 261, a motion information derivation unit 262, and a prediction sample derivation unit 263. The prediction mode determination unit 181 determines the prediction mode for the current block based on the prediction mode information received, the motion information derivation unit 182 derives motion information (motion vector and / or reference picture index, etc.) for the current block based on the motion information information received, and the prediction sample derivation unit 183 derives the prediction sample for the current block.

[0104] The image decoding device 200 can generate a residual sample for the current block based on the received residual information (S740). The image decoding device 200 can generate a reconstructed sample for the current block based on the predicted sample and the residual sample, and generate a reconstructed picture based on this (S750). As described above, further procedures such as in-loop filtering can be applied to the reconstructed picture thereafter.

[0105] Referring to Figure 8, as described above, the interpretation procedure may include an interpretation mode determination step (S810), a motion information derivation step (S820) based on the determined prediction mode, and a prediction execution (prediction sample generation) step (S830) based on the derived motion information. The interpretation procedure can be performed by the image encoding device 100 and the image decoding device (200) as described above.

[0106] Interpretation mode determination

[0107] Various interpretation modes can be used to predict the current block within a picture. For example, various modes such as merge mode, skip mode, MVP (motion vector prediction) mode, affine mode, subblock merge mode, and MVVD (merge with MVD) mode can be used. DMVR (Decoder side motion vector refinement) mode, AMVR (adaptive motion vector resolution) mode, Bi-prediction with CU-level weight (BCW), and Bi-directional optical flow (BDOF) can be used as additional or alternative modes. Affine mode is sometimes called affine motion prediction mode. MVP mode is sometimes called AMVP (advanced motion vector prediction) mode. Motion information candidates derived by some modes and / or some modes in this document may also be included as one of the motion information related candidates in other modes. For example, an HMVP candidate may be added as a merge candidate in the merge / skip mode, or as an mvp candidate in the MVP mode.

[0108] Prediction mode information indicating the inter-prediction mode of the current block can be signaled from the image coding device 100 to the image decoding device 200. The prediction mode information can be received by the image decoding device 200 in the bitstream. The prediction mode information may include index information indicating one of a number of candidate modes. Alternatively, the inter-prediction mode may be indicated via hierarchical signaling of flag information. In this case, the prediction mode information may include one or more flags. For example, a skip flag may be signaled to indicate whether a skip mode is applied, and if the skip mode is not applied, a merge flag may be signaled to indicate whether a merge mode is applied, and if the merge mode is not applied, it may indicate that the MVP mode is applied, or flags for additional distinctions may be further signaled. Affine modes may be signaled as independent modes, or as modes dependent on merge modes or MVP modes, etc. For example, affine modes may include affine merge mode and affine MVP mode.

[0109] Derivation of motion information

[0110] Interpretation can be performed using the motion information of the current block. The image coding device 100 can derive optimal motion information for the current block through a motion estimation procedure. For example, the image coding device 100 can use the original block in the original picture for the current block to search for a highly correlated similar reference block in fractional pixel units within a predetermined search range in the reference picture, thereby deriving motion information. Block similarity can be derived based on the difference in phase-based sample values. For example, block similarity can be calculated based on the SAD between the current block (or template of the current block) and the reference block (or template of the reference block). In this case, motion information can be derived based on the reference block with the smallest SAD in the search area. The derived motion information can be signaled to the image decoding device 200 according to various methods based on the interpretation mode.

[0111] Generation of Predictive Samples

[0112] Based on motion information derived according to the prediction mode, a predicted block can be derived for the current block. The predicted block may include predicted samples (predicted sample arrays) of the current block. If the motion vector of the current block points to fractional sample units, an interpolation procedure can be performed to derive predicted samples of the current block based on reference samples in fractional sample units within the reference picture. When affine interpretation is applied to the current block, predicted samples can be generated based on sample / subblock unit MV. When biprediction is applied, predicted samples derived via a (phase-weighted) sum or weighted average of predicted samples derived based on L0 prediction (i.e., prediction using reference pictures in reference picture list L0 and MVL0) and predicted samples derived based on L1 prediction (i.e., prediction using reference pictures in reference picture list L1 and MVL1) may be used as predicted samples for the current block. When biprediction is applied, if the reference picture used for L0 prediction and the reference picture used for L1 prediction are located in different temporal directions relative to the current picture (i.e., it is biprediction but corresponds to bidirectional prediction), this can be called true biprediction.

[0113] As mentioned above, reconstructed samples and pictures can be generated based on the derived predicted samples, and then procedures such as in-loop filtering can be performed.

[0114] MMVD(merge mode with motion vector difference)

[0115] In addition to the merge mode, in which implicitly induced motion information is immediately used for predictive sample generation of the current block, the MMVD mode has been proposed. Since similar motion information induction methods are used for the skip mode and merge mode, the MMVD mode can also be applied to the skip mode. An mmvd flag (e.g., mmvd_flag) indicating whether the MMVD mode is applied to the current block can be signaled immediately after the skip flag and merge flag have been transmitted.

[0116] In MMVD mode, after a merge candidate is selected, this selected merge candidate can be enhanced by signaled MVD information. If MMVD mode is applied to the current block (i.e., the value of mmvd_flag is 1), additional information for MMVD mode can be signaled. This additional information may include a merge candidate flag (e.g., mmvd_merge_flag) indicating whether the first (0) or second (1) merge candidate in the merge candidate list is used with the MVD, an index indicating the magnitude of the movement (e.g., mmvd_distance_idx), and an index indicating the direction of the movement (e.g., mmvd_direction_idx). In MMVD mode, one of the first two candidates in the merge candidate list is selected and can be used as the MV basis. The merge candidate flag can be signaled to indicate which of the first two candidates is used.

[0117] The distance index (e.g., mmvd_distance_idx) indicates the magnitude of the motion and can represent a predetermined offset from the starting point. As shown in Figure 9, the offset can be added to the horizontal or vertical component of the starting MV (starting point). The relationship between the distance index and the predetermined offset can be shown in Table 1.

[0118] [Table 1]

[0119] In Table 1, a value of 1 for slice_fpel_mmvd_enabled_flag can indicate that the MMVD mode currently uses integer sample accuracy in the slice. A value of 0 for slice_fpel_mmvd_enabled_flag can indicate that the MMVD mode currently uses fractional sample accuracy in the slice. If slice_fpel_mmvd_enabled_flag is not present, its value can be inferred to be 0. The syntax elements of slice_fpel_mmvd_enabled_flag can be signaled via (or included in) the slice header.

[0120] The direction index (e.g., mmvd_direction_idx) can indicate the MVD direction relative to the starting point. The direction index can indicate four directions, as shown in Table 2.

[0121] [Table 2]

[0122] In Table 2, the meaning of the MVD code (MmvdSign) is variable depending on the information of the starting MVs. If the starting MVs are uni-prediction MVs, or if the two prediction lists are bi-prediction MVs pointing in the same direction of the current picture (i.e., the POC of the reference picture is greater than the POC of the current picture, or the POC of the reference picture is less than the POC of the current picture), the MVD code in Table 2 can represent the sign of the MV offset added to the starting MV. If the two prediction lists are bi-prediction MVs pointing in different directions of the current picture (i.e., the POC of one reference picture is greater than the POC of the current picture, and the POC of the other reference picture is less than the POC of the current picture), the MVD code in Table 2 can represent the sign of the MV offset added to the list 0 MV component of the starting MV, and the MVD for list 1 MV may have the opposite sign.

[0123] The two components of Mmvdoffset[x0][y0] can be derived as shown in equation 1.

[0124] [Formula 1]

number

[0125] Affine prediction

[0126] Existing video coding systems use a single motion vector (MV) to represent the motion of a coded block (using translation MV). However, while the above method can represent the optimal motion at the block level, this does not represent the optimal motion of each actual pixel. Therefore, if the optimal MV can be determined at the pixel level, coding efficiency can be improved. For this purpose, this embodiment describes an affine motion prediction method that encodes using an affine motion model. The affine motion prediction method can represent the MV at the pixel level of a block using two, three, or four MVs.

[0127] As shown in Figure 10, the affine motion model can represent four types of motion. Of the motions that the affine motion model can represent, an affine motion model that represents three types of motion (translation, scale, and rotate) is called a similarity (or simplified) affine motion model, and in this application, the proposed method will be explained based on the similarity (or simplified) affine motion model. However, the proposed method is not limited to this affine motion model.

[0128] As shown in Figure 11, affine motion prediction can determine the MV of the pixel positions contained within a block using two or more control point motion vectors (CPMVs). In this case, the set of MVs is called the affine motion vector field (MVF), and the MVF can be determined by the following equations 2 and 3.

[0129] For a 4-parameter affine motion model, the MV of the sample position (x, y) within the block can be derived by equation 3.

[0130] [Formula 2]

number

[0131] For a 6-parameter affine motion model, the MV of the sample position (x, y) within the block can be derived by equation 3.

[0132] [Formula 3]

number

[0133] In Figure 11, in equations 2 and 3, JPEG2026099991000007.jpg5136 is a CPMV of the top-left corner position of the encoded block, JPEG2026099991000008.jpg5136 is a CPMV with the top-right corner position. JPEG2026099991000009.jpg5136 could be a CPMV of the CP at the bottom-left corner position. Then, W corresponds to the current block width, and H corresponds to the current block height. JPEG2026099991000010.jpg5136 can correspond to the MV at position {x, y}.

[0134] During the encoding / decoding process, the affine MVF can be determined on a pixel-by-pixel basis or in predefined subblock units. When determined on a pixel-by-pixel basis, the MV is obtained based on each pixel value. When determined on a subblock basis, the MV for a subblock can be obtained based on the pixel value of the center of the subblock (the lower right side of the center, i.e., the lower right sample of the four central samples). In this application, we will explain the case where the affine MVF is determined in 4x4 subblock units, as shown in Figure 12. However, this is merely for the sake of explanation, and the size of the subblocks in which the affine MVF is determined can be varied in various ways.

[0135] If affine prediction is available, the motion models currently applicable to a block may include three: a translational motion model, a four-parameter affine motion model, and a six-parameter affine motion model. Here, the translational motion model may refer to a model in which existing block unit motion vectors are used, the four-parameter affine motion model may refer to a model in which two CPMVs are used, and the six-parameter affine motion model may refer to a model in which three CPMVs are used.

[0136] The affine motion prediction may include affine MVP (or affine inter) mode and affine smerge mode. In affine motion prediction, the MVs of the current block can be induced on a sub-block or sample basis.

[0137] Template matching(TM)

[0138] Template matching (TM) is a method for inducing motion vectors at the decoder end that can refine the motion information of a current block (e.g., current coding unit, current CU) by finding the template in a reference picture that is most similar to the template adjacent to the current block (hereinafter referred to as the "current template") (hereinafter referred to as the "reference template"). The current template may be the block adjacent to the top edge of the current block and / or the block adjacent to the left, or a part of these adjacent blocks. The reference template may be determined to be the same size as the current template.

[0139] As shown in Figure 13, once the initial motion vector of the current block is induced, a search for a better motion vector can be performed in the region surrounding the initial motion vector. For example, the range of the region in which the search is performed may be within the [-8, +8]-pel search region centered on the initial motion vector. The size of the search step can be determined based on the AMVR mode of the current block. Furthermore, template matching may be performed sequentially with the bilateral matching process in merge mode.

[0140] Adaptive reordering of merge candidates with template matching (ARMC-TM)

[0141] Merge candidates can be adaptively rearranged by template matching. This rearrangement method can be applied to general merge mode, template matching (TM) merge mode, and affine merge mode (excluding SbTMVP candidates). In TM merge mode, the rearrangement of merge candidates can be performed before the motion vector improvement process described above.

[0142] After the merge candidate list is constructed, the merge candidates can be divided into one or more subgroups. The subgroup size for general merge mode and TM merge mode can be 5, while the subgroup size for affine merge mode can be 3. The merge candidates within each subgroup can be rearranged in ascending order by cost values ​​based on template matching. For simplicity, the merge candidates in the last subgroup, rather than the first subgroup, do not need to be rearranged.

[0143] The template matching cost for a merge candidate can be measured by the sum of absolute difference (SAD) between the template sample of the current block and its corresponding reference sample. The template may include a set of reconstructed samples adjacent to the current block. The reference samples of the template can be located by the motion information of the merge candidate.

[0144] Figure 14 illustrates the template of the current block and the reference sample of the template in the reference picture. When the merge candidate uses bidirectional prediction, the reference sample of the template of the merge candidate can be generated as shown in Figure 14. Specifically, after the reference block in the list0 reference picture is identified based on the list0 (L0) motion vector of the current block's merge candidate, the reference template (reference template 0, RT0) in the list0 reference picture can be identified. Similarly, after the reference block in the list1 reference picture is identified based on the list1 (L1) motion vector of the current block's merge candidate, the reference template (reference template 1, RT1) in the list1 reference picture can be identified.

[0145] For merge candidates based on subblocks with a subblock size of Wsub×Hsub, the top template can contain one or more subtemplates of size Wsub×1, and the left template can contain one or more subtemplates of size 1×Hsub.

[0146] Figure 15 illustrates how to identify a template using subblock movement information.

[0147] In the example shown in Figure 15, the motion information of the subblocks contained in the first column and first row of the current block can be used to derive the reference samples for each subtemplate. More specifically, the motion vectors of the subblocks (A, B, C, D, E, F, G) contained in the first column and first row of the current block in the current picture can be used to identify the reference subblocks (A_ref, B_ref, C_ref, D_ref, E_ref, F_ref, G_ref) corresponding to the reference picture. For example, the position of the corresponding reference subblock can be identified from the collocated block in the reference picture based on the motion vector of each subblock. Subsequently, a reference template can be constructed from the restoration area adjacent to each reference subblock. As shown in Figure 15, when the size of a subblock is Wsev × Hsub, the top reference template can contain one or more subreference templates of size Wsub × 1, and the left reference template can contain one or more subreference templates of size 1 × Hsub.

[0148] TMVP / NAMVP candidate reordering

[0149] Merger candidates of a single type (e.g., TVMP or NAMVP) can be reordered based on ARMC TM cost values. Here, TMVP (temporal motion vector prediction) is a temporal prediction candidate for the current block, and NAMVP (non-adjacent) may be a prediction candidate that is not adjacent to the current block. The reordered candidates can be added to the merge candidate list. The TMVP candidate type can have more TMVP candidates with different interpretation directions at more temporal positions added for reordering and selection. The NAMVP candidate type can also be expanded to include positions that are not spatially adjacent.

[0150] MMVD / Affine MMVD candidate reordering

[0151] The number of directions can be increased from 4 to 16 by adding additional refinement positions along the kXπ / 8 diagonal angle. Also, the number of distance positions per direction can be reduced from 8 to 6. Based on the SAD cost between the template for each refinement position (one above the current block, the other to the left of the current block) and the reference template, all 96 (16*6) MMVD refinement positions (i.e., MMVD prediction candidates) for each base candidate can be rearranged. The top 1 / 8 MMVD prediction candidates with the smallest template SAD cost can be retained as available candidates for MMVD index coding. The MMVD index can be binarized with a rice code with parameter 2.

[0152] An extended affine MMVD realignment has been added, which adds additional improvement positions along the kXπ / 8 diagonal angle. After the affine MMVD prediction candidates are realigned, the top half of the affine MMVD prediction candidates with the smallest template SAD cost can be retained.

[0153] Reference Picture Resampling (RPR)

[0154] Image compression techniques can support Adaptive Resolution Change (Arc) in Coded Layer Video Sequences (CLVS). When ARC is permitted, a reference picture with a different resolution than the current picture may be resampled. The reference picture may be a picture contained in the same layer as the current picture. Resampling is sometimes called Reference Picture Resampling (RPR). For example, reference picture resampling may include scaling and interpolation of the reference picture.

[0155] If ARC is permitted, interpretation can be performed based on a reference picture with a different resolution. The reference picture may have different widths and / or heights of lumens than the current picture. For interpretation, the reference picture may be resampled. Predicted samples of the current block in the current picture can be derived based on the motion vector of the current block and the resampled reference picture.

[0156] [Examples]

[0157] In this disclosure, template matching may be the process of searching for the reference template that is currently most similar to the template. According to this disclosure, a template matching cost (TM cost) can be calculated to measure the similarity, and for this purpose, a cost function such as SAD can be used. A high template matching cost means that the template matching error is large, and therefore may mean that the similarity between templates is low. Conversely, a low template matching cost means that the template matching error is small, and therefore may mean that the similarity between templates is high.

[0158] In this disclosure, the cost function for calculating template matching costs may be a function that utilizes the difference between the sample values ​​in the current template and the corresponding sample values ​​in the reference template. Therefore, the cost function may be referred to as a "difference (error)-based function" or "difference (error)-based equation" between the corresponding samples in the two templates. The template matching cost calculated by the cost function may also be referred to as a "difference (error)-based function value" or "difference (error)-based value" between the corresponding samples in the two templates.

[0159] In this disclosure, "cost" may be referred to as "error," "error," etc. In other words, in this disclosure, "cost," "error," and "error" may have the same meaning. Therefore, the "cost function" may also be referred to as the "error function," "error function," etc. Since the cost can be derived based on the difference between the current template and the reference template, it can correspond to the error value for the current block.

[0160] This application proposes a method for adaptively inducing an error value for the current block based on whether or not reference picture resampling (RPR) has been applied to the reference picture. In other words, this application proposes a method for adaptively inducing the TM cost based on whether or not RPR has been applied to the reference picture. Hereinafter, a reference picture to which RPR has been applied will be referred to as an "RPR reference picture," and a reference picture to which RPR has not been applied will be referred to as a "non-RPR reference picture."

[0161] Figure 16 is a flowchart showing an image encoding / decoding method according to one embodiment of the present invention.

[0162] Referring to Figure 16, the image encoding device 100 and the image decoding device 200 can generate a list of prediction candidates for the current block (S1610).

[0163] Prediction candidates can be any of the following: merge candidates, TMVP (temporal motion vector prediction) candidates, non-adjacent (NAMVP) candidates (prediction candidates that are not currently adjacent to a block), MMVD candidates, and affine MMVD candidates. Prediction candidate lists can be any of the following: merge candidate lists (containing merge candidates), TMVP candidate lists (containing TMVP candidates), NAMVP candidate lists (containing NAMVP candidates), MMVD candidate lists (containing MMVD correct candidates), and affine MMVD candidate lists (containing affine MMVD candidates).

[0164] The image encoding device 100 and the image decoding device 200 can induce an error value for the current block (S1620). The error value may be the TM cost. The error value can be induced based on whether or not the reference picture referenced by the prediction candidate is an RPR reference picture. For example, if the reference picture is an RPR reference picture, the error value can be induced to a predetermined value, and if the reference picture is a non-RPR reference picture, the error value can be induced based on the difference between the current template and the reference template. That is, if the reference picture is an RPR reference picture, the process of calculating the error value based on the difference between the current template and the reference template is not performed (bypassed), and the error value can be induced to a predetermined value. Here, the predetermined value may be a maximum value predetermined for the error value.

[0165] The image encoding device 100 and the image decoding device 200 can rearrange the order of the prediction candidates based on the error value (S1630).

[0166] The rearrangement order of prediction candidates may be in ascending order based on the error value of each prediction candidate. In this case, if the error value for an RPR reference picture is guided to a predetermined maximum value, that prediction candidate will have a lower priority than other prediction candidates (prediction candidates for non-RPR reference pictures). Therefore, prediction candidates for RPR reference pictures may no longer be used for prediction of the current block.

[0167] To calculate the error value, a process may be performed in which the reference picture is scaled to the same size as the current picture and interpolated. However, the error generated in this process may be relatively larger for RPR reference pictures compared to non-RPR reference pictures. Therefore, as in the present invention, by lowering the priority of prediction candidates for RPR reference pictures and inducing them not to be used, the scaling and interpolation process is avoided. As a result, large errors that occur in the scaling and interpolation process can be prevented, thereby enabling more accurate prediction of the current block.

[0168] Example 1

[0169] The conventional merge candidate rearrangement method sets the priority of merge candidates higher than those referencing non-RPR reference pictures by setting the TM cost (error value) of a merge candidate to 0 when the reference picture is an RPR reference picture, and by bypassing the process of calculating the TM cost.

[0170] However, with such conventional merge candidate rearrangement methods, in order to calculate the TM cost, the merge candidates for RPR reference pictures may have higher errors than those for non-RPR reference pictures during the process of scaling the RPR reference picture to the same size as the current picture and additionally interpolating. Therefore, to solve the above-mentioned problems, the present invention proposes, through Example 1, a method in which merge candidates for RPR reference pictures are configured in the candidate list with the lowest priority.

[0171] Figure 17 is a flowchart showing the image encoding / decoding method according to Example 1.

[0172] Referring to Figure 17, the image encoding device 100 and the image decoding device 200 can derive a merge candidate list (S1702), set the value of index idx which indicates one of the merge candidates in the merge candidate list to 0 (S1704), and then determine whether the merge candidate indicated by the index is the last merge candidate in the merge candidate list (i.e., the number of merge candidates, numValidMergeCand) (S1706).

[0173] If the merge candidate indicated by the index is not the last merge candidate, the image encoding device 100 and the image decoding device 200 can determine the predicted direction (interDir) of the merge candidate indicated by the index (S1708), and determine the values ​​of the start reference list (startRefList) and end reference list (endRefList) according to the determined direction (S1710~S1714).

[0174] The image encoding device 100 and the image decoding device 200 can set the value of the variable refPicList to be the same as the value of the starting reference list and set the value of the variable (bRPR) indicating whether or not it is an RPR reference picture to 0 (S1716). The image encoding device 100 and the image decoding device 200 can also determine whether or not the reference picture corresponding to refPicidx in refPicList is an RPR reference picture (S1728), increment the value of the reference list by 1 (S1730), and set the next reference list as the target for determination. By performing the S1728 and S1730 processes for all reference lists (S1718), the image encoding device 100 and the image decoding device 200 can determine whether or not each reference picture in each reference list is an RPR reference picture.

[0175] The image encoding device 100 and the image decoding device 200 can set the value of the TM cost of the merge candidate to a predetermined maximum value if the reference picture is an RPR reference picture (S1720) (S1724), and can calculate and induce the TM cost of the merge candidate if the reference picture is a non-RPR reference picture (S1720) (S1722).

[0176] The image encoding device 100 and the image decoding device 200 can repeatedly perform processes S1706 to S1724 for all merge candidates in the merge candidate list by performing processes S1706 to S1724 for the next merge candidate (idx++, S1726).

[0177] If processes S1706 through S1724 have all been performed for all merge candidates in the merge candidate list (S1706), the image encoding device 100 and the image decoding device 200 can rearrange the merge candidates in the merge candidate list based on the error value of each merge candidate (S1732). In this process, merge candidates having a predetermined maximum value as an error value (merging candidates that refer to an RPR reference picture) can be rearranged with a lower priority. The image encoding device 100 can determine which merge candidate to use for predicting the current block from among the rearranged merge candidates, and can signal the determined merge candidate by encoding an index (merge index) into a bitstream. The image decoding device 200 can derive motion information based on the signaled merge index (S1734).

[0178] Example 2

[0179] Conventional TMVP candidate rearrangement methods calculated the TM cost for all TMVP candidates without considering whether the reference picture was an RPR reference picture or not.

[0180] However, with such conventional TMVP candidate rearrangement methods, the process of scaling the RPR reference picture to the same size as the current picture and additionally interpolating it in order to calculate the TM cost can result in higher errors in the TMVP candidates of the RPR reference picture compared to the TMVP candidates of the non-RPR reference picture. Therefore, to solve the above-mentioned problem, the present invention proposes, through Example 2, a method in which the TMVP candidates of the RPR reference picture are configured in the candidate list with the lowest rank.

[0181] Figure 18 is a flowchart showing the image encoding / decoding method according to Example 2.

[0182] Referring to Figure 18, the image encoding device 100 and the image decoding device 200 can derive a TMVP candidate list (S1802), set the value of an index (idx) that indicates one of the TMVP candidates in the TMVP candidate list to 0 (S1804), and then determine whether the TMVP candidate indicated by the index is the last TMVP candidate in the TMVP candidate list (i.e., the number of TMVP candidates, numValidMergeCand) (S1806).

[0183] If the TMVP candidate indicated by the index is not the last TMVP candidate, the image encoding device 100 and the image decoding device 200 can determine the predicted direction (interDir) of the TMVP candidate indicated by the index (S1808), and determine the values ​​of the start reference list (startRefList) and end reference list (endRefList) according to the determined direction (S1810~S1814).

[0184] The image encoding device 100 and the image decoding device 200 can set the value of the variable refPicList to be the same as the value of the starting reference list and set the value of the variable (bRPR) indicating whether or not it is an RPR reference picture to 0 (S1816). The image encoding device 100 and the image decoding device 200 can also determine whether or not the reference picture corresponding to refPicidx in refPicList is an RPR reference picture (S1828), increment the value of the reference list by 1 (S1830), and set the next reference list as the target for determination. By performing the S1828 and S1830 processes for all reference lists (S1818), the image encoding device 100 and the image decoding device 200 can determine whether or not each reference picture in each reference list is an RPR reference picture.

[0185] The image encoding device 100 and the image decoding device 200 can set the TM cost value of the TMVP candidate to a predetermined maximum value (S1824) if the reference picture is an RPR reference picture (S1820), and can calculate and induce the TM cost of the TMVP candidate (S1822) if the reference picture is a non-RPR reference picture (S1820).

[0186] The image encoding device 100 and the image decoding device 200 can repeatedly perform processes S1806 to S1824 for all TMVP candidates in the TMVP candidate list by performing processes S1806 to S1824 for the next TMVP candidate (idx++, S1826).

[0187] If all processes S1806 through S1824 have been performed for all TMVP candidates in the TMVP candidate list (S1806), the image encoding device 100 and the image decoding device 200 can rearrange the TMVP candidates in the TMVP candidate list based on the error value of each TMVP candidate (S1832). In this process, TMVP candidates with a predetermined maximum value as their error value (TMVP candidates that refer to an RPR reference picture) can be rearranged with a lower priority. The image encoding device 100 and the image decoding device 200 can determine the first TMVP candidate among the rearranged TMVP candidates as the TMVP information for the current block (S1834).

[0188] Example 3

[0189] Conventional NAMVP candidate rearrangement methods calculated the TM cost for all NAMVP candidates without considering whether the reference picture was an RPR reference picture or not.

[0190] However, with such conventional NAMVP candidate rearrangement methods, the NAMVP candidates for RPR reference pictures may have higher errors than those for non-RPR reference pictures in the process of scaling the RPR reference picture to the same size as the current picture and additionally interpolating in order to calculate the TM cost. Therefore, to solve the above-mentioned problem, the present invention proposes, through Example 3, a method in which the NAMVP candidates for RPR reference pictures are configured in the candidate list with the lowest rank.

[0191] Figure 19 is a flowchart showing the image encoding / decoding method according to Example 3.

[0192] Referring to Figure 19, the image encoding device 100 and the image decoding device 200 can derive a NAMVP candidate list (S1902), set the value of an index (idx) that indicates one of the NAMVP candidates in the NAMVP candidate list to 0 (S1904), and then determine whether the NAMVP candidate indicated by the index is the last NAMVP candidate in the NAMVP candidate list (i.e., the number of NAMVP candidates, numValidMergeCand) (S1906).

[0193] If the NAMVP candidate indicated by the index is not the last NAMVP candidate, the image encoding device 100 and the image decoding device 200 can determine the predicted direction (interDir) of the NAMVP candidate indicated by the index (S1908), and determine the values ​​of the start reference list (startRefList) and end reference list (endRefList) according to the determined direction (S1910~S1914).

[0194] The image encoding device 100 and the image decoding device 200 can set the value of the variable refPicList to be the same as the value of the starting reference list and set the value of the variable (bRPR) indicating whether or not it is an RPR reference picture to 0 (S1916). The image encoding device 100 and the image decoding device 200 can also determine whether or not the reference picture corresponding to refPicidx in refPicList is an RPR reference picture (S1928), increment the value of the reference list by 1 (S1930), and set the next reference list as the target for determination. By performing the S1928 and S1930 processes for all reference lists (S1918), the image encoding device 100 and the image decoding device 200 can determine whether or not each reference picture in each reference list is an RPR reference picture.

[0195] The image encoding device 100 and the image decoding device 200 can set the value of the TM cost of the NAMVP candidate to a predetermined maximum value if the reference picture is an RPR reference picture (S1920) (S1924), and can calculate and induce the TM cost of the NAMVP candidate if the reference picture is a non-RPR reference picture (S1920) (S1922).

[0196] The image encoding device 100 and the image decoding device 200 can repeatedly perform processes S1906 to S1924 for all NAMVP candidates in the NAMVP candidate list by performing processes S1906 to S1924 for the next NAMVP candidate (idx++, S1926).

[0197] If all processes S1906 through S1924 have been performed for all NAMVP candidates in the NAMVP candidate list (S1906), the image encoding device 100 and the image decoding device 200 can rearrange the NAMVP candidates in the NAMVP candidate list based on the error value of each NAMVP candidate (S1932). In this process, NAMVP candidates with a predetermined maximum value as their error value (NAMVP candidates that reference an RPR reference picture) can be rearranged with a lower priority. The image encoding device 100 and the image decoding device 200 can set up the NAMVP list based on a predetermined number (N) of candidates having the optimal TM cost (S1934).

[0198] Example 4

[0199] The conventional MMVD candidate rearrangement method and the affine MMVD candidate rearrangement method calculated the TM cost for all predicted candidates without considering whether the base candidate's reference picture was an RPR reference picture or not. Here, the predicted candidates can be derived from the motion information of the base candidate via MMVD and affine MMVD. Furthermore, the conventional MMVD candidate rearrangement method and the affine MMVD candidate rearrangement method signaled 12.5% ​​of the MMVD candidates and 50% of the affine MMVD candidates in order of the smallest calculated TM cost.

[0200] However, with these conventional MMVD candidate rearrangement methods and affine MMVD candidate rearrangement methods, the error in the RPR reference picture may be even greater than that of the non-RPR reference picture when scaling the RPR reference picture to the same size as the current picture and additionally interpolating in order to calculate the TM cost. Therefore, to solve the above-mentioned problem, the present application proposes, through Example 4, a method such that when the reference picture of the basic candidate is an RPR reference picture, the MMVD candidate and affine MMVD candidate are configured in the candidate list with the lowest rank. Below, Example 4 will be described focusing on the MMVD candidate, but Example 4 can also be applied to the affine MMVD candidate.

[0201] Figure 20 is a flowchart showing the image encoding / decoding method according to Example 4.

[0202] Referring to Figure 20, the image encoding device 100 and the image decoding device 200 can derive a merge candidate list (S2002). The image encoding device 100 can set a base candidate for MMVD (basdCand) and signal the index for the set base candidate (base candidate index) in a bitstream. The image decoding device 200 can set a base candidate for MMVD using the signaled base candidate index (S2004). The base candidate can represent any of the merge candidates in the merge candidate list. The merge candidate list is sometimes called the "first prediction candidate list," and the base candidate is sometimes called the "first prediction candidate."

[0203] The image encoding device 100 and the image decoding device 200 determine the predicted direction (interDir) of the basic candidate (S2006), and can determine the values ​​of the start reference list (startRefList) and end reference list (endRefList) according to the determined direction (S2008~S2010).

[0204] The image encoding device 100 and the image decoding device 200 can set the value of the variable refPicList to be the same as the value of the starting reference list and set the value of the variable (bRPR) indicating whether or not it is an RPR reference picture to 0 (S2014). The image encoding device 100 and the image decoding device 200 can also determine whether or not the reference picture corresponding to refPicidx in refPicList is an RPR reference picture (S2018), increment the value of the reference list by 1 (S2020), and set the next reference list as the target for determination. By performing the S2018 and S2020 processes for all reference lists (S2016), the image encoding device 100 and the image decoding device 200 can determine whether or not each reference picture in each reference list is an RPR reference picture.

[0205] The image encoding device 100 and the image decoding device 200 can set the TM cost value of the MMVD candidate for the basic candidate to a predetermined maximum value when the reference picture of the basic candidate is an RPR reference picture (S2022) (S2032).

[0206] In contrast, if the reference picture of the basic candidate is a non-RPR reference picture (S2022), the image encoding device 100 and the image decoding device 200 can set the index mmvdIdx, which indicates the MMVD candidate, to an initial value (0) (S2024), and then calculate the TM cost value for all MMVD candidates for the basic candidate (S2026, S2030) (S2028). Furthermore, the image encoding device 100 and the image decoding device 200 can rearrange all MMVD candidates based on the TM cost value calculated for each of them (S2034).

[0207] The image encoding device 100 and the image decoding device 200 can set up an MMVD list based on MMVD candidates whose TM cost value is set to a predetermined maximum value via the S2032 process, and a predetermined number (N) of MMVD candidates having the optimal TM cost among the rearranged MMVD candidates via the S2034 process (S2036).

[0208] MMVD candidates are sometimes called "secondary prediction candidates," and the MMVD list is sometimes called the "secondary prediction candidate list." MMVD candidates can be derived by applying an offset to the primary candidate.

[0209] Figure 21 is a diagram illustrating an exemplary content streaming system to which the embodiments of this disclosure can be applied.

[0210] As shown in Figure 21, a content streaming system to which an embodiment of the present disclosure is applied may broadly include an encoding server, a streaming server, a web server, media storage, user equipment, and multimedia input devices.

[0211] The encoding server is responsible for compressing content input from multimedia input devices such as smartphones, cameras, and camcorders into digital data to generate a bitstream, and transmitting this bitstream to the streaming server. In other cases, if a multimedia input device such as a smartphone, camera, or video camera directly generates the bitstream, the encoding server can be omitted.

[0212] The bitstream can be generated by an image encoding method and / or image encoding apparatus to which an embodiment of the present disclosure is applied, and the streaming server can temporarily store the bitstream in the process of transmitting or receiving the bitstream.

[0213] The streaming server transmits multimedia data to the user's device based on the user's request via a web server, and the web server can act as an intermediary to inform the user of available services. When a user requests a desired service from the web server, the web server transmits this to the streaming server, and the streaming server can transmit multimedia data to the user. In this case, the content streaming system may include a separate control server, in which case the control server can play a role in controlling the commands and responses between the devices within the content streaming system.

[0214] The streaming server can receive content from media storage and / or encoding servers. For example, when receiving content from the encoding server, the content can be received in real time. In this case, in order to provide a smooth streaming service, the streaming server can store the bitstream for a certain period of time.

[0215] Examples of user devices include mobile phones, smartphones, laptop computers, digital broadcasting terminals, PDAs (personal digital assistants), PMPs (portable multimedia players), navigation systems, slate PCs, tablet PCs, ultrabooks, wearable devices such as smartwatches, smart glasses, HMDs (head-mounted displays), digital TVs, desktop computers, and digital signage.

[0216] Each server within the aforementioned content streaming system can be operated as a distributed server, in which case the data received from each server can be processed in a distributed manner.

[0217] The scope of this disclosure includes software or machine-executable commands (e.g., operating systems, applications, firmware, programs, etc.) that enable the operation of various embodiments to be performed on a device or computer, and non-transitory computer-readable medium on which such software or commands etc. are stored and can be executed on a device or computer. [Industrial applicability]

[0218] The embodiments described herein can be used for encoding / decoding images.

Claims

1. An image decoding method performed by an image decoding device, The steps include: constructing a list of prediction candidates for the current block, The steps include: deriving an error value for the current block based on whether the reference picture referenced by the prediction candidate in the prediction candidate list is a resampled reference picture; The step includes rearranging the order of the prediction candidates in the prediction candidate list based on the error value, An image decoding method wherein, based on the fact that the reference picture is the resampled reference picture, the error value is derived to be a predetermined value.

2. The order of the prediction candidates in the prediction candidate list is rearranged in ascending order based on the error value. The image decoding method according to claim 1, wherein the error value is derived to be a predetermined maximum value based on the fact that the reference picture is the resampled reference picture.

3. The image decoding method according to claim 1, wherein, based on the fact that the reference picture is not the resampled reference picture, the error value is derived based on the difference between the template of the current block and the reference template in the reference picture.

4. The image decoding method according to claim 1, wherein the predicted candidate is one of the merge candidate for the current block, the temporal candidate for the current block, or the non-adjacent candidate for the current block.

5. The prediction candidates include a first prediction candidate and a second prediction candidate obtained by applying an offset to the first prediction candidate. The aforementioned list of prediction candidates includes a first list of prediction candidates containing the first prediction candidate, and a second list of prediction candidates containing the second prediction candidate. The steps described above include the step of constructing the first prediction candidate list, The derivation step includes deriving an error value between the current block and the second prediction candidate based on the fact that the reference picture referenced by the first prediction candidate is not the resampled reference picture, The image decoding method according to claim 1, wherein the rearrangement step includes rearranging the order of the second prediction candidates in the second prediction candidate list based on the error value.

6. An image encoding method performed by an image encoding device, The steps include: constructing a list of prediction candidates for the current block, The steps include: deriving an error value for the current block based on whether the reference picture referenced by the prediction candidate in the prediction candidate list is a resampled reference picture; The step includes rearranging the order of the prediction candidates in the prediction candidate list based on the error value, An image encoding method in which, based on the fact that the reference picture is the resampled reference picture, the error value is derived to be a predetermined value.

7. A method for transmitting a bitstream generated by an image encoding method, The aforementioned image encoding method is The steps include: constructing a list of prediction candidates for the current block, The steps include: deriving an error value for the current block based on whether the reference picture referenced by the prediction candidate in the prediction candidate list is a resampled reference picture; The step includes rearranging the order of the prediction candidates in the prediction candidate list based on the error value, A method for deriving the error value to a predetermined value based on the fact that the reference picture is the resampled reference picture.

Images