Method of coding and computer readable medium

Abstract

A method of coding a video signal and a computer readable medium are provided. The method for processing a video signal according to an embodiment of the present specification includes a step of forming a reference picture list of a current picture in the video signal, and a step of performing prediction on the current picture by using the reference picture list, wherein the step of forming the reference picture list includes a step of obtaining a picture order count (POC) difference between a picture related to a first entry of the reference picture list and another picture when the first entry corresponds to a short-term reference picture (STRP) entry, and a step of obtaining a POC modulo value of a picture related to a second entry of the reference picture list when the second entry corresponds to a long-term reference picture (LTRP) entry. A reference picture list for simply and effectively identifying a picture can be generated to improve compression performance and reduce computation time.

Description

[0001] This application is a divisional application of the original application No. 202080002063.4 (International Application No. PCT / KR2020 / 000032, filed on January 2, 2020, entitled "Method and apparatus for processing video signal by using inter prediction"). TECHNICAL FIELD TECHNICAL FIELD

[0002] Embodiments of the disclosure relate to a method and apparatus for processing a video signal using inter prediction, and more particularly, to a method and apparatus for constructing a reference picture list to predict a current picture. BACKGROUND

[0003] Compression coding refers to a series of signal processing techniques for transmitting digitized information via a communication line or storing digitized information in a suitable format in a storage medium. Video, image, sound, or other media can be the target of compression coding. In particular, a compression coding technique for video is referred to as video compression.

[0004] Next-generation video content will be characterized by high spatial resolution, high frame rate, and high dimensionality in scene representation. Processing for such content will result in a large increase in memory space, memory access rate, and processing power.

[0005] Accordingly, there is a need for the design of coding tools for efficiently processing next-generation video content. In particular, the High Efficiency Video Coding (HEVC) video codec standard requires a prediction technique capable of accurately generating predicted samples while more efficiently using resources. SUMMARY

[0006] TECHNICAL PROBLEM

[0007] Embodiments of the disclosure provide a method and apparatus for constructing a reference picture list necessary to predict a current picture.

[0008] Embodiments of the disclosure provide a method and apparatus for obtaining information necessary to construct a reference picture list.

[0009] Embodiments of the disclosure provide a method and apparatus for limiting the number of entries necessary to construct a reference picture list.

[0010] Embodiments of the disclosure provide a method and apparatus for reducing the amount of syntax elements necessary to construct a reference picture list.

[0011] Embodiments of the present disclosure provide a method and apparatus for counting the number of reference picture marking during a process of constructing a reference picture list.

[0012] The objects of the present disclosure are not limited to the aforementioned objects, and other objects not mentioned will be apparent to those skilled in the art from the following description.

[0013] Technical solutions

[0014] Embodiments of the present disclosure provide a method and apparatus for decoding a video signal using inter prediction. According to embodiments of the present disclosure, a method for processing a video signal includes the steps of constructing a reference picture list of a current picture in the video signal, and performing prediction on the current picture by using the reference picture list, wherein the step of constructing the reference picture list includes obtaining a picture order count (POC) difference between a picture related to a first entry of the reference picture list and another picture if the first entry corresponds to a short-term reference picture (STRP), and obtaining a POC modulo value of a picture related to a second entry of the reference picture list if the second entry corresponds to a long-term reference picture (LTRP).

[0015] According to embodiments, the step of constructing the reference picture list can include obtaining a number of entries of the reference picture list, and performing reference picture marking on each entry of the reference picture list by as many times as the number of entries, as an LTRP or an STRP.

[0016] According to embodiments, a maximum value of the number of entries can be a maximum required size of a decoded picture buffer plus 14.

[0017] According to embodiments, the step of performing the reference picture marking can include obtaining a number of STRP entries and a number of LTRP entries, and performing STRP marking by as many times as the number of STRP entries and performing LTRP marking by as many times as the number of LTRP entries.

[0018] According to embodiments, the step of constructing the reference picture list can include obtaining a first flag, and determining whether an entry of the reference picture list corresponds to an STRP entry or an LTRP entry based on the first flag.

[0019] According to embodiments, the POC difference can be an absolute value of a POC of the other picture minus a POC of the picture related to the first entry.

[0020] According to embodiments, the step of constructing the reference picture list can include obtaining a second flag indicating a sign of the POC difference if the absolute value is greater than 0.

[0021] According to embodiments of the disclosure, a method for encoding a video signal includes the steps of constructing a reference picture list for a current picture in the video signal, performing prediction for the current picture by using the reference picture list, and encoding information for constructing the reference picture list. The information for constructing the reference picture list can include a picture order count (POC) difference between a picture related to a short-term reference picture (STRP) entry and another picture in the reference picture list and a POC modulo value of a picture related to a second entry in the reference picture list to a long-term reference picture (LTRP) entry.

[0022] According to embodiments, the encoded information for constructing the reference picture list can further include a number of entries of the reference picture list. A maximum value of the number of entries can be a maximum required size of a decoded picture buffer plus 14.

[0023] According to embodiments, the encoded information for constructing the reference picture list can further include a first flag indicating whether an entry of the reference picture list corresponds to a STRP entry.

[0024] According to embodiments, the encoded information for constructing the reference picture list can further include a second flag indicating a sign of a POC difference for an entry with a POC difference greater than 0.

[0025] According to embodiments, the encoded information for constructing the reference picture list can be included in a sequence parameter set (SPS), a picture parameter set (PPS), or a slice header in a bitstream related to the video signal.

[0026] According to another embodiment of the disclosure, an apparatus for decoding a video signal includes a memory for storing the video signal and a processor coupled to the memory. The processor is configured to construct a reference picture list for a current picture in the video signal and perform prediction for the current picture using the reference picture list. To construct the reference picture list, the processor is configured to obtain a picture order count (POC) difference between a picture related to a first entry of the reference picture list and another picture if the first entry corresponds to a short-term reference picture (STRP) and obtain a POC modulo value of a picture related to a second entry of the reference picture list if the second entry corresponds to a long-term reference picture (LTRP).

[0027] According to another embodiment of the disclosure, an apparatus for encoding a video signal includes a memory for storing a video signal and a processor coupled to the memory. The processor is configured to construct a reference picture list of a current picture in the video signal and encode information for constructing the reference picture list. The encoded information for constructing the reference picture list can include a picture order count (POC) difference between a picture related to a first entry and another picture in the reference picture list and a POC modulo value of a picture related to a second entry in the reference picture list.

[0028] According to another embodiment of the disclosure, a non-transitory computer readable medium storing one or more instructions is provided. The one or more instructions, when executed by one or more processors, control a video signal processing apparatus to construct a reference picture list of a current picture in a video signal and perform prediction for the current picture using the reference picture list. The one or more instructions control the video signal processing apparatus to obtain a picture order count (POC) difference between a picture related to a first entry and another picture if the first entry of the reference picture list corresponds to a short-term reference picture (STRP) and obtain a POC modulo value of a picture related to a second entry if the second entry of the reference picture list corresponds to a long-term reference picture (LTRP) for constructing the reference picture list.

[0029] Advantageous Effects

[0030] According to embodiments of the disclosure, a reference picture list necessary for predicting a current picture can be constructed by marking the reference pictures.

[0031] According to embodiments of the disclosure, appropriate information can be obtained according to the type of the reference pictures.

[0032] According to embodiments of the disclosure, the number of entries necessary for constructing a reference picture list can be limited so that a memory necessary for storing pictures can be efficiently managed.

[0033] According to embodiments of the disclosure, the amount of data necessary for constructing a reference picture list can be omitted to parse some syntax elements.

[0034] According to embodiments of the disclosure, syntax for constructing a reference picture list can be efficiently implemented by counting a reference picture marking count.

[0035] Effects of the disclosure are not limited to the aforementioned effects, and other unmentioned effects will be clearly understood by those skilled in the art from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0036] The present disclosure will become better understood when considered in conjunction with the following detailed description and with reference to the following drawings, in which:

[0037] Figure 1 FIG. 1 is a diagram illustrating an example video encoding system according to an embodiment of the present disclosure;

[0038] Figure 2 FIG. 2 is a block diagram schematically illustrating an encoding apparatus for encoding a video / image signal according to an embodiment of the present disclosure;

[0039] Figure 3 FIG. 3 is a block diagram schematically illustrating a decoding apparatus for decoding a video signal according to an embodiment of the present disclosure;

[0040] Figure 4 FIG. 4 is a diagram illustrating a structure of a content streaming system according to an embodiment of the present disclosure;

[0041] Figure 5 FIG. 5 is a block diagram illustrating an example apparatus for processing a video signal according to an embodiment of the present disclosure;

[0042] Figure 6a , Figure 6b , Figure 6c and Figure 6d are diagrams illustrating example block split structures obtained by a quad tree (QT), a binary tree (BT), a ternary tree (TT), and an asymmetric tree (AT), respectively, according to an embodiment of the present disclosure;

[0043] Figure 7 and Figure 8 illustrate an inter prediction-based video / image encoding procedure and an inter prediction unit in an encoding apparatus according to an embodiment of the present disclosure;

[0044] Figure 9 and Figure 10 illustrate an inter prediction-based video / image decoding procedure and an inter prediction unit in a decoding apparatus according to an embodiment of the present disclosure;

[0045] Figure 11 is a flowchart illustrating an example of constructing a reference picture list according to an embodiment of the present disclosure;

[0046] Figure 12 and Figure 13 are flowcharts illustrating examples of marking a reference picture according to an embodiment of the present disclosure;

[0047] Figure 14 is a flowchart illustrating another example of marking a reference picture according to an embodiment of the present disclosure; and

[0048] Figure 15 FIG. 1 is a flowchart illustrating an example of processing a video signal according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0049] Hereinafter, preferred embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. The following detailed description is intended to describe examples of the present disclosure, not to present the only embodiment of the present disclosure. The following detailed description includes specific details to convey a thorough understanding of the present disclosure. However, those skilled in the art will readily appreciate that the embodiments of the present disclosure can be practiced without these specific details.

[0050] In some cases, in order to avoid obscuring concepts, known structures or devices can be omitted or shown in the form of a block diagram while focusing on core features of each structure and device.

[0051] Although the embodiments of the present disclosure are mainly described using general terms that are widely used at present, some specific terms can be selected by the inventors. In such a case, the definition of the term by the inventors can be given. Accordingly, the terms used herein should be interpreted based on the definition made by the inventors, not the simple definition in the dictionary.

[0052] The terms used herein are provided for better understanding of the present disclosure, and changes can be made thereto without departing from the technical spirit of the present disclosure. For example, the terms "signal", "data", "sample", "picture", "slice", "tile", "frame", and "block" can be appropriately replaced and interpreted in each encoding process.

[0053] As used herein, the term "processing unit" refers to a unit of processing such as encoding / decoding (e.g., prediction, transform, and / or quantization). The "processing unit" can be interpreted to include a unit for a luma component and a unit for a chroma component. For example, the processing unit can correspond to a block, a coding unit (CU), a prediction unit (PU), or a transform unit (TU).

[0054] The processing unit can also be interpreted as a unit for a luma component or a unit for a chroma component. For example, the processing unit can correspond to a coding tree block, a coding block (CB), a PU, or a transform block (TB) for a luma component. Alternatively, the processing unit can correspond to a CTB, a CB, a PU, or a TB for a chroma component. However, it is not limited thereto, and the processing unit can also be interpreted to include a unit for a luma component and a unit for a chroma component.

[0055] Furthermore, the processing unit is not necessarily limited to a square block, but rather can be a polygon having three or more vertices.

[0056] As used herein, "pixel" and "coefficient" (e.g., transform coefficient or transform coefficient having undergone a first transform) can be collectively referred to as sample. When sample is used, this can mean using, for example, a pixel value or a coefficient (e.g., transform coefficient or transform coefficient having undergone a first transform).

[0057] Figure 1 An example video coding system according to an embodiment of the disclosure is shown.

[0058] The video coding system can include a source device 10 and a receiving device 20. The source device 10 can transmit encoded video / image information or data in the form of a file or a stream to the receiving device 20 via a digital storage medium or a network.

[0059] The source device 10 can include a video source 11, an encoding apparatus 12, and a transmitter 13. The receiving device 20 can include a receiver 21, a decoding apparatus 22, and a renderer 23. The encoding apparatus 12 can be referred to as a video / image encoding apparatus, and the decoding apparatus 22 can be referred to as a video / image decoding apparatus. The transmitter 13 can be included in the encoding apparatus 12. The receiver 21 can be included in the decoding apparatus 22. The renderer 23 can include a display unit, and the display unit can be configured as a separate device or an external component.

[0060] The video source can obtain a video / image by capturing, synthesizing, or generating a video / image. The video source can include a video / image capturing device and / or a video / image generating device. The video / image capturing device can include, for example, one or more cameras and a video / image archive including previously captured video / images. The video / image generating device can include, for example, a computer, a tablet PC, or a smart phone, and can (electronically) generate a video / image. For example, a virtual video / image can be generated via, for example, a computer, in which case a process for generating its related data can replace a video / image capturing process.

[0061] The encoding apparatus 12 can encode an input video / image. For compression and coding efficiency, the encoding apparatus 12 can perform a series of processes such as prediction, transform, and quantization. The encoded data (encoded video / image information) can be output in the form of a bitstream.

[0062] The transmitter 13 can transmit the encoded video / image information or data, which has been output in the form of a bitstream, to the receiver of the receiving apparatus in the form of a file or a stream via a digital storage medium or a network. The digital storage medium can include various storage media such as a universal serial bus (USB), a secure digital (SD), a compact disc (CD), a digital video disc (DVD), a Blu-ray, a hard drive, or a solid state drive (SSD) medium. The transmitter 13 can include elements for generating a media file in a predetermined file format and elements for transmitting over a broadcasting / communication network. The receiver 21 can extract the bitstream and transmit the bitstream to the decoding device 22.

[0063] The decoding device 22 can perform a series of procedures such as inverse quantization, inverse transform, and prediction corresponding to the operations of the encoding device 12, thereby decoding the video / image.

[0064] The renderer 23 can render the decoded video / image. The rendered video / image can be displayed on a display unit.

[0065] Figure 2 is a block diagram schematically illustrating an encoding device for encoding a video / image signal according to an embodiment of the disclosure.

[0066] Referring to Figure 2 The encoding device 100 can include an image partitioning unit 110, a subtractor 115, a transform unit 120, a quantization unit 130, an inverse quantization unit 140, an inverse transform unit 150, an adder 155, a filter unit 160, a memory 170, an inter prediction unit 180, an intra prediction unit 185, and an entropy encoding unit 190. The inter prediction unit 180 and the intra prediction unit 185 can be collectively referred to as a predictor. In other words, the predictor can include the inter prediction unit 180 and the intra prediction unit 185. The transform unit 120, the quantization unit 130, the inverse quantization unit 140, and the inverse transform unit 150 can be included in a residual processor. The residual processor can further include the subtractor 115. According to an embodiment, the above-described image partitioning unit 110, subtractor 115, transform unit 120, quantization unit 130, inverse quantization unit 140, inverse transform unit 150, adder 155, filter unit 160, inter prediction unit 180, intra prediction unit 185, and entropy encoding unit 190 can be configured as a single hardware component (e.g., an encoder or a processor). According to an embodiment, the memory 170 can be configured as a single hardware component (e.g., a memory or a digital storage medium), and the memory 170 can include a decoded picture buffer (DPB) 175.

[0067] The image segmentation unit 110 can split an image (or a picture or a frame) input to the encoding apparatus 100 into one or more processing units. As an example, the processing units can be referred to as coding units (CUs). In this case, the coding units can be recursively split from a coding tree unit (CTU) or a largest coding unit (LCU) according to a quadtree binary tree (QTBT) structure. For example, one coding unit can be split into multiple coding units of a deeper depth based on a quadtree structure and / or a binary tree structure. In this case, for example, the quadtree structure can be first applied, and then the binary tree structure can be applied. Or the binary tree structure can be first applied. An encoding procedure according to an embodiment of the disclosure can be performed based on a final coding unit that is no longer split. In this case, the largest coding unit can be directly used as the final coding unit based on, for example, coding efficiency per image property, or the coding unit can be recursively split into a coding unit of a lower depth when necessary, and a coding unit of an optimal size can be used as the final coding unit. The encoding procedure can include, for example, prediction, transform, or reconstruction described below. As an example, the processing units can also include prediction units (PUs) or transform units (TUs). In this case, the prediction units and the transform units can each be split into or partitioned from the final coding units described above. The prediction units can be units of sample prediction, and the transform units can be units for deriving transform coefficients and / or units for deriving a residual signal from the transform coefficients.

[0068] In some cases, the term "unit" can be used interchangeably with "block" or "area". In general, an MxN block can denote a set of samples or transform coefficients consisting of M columns and N rows. In general, a sample can denote a pixel or a pixel value or can denote a pixel / pixel value of only a luminance component or a pixel / pixel value of only a chrominance component. A sample can be used as a term corresponding to a pixel or a pel of one picture (or image).

[0069] The encoding device 100 can generate a residual signal (a residual block or a residual array of samples) by subtracting a prediction signal (a prediction block or a prediction array of samples) output from the inter prediction unit 180 or the intra prediction unit 185 from an input image signal (an original block or an original array of samples), and transmit the generated residual signal to the transform unit 120. In this case, as illustrated, a unit for subtracting a prediction signal (a prediction block or a prediction array of samples) from an input image signal (an original block or an original array of samples) in the encoding device 100 can be referred to as a subtractor 115. The predictor can perform prediction on a target block (hereinafter, a current block) for processing and generate a prediction block including prediction samples for the current block. The predictor can determine whether to apply intra prediction or inter prediction in each block or CU unit. The predictor can generate pieces of information for prediction (e.g., prediction mode information), as described below in connection with various prediction modes, and transfer the generated information to the entropy encoding unit 190. The prediction-related information can be encoded by the entropy encoding unit 190 and output in the form of a bitstream.

[0070] The intra prediction unit 185 can predict the current block by referring to samples in the current picture. Depending on the prediction mode, the referred samples can be located adjacent to or away from the current block. In intra prediction, the prediction mode can include a plurality of non-directional modes and a plurality of directional modes. The non-directional modes can include, for example, a DC mode and a planar mode. Depending on the degree of precision of the prediction direction, the directional modes can include, for example, 33 directional prediction modes or 65 directional prediction modes. However, this is merely an example, and more or less directional prediction modes can be used. The intra prediction unit 185 can determine a prediction mode applied to the current block using a prediction mode applied to a neighboring block.

[0071] The inter prediction unit 180 can derive a prediction block of the current block based on reference blocks (reference sample arrays) specified by motion vectors on reference pictures. Here, to reduce the amount of motion information transmitted in the inter prediction mode, the motion information can be predicted in a block, sub-block, or sample basis based on correlation in the motion information between neighboring blocks and the current block. The motion information can include a motion vector and a reference picture index. The motion information can also include inter prediction direction (L0 prediction, L1 prediction, or Bi prediction) information. In the case of inter prediction, the neighboring blocks can include spatial neighboring blocks existing in the current picture and temporal neighboring blocks existing in the reference pictures. The reference pictures including the reference blocks can be the same as or different from the reference pictures including the temporal neighboring blocks. The temporal neighboring blocks can be referred to as, for example, co-located reference blocks or colCUs, and the reference pictures including the temporal neighboring blocks can be referred to as colPics. For example, the inter prediction unit 180 can construct a motion information candidate list based on the neighboring blocks and generate information indicating which candidate is used to derive the motion vector and / or the reference picture index of the current block. The inter prediction can be performed based on various prediction modes. For example, in a skip mode or a merge mode, the inter prediction unit 180 can use motion information of the neighboring blocks as motion information of the current block. In the skip mode, unlike in the merge mode, a residual signal can not be transmitted. In a motion vector prediction (MVP) mode, a motion vector of the neighboring blocks can be used as a motion vector predictor, and a motion vector difference can be signaled, indicating the motion vector of the current block.

[0072] The prediction signal generated via the inter prediction unit 180 or the intra prediction unit 185 can be used to generate a reconstructed signal or a residual signal.

[0073] The transform unit 120 can apply a transform scheme to the residual signal, thereby generating transform coefficients. For example, the transform scheme can include at least one of a discrete cosine transform (DCT), a discrete sine transform (DST), a Karhunen-Loeve transform (KLT), a graph-based transform (GBT), or a conditional non-linear transform (CNT). The GBT refers to a transform obtained from a graph representing information on a relationship between pixels. The CNT refers to a transform obtained based on generating a prediction signal using all previously reconstructed pixels. Further, the transform process can be applied to square blocks of pixels having the same size or can also be applied to blocks of variable sizes that are not square.

[0074] The quantization unit 130 can quantize the transform coefficients and transmit the quantized transform coefficients to the entropy encoding unit 190, and the entropy encoding unit 190 can encode and output the quantized signal (information of the quantized transform coefficients) in a bitstream. The information of the quantized transform coefficients can be referred to as residual information. The quantization unit 130 can re-sort the quantized transform coefficients in a block form in a one-dimensional vector form based on a coefficient scan order, and generate the information of the quantized transform coefficients based on the one-dimensional form of the quantized transform coefficients. The entropy encoding unit 190 can perform various encoding methods such as, for example, exponential Golomb, context adaptive variable length coding (CAVLC), or context adaptive binary arithmetic coding (CABAC). The entropy encoding unit 190 can encode the values of pieces of information (e.g., syntax elements) necessary for reconstructing a video / picture together with or separately from the quantized transform coefficients. The encoded information (e.g., video / image information) can be transmitted or stored in a bitstream form based on each network abstraction layer (NAL) unit. The bitstream can be transmitted via a network or stored in a digital storage medium. The network can include, for example, a broadcasting network and / or a communication network, and the digital storage medium can include, for example, a USB, an SD, a CD, a DVD, a Blu-ray, an HDD, an SSD, or other various storage media. A transmitter (not shown) for transmitting the signal output from the entropy encoding unit 190 and / or a storage unit (not shown) for storing the signal output from the entropy encoding unit 190 can be configured as an internal / external element of the encoding apparatus 100, or the transmitter can be a component of the entropy encoding unit 190.

[0075] The quantized transform coefficients output from the quantization unit 130 can be used to generate a prediction signal. For example, a residual signal can be reconstructed by applying inverse quantization and inverse transformation to the quantized transform coefficients via the inverse quantization unit 140 and the inverse transformation unit 150 in a loop. The adder 155 can add the reconstructed residual signal to the prediction signal output from the inter prediction unit 180 or the intra prediction unit 185, thereby generating a reconstructed signal (a reconstructed picture, a reconstructed block, or a reconstructed sample array). As in the case of applying a skip mode, when there is no residual for a target block for processing, a prediction block can be used as a reconstructed block. The adder 155 can be expressed as a reconstructor or a reconstructed block generator. The reconstructed signal can be used for intra prediction of a next target processing block in a current picture, and (as described below) filtered and then used for inter prediction of a next picture.

[0076] The filtering unit 160 can enhance subjective / objective picture quality by applying filtering to the reconstructed signal. For example, the filtering unit 160 can generate a modified reconstructed picture by applying various filtering methods to the reconstructed picture and transmit the modified reconstructed picture to the decoded picture buffer 170. The various filtering methods can include, for example, a deblocking filter, a sample adaptive offset, an adaptive loop filter, or a bilateral filter. The filtering unit 160 can generate pieces of information for filtering as described below in connection with each filtering method and transfer the resulting information to the entropy encoding unit 190. The filtering-related information can be encoded by the entropy encoding unit 190 and output in the form of a bitstream.

[0077] The modified reconstructed picture transmitted to the decoded picture buffer 170 can be used as a reference picture in the inter prediction unit 180. The encoding apparatus 100 (when inter prediction is applied thereto) can avoid prediction mismatch between the encoding apparatus 100 and the decoding apparatus 200 and improve coding efficiency.

[0078] The decoded picture buffer 170 can store the modified reconstructed picture to be used as a reference picture in the inter prediction unit 180.

[0079] Figure 3 is a block diagram schematically illustrating a decoding apparatus for decoding a video signal according to an embodiment of the disclosure.

[0080] Referring to Figure 3 The decoding apparatus 200 can include an entropy decoding unit 210, an inverse quantization unit 220, an inverse transform unit 230, an adder 235, a filtering unit 240, a memory 250, an inter prediction unit 260, and an intra prediction unit 265. The inter prediction unit 260 and the intra prediction unit 265 can be collectively referred to as a predictor. In other words, the predictor can include the inter prediction unit 180 and the intra prediction unit 185. The inverse quantization unit 220 and the inverse transform unit 230 can be collectively referred to as a residue processor. In other words, the residue processor can include the inverse quantization unit 220 and the inverse transform unit 230. According to an embodiment, the entropy decoding unit 210, the inverse quantization unit 220, the inverse transform unit 230, the adder 235, the filtering unit 240, the inter prediction unit 260, and the intra prediction unit 265 can be configured in a single hardware component (e.g., a decoder or a processor). According to an embodiment, the decoded picture buffer 250 can be implemented as a single hardware component (e.g., a memory or a digital storage medium). The memory can include the DPB 250 and be configured by the digital storage medium.

[0081] When a bitstream including video / image information is input, the decoding apparatus 200 can reconstruct a picture corresponding to the picture included in the bitstream by decoding the bitstream. Figure 2The image corresponds to the video / image information process in the encoding device 100. For example, the decoding device 200 can perform decoding using a processing unit applied in the encoding device 100. Therefore, during decoding, the processing unit can be, for example, an encoding unit, and the encoding unit can be split from the encoding tree unit or the maximum encoding unit according to a quadtree structure and / or a binary tree structure. The reconstructed image signal decoded and output by the decoding device 200 can be played via a player.

[0082] Decoding device 200 can receive data from a source in the form of a bitstream. Figure 2 The signal output by the encoding device 100 can be decoded by the entropy decoding unit 210. For example, the entropy decoding unit 210 can parse the bitstream and extract the information necessary for image reconstruction (or picture reconstruction) (e.g., video / image information). For example, the entropy decoding unit 210 can decode the information in the bitstream based on encoding methods such as exponential Golomb coding, CAVLC, or CABAC and can output the values ​​of the syntax elements necessary for image reconstruction and the quantized values ​​of the transform coefficients of the residuals. Specifically, the CABAC entropy decoding method can receive a binary number (bin) corresponding to each syntax element in the bitstream, use the decoding target syntax element information, the decoding information of the neighboring and decoding target blocks, or the information of the symbol / binary number decoded in a previous step to determine the context model, predict the probability of occurrence of the binary number based on the determined context model, and perform arithmetic decoding of the binary number. At this time, after determining the context model, the CABAC entropy decoding method can update the context model using the information of the symbol / binary number decoded for the context model of the next symbol / binary number. Among the information decoded by the entropy decoding unit 210, information for prediction can be provided to the predictor (e.g., inter-frame prediction unit 260 and intra-frame prediction unit 265), and the residual values ​​(i.e., quantized transform coefficients and related processor information) entropy-decoded by the entropy decoding unit 210 can be input to the inverse quantization unit 220. Among the information decoded by the entropy decoding unit 210, information for filtering can be provided to the filtering unit 240. Furthermore, the receiver (not shown) for receiving the signal output from the encoding device 100 can be configured as an internal / external component of the decoding device 200, or the receiver can be a component of the entropy decoding unit 210.

[0083] The inverse quantization unit 220 can inverse quantize the quantized transform coefficients and output the transform coefficients. The inverse quantization unit 220 can reorder the quantized transform coefficients in the form of two-dimensional blocks. In this case, the reordering can be performed based on a coefficient scan order in which the encoding device 100 has performed. The inverse quantization unit 220 can inverse quantize the quantized transform coefficients using a quantization parameter (e.g., quantization step information), thereby obtaining transform coefficients.

[0084] The inverse transform unit 230 can output a residual signal (a residual block or a residual sample array) by applying an inverse transform to the transform coefficients.

[0085] The predictor can perform prediction for the current block and generate a prediction block including prediction samples of the current block. The predictor can determine which of intra prediction or inter prediction is to be applied to the current block and determine a specific intra / inter prediction mode based on information for prediction output from the entropy decoding unit 210.

[0086] The intra prediction unit 265 can predict the current block by referring to samples in the current picture. The referred samples can be located adjacent to or away from the current block depending on the prediction mode. In intra prediction, the prediction mode can include various non-directional modes and various directional modes. The intra prediction unit 265 can determine the prediction mode applied to the current block using the prediction mode applied to the neighboring block.

[0087] The inter prediction unit 260 can derive a prediction block of the current block based on a reference block (a reference sample array) specified by a motion vector on a reference picture. Here, to reduce the amount of motion information transmitted in the inter prediction mode, the motion information can be predicted in a block, sub-block, or sample basis based on correlation in the motion information between neighboring blocks and the current block. The motion information can include a motion vector and a reference picture index. The motion information can further include information of an inter prediction direction (L0 prediction, L1 prediction, or Bi prediction). In the case of inter prediction, the neighboring blocks can include spatial neighboring blocks existing in the current picture and temporal neighboring blocks existing in the reference picture. The inter prediction unit 260 can construct a motion information candidate list based on the neighboring blocks and derive a motion vector and / or a reference picture index of the current block based on received candidate selection information. The inter prediction can be performed based on various prediction modes. The information for prediction can include information indicating a mode of inter prediction for the current block.

[0088] The adder 235 can add the obtained residual signal to a prediction signal (e.g., a prediction block or a prediction sample array) output from the inter prediction unit 260 or the intra prediction unit 265, thereby generating a reconstructed signal (a reconstructed picture, a reconstructed block, or a reconstructed sample array). As in the case of applying the skip mode, when there is no residual for a target block to be processed, the prediction block can be used as the reconstructed block.

[0089] The adder 235 can be denoted as a reconstructor or a reconstructed block generator. The generated reconstructed signal can be used for intra prediction of a next target processing block in the current picture, and (as described below) filtered and then used for inter prediction of a next picture.

[0090] The filter unit 240 can enhance subjective / objective image quality by applying filtering to the reconstructed signal. For example, the filter unit 240 can generate a modified reconstructed picture by applying various filtering methods to the reconstructed picture and transmit the modified reconstructed picture to the decoded picture buffer 250. The various filtering methods can include, for example, deblocking filtering, sample adaptive offset (SAO), adaptive loop filter (ALF), or bilateral filter.

[0091] The modified reconstructed picture transmitted to the decoded picture buffer 250 can be used as a reference picture by the inter prediction unit 260.

[0092] In the present disclosure, the embodiments described above in connection with the filter unit 160, the inter prediction unit 180, and the intra prediction unit 185 of the encoding apparatus 100 can be applied in the same manner as or corresponding to the filter unit 240, the inter prediction unit 260, and the intra prediction unit 265 of the decoding apparatus.

[0093] Figure 4 FIG. 1 is a diagram illustrating a structure of a content streaming system according to an embodiment of the present disclosure.

[0094] The content streaming system to which the present disclosure is applied can mainly include an encoding server 410, a web server 430, a media storage 440, a user device 450, and a multimedia input device 460.

[0095] The encoding server 410 can compress content input from a multimedia input device (e.g., a smartphone, a camera, or a camcorder) into digital data, generate a bitstream, and transmit the bitstream to a streaming server 420. As an example, when the multimedia input device 460 (e.g., a smartphone, a camera, or a camcorder) itself generates a bitstream, the encoding server 410 can be omitted.

[0096] A bitstream can be generated by applying the encoding or bitstream generation method of the present disclosure, and the streaming server 420 can temporarily store the bitstream while transmitting or receiving the bitstream.

[0097] The streaming server 420 can transmit multimedia data to the user device 450 based on a user request through the web server 430, and the web server 430 acts as an agent to inform the user of what service is provided. When the user transmits a request for a desired service to the web server 430, the web server 430 transfers the request to the streaming server 420, and the streaming server 420 transmits multimedia data to the user. The content streaming system can include a separate control server, in which case the control server controls commands / responses between devices in the content streaming system.

[0098] The streaming server 420 can receive content from the media storage device 440 and / or the encoding server 410. For example, the streaming server 420 can receive content from the encoding server 410 in real time. In this case, in order to provide a seamless service, the streaming server 420 can store a bitstream for a predetermined time.

[0099] For example, examples of the user device 450 can include a mobile phone, a smart phone, a laptop computer, a digital broadcasting terminal, a personal digital assistant (PDA), a portable multimedia player (PMP), a navigation device, a touch screen tablet PC, a tablet PC, an ultrabook, a wearable device (e.g., a smart watch, smart glasses, or a head-mounted display (HMD)), a digital TV, a desktop computer, or a digital signage device.

[0100] In the content streaming system, the server can be a distributed server, in which case each server received data can be distributed and processed.

[0101] Figure 5 is a block diagram illustrating an example device for processing a video signal according to an embodiment of the present disclosure. Figure 5 The video signal processing device of Figure 2 The encoding apparatus 100 of Figure 3 The decoding apparatus 200 of

[0102] According to an embodiment of the present disclosure, the video signal processing device 500 can include a memory 520 for storing a video signal and a processor 510 coupled with the memory to process the video signal.

[0103] According to an embodiment of the present disclosure, the processor 510 can be configured as at least one processing circuitry for processing a video signal and can execute instructions for encoding or decoding the video signal to process the video signal. In other words, the processor 510 can encode the video signal or decode the encoded video signal by executing the encoding or decoding method described below.

[0104] According to an embodiment of the present disclosure, the processor 510 can be configured as at least one processing circuitry for processing an image signal and can execute instructions for encoding or decoding the image signal to process the image signal. In other words, the processor 510 can encode the original image data or decode the encoded image signal by executing the encoding or decoding method described above.

[0105] Figure 6a Figure 6b Figure 6c Figure 6d FIGS. 1 to 4 are diagrams illustrating example block split structures obtained by a quad tree (QT), a binary tree (BT), a ternary tree (TT), and an asymmetric tree (AT), respectively, according to an embodiment of the present disclosure.

[0106] In video encoding, one block can be split by QT. A sub-block split by QT can be further split by QT recursively. A leaf block which is not split by QT any more can be split by at least one of BT, TT, or AT. BT can have two types of split such as horizontal BT (2N×N, 2N×N) and vertical BT (N×2N, N×2N). TT can have two types of split such as horizontal TT (2N×1 / 2N, 2N×N, 2N×1 / 2N) and vertical TT (1 / 2N×2N, N×2N, 1 / 2N×2N). AT can have four types of split such as horizontal-up AT (2N×1 / 2N, 2N×3 / 2N), horizontal-down AT (2N×3 / 2N, 2N×1 / 2N), vertical-left AT (1 / 2N×2N, 3 / 2N×2N), and vertical-right AT (3 / 2N×2N, 1 / 2N×2N). BT, TT, and AT can each use BT, TT, and AT to further split recursively.

[0107] Figure 6a An example of QT split is illustrated. Block A can be split into four sub-blocks (A0, A1, A2, A3) by QT. Sub-block A1 can be split again into four sub-blocks (B0, B1, B2, B3) by QT.

[0108] Figure 6b ​​​An example of BT splitting is shown. Block B3, which is no longer split by QT, can be split into vertical BT (C0, C1) or horizontal BT (D0, D1). Like block C0, each sub-block can be further recursively split, for example, in the form of horizontal BT (E0, E1) or vertical BT (F0, F1).

[0109] Figure 6c An example of TT splitting is shown. Block B3, which is no longer split by QT, can be split into vertical TT (C0, C1, C2) or horizontal TT (D0, D1, D2). Like block C1, each sub-block can be further recursively split, for example, in the form of horizontal TT (E0, E1, E2) or vertical TT (F0, F1, F2).

[0110] Figure 6d An example of AT splitting is shown. Block B3, which is no longer split by QT, can be split into vertical AT (C0, C1) or horizontal AT (D0, D1). Like block C1, each sub-block can be further recursively split, for example, in the form of horizontal AT (E0, E1) or vertical TT (F0, F1).

[0111] Meanwhile, BT, TT, and AT can be used together for splitting. For example, a sub-block split by BT can be split by TT or AT. Also, a sub-block split by TT can be split by BT or AT. A sub-block split by AT can be split by BT or TT. For example, after split by horizontal BT, each sub-block can be split by vertical BT, or after split by vertical BT, each sub-block can be split by horizontal BT. In this case, although different split orders are applied, the final shape after splitting can be the same.

[0112] When a block is split, various orders of searching blocks can be defined. Generally, searching is performed from left to right or from top to bottom. The order of searching blocks can refer to an order of determining whether to further split each sub-block split, an order of encoding each sub-block (in the case where a block is no longer split), or an order of searching when a sub-block refers to other neighboring blocks.

[0113] Figure 7 and Figure 8 A video / image encoding program based on inter prediction in an encoding apparatus according to an embodiment of the disclosure and an inter prediction unit are shown.

[0114] The encoding device 100 performs inter prediction on the current block (S710). The encoding device 100 can derive the motion i and the inter prediction mode of the current block and generate the prediction samples of the current block. Here, the determination of the inter prediction mode, the derivation of the motion information, and the generation of the prediction samples can be performed simultaneously, or any one of them can be performed earlier than the rest. For example, the inter prediction unit 180 of the encoding device 100 can include a prediction mode determination unit 181, a motion information derivation unit 182, and a prediction sample derivation unit 183. The prediction mode determination unit 181 can determine the prediction mode of the current block, the motion information derivation unit 182 can derive the motion information of the current block, and the prediction sample derivation unit 183 can derive the prediction samples of the current block. For example, the inter prediction unit 180 of the encoding device 100 can search for a block similar to the current block in a predetermined region (search region) of a reference picture via motion estimation and derive a reference block having the minimum difference or a predetermined difference or less from the current block. Based on this, a reference picture index indicating the reference picture in which the reference block is located can be derived, and a motion vector can be derived based on the positional difference between the reference block and the current block. The encoding device 100 can determine the mode applied to the current block among various prediction modes. The encoding device 100 can compare the RD costs of various prediction modes and determine the best prediction mode of the current block.

[0115] For example, when the skip mode or the merge mode is applied to the current block, the encoding device 100 can construct a merge candidate list described below, and derive a reference block having the minimum difference or a predetermined difference or less from the current block among the reference blocks indicated by the merge candidates included in the merge candidate list. In this case, the merge candidate associated with the derived reference block can be selected, and merge index information indicating the selected merge candidate can be generated and signaled to the decoding device 200. The motion information of the selected merge candidate can be used to derive the motion information of the current block.

[0116] As an example, when the (A)MVP mode is applied to the current block, the encoding apparatus 100 can construct an (A)MVP candidate list, and use a motion vector of an MVP candidate selected from among the motion vector predictors (MVPs) included in the (A)MVP candidate list as an MVP of the current block. In this case, for example, a motion vector indicating a reference block derived through the above-described motion estimation can be used as a motion vector of the current block, and an MVP candidate having a motion vector that is least different from the motion vector of the current block among the MVP candidates can be the selected MVP candidate. A motion vector difference (MVD) that is the motion vector of the current block minus the motion vector of the MVP can be derived. In this case, information of the MVD can be signaled to the decoding apparatus 200. Also, when the (A)MVP mode is applied, a value of a reference picture index can be configured as reference picture index information and signaled to the decoding apparatus 200 separately.

[0117] The encoding apparatus 100 can derive residual samples based on the prediction samples (S720). The encoding apparatus 100 can derive the residual samples via a comparison between original samples of the current block and the prediction samples.

[0118] The encoding apparatus 100 encodes image information including prediction information and residual information (S730). The encoding apparatus 100 can output the encoded image information in the form of a bitstream. The prediction information can include motion information and prediction mode information (e.g., a skip flag, a merge flag, or a mode index) as various items of information related to a prediction procedure. The motion information can include candidate selection information (e.g., a merge index, an mvp flag, or an mvp index) that is information for deriving a motion vector. Also, the motion information can include information of the above-described MVD and / or reference picture index information. The motion information can include information indicating whether to apply L0 prediction, L1 prediction, or Bi prediction. The residual information is information of residual samples. The residual information can include information of quantized transform coefficients for the residual samples.

[0119] The output bitstream can be stored in a (digital) storage medium and transmitted to a decoding apparatus, or can be transmitted to a decoding apparatus via a network.

[0120] Meanwhile, as described above, the encoding apparatus can generate a reconstructed picture (including reconstructed samples and reconstructed blocks) based on the reference samples and the residual samples. This is to make the encoding apparatus 100 derive the same prediction result as a prediction result obtained by the decoding apparatus 200, so that encoding efficiency can be improved. Accordingly, the encoding apparatus 100 can store the reconstructed picture (or the reconstructed samples or the reconstructed blocks) in a memory and use it as a reference picture for inter prediction. As described above, for example, in-loop filtering can be further performed on the reconstructed picture.

[0121] Figure 9 and Figure 10 A video / image decoding procedure based on inter prediction in a decoding device and an inter prediction unit are shown according to embodiments of the present disclosure.

[0122] The decoding device 200 can perform operations corresponding to those performed by the encoding device 100. The decoding device 200 can perform prediction on the current block based on the received prediction information and derive prediction samples.

[0123] In particular, the decoding device 200 can determine a prediction mode of the current block based on the received prediction information (S910). The decoding device 200 can determine which inter prediction mode is applied to the current block based on the prediction mode information in the prediction information.

[0124] For example, the decoding device 200 can determine which one of the merge mode or the (A)MVP mode is applied to the current block based on the merge flag. Or the decoding device 200 can select one from among various inter prediction mode candidates based on the mode index. The inter prediction mode candidates can include the skip mode, the merge mode, and / or the (A)MVP mode or other various inter prediction modes described below.

[0125] The decoding device 200 derives the motion information of the current block based on the determined inter prediction mode (S920). For example, when the skip mode or the merge mode is applied to the current block, the decoding device 200 can construct a merge candidate list described below and select one from among merge candidates included in the merge candidate list. The selection of the merge candidate can be performed based on a merge index. The motion information of the current block can be derived from the motion information of the selected merge candidate. The motion information of the selected merge candidate can be used as the motion information of the current block.

[0126] As an example, when the (A)MVP mode is applied to the current block, the decoding device 200 can construct an (A)MVP candidate list and use a motion vector of an MVP candidate selected from among MVP candidates included in the (A)MVP candidate list as an MVP of the current block. The selection of the MVP can be performed based on the above-described selection information (MVP flag or MVP index). In this case, the decoding device 200 can derive a MVD of the current block based on information of the MVD and derive a motion vector of the current block based on the MVD of the current block and the MVP. In addition, the decoding device 200 can derive a reference picture index of the current block based on the reference picture index information. A picture indicated by the reference picture index in a reference picture list of the current block can be derived as a reference picture referred to for inter prediction of the current block.

[0127] Meanwhile, as described below, the motion information of the current block can be derived without configuring the candidate list, in which case the motion information of the current block can be derived according to a procedure initiated in the following prediction mode. In this case, the above-described candidate list configuration can be omitted.

[0128] The decoding device 200 can generate the prediction samples of the current block based on the motion information of the current block (S930). In this case, the decoding device 200 can derive the reference picture based on the reference picture index of the current block, and derive the prediction samples of the current block using the samples of the reference block indicated by the motion vector of the current block on the reference picture. In this case, as described below, all or some of the prediction samples of the current block can be further subjected to prediction sample filtering depending on the situation.

[0129] For example, the inter prediction unit 260 of the decoding device 200 can include a prediction mode determination unit 261, a motion information derivation unit 262, and a prediction sample derivation unit 263. The prediction mode determination unit 181 can determine the prediction mode of the current block based on the received prediction mode information, the motion information derivation unit 182 can derive the motion information (motion vector and / or reference picture index) of the current block based on the received information of the motion information, and the prediction sample derivation unit 183 can derive the prediction samples of the current block.

[0130] The decoding device 200 generates the residual samples of the current block based on the received residual information (S940). The decoding device 200 can generate the reconstructed samples of the current block based on the prediction samples and the residual samples, and based thereon, generate the reconstructed picture (S950). Thereafter, the reconstructed picture can be further subjected to in-loop filtering as described above.

[0131] As described above, the inter prediction procedure can include the steps of determining an inter prediction mode, deriving motion information according to the determined prediction mode, and performing prediction (generating prediction samples) based on the derived motion information.

[0132] Various inter prediction modes can be used to predict the current block in the picture. For example, the merge mode, the skip mode, the MVP mode, and the affine mode can be used. The decoder-side motion vector refinement (DMVR) mode and the adaptive motion vector resolution (AMVR) mode can be further used as additional modes. The affine mode can also be referred to as an affine motion prediction mode. The MVP mode can also be referred to as an advanced motion vector prediction (AMVP) mode.

[0133] The prediction mode information indicating the inter prediction mode of the current block can be signaled from the encoding apparatus to the decoding apparatus 200. The prediction mode information can be included in the bitstream and received by the decoding apparatus 200. The prediction mode information can include index information indicating one of a plurality of candidate modes. Or the inter prediction mode can be indicated via hierarchical signaling of flag information. In this case, the prediction mode information can include one or more flags. For example, the encoding apparatus 100 can signal a skip flag to indicate whether the skip mode is applied, and when the skip mode is not applied, signal a merge flag to indicate whether the merge mode is applied, and when the merge mode is not applied, indicate that the MVP mode is applied or signal a flag for further discrimination. The affine mode can be signaled as an independent mode or as a mode dependent on the merge mode or the MVP mode. For example, the affine mode can be configured as one of the candidates in the merge candidate list or the MVP candidate list as described below.

[0134] The encoding apparatus 100 or the decoding apparatus 200 can perform inter prediction using the motion information of the current block. The encoding apparatus 100 can derive the optimal motion information of the current block via a motion estimation procedure. For example, the encoding apparatus 100 can use an original block in an original picture of the current block to search for a similar reference block having high correlation in a predetermined search range in a reference picture on a per-fractional pixel basis, and derive the motion information based thereon. The similarity of the blocks can be derived based on a difference between sample values based on phase. For example, the similarity of the blocks can be calculated based on a sum of absolute differences (SAD) between the current block (or a template of the current block) and the reference block (or a template of the reference block). In this case, the motion information can be derived based on the reference block having the smallest SAD in the search area. The derived motion information can be signaled to the decoding apparatus based on the inter prediction mode by various methods.

[0135] When the merge mode is applied, the motion information of the current prediction block is not directly transmitted, but rather the motion information of its neighboring prediction blocks is used to derive. Accordingly, the encoding apparatus 100 can indicate the motion information of the current prediction block by transmitting flag information indicating that the merge mode has been used and a merge index indicating which neighboring prediction blocks have been used.

[0136] The encoding apparatus 100 needs to search for merge candidate blocks for deriving motion information of a current prediction block in order to implement a merge mode. For example, a maximum of five merge candidate blocks can be used, but embodiments of the present disclosure are not limited thereto. The maximum number of merge candidate blocks can be transmitted in a slice header, but embodiments of the present disclosure are not limited thereto. After finding the merge candidate blocks, the encoding apparatus 100 can generate a merge candidate list, and among them, select a merge candidate block having a minimum cost as a final merge candidate block.

[0137] Various embodiments of the merge candidate blocks constituting the merge candidate list are described herein.

[0138] The merge candidate list can use, for example, five merge candidate blocks. For example, four spatial merge candidates and one temporal merge candidate can be utilized.

[0139] A method for configuring a reference picture list for inter prediction described above is described below.

[0140] There is a high level syntax (HLS) in a video coding system to control the overall sharing of common information. The HLS aims to manage frames to reduce the bit count, thereby enabling efficient use of memory. The HLS uses a reference picture management system that invokes a process denoted as a reference picture marking. This process marks each frame according to the availability for decoding of the current frame. The design and implementation of the HLS aim to have a simple and efficient means to identify pictures necessary for decoding the current picture.

[0141] Embodiment 1

[0142] In this embodiment, a reference picture list (RPL) management system is introduced that is directly signaled and derived. Two lists are used (e.g., reference picture list 0 and reference picture list 1). The information for deriving the two reference picture lists is signaled as syntax elements in the slice header, sequence parameter set (SPS), and picture parameter set (PPS). The two lists are generated on all types of slices. However, while I slices do not use the information from the lists, P slices use information from only list 0, and B slices use information from both lists. Pictures are identified using picture order count (POC) numbers. Each picture can be classified as a long-term reference picture (LTRP) or a short-term reference picture (STRP). Pictures classified as LTRPs or STRPs represent that they are active pictures and can be used at the time of decoding, and inactive pictures are pictures that are not used during the decoding process.

[0143] Figure 11 is a flowchart illustrating an example of constructing a reference picture list according to an embodiment of the present disclosure. Figure 11The operations of FIG. 10 can be performed by the decoding device 200.

[0144] Figure 11 The front part of the reference picture list configuration process is shown. First, in step S1105, the decoding device 200 parses the total number of entries in the list. Then, for each entry, the pictures are marked as LTRP or STRP via the reference picture marking process (S1120). In the present disclosure, marking a picture refers to adding or updating information (parameters) related to the picture. Specifically, the decoding device 200 initializes the entry index (i) (S1110), and when the entry index (i) is equal to or less than the total number of entries (S1115), the reference picture marking is performed (S1120). When the reference picture marking is completed, the decoding device updates the entry index (i) (S1125). According to the embodiments of the present disclosure, a brand new architecture is proposed to enhance the reference picture management system.

[0145] Figure 12 and Figure 13 is a flowchart showing an example of marking reference pictures according to the embodiments of the present disclosure.

[0146] [Table 1]

[0147]

[0148] Table 1 shows an example of syntax for configuring reference picture lists. Here, the present embodiment provides a structure that can manage reference pictures. The ref_pic_list_struct function receives listldx indicated by rpl1_same_as_rpl0_flag as a first input. In other words, when the flag (rpl1_same_as_rpl0_flag) is true, this indicates that the lists are the same, and one list (L0 or L1) can be inferred from the other list (L1 or L0). The second input rplsldx of the ref_pic_list_struct function indicates the number of reference picture lists used in the coded video sequence specified in the SPS. num_ref_entries indicates the number of entries in ref_pic_list_struct(listldx, rplsldx). When an LRTP frame is used for each entry, a flag is parsed from the list to read whether the frame is LTRP or STRP. When the entry is a STRP entry and is the first STRP entry in the syntax structure ref_pic_list_struct(listldx, rplsldx), this indicates the absolute difference between the POC of the current picture and the POC of the picture referred to by the i-th entry, or, if the i-th entry is a STRP but not the first entry, this indicates the absolute difference between the POC of the picture referred to by the i-th entry and the POC of the picture indicated by the previous STRP. (If the entry is a STRP, it is the first STRP entry in the ref_pic_list_struct(listldx, rplsldx) syntax structure and it indicates the absolute difference of the POC of the current picture and the POC of the picture referred to by the i-th entry, or when the i-th entry is a STRP entry but not the first entry, then it indicates the absolute difference between the POC of the picture referred to by the i-th entry and the POC of the picture indicated by the previous STRP). When abs_delta_poc of the i-th entry is greater than 0, a flag that signals the code information of the STRP is signaled. However, when st_ref_pic_flag is 0, the i-th entry indicates a long-term reference picture. The LTRP is signaled by the POC least significant bits (LSB).

[0149] The syntax elements related to Table 1 can be defined as follows:

[0150] num_ref_entries[ listldx ][ rplsldx ] specifies the number of entries in the ref_pic_list_struct( listldx, rplsldx ) syntax structure. num_ref_entries[ listldx ][ rplsldx ] can have a value in the range of 0 to sps_max_dec_pic_buffering_minusl + 14.

[0151] When rpl1_same_as_rpl0_flag is equal to 1, the syntax structure num_ref_pic_lists_in_sps[ 1 ] and ref_pic_list_struct( 1, rplsldx ) are not present, the following applies.

[0152] - num_ref_pic_lists_in_sps[ 1 ] is inferred to have the same value as num_ref_pic_lists_in_sps[ 0 ].

[0153] - For rplsldx, which can be a value in the range of 0 to num_ref_pic_lists_in_sps[ 0 ] - 1, in ref_pic_list_struct( 1, rplsldx ), the value of each syntax element is inferred to be the same as the value of the syntax element in ref_pic_list_struct( 0, rplsldx ).

[0154] When long_term_ref_pics_flag is equal to 0, this indicates that LTRPs are not used for inter prediction of any coded picture in the coded video sequence (CVS). When long_term_ref_pics_flag is equal to 1, this indicates that LTRPs are used for inter prediction of one or more coded pictures in the CVS.

[0155] When st_ref_pic_flag[ listldx ][ rplsldx ][ i ] is equal to 1, the i-th entry in the ref_pic_list_struct( listldx, rplsldx ) syntax structure is an STRP entry. When st_ref_pic_flag[ listldx ][ rplsldx ][ i ] is equal to 0, the i-th entry in the ref_pic_list_struct( listldx, rplsldx ) syntax structure is an LTRP entry. The value of st_ref_pic_flag[ listldx ][ rplsldx ][ i ] is inferred to be equal to 1 unless present.

[0156] When the i-th entry is the first STRP in the syntax structure ref_pic_list_struct( listldx, rplsldx ), abs_delta_poc_st[ listldx ][ rplsldx ][ i ] specifies the absolute difference of the picture order count value between the picture referred to by the i-th entry and the current picture, unless the i-th entry is a STRP but not the first entry in the syntax structure ref_pic_list_struct( listldx, rplsldx ), in which case abs_delta_poc_st[ listldx ][ rplsldx ][ i ] specifies the absolute difference of the picture order count value between the picture referred to by the i-th entry and the picture referred to by the previous STRP in the syntax structure ref_pic_list_struct( listldx, rplsldx ).

[0157] abs_delta_poc_st[ listldx ][ rplsldx ][ i ] has a value in the range of 0 to 2 15 - 1, inclusive.

[0158] When strp_entry_sign_flag[ listldx ][ rplsldx ][ i ] is equal to 1, it specifies that the i-th entry in the syntax structure ref_pic_list_struct( listldx, rplsldx ) has a value equal to or greater than 0. When strp_entry_sign_flag[ listldx ][ rplsldx ][ i ] is equal to 0, it specifies that the i-th entry in the syntax structure ref_pic_list_struct( listldx, rplsldx ) has a value less than 0. The value of strp_entry_sign_flag[ i ][ j ] is inferred to be equal to 1 unless it is present.

[0159] poc_lsb_lt[ listldx ][ rplsldx ][ i ] specifies the picture order count modulo (modulo) MaxPicOrderCntLsb of the picture referred to by the i-th entry in ref_pic_list_struct( listldx, rplsldx ). The length of poc_lsb_lt[ listldx ][ rplsldx ][ i ] is log2_max_pic_order_cnt_lsb_minus4 + 4 bits.

[0160] The above description of the syntax elements can be summarized as in Table 2 below:

[0161] [table 2]

[0162]

[0163] ReferringFigure 12 The decoding device 200 identifies whether a LTRP is used in the reference picture list entry by long_term_ref_pics_flag (S1205), and when a LTRP is used (when long_term_ref_pics_flag is 1), parses st_ref_pic_flag (S1210) and identifies whether st_ref_pic_flag is true (S1215). When st_ref_pic_flag is true (1) (i.e., when the current entry corresponds to a STRP entry), the decoding device 200 parses abs_delta_poc_st. In other words, when the current entry corresponds to a STRP entry, the decoding device 200 obtains a POC difference between a picture related to the current entry and a previous picture.

[0164] When long_term_ref_pics_flag is false in step S1205, the decoding device directly parses abs_delta_poc_st. Thereafter, the decoding device 200 identifies whether abs_delta_poc_st is greater than 0 (S1225), and when abs_delta_poc_st is greater than 0, parses strp_entry_sign_flag (S1230).

[0165] When st_ref_pic_flag is false (0) in step S1215, the decoding device 200 parses poc_lsb_lt (S1235). In other words, when the current entry corresponds to a LTRP entry, the decoding device 200 obtains a POC least significant bit (LSB) of a picture related to the current entry. The POC LSB of the picture is used to derive a POC value of the picture and can be combined with a POC most significant bit (MSB) to thereby derive the POC value of the picture.

[0166] The above process of configuring a reference picture list can be represented in two blocks as shown in FIG. 13B. The operations of each block can be implemented by the same or different hardware. The decoding device 200 can obtain a flag indicating whether a long-term reference picture is used (long_term_ref_pics_flag) and the number of entries (num_ref_entries) (1310), and determine whether each entry is a LTRP entry or a STRP entry (1320). Figure 13

[0167] Embodiment 2

[0168] ​In this embodiment, the num_ref_entries parameter described above in connection with Table 1 is proposed to be limited. Generally, num_ref_entries[ listldx ][ rplIdx ] indicates the number of entries in ref_pic_list_struct( listldx, rplldx ). It is recognized that num_ref_entries can have a value from 0 to sps_max_dec_pic_buffering_minusl + 14. In order to use less frames, the range of num_ref_entries can be limited. The worst case occurs when 14 frames more than the value indicated by sps_max_dec_pic_buffering_minusl are required. This means that the decoded picture buffer needs to be decoded to have enough memory to store so many frames. Instead of this operation, a restriction can be imposed on hierarchical groups of structures (GOP) taking into account the temporalld of the layer.

[0169] The additional syntax can be defined as shown in Table 3. sps_max_dec_pic_buffering_minusl plus 1 indicates the maximum required size of one decoded picture buffer in the CVS as a unit of picture storage buffer. sps_max_dec_pic_buffering_minusl can have a value from 0 to MaxDpbSize - 1, and MaxDpbSize can be defined additionally.

[0170] [Table 3]

[0171]

[0172] In other words, the maximum value of the number of entries in the reference picture list can be the maximum required size of the decoded picture buffer plus 14. Further, according to the present embodiment (Embodiment 2), the maximum value of the number of entries in the reference picture list can be the maximum required size of the decoded picture buffer.

[0173] Embodiment 3

[0174] In the present embodiment, a method of removing the syntax element long_term_ref_pics_flag from ref_pic_list_struct(listldx, rpIsIdx) is proposed. When the syntax element long_term_ref_pics_flag is 0, this indicates that LTRPs are not used for inter prediction of any coded picture in the CVS. When long_term_ref_pics_flag is 1, this indicates that LTRPs are used for inter prediction of one or more coded pictures in the CVS. By removing the syntax element long_term_ref_pics_flag, it can be inferred that LTRPs are used by default in the CVS. This is useful because LTRPs are typically high-reliability pictures in the DPB.

[0175] Embodiment 4

[0176] Figure 14 is a flowchart illustrating another example of signaling reference pictures according to an embodiment of the disclosure.

[0177] Another embodiment of ref_pic_list_struct(listldx, rpIsIdx) is provided. In particular, ref_pic_list_struct(listldx, rpIsIdx) can be configured considering the case of signaling LTRPs and STRPs separately. This can be represented as shown in Table 4 below. Here, according to an embodiment of the disclosure, a method of providing num_strp_entries[listldx][rpIsIdx] that limits the total number of STRPs used for the indication and num_ltrp_entries[listldx][rpIsIdx] that indicates the total number of LTRPs is provided.

[0178] Figure 14 An overview of the proposed structure of signaling LTRP count and STRP count separately is illustrated. Here, first, STRP and LTRP counts are parsed using LTRPs according to long_term_ref_pics_flag. Thereafter, for each entry, lt_ref_pic_flag that indicates whether the picture is an LTRP is parsed. When lt_ref_pic_flag is true, the LTRP count is increased. Otherwise, the STRP count is increased. When lt_ref_pic_flag is false, STRP is identified by parsing delta_poc_st and its code information (when delta_poc_st is greater than 0). Otherwise, LTRP is identified by LSB POC.

[0179] Table 4 below shows an example of syntax for configuring a reference picture list according to an embodiment.

[0180] [Table 4]

[0181]

[0182] num_strp_entries[listIdx][rplsIdx] indicates the number of STRP entries in ref_pic_list_struct( listIdx, rplsIdx ).

[0183] num_ltrp_entries[listIdx][rplsIdx] indicates the number of LTRP entries in ref_pic_list_struct( listIdx, rplsIdx ).

[0184] When lt_ref_pic_flag[ listIdx ][ rplsIdx ] is not present, the value of lt_ref_pic_flag[ listIdx ][ rplsIdx ] is inferred as follows.

[0185] When num_ltrp_entries[ listIdx ][ rplsIdx ] is equal to 0, the value ( lt_ref_pic_flag[ listIdx ][ rplsIdx ]) is inferred to be 0. Otherwise, it is inferred to be 0 when numLtrps (the number of LTRPs) is equal to num_ltrp_entries[ listIdx ][ rplsIdx ], and it is inferred to be 0 when numStrps (the number of STRPs) is equal to num_strp_entries[ listIdx ][ rplsIdx ].

[0186] [Table 5]

[0187]

[0188] Referring to Figure 14, the decoding device 200 starts to build the reference list (S1405) and parses the number of STRP entries (num_strp_entries) (S1410). The decoding device identifies whether LTRP is used (S1415), and parses the number of LTRP (num_ltrp_entries) when LTRP is used (S1420). The decoding device 200 initializes the variables i, numLtrps and numStrps to 0 (S1425), determines whether the variable i is equal to or less than the number of entries (num_ref_entires) (S1430), and performs the reference picture marking operation (S1435) and updates i (S1440) when i is equal to or less than the number of entries (num_ref_entires).

[0189] Figure 15 is a flowchart illustrating an example of processing a video signal according to an embodiment of the disclosure. Figure 15 The operations of can be performed by the decoding device 200.

[0190] In step S1510, the decoding device 200 builds a reference picture list of a current picture in a video signal. The decoding device 200 obtains a POC difference (e.g., abs_delta_poc_st) between a picture related to a first entry and a prior picture when the first entry of the reference picture list corresponds to an STRP entry, and obtains a POC modulo value (e.g., poc_lsb_lt) of the picture related to a second entry when the second entry of the reference picture list corresponds to an LTRP entry.

[0191] According to an embodiment, in the process of configuring the reference picture list, the decoding device 200 can obtain the number of entries (num_ref_entries) in the reference picture list, and perform the reference picture marking on each entry in the reference picture list as many times as the number of entries in the LTRP or the STRP.

[0192] According to an embodiment, the maximum number of entries can be the maximum required size of the decoded picture buffer plus 14 (e.g., sps_max_dec_pic_buffering_minus1 + 14) or the maximum required size of the decoded picture buffer (e.g., sps_max_dec_pic_buffering_minus1).

[0193] According to embodiments, in performing the process of reference picture marking, the decoding device 200 can obtain a number of STRP entries (e.g., num_strp_entries) and a number of LTRP entries (e.g., num_ltrp_entries) as shown in Table 4, perform STRP marking as many times as the number of STRP entries, and perform LTRP marking as many times as the number of LTRP entries.

[0194] According to embodiments, in configuring the reference picture list, the decoding device 200 can obtain a first flag and determine whether an entry in the reference picture list is an STRP entry or an LTRP entry based on the first flag. For example, the decoding device 200 can use a flag (e.g., st_ref_pic_flag) indicating whether a picture related to a current entry corresponds to an STRP to determine whether the current entry is an STRP entry or an LTRP entry.

[0195] According to embodiments, the POC difference represents an absolute value of a POC of a preceding picture minus a POC of a picture related to a first entry. When the absolute value (absolute POC difference) (e.g., abs_delta_poc_st) is greater than 0, the decoding device 200 can obtain a second flag (e.g., strp_entry_sign_flag) indicating a sign of the POC difference.

[0196] In step S1520, the decoding device 200 performs prediction on the current picture using the reference picture list. For example, the decoding device 200 can generate predicted samples using sample values indicated by motion vectors in the reference pictures included in the first reference picture list (L0) or the second reference picture list (L1).

[0197] Embodiments of the disclosure provide a method and an apparatus for encoding information necessary for constructing a reference picture list. The encoding device 100 can construct a reference picture list and perform prediction using reference pictures in the reference picture list.

[0198] Further, the encoding device 100 can encode information for configuring the reference picture list per entry in the reference picture list. For example, the information for configuring the reference picture list includes a POC difference (e.g., abs_delta_poc_st) between a picture corresponding to an STRP and a preceding picture and includes a POC least significant bit (e.g., poc_lsb_lt) of a picture corresponding to an LTRP.

[0199] The encoding apparatus 100 can include a first flag (e.g., st_ref_pic_flag) for indicating whether a picture in a reference picture list is an STRP or an LTRP in the information for configuring the reference picture list. In addition, when the POC difference (i.e., abs_delta_poc_st) is greater than 0, a second flag (e.g., strp_entry_sign_flag) can be included to indicate the sign of the POC difference. The information for configuring the reference picture list can be included in an SPS, a PPS, or a slice header and signaled.

[0200] The processing method to which the embodiments of the disclosure are applied can be manufactured in the form of a program executed on a computer and can be stored in a computer-readable recording medium. Multimedia data having a data structure according to the disclosure can also be stored in a computer-readable recording medium. The computer-readable recording medium includes all kinds of storage devices and distributed storage devices that can store computer-readable data. The computer-readable recording medium can include, for example, a Blu-ray disc (BD), a universal serial bus (USB) drive, a ROM, a PROM, an EPROM, an EEPROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, and an optical data storage device. The computer-readable recording medium can include a medium implemented in the form of a carrier wave (e.g., transmission through the Internet). The bitstream generated by the encoding method can be stored in a computer-readable recording medium or transmitted via a wired / wireless communication network.

[0201] According to one embodiment of the disclosure, the embodiments of the disclosure can be implemented as a computer program by program codes that can be executed on a computer. The computer codes can be stored on a computer-readable carrier.

[0202] The decoding apparatus and the encoding apparatus to which the embodiments of the disclosure are applied can include a digital device. The digital device includes all kinds or types of digital devices capable of performing at least one of, for example, transmission, reception, processing, and output of data, content, or services. Processing data, content, or services by the digital device includes encoding and / or decoding the data, content, or services. Such a digital device can be paired or connected with other digital devices or external servers via a wired / wireless network, thereby transmitting or receiving data, or converting data as necessary.

[0203] The digital device can include, for example, a network TV, a hybrid broadcast broadband TV, a smart TV, an Internet protocol TV (IPTV), a personal computer, or other standing devices or mobile or handheld devices such as a personal digital assistant (PDA), a smart phone, a tablet PC, or a laptop computer.

[0204] As used herein, a "wired / wireless network" collectively refers to communication networks supporting various communication standards or protocols for data communication and / or interconnection between digital devices or between a digital device and an external server. Such wired / wireless networks can include currently existing or to-be-developed communication networks and communication protocols for such communication networks, and can be formed of, for example, communication standards for wired connection including USB (Universal Serial Bus), CVBS (Composite Video Blank Sync), component, S-video (analog), DVI (Digital Visual Interface), HDMI (High Definition Multimedia Interface), RGB, or D-SUB, and communication standards for wireless connection including Bluetooth, RFID (Radio Frequency Identification), IrDA (Infrared Data Association), UWB (Ultra Wideband), ZigBee, DLNA (Digital Living Network Alliance), WLAN (Wireless LAN) (Wi-Fi), Wibro (Wireless Broadband), Wimax (Worldwide Interoperability for Microwave Access), HSDPA (High Speed Downlink Packet Access), LTE (Long Term Evolution), or Wi-Fi Direct.

[0205] The above-described embodiments relate to predetermined combinations of components and features of the present disclosure. Unless explicitly mentioned otherwise, each component or feature should be considered optional. Each component or feature can be practiced without being combined with other components or features. Further, some components and / or features can be combined together to configure embodiments of the present disclosure. The order of operations described in connection with embodiments of the present disclosure can be varied. Some components or features in an embodiment can be included in another embodiment or can be replaced with corresponding components or features of another embodiment. It is obvious that claims can be combined to constitute embodiments or can be added in new claims through amendments after filing, unless explicitly mentioned otherwise.

[0206] When implemented in firmware or software, embodiments of the present disclosure can be implemented as a module, a procedure, or a function for performing the above-described functions or operations. Software code can be stored in a memory and driven by a processor. The memory can be disposed inside or outside the processor to exchange data with the processor through various known means.

[0207] It is obvious to those of ordinary skill in the art that the present disclosure can be implemented in other specific forms without departing from the essential characteristics of the present disclosure. Therefore, the above description should not be construed as limiting in all respects, but should be construed as exemplary. The scope of the present disclosure should be determined by a reasonable interpretation of the appended claims, and all equivalents of the present disclosure belong to the scope of the present disclosure.

[0208] Industrial applicability

[0209] The above preferred embodiments of the present disclosure have been provided for illustrative purposes, and those of ordinary skill in the art will readily understand that various changes or variations, or additions or replacements of other embodiments, can be made thereto without departing from the technical spirit and scope of the present disclosure as defined in the appended claims.

Claims

1. A method of decoding a video signal by a device, the method comprising the steps of: obtaining a number of entries included in a reference picture list of a current picture; labeling each entry in the reference picture list as a short-term reference picture (STRP) or a long-term reference picture (LTRP) by as many times as the number of entries; deriving prediction samples of the current picture based on the reference picture list; deriving residual samples of the current picture; and generating reconstructed samples of the current picture based on the prediction samples and the residual samples, and wherein the step of labeling each entry comprises the steps of: obtaining a picture order count (POC) difference between a reference picture related to a first entry of the reference picture list and another picture based on the first entry corresponding to the STRP; and obtaining a POC modulo value of a picture related to a second entry of the reference picture list based on the second entry corresponding to the LTRP, wherein the first specific entry is a first STRP entry in one reference picture list, the other picture is the current picture, or if the first specific entry is not the first STRP entry in the one reference picture list, the other picture is a previous STRP entry in the one reference picture list, and wherein a maximum of the number of entries is a maximum required size of a decoded picture buffer plus 13. the POC difference is an absolute value of a POC of the other picture minus a POC of the reference picture related to the entry labeled as the STRP.

2. The method of claim 1, wherein, the step of labeling each entry further comprises the step of:

3. The method of claim 2, wherein, obtaining a flag indicating a sign of the POC difference based on the absolute value being greater than 0. the other picture is the current picture based on the entry labeled as the STRP being a first STRP entry of the labeling of each entry of the reference picture, or the other picture is a previous STRP in the reference picture list based on the entry labeled as the STRP not being the first STRP of the labeling of each entry of the reference picture.

4. The method of claim 2, wherein, the step of labeling each entry comprises the steps of:

5. The method of claim 1, wherein, obtaining a number of STRP entries and a number of LTRP entries; and performing STRP labeling by as many times as the number of STRP entries and performing LTRP labeling by as many times as the number of LTRP entries. the step of labeling each entry comprises the steps of:

6. The method of claim 1, wherein, obtaining a flag including information on whether an entry of the reference picture list corresponds to a STRP entry or a LTRP entry; and determining whether the entry of the reference picture list corresponds to the STRP entry or the LTRP entry based on the flag.

7. A method of encoding a video signal by a device, the method comprising the steps of: constructing a reference picture list of a current picture; generating prediction samples of the current picture based on the reference picture list; deriving residual samples of the current picture based on the prediction samples; and ​ ​ generating coding information for constructing the reference picture list and for deriving the residual samples, wherein, based on a first entry of the reference picture list corresponding to a short-term reference picture STRP, the coding information includes a picture order count POC difference between a reference picture related to the first entry and another picture, and wherein, based on a second entry of the reference picture list corresponding to a long-term reference picture LTRP, the coding information includes a POC modulus of a picture related to the second entry, wherein, based on the first specific entry being a first STRP entry in one reference picture list, the another picture is the current picture, or if the first specific entry is not the first STRP entry in the one reference picture list, the another picture is a previous STRP entry in the one reference picture list, wherein, the coding information includes a number of entries of the reference picture list, and wherein, a maximum value of the number of entries is a maximum required size of a decoded picture buffer plus 13.

8. The method of claim 7, wherein, the POC difference is an absolute value of a POC of the another picture minus a POC of a reference picture related to an entry marked as the STRP.

9. The method of claim 8, wherein, the coding information further includes: a flag indicating a sign of the POC difference of the entry whose absolute value of the POC difference is greater than 0.

10. The method of claim 7, wherein, based on the entry marked as the STRP being a first STRP entry in the coding information, the another picture is the current picture, or based on the entry marked as the STRP not being the first STRP in the coding information, the another picture is a previous STRP in the reference picture list.

11. The method of claim 7, wherein, the coding information further includes: a flag including information about whether an entry of the reference picture list corresponds to a STRP entry or a LTRP entry.

12. The method of claim 7, further comprising the steps of: loading the coding information into a sequence parameter set SPS, a picture parameter set PPS, or a slice header in a bitstream related to the video signal.

13. A non-transitory computer readable medium storing a computer program and a bitstream, the computer program, when executed by one or more processors, causing the one or more processors to implement a method for generating the bitstream, the method comprising: generating coding information for constructing a reference picture list, for marking each entry of the reference picture list as a short-term reference picture STRP or a long-term reference picture LTRP, and for deriving residual samples of a current picture, wherein, based on a first entry of the reference picture list corresponding to the STRP, the coding information includes a picture order count POC difference between a reference picture related to the first entry and another picture, and wherein, based on a second entry of the reference picture list corresponding to the LTRP, the coding information includes a POC modulus of a picture related to the second entry, wherein, based on the first particular entry being a first STRP entry in one reference picture list, the other picture is the current picture, or if the first particular entry is not the first STRP entry in the one reference picture list, the other picture is a previous STRP entry in the one reference picture list, wherein the coding information includes a number of entries of the reference picture list, and wherein a maximum value of the number of entries is a maximum required size of a decoded picture buffer plus 13.

Images