Method and apparatus for processing a video signal using inter-prediction

Abstract

To provide methods and devices for decoding video signals using inter prediction.SOLUTION: A method comprises constructing a reference picture list of a current picture in the video signal, and performing a prediction for a current picture by using the reference picture list. The step of constructing the reference picture list comprises, if a first entry of the reference picture list corresponds to an STRP entry, obtaining a POC difference value between a picture related to the first entry and another picture, and if a second entry of the reference picture list corresponds to an LTRP entry, obtaining a POC modulo value of a picture related to the second entry. A reference picture list for identifying a picture may be generated in a simple and effective manner, and thus, the compression performance may be increased and the computation time may be decreased.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] SUMMARY OF THE INVENTION The embodiments herein relate to methods and apparatus for processing video signals using inter prediction, and more particularly to methods and apparatus for constructing a reference picture list for prediction of a current picture. [Background technology]

[0002] Compression coding refers to a series of signal processing techniques used to transmit digitized information over a communication line or store it in a suitable format for storage. Media such as pictures, images, and audio can be compressed and coded, and the technique of compressing and coding images in particular is called video image compression.

[0003] Next-generation video content will be characterized by high spatial resolution, high frame rate, and high dimensionality of scene representation, which will bring about significant increases in memory storage, memory access rate, and processing power to process such content.

[0004] Therefore, it is necessary to design coding tools to process next-generation image content more efficiently. In particular, video codec standards such as the high efficiency video coding (HEVC) standard and beyond require prediction techniques that can generate accurate prediction samples while using resources more efficiently. Summary of the Invention [Problem to be solved by the invention]

[0005] The embodiments herein provide a method and apparatus for constructing a reference picture list required for prediction of a current picture.

[0006] Additionally, embodiments herein provide a method and apparatus for obtaining information necessary to construct a reference picture list.

[0007] The embodiments herein also provide a method and apparatus for defining the number of entries required for constructing a reference picture list.

[0008] Additionally, embodiments herein provide methods and apparatus for reducing the amount of syntax elements required to construct a reference picture list.

[0009] Furthermore, the embodiments of the present specification provide a method and apparatus for counting the number of times a reference picture is marked in the process of constructing a reference picture list.

[0010] The technical problems to be solved by the present invention are not limited to the technical problems mentioned above, and other technical problems not mentioned should be clearly understood by a person having ordinary skill in the art to which this specification pertains from the following description. [Means for solving the problem]

[0011] An embodiment of the present specification provides a method and apparatus for decoding a video signal using inter prediction. A method for processing a video signal according to an embodiment of the present specification includes: constructing a reference picture list for a current picture in the video signal; and performing prediction for the current picture using the reference picture list, wherein constructing the reference picture list includes, when a first entry of the reference picture list corresponds to a short-term reference picture (STRP) entry, obtaining a POC difference value between a picture related to the first entry and another picture; and, when a second entry of the reference picture list corresponds to a long-term reference picture (LTRP) entry, obtaining a POC modulo value of a picture related to the second entry.

[0012] In one embodiment, the step of constructing the reference picture list may include the steps of obtaining the number of entries in the reference picture list and marking the LTRP or STRP as reference pictures for each of the entries in the reference picture list by the number of entries.

[0013] In one embodiment, the number of entries may have a maximum value of 14 plus the maximum required size of the decoded picture buffer.

[0014] In one embodiment, the step of marking the reference picture may include the steps of obtaining the number of STRP entries and the number of LTRP entries, marking STRPs for the number of STRP entries, and marking LTRPs for the number of LTRP entries.

[0015] In one embodiment, the step of constructing the reference picture list may include the steps of acquiring a first flag and determining whether an entry in the reference picture list corresponds to the STRP entry or the LTRP entry based on the first flag.

[0016] In one embodiment, the POC difference value may be determined as the absolute value of the POC of the other picture minus the POC of the picture associated with the first entry.

[0017] In one embodiment, constructing the reference picture list may include obtaining a second flag indicating a sign of the difference value of the POC if the absolute value is greater than zero.

[0018] A method for encoding a video signal according to another embodiment of the present specification includes constructing a reference picture list for a current picture in the video signal, performing prediction for the current picture using the reference picture list, and coding information for constructing the reference picture list. The information for constructing the reference picture list may include a POC difference between a picture associated with a short-term reference picture (STRP) entry in the reference picture list and another picture, and a modulo value of the POC of a picture associated with a long-term reference picture (LTRP) entry in the reference picture list.

[0019] In one embodiment, the information for constructing the reference picture list further includes the number of entries in the reference picture list, and the number of entries may have a maximum value equal to the maximum required size of the decoded picture buffer plus 14.

[0020] In one embodiment, the information for constructing the reference picture list may further include a first flag indicating whether an entry of the reference picture list corresponds to the STRP entry.

[0021] In one embodiment, the information for constructing the reference picture list may further include a second flag indicating a sign of the POC difference value for an entry whose POC difference value is greater than zero.

[0022] In one embodiment, information for constructing the reference picture list may be included in a sequence parameter set (SPS), a picture parameter set (PPS), or a slice header in a bitstream for the video signal.

[0023] According to another embodiment of the present specification, an apparatus for decoding a video signal includes a memory for storing the video signal and a processor coupled to the memory. The processor is configured to construct a reference picture list for a current picture in the video signal and perform prediction for the current picture using the reference picture list. The processor is further configured to, in constructing the reference picture list, obtain a difference value of POC between a picture associated with a first entry of the reference picture list and another picture when a first entry of the reference picture list corresponds to an STRP entry, and obtain a modulo value of POC of a picture associated with the second entry of the reference picture list when a second entry of the reference picture list corresponds to an LTRP entry.

[0024] According to another embodiment of the present disclosure, an apparatus for encoding a video signal includes a memory for storing the video signal, and a processor coupled to the memory, wherein the processor is configured to construct a reference picture list for a current picture in the video signal, perform prediction for the current picture using the reference picture list, and code information for constructing the reference picture list, the information for constructing the reference picture list including a picture order count (POC) difference between a picture associated with a short-term reference picture (STRP) entry in the reference picture list and another picture, and a modulo value of the POC of a picture associated with a long-term reference picture (LTRP) entry in a second entry in the reference picture list.

[0025] Another embodiment of the present disclosure provides a non-transitory computer-readable medium storing one or more commands, the one or more commands executed by one or more processors to control a video signal processing device to construct a reference picture list for a current picture in a video signal and perform prediction for the current picture using the reference picture list. The one or more commands also control the video signal processing device to, in constructing the reference picture list, obtain a picture order count (POC) difference between a picture associated with a first entry of the reference picture list and another picture when the first entry corresponds to a short-term reference picture (STRP) entry, and to obtain a modulo value of the POC of a picture associated with a second entry of the reference picture list when the second entry corresponds to a long-term reference picture (LTRP) entry. [Effects of the Invention]

[0026] According to the embodiments of the present specification, a list of reference pictures required for prediction of the current picture can be constructed through marking of reference pictures.

[0027] Furthermore, according to the embodiments of the present specification, appropriate information can be obtained for each type of reference picture.

[0028] Furthermore, according to the embodiments of the present specification, by defining the number of entries required for constructing a reference picture list, the memory required for storing pictures can be managed efficiently.

[0029] Additionally, according to the embodiments of the present specification, the amount of data required to construct a reference picture list can be reduced by omitting the parsing of some syntax elements.

[0030] Furthermore, according to the embodiments of the present specification, by counting the number of times a reference picture is marked, it is possible to efficiently implement a syntax for constructing a reference picture list.

[0031] The effects obtained by the present invention are not limited to those mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art to which the present invention pertains from the following description. [Brief explanation of the drawings]

[0032] The accompanying drawings, which are included as part of the detailed description to aid in understanding the present specification, provide examples for the present specification and, together with the detailed description, explain the technical features of the present specification.

[0033] [Figure 1] 1 illustrates an example video coding system according to an embodiment of the present disclosure. [Figure 2]1 shows a schematic block diagram of an encoding device for encoding a video / image signal according to an embodiment of the present disclosure; [Figure 3] As an example of the present disclosure, a schematic block diagram of a decoding device for decoding an image signal is shown. [Figure 4] 1 illustrates an example of a structural diagram of a content streaming system according to an embodiment of the present disclosure; [Figure 5] 1 shows an example of a block diagram of an apparatus for processing a video signal according to an embodiment of the present disclosure; [Figure 6a] 1 shows an example of a block division structure of an image according to an embodiment of the present specification, which is an example of a block division structure based on QT (QuadTree, QT). [Figure 6b] 1 shows an example of a block division structure of an image according to an embodiment of the present specification, which is an example of a block division structure based on a Binary Tree (BT). [Figure 6c] 1 shows an example of a block division structure of an image according to an embodiment of the present specification, which is an example of a block division structure based on a TT (Ternary Tree, TT). [Figure 6d] 1 shows an example of a block division structure of an image according to an embodiment of the present specification, which is an example of a block division structure based on an Asymmetric Tree (AT). [Figure 7] 1 illustrates a video / image encoding procedure based on inter-prediction and an inter-prediction unit in an encoding device according to an embodiment of the present specification; [Figure 8] 1 illustrates a video / image encoding procedure based on inter-prediction and an inter-prediction unit in an encoding device according to an embodiment of the present specification; [Figure 9] 1 illustrates a video / image decoding procedure based on inter-prediction and an inter-prediction unit in a decoding device according to an embodiment of the present specification; [Figure 10] 1 illustrates a video / image decoding procedure based on inter-prediction and an inter-prediction unit in a decoding device according to an embodiment of the present specification; [Figure 11] 1 illustrates an example of a flowchart for constructing a reference picture list according to an embodiment of the present disclosure. [Figure 12] 1 illustrates an example of a flowchart for marking reference pictures according to an embodiment of the present disclosure. [Figure 13] 1 illustrates an example of a flowchart for marking reference pictures according to an embodiment of the present disclosure. [Figure 14] 10 illustrates another example of a flowchart for marking reference pictures according to an embodiment of the present disclosure. [Figure 15] 1 illustrates an example of a flowchart for processing a video signal according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0034] Preferred embodiments of the present specification will be described in detail below with reference to the accompanying drawings. The detailed description disclosed below together with the accompanying drawings is intended to describe exemplary embodiments of the present specification and is not intended to show the only embodiments in which the present specification can be implemented. The detailed description below includes specific details to provide a complete understanding of the present specification. However, those skilled in the art will recognize that the present specification can be implemented without such specific details.

[0035] In some cases, to avoid obscuring the concepts of this specification, well-known structures and devices may be omitted or shown in block diagram form, focusing on the core functions of each structure and device.

[0036] Furthermore, the terms used in this specification have been selected to be as widely used as possible, but in certain cases, terms arbitrarily selected by the applicant will be used for explanation. In such cases, the meanings will be clearly stated in the detailed explanation of the relevant part, so it is made clear that the terms used in this specification should not be interpreted simply based on their names, but should also be interpreted by understanding the meanings of the relevant terms.

[0037] The specific terms used in the following description are provided to facilitate understanding of the present specification, and the use of such specific terms may be changed to other forms within the scope of the technical concept of the present specification. For example, in the case of a signal, data, sample, picture, slice, tile, frame, and block, they may be appropriately interpreted as interchangeable in each coding process.

[0038] Hereinafter, the term "processing unit" as used herein refers to a unit in which encoding / decoding processes such as prediction, transformation, and / or quantization are performed. The term "processing unit" may also be interpreted to include a unit for a luma component and a unit for a chroma component. For example, the processing unit may correspond to a block, a coding unit (CU), a prediction unit (PU), or a transform unit (TU).

[0039] Furthermore, the processing unit may be interpreted as a unit for a luma component or a unit for a chroma component. For example, the processing unit may correspond to a coding tree block (CTB), a coding block (CB), a PU, or a transform block (TB) for a luma component. Alternatively, the processing unit may correspond to a CTB, a CB, a PU, or a TB for a chroma component. Furthermore, the processing unit is not limited thereto, and may be interpreted to include a unit for a luma component and a unit for a chroma component.

[0040] Furthermore, the processing units are not necessarily limited to square blocks, but may also be configured in the form of polygons having three or more vertices.

[0041] In the following description, pixels, picture elements, or coefficients (transform coefficients or transform coefficients that have undergone a linear transformation) are commonly referred to as samples. Using a sample may mean using a pixel value, picture element value, or coefficient (transform coefficients or transform coefficients that have undergone a linear transformation).

[0042] FIG. 1 illustrates an example of a video coding system according to an embodiment of the present disclosure.

[0043] The video coding system may include a source device 10 and a receiving device 20. The source device 10 may transmit encoded video / image information or data to the receiving device 20 in the form of a file or streaming data via a digital storage medium or a network.

[0044] The source device 10 may include a video source 11, an encoding device 12, and a transmitter 13. The receiving device 20 may include a receiver 21, a decoding device 22, and a renderer 23. The encoding device 10 is referred to as a video / image encoding device, and the decoding device 20 is referred to as a video / image decoding device. The transmitter 13 may be included in the encoding device 12. The receiver 21 may be included in the decoding device 22. The renderer 23 may include a display unit, which may be configured as a separate device or an external component.

[0045] A video source can acquire video / images through a video / image capture, synthesis, or generation process, etc. A video source can include a video / image capture device and / or a video / image generation device. A video / image capture device can include, for example, one or more cameras, a video / image archive containing previously captured video / images, etc. A video / image generation device can include, for example, a computer, a tablet, a smartphone, etc., and can (electronically) generate video / images. For example, a virtual video / image can be generated via a computer, etc., in which case the video / image capture process can be replaced by a process in which the associated data is generated.

[0046] The encoding device 12 can encode the input video / image. The encoding device 12 can perform a series of procedures such as prediction, transformation, and quantization for compression and coding efficiency. The encoded data (encoded video / image information) can be output in the form of a bitstream.

[0047] The transmitter 13 may transmit the encoded video / image information or data output in the form of a bitstream to a receiver of a receiving device via a digital storage medium or a network in the form of a file or streaming. The digital storage medium may include various storage media such as a universal serial bus (USB), secure digital (SD), compact disk (CD), digital video disk (DVD), Blu-ray, hard disk drive (HDD), and solid state drive (SSD). The transmitter 13 may include elements for generating a media file in a predetermined file format and elements for transmission via a broadcast / communication network. The receiver 21 may extract the bitstream and transmit it to a decoding device 22.

[0048] The decoding device 22 can decode the video / image by performing a series of procedures such as inverse quantization, inverse transform, and prediction corresponding to the operations of the encoding device 12.

[0049] The renderer 23 can render the decoded video / images, and the rendered video / images can be displayed via a display unit.

[0050] FIG. 2 shows a schematic block diagram of an encoding device for encoding a video / image signal according to an embodiment of the present disclosure.

[0051] 2, the encoding apparatus 100 may include an image division unit 110, a subtraction unit 115, a transform unit 120, a quantization unit 130, an inverse quantization unit 140, an inverse transform unit 150, an addition unit 155, a filtering unit 160, a memory 170, an inter prediction unit 180, an intra prediction unit 185, and an entropy encoding unit 190. The inter prediction unit 180 and the intra prediction unit 185 may be commonly referred to as a prediction unit. That is, the prediction unit may include the inter prediction unit 180 and the intra prediction unit 185. The transform unit 120, the quantization unit 130, the inverse quantization unit 140, and the inverse transform unit 150 may be included in a residual processing unit. The residual processing unit may further include a subtraction unit 115. The image division unit 110, subtraction unit 115, transformation unit 120, quantization unit 130, inverse quantization unit 140, inverse transformation unit 150, addition unit 155, filtering unit 160, inter prediction unit 180, intra prediction unit 185, and entropy encoding unit 190 may be configured as a single hardware component (e.g., an encoder or processor) depending on the embodiment. Also, the memory 170 may be configured as a single hardware component (e.g., a memory or digital storage medium) depending on the embodiment, and the memory 170 may include a decoded picture buffer (DPB) 175.

[0052] The image division unit 110 may divide an input image (or picture, frame) input to the encoding device 100 into one or more processing units. For example, the processing units may be called coding units (CUs). In this case, the coding units may be recursively divided from a coding tree unit (CTU) or a largest coding unit (LCU) using a quad-tree binary-tree (QTBT) structure. For example, one coding unit may be divided into multiple coding units of deeper depths based on a quad-tree structure and / or a binary tree structure. In this case, for example, the quad-tree structure may be applied first, and then the binary tree structure may be applied. Alternatively, the binary tree structure may be applied first. The coding procedure described herein is performed based on the final coding unit that is not further divided. In this case, based on coding efficiency according to image characteristics, the largest coding unit may be immediately used as the final coding unit, or, if necessary, the coding unit may be recursively divided into coding units of lower depth, and the coding unit of the optimal size may be used as the final coding unit. Here, the coding procedure may include procedures such as prediction, transformation, and restoration, which will be described later. As another example, the processing unit may further include a prediction unit (PU) or a transform unit (TU). In this case, the prediction unit and the transform unit may be divided or partitioned from the final coding unit. The prediction unit may be a unit of sample prediction, and the transform unit may be a unit for deriving transform coefficients and / or a unit for deriving a residual signal from the transform coefficients.

[0053] The term "unit" may be used interchangeably with terms such as "block" or "area." In general, an MxN block can refer to a set of samples or transform coefficients consisting of M columns and N rows. A sample can generally refer to a pixel or a pixel value, and can refer to only a pixel / pixel value of a luma component, or only a pixel / pixel value of a chroma component. A sample can be used as a term that corresponds to one pixel or pel of a picture (or image).

[0054] The encoding apparatus 100 may subtract a prediction signal (predicted block, prediction sample array) output from the inter prediction unit 180 or the intra prediction unit 185 from an input image signal (original block, original sample array) to generate a residual signal (residual block, residual sample array), and the generated residual signal is transmitted to the conversion unit 120. In this case, as shown, a unit in the encoding apparatus 100 that subtracts a prediction signal (predicted block, prediction sample array) from an input image signal (original block, original sample array) is called a subtraction unit 115. The prediction unit may perform prediction on a block to be processed (hereinafter, referred to as a current block) and generate a predicted block including prediction samples for the current block. The prediction unit may determine whether intra prediction or inter prediction is applied on a block or CU basis. The prediction unit may generate various information related to prediction, such as prediction mode information, and transmit the information to the entropy encoding unit 190, as will be described later in the description of each prediction mode. The prediction information can be encoded by the entropy encoding unit 190 and output in the form of a bitstream.

[0055] The intra prediction unit 185 may predict the current block by referring to samples in the current picture. The referenced samples may be located in the neighborhood of the current block or may be located far away, depending on the prediction mode. Prediction modes in intra prediction may include a plurality of non-directional modes and a plurality of directional modes. The non-directional modes may include, for example, a DC mode and a planar mode. The directional modes may include, for example, 33 directional prediction modes or 65 directional prediction modes depending on the degree of precision of the prediction direction. However, this is merely an example, and more or less directional prediction modes may be used depending on the settings. The intra prediction unit 185 may also determine the prediction mode to be applied to the current block using the prediction modes applied to neighboring blocks.

[0056] The inter prediction unit 180 may derive a predicted block for a current block based on a reference block (reference sample array) identified by a motion vector on a reference picture. To reduce the amount of motion information transmitted in inter prediction mode, the motion information may be predicted in units of blocks, sub-blocks, or samples based on the correlation of motion information between neighboring blocks and the current block. The motion information may include a motion vector and a reference picture index. The motion information may further include information on the inter prediction direction (such as L0 prediction, L1 prediction, or Bi prediction). In the case of inter prediction, the neighboring blocks may include spatial neighboring blocks present in the current picture and temporal neighboring blocks present in the reference picture. The reference picture including the reference block and the reference picture including the temporal neighboring blocks may be the same or different. The temporal neighboring blocks are also called collocated reference blocks, collocated CUs (colCUs), etc., and the reference picture including the temporal neighboring blocks is also called a collocated picture (colPic). For example, the inter predictor 180 may construct a candidate list of motion information based on neighboring blocks and generate information indicating which candidate is used to derive a motion vector and / or a reference picture index for the current block. Inter prediction is performed based on various prediction modes. For example, in the case of skip mode and merge mode, the inter predictor 180 may use motion information of neighboring blocks as motion information for the current block. In the case of skip mode, unlike in merge mode, a residual signal may not be transmitted.In the case of motion vector prediction (MVP) mode, the motion vector of the current block can be indicated by using the motion vector of a neighboring block as a motion vector predictor and signaling the motion vector difference.

[0057] The prediction signal generated via the inter predictor 180 or the intra predictor 185 can be used to generate a reconstructed signal or a residual signal.

[0058] The transform unit 120 may generate transform coefficients by applying a transform technique to the residual signal. For example, the transform technique may include at least one of a discrete cosine transform (DCT), a discrete sine transform (DST), a Karhunen-Loeve transform (KLT), a graph-based transform (GBT), or a conditionally non-linear transform (CNT). Here, GBT refers to a transform obtained from a graph that represents inter-pixel relationship information. CNT refers to a transform obtained based on a prediction signal generated using all previously reconstructed pixels. In addition, the transform process may be applied to pixel blocks having the same square size, or to non-square blocks of variable size.

[0059] The quantization unit 130 quantizes the transform coefficients and transmits them to the entropy encoding unit 190. The entropy encoding unit 190 encodes the quantized signal (information about the quantized transform coefficients) and outputs it as a bitstream. The information about the quantized transform coefficients is called residual information. The quantization unit 130 may rearrange the quantized transform coefficients in a block form into a one-dimensional vector form based on the coefficient scan order, and may generate information about the quantized transform coefficients based on the quantized transform coefficients in a one-dimensional vector form. The entropy encoding unit 190 may perform various encoding methods, such as exponential Golomb, context-adaptive variable length coding (CAVLC), context-adaptive binary arithmetic coding (CABAC), etc. The entropy encoding unit 190 may also encode information required for video / image restoration other than the quantized transform coefficients (e.g., values ​​of syntax elements, etc.) together with or separately from the quantized transform coefficients. The encoded information (e.g., video / image information) may be transmitted or stored in the form of a bitstream in network abstraction layer (NAL) units. The bitstream may be transmitted over a network or stored in a digital storage medium. Here, the network may include a broadcasting network and / or a communication network, and the digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. A transmitter (not shown) for transmitting and / or a storage unit (not shown) for storing the signal output from the entropy encoding unit 190 may be configured as an internal / external element of the encoding apparatus 100, or the transmitter may be a component of the entropy encoding unit 190.

[0060] The quantized transform coefficients output from the quantization unit 130 may be used to generate a prediction signal. For example, a residual signal may be reconstructed by applying inverse quantization and inverse transform to the quantized transform coefficients via the inverse quantization unit 140 and the inverse transform unit 150 in a loop. The adder 155 may generate a reconstructed signal (reconstructed picture, reconstructed block, reconstructed sample array) by adding the reconstructed residual signal to the prediction signal output from the inter prediction unit 180 or the intra prediction unit 185. When there is no residual for the current block, such as when skip mode is applied, the predicted block may be used as the reconstructed block. The adder 155 may be referred to as a reconstruction unit or a reconstructed block generator. The reconstructed signal may be used for intra prediction of the next current block in the current picture, or may be used for inter prediction of the next picture after filtering, as described below.

[0061] The filtering unit 160 may apply filtering to the reconstructed signal to improve subjective / objective image quality. For example, the filtering unit 160 may apply various filtering methods to the reconstructed picture to generate a modified reconstructed picture and transmit the modified reconstructed picture to the decoded picture buffer 170. The various filtering methods may include, for example, deblocking filtering, sample adaptive offset, an adaptive loop filter, and a bilateral filter. The filtering unit 160 may generate various information related to filtering and transmit the information to the entropy encoding unit 190, as will be described later in the description of each filtering method. The information related to filtering may be encoded by the entropy encoding unit 190 and output in the form of a bitstream.

[0062] The modified reconstructed picture transmitted to the decoded picture buffer 170 can be used as a reference picture in the inter predictor 180. Through this, when inter prediction is applied, the encoding apparatus 100 can avoid prediction mismatch between the encoding apparatus 100 and the decoding apparatus 200, and can also improve encoding efficiency.

[0063] The decoded picture buffer 170 may store the modified reconstructed picture for use as a reference picture by the inter predictor 180 .

[0064] FIG. 3 shows a schematic block diagram of a decoding device for decoding an image signal as an embodiment of the present invention.

[0065] 3, the decoding device 200 may include an entropy decoding unit 210, an inverse quantization unit 220, an inverse transform unit 230, an adder 235, a filtering unit 240, a memory 250, an inter prediction unit 260, and an intra prediction unit 265. The inter prediction unit 260 and the intra prediction unit 265 may be commonly referred to as a prediction unit. That is, the prediction unit may include an inter prediction unit 180 and an intra prediction unit 185. The inverse quantization unit 220 and the inverse transform unit 230 may be commonly referred to as a residual processing unit. That is, the residual processing unit may include an inverse quantization unit 220 and an inverse transform unit 230. Depending on the embodiment, the entropy decoding unit 210, the inverse quantization unit 220, the inverse transform unit 230, the adder 235, the filtering unit 240, the inter prediction unit 260, and the intra prediction unit 265 may be configured as a single hardware component (e.g., a decoder or a processor). Also, the decoded picture buffer 250 can be implemented by a single hardware component (e.g., a memory or a digital storage medium) depending on the embodiment. Also, the memory 250 can include the DPB 175 and can be implemented by a digital storage medium.

[0066] When a bitstream including video / image information is input, the decoding apparatus 200 can reconstruct an image by processing the video / image information in accordance with the process performed by the encoding apparatus 100 of Fig. 2. For example, the decoding apparatus 200 can perform decoding using the processing unit applied in the encoding apparatus 100. Therefore, the processing unit during decoding may be, for example, a coding unit, and the coding unit may be divided into coding tree units or the largest coding unit according to a quad tree structure and / or a binary tree structure. In addition, the reconstructed image signal decoded and output by the decoding apparatus 200 can be reproduced through a reproduction device.

[0067] The decoding apparatus 200 may receive a signal output from the encoding apparatus 100 of FIG. 2 in the form of a bitstream, and the received signal may be decoded via an entropy decoding unit 210. For example, the entropy decoding unit 210 may parse the bitstream to derive information (e.g., video / image information) necessary for image restoration (or picture restoration). For example, the entropy decoding unit 210 may decode information in the bitstream based on a coding method such as Exponential-Golomb coding, CAVLC, or CABAC, and output values ​​of syntax elements necessary for image restoration and quantized values ​​of transform coefficients related to the residual. More specifically, the CABAC entropy decoding method receives bins corresponding to each syntax element from a bitstream, determines a context model using information about the syntax element to be decoded and decoding information about neighboring and target blocks, or information about symbols / bins decoded in a previous step, predicts the occurrence probability of bins according to the determined context model, and performs arithmetic decoding of the bins to generate symbols corresponding to the values ​​of each syntax element. After determining the context model, the CABAC entropy decoding method may update the context model using information about the decoded symbols / bins for the context model of the next symbol / bin. Prediction information from the information decoded by the entropy decoding unit 210 is provided to a prediction unit (inter prediction unit 260 and intra prediction unit 265), and residual values ​​entropy decoded by the entropy decoding unit 210, i.e., quantized transform coefficients and related parameter information, may be input to the inverse quantization unit 220. In addition, information regarding filtering among the information decoded by the entropy decoding unit 210 can be provided to the filtering unit 240 .Meanwhile, a receiving unit (not shown) that receives the signal output from the encoding device 100 may be further configured as an internal / external element of the decoding device 200, or the receiving unit may be a component of the entropy decoding unit 210.

[0068] The inverse quantization unit 220 may output transform coefficients by inverse quantizing the quantized transform coefficients. The inverse quantization unit 220 may rearrange the quantized transform coefficients in the form of two-dimensional blocks. In this case, the rearrangement may be performed based on the coefficient scanning order performed in the encoding apparatus 100. The inverse quantization unit 220 may obtain transform coefficients by inverse quantizing the quantized transform coefficients using a quantization parameter (e.g., quantization step size information).

[0069] The inverse transform unit 230 can output a residual signal (residual block, residual sample array) by applying an inverse transform to the transform coefficients.

[0070] The prediction unit may perform prediction on the current block and generate a predicted block including prediction samples for the current block. The prediction unit may determine whether intra prediction or inter prediction is applied to the current block based on prediction information output from the entropy decoding unit 210, and may determine a specific intra / inter prediction mode.

[0071] The intra prediction unit 265 may predict the current block by referencing samples in the current picture. The referenced samples may be located in the neighborhood of the current block or may be located far away from the current block depending on the prediction mode. Prediction modes for intra prediction may include a plurality of non-directional modes and a plurality of directional modes. The intra prediction unit 265 may also determine the prediction mode to be applied to the current block using the prediction modes applied to neighboring blocks.

[0072] The inter prediction unit 260 may derive a predicted block for a current block based on a reference block (reference sample array) identified by a motion vector on a reference picture. To reduce the amount of motion information transmitted in inter prediction mode, the motion information may be predicted in units of blocks, sub-blocks, or samples based on the correlation of motion information between neighboring blocks and the current block. The motion information may include a motion vector and a reference picture index. The motion information may further include information regarding the inter prediction direction (e.g., L0 prediction, L1 prediction, Bi prediction, etc.). In the case of inter prediction, the neighboring blocks may include spatial neighboring blocks present in the current picture and temporal neighboring blocks present in the reference picture. For example, the inter prediction unit 260 may construct a candidate list of motion information based on the neighboring blocks and derive a motion vector and / or a reference picture index for the current block based on received candidate selection information. Inter prediction is performed based on various prediction modes, and the prediction information may include information indicating the inter prediction mode for the current block.

[0073] The adder 235 may generate a reconstructed signal (reconstructed picture, reconstructed block, reconstructed sample array) by adding the obtained residual signal to a prediction signal (predicted block, predicted sample array) output from the inter prediction unit 260 or the intra prediction unit 265. When there is no residual for the block to be processed, such as when skip mode is applied, the predicted block can be used as the reconstructed block.

[0074] The adder 235 is called a reconstruction unit or a reconstruction block generator. The generated reconstruction signal can be used for intra prediction of the next block to be processed in the current picture, and can also be used for inter prediction of the next picture after filtering, as will be described later.

[0075] The filtering unit 240 may apply filtering to the reconstructed signal to improve subjective / objective image quality. For example, the filtering unit 240 may apply various filtering methods to the reconstructed picture to generate a modified reconstructed picture and transmit the modified reconstructed picture to the decoded picture buffer 250. The various filtering methods may include, for example, deblocking filtering, sample adaptive offset (SAO), adaptive loop filter (ALF), bilateral filter, etc.

[0076] The modified reconstructed picture sent to the decoded picture buffer 250 can be used as a reference picture by the inter predictor 260 .

[0077] In this specification, the embodiments described for the filtering unit 160, inter prediction unit 180, and intra prediction unit 185 of the encoding device 100 can also be applied identically or correspondingly to the filtering unit 240, inter prediction unit 260, and intra prediction unit 265 of the decoding device, respectively.

[0078] FIG. 4 illustrates an example of a structural diagram of a content streaming system according to an embodiment of the present disclosure.

[0079] The content streaming system to which this specification applies can broadly include an encoding server 410 , a streaming server 420 , a web server 430 , a media storage 440 , a user device 450 , and a multimedia input device 460 .

[0080] The encoding server 410 can compress content input from a multimedia input device such as a smartphone, camera, camcorder, etc. into digital data to generate a bitstream and transmit it to the streaming server 420. As another example, if a multimedia input device 460 such as a smartphone, camera, camcorder, etc. directly generates a bitstream, the encoding server 410 can be omitted.

[0081] The bitstream can be generated by an encoding method or a bitstream generation method to which this specification applies, and the streaming server 420 can temporarily store the bitstream in the process of transmitting or receiving the bitstream.

[0082] The streaming server 420 transmits multimedia data to the user device 450 based on a user request via the web server 430, and the web server 430 acts as an intermediary that informs the user of available services. When a user requests a desired service from the web server 430, the web server 430 transmits the request to the streaming server 420, which then transmits the multimedia data to the user. In this case, the content streaming system may include a separate control server, which controls commands and responses between devices in the content streaming system.

[0083] The streaming server 420 can receive content from the media storage 440 and / or the encoding server 410. For example, the streaming server 420 can receive content in real time from the encoding server 410. In this case, to provide a smooth streaming service, the streaming server 420 can store the bitstream for a certain period of time.

[0084] For example, the user device 450 may include a mobile phone, a smartphone, a laptop computer, a digital broadcasting terminal, a personal digital assistant (PDA), a portable multimedia player (PMP), a navigation system, a slate PC, a tablet PC, an ultrabook, a wearable device such as a smartwatch, a smart glass, a head mounted display (HMD), a digital TV, a desktop computer, and a digital signage.

[0085] Each server in the content streaming system can be operated as a distributed server, in which case data received by each server can be processed in a distributed manner.

[0086] 5 shows an example of a block diagram of an apparatus for processing a video signal according to an embodiment of the present disclosure. The video signal processing apparatus of FIG. 5 may correspond to the encoding apparatus 100 of FIG. 2 or the decoding apparatus 200 of FIG. 3.

[0087] A video signal processing device 500 according to an embodiment of the present disclosure may include a memory 520 for storing a video signal, and a processor 510 coupled to the memory for processing the video signal.

[0088] The processor 510 according to the embodiment of the present specification may be configured with at least one processing circuit for processing a video signal, and may process an image signal by executing a command for encoding or decoding the video signal. That is, the processor 510 may encode an original video signal or decode an encoded video signal by executing an encoding or decoding method described below.

[0089] The processor 510 according to an embodiment of the present invention may be configured with at least one processing circuit for processing an image signal, and may process the image signal by executing a command for encoding or decoding the image signal. That is, the processor 510 may encode original image data or decode an encoded image signal by executing the encoding or decoding method described above.

[0090] Figure 6 shows examples of block division structures of an image according to an embodiment of this specification, with Figure 6a showing an example of a QT (QuadTree, QT), Figure 6b showing an example of a BT (Binary Tree, BT), Figure 6c showing an example of a TT (Ternary Tree, TT), and Figure 6d showing an example of an AT (Asymmetric Tree, AT).

[0091] In video coding, a block can be divided based on the QT. Furthermore, a subblock divided by the QT can be further divided recursively using the QT. A leaf block that is not further divided by the QT can be divided by at least one of BT, TT, or AT. BT can have two types of division: horizontal BT (2NxN, 2NxN) and vertical BT (Nx2N, Nx2N). TT can have two types of division: horizontal TT (2Nx1 / 2N, 2NxN, 2Nx1 / 2N) and vertical TT (1 / 2Nx2N, Nx2N, 1 / 2Nx2N). AT can have four forms of partitioning: horizontal-up AT (2Nx1 / 2N, 2Nx3 / 2N), horizontal-down AT (2Nx3 / 2N, 2Nx1 / 2N), vertical-left AT (1 / 2Nx2N, 3 / 2Nx2N), and vertical-right AT (3 / 2Nx2N, 1 / 2Nx2N). Each BT, TT, and AT can be further partitioned recursively using BT, TT, and AT.

[0092] Figure 6a shows an example of QT division. Block A can be divided into four sub-blocks (A0, A1, A2, A3) by QT. Sub-block A1 can be divided into four sub-blocks (B0, B1, B2, B3) again by QT.

[0093] Figure 6b shows an example of BT division. Block B3, which is not further divided by QT, can be divided into vertical BT (C0, C1) or horizontal BT (D0, D1). Like block C0, each sub-block can be further divided recursively, such as into horizontal BT (E0, E1) or vertical BT (F0, F1).

[0094] Figure 6c shows an example of TT division. Block B3, which is not further divided by QT, can be divided into vertical TTs (C0, C1, C2) or horizontal TTs (D0, D1, D2). Like block C1, each sub-block can be further divided recursively into horizontal TTs (E0, E1, E2) or vertical TTs (F0, F1, F2).

[0095] Figure 6d shows an example of AT division. Block B3, which is not further divided by QT, can be divided into vertical AT(C0, C1) or horizontal AT(D0, D1). Each sub-block, such as block C1, can be further divided recursively, such as into horizontal AT(E0, E1) or vertical TT(F0, F1).

[0096] Meanwhile, division using BT, TT, and AT is possible. For example, sub-blocks divided by BT can be divided by TT or AT. Also, sub-blocks divided by TT can be divided by BT or AT. Sub-blocks divided by AT can be divided by BT or TT. For example, after horizontal BT division, each sub-block can be divided into vertical BT, or after vertical BT division, each sub-block can be divided into horizontal BT. In this case, the order of division is different, but the final divided shape is the same.

[0097] In addition, when a block is divided, the order of searching the block can be defined in various ways. Generally, searching a block from left to right and from top to bottom can refer to the order of determining whether each divided sub-block can be further divided into blocks, the order of encoding each sub-block if the block is not further divided, or the order of searching when a sub-block refers to information on other adjacent blocks.

[0098] 7 and 8 illustrate a video / image encoding procedure based on inter prediction and an inter prediction unit in an encoding device according to an embodiment of the present specification.

[0099] The encoding apparatus 100 performs inter prediction on a current block (S710). The encoding apparatus 100 may derive an inter prediction mode and motion information for the current block and generate a predicted sample for the current block. Here, the procedures for determining the inter prediction mode, deriving the motion information, and generating the predicted sample may be performed simultaneously, or one procedure may be performed before the other. For example, the inter prediction unit 180 of the encoding apparatus 100 may include a prediction mode determination unit 181, a motion information derivation unit 182, and a predicted sample derivation unit 183. The prediction mode determination unit 181 may determine a prediction mode for the current block, the motion information derivation unit 182 may derive motion information for the current block, and the predicted sample derivation unit 183 may derive a predicted sample for the current block. For example, the inter-prediction unit 180 of the encoding apparatus 100 may search for blocks similar to the current block within a certain region (search region) of a reference picture through motion estimation and derive a reference block whose difference from the current block is minimum or equal to or less than a certain criterion. Based on this, a reference picture index indicating a reference picture in which the reference block is located may be derived, and a motion vector may be derived based on the difference between the positions of the reference block and the current block. The encoding apparatus 100 may determine a mode to be applied to the current block from among various prediction modes. The encoding apparatus 100 may compare RD costs for various prediction modes to determine an optimal prediction mode for the current block.

[0100] For example, when a skip mode or a merge mode is applied to the current block, the encoding apparatus 100 may construct a merge candidate list (to be described later) and derive a reference block whose difference from the current block is minimum or equal to or less than a certain criterion among reference blocks indicated by merge candidates included in the merge candidate list. In this case, a merge candidate associated with the derived reference block may be selected, and merge index information indicating the selected merge candidate may be generated and signaled to the decoding apparatus 200. Motion information of the current block may be derived using motion information of the selected merge candidate.

[0101] As another example, when the (A)MVP mode is applied to the current block, the encoding apparatus 100 may construct an (A)MVP candidate list (described later) and use a motion vector of a selected MVP candidate from among the MVP (motion vector predictor) candidates included in the (A)MVP candidate list as the MVP of the current block. In this case, for example, a motion vector pointing to a reference block derived by the motion estimation described above may be used as the motion vector of the current block, and the MVP candidate having the smallest difference from the motion vector of the current block may be selected as the MVP candidate. A motion vector difference (MVD), which is the difference obtained by subtracting the MVP from the motion vector of the current block, may be derived. In this case, information regarding the MVD may be signaled to the decoding apparatus 200. Furthermore, when the (A)MVP mode is applied, the value of the reference picture index may be configured as reference picture index information and separately signaled to the decoding apparatus 200.

[0102] The encoding apparatus 100 may derive residual samples based on the predicted samples (S720). The encoding apparatus 100 may derive residual samples through comparison between the original samples of the current block and the predicted samples.

[0103] The encoding apparatus 100 encodes image information including prediction information and residual information (S730). The encoding apparatus 100 may output the encoded image information in the form of a bitstream. The prediction information may include prediction mode information (e.g., a skip flag, a merge flag, or a mode index) and motion information as information related to a prediction procedure. The motion information may include candidate selection information (e.g., a merge index, an MVP flag, or an MVP index) for deriving a motion vector. The motion information may also include information related to the MVD and / or reference picture index information. The motion information may also include information indicating whether L0 prediction, L1 prediction, or bi-prediction is applied. The residual information is information related to residual samples. The residual information may include information related to quantized transform coefficients for the residual samples.

[0104] The output bitstream can be stored in a (digital) storage medium and then transmitted to the decoding device, or can be transmitted to the decoding device via a network.

[0105] Meanwhile, as described above, the encoding apparatus can generate a reconstructed picture (including reconstructed samples and reconstructed blocks) based on the reference samples and the residual samples. This is because the encoding apparatus 100 can derive the same prediction result as that performed by the decoding apparatus 200, thereby improving coding efficiency. Therefore, the encoding apparatus 100 can store the reconstructed picture (or reconstructed samples, reconstructed blocks) in a memory and use it as a reference picture for inter prediction. As described above, an in-loop filtering procedure, etc. can be further applied to the reconstructed picture.

[0106] 9 and 10 illustrate a video / picture decoding procedure based on inter prediction and an inter prediction unit in a decoding device according to an embodiment of the present specification.

[0107] The decoding apparatus 200 may perform operations corresponding to those performed by the encoding apparatus 100. The decoding apparatus 200 may perform prediction on the current block based on received prediction information and derive predicted samples.

[0108] Specifically, the decoding apparatus 200 may determine a prediction mode for the current block based on the received prediction information (S910). The decoding apparatus 200 may determine which inter-prediction mode is applied to the current block based on prediction mode information in the prediction information.

[0109] For example, the decoding apparatus 200 may determine whether a merge mode is applied to the current block or whether an (A)MVP mode is to be determined based on a merge flag. Alternatively, the decoding apparatus 200 may select one of various candidate inter prediction modes based on a mode index. The candidate inter prediction modes may include skip mode, merge mode, and / or (A)MVP mode, or may include various inter prediction modes described below.

[0110] The decoding apparatus 200 derives motion information of the current block based on the determined inter prediction mode (S920). For example, when a skip mode or a merge mode is applied to the current block, the decoding apparatus 200 may construct a merge candidate list (described below) and select one merge candidate from among the merge candidates included in the merge candidate list. The merge candidate is selected based on a merge index. Motion information of the current block may be derived from motion information of the selected merge candidate. The motion information of the selected merge candidate may be used as motion information of the current block.

[0111] As another example, when the (A)MVP mode is applied to the current block, the decoding apparatus 200 may construct an (A)MVP candidate list (described below) and use a motion vector of an MVP candidate selected from the MVP candidates included in the (A)MVP candidate list as the MVP of the current block. The MVP selection is performed based on the selection information (MVP flag or MVP index). In this case, the decoding apparatus 200 may derive the MVD of the current block based on information related to the MVD, and may derive a motion vector of the current block based on the MVP and MVD of the current block. In addition, the decoding apparatus 200 may derive a reference picture index of the current block based on reference picture index information. In the reference picture list for the current block, a picture pointed to by a reference picture index may be derived as a reference picture referenced for inter-prediction of the current block.

[0112] On the other hand, as will be described later, the motion information of the current block can be derived without constructing a candidate list, in which case the motion information of the current block can be derived by a procedure started in a prediction mode, as will be described later. In this case, the construction of the candidate list as described above can be omitted.

[0113] The decoding apparatus 200 may generate predictive samples for the current block based on the motion information of the current block (S930). In this case, the decoding apparatus 200 may derive a reference picture based on the reference picture index of the current block, and derive predictive samples for the current block using samples of the reference block to which the motion vector of the current block points in the reference picture. In this case, as described below, a predictive sample filtering procedure may further be performed on all or some of the predictive samples of the current block, depending on the circumstances.

[0114] For example, the inter prediction unit 260 of the decoding device 200 may include a prediction mode determination unit 261, a motion information derivation unit 262, and a prediction sample derivation unit 263, and may determine a prediction mode for the current block based on prediction mode information received by the prediction mode determination unit 181, derive motion information (such as a motion vector and / or a reference picture index) of the current block based on information regarding the motion information received by the motion information derivation unit 182, and derive a prediction sample of the current block from the prediction sample derivation unit 183.

[0115] The decoding apparatus 200 generates residual samples for the current block based on the received residual information (S940). The decoding apparatus 200 generates reconstructed samples for the current block based on the predicted samples and the residual samples, and generates a reconstructed picture based on the reconstructed samples (S950). As described above, an in-loop filtering procedure can then be applied to the reconstructed picture.

[0116] As described above, the inter prediction procedure may include an inter prediction mode determination step, a motion information derivation step based on the determined prediction mode, and a prediction execution step (generation of prediction samples) based on the derived motion information.

[0117] Various inter-prediction modes can be used for predicting the current block in a picture. For example, merge mode, skip mode, MVP mode, affine mode, etc. can be used. Decoder side motion vector refinement (DMVR) mode, adaptive motion vector resolution (AMVR) mode, etc. can also be used as additional modes. Affine mode is also called affine motion prediction mode. MVP mode is also called advanced motion vector prediction mode.

[0118] Prediction mode information indicating the inter prediction mode of the current block may be signaled from the encoding apparatus to the decoding apparatus 200. The prediction mode information may be included in a bitstream and received by the decoding apparatus 200. The prediction mode information may include index information indicating one of multiple candidate modes. Alternatively, the inter prediction mode may be indicated through hierarchical signaling of flag information. In this case, the prediction mode information may include one or more flags. For example, the encoding apparatus 100 may signal a skip flag to indicate whether the skip mode is applied, and if the skip mode is not applied, signal a merge flag to indicate whether the merge mode is applied. If the merge mode is not applied, the encoding apparatus 100 may indicate that the MVP mode is applied, or may further signal a flag for further classification. The affine mode may be signaled as an independent mode or as a mode dependent on the merge mode, MVP mode, etc. For example, the affine mode may be configured as one candidate in a merge candidate list or an MVP candidate list, as described below.

[0119] The encoding apparatus 100 or the decoding apparatus 200 may perform inter prediction using motion information of a current block. The encoding apparatus 100 may derive optimal motion information for the current block through a motion estimation procedure. For example, the encoding apparatus 100 may use an original block in an original picture for the current block to search for a similar reference block with high correlation within a predetermined search range in the reference picture in fractional pixel units, thereby deriving motion information. Block similarity may be derived based on a difference in phase-based sample values. For example, block similarity may be calculated based on the sum of absolute differences (SAD) between the current block (or a template of the current block) and the reference block (or a template of the reference block). In this case, motion information may be derived based on the reference block with the smallest SAD within the search range. The derived motion information may be signaled to a decoding apparatus in various ways based on the inter prediction mode.

[0120] When the merge mode is applied, the motion information of the current prediction block is not directly transmitted, but is derived using the motion information of the surrounding prediction blocks. Therefore, the encoding apparatus 100 can indicate the motion information of the current prediction block by transmitting flag information indicating that the merge mode is used and a merge index indicating which surrounding prediction block is used.

[0121] To perform the merge mode, the encoding apparatus 100 should search for merge candidate blocks to be used to derive motion information of the current prediction block. For example, up to five merge candidate blocks may be used, but this specification is not limited to this. The maximum number of merge candidate blocks may be transmitted in the slice header, but this specification is not limited to this. After finding the merge candidate blocks, the encoding apparatus 100 may generate a merge candidate list and select the merge candidate block with the smallest cost as the final merge candidate block.

[0122] This specification provides various examples for the merge candidate blocks that make up the merge candidate list.

[0123] The merge candidate list may, for example, utilize five merge candidate blocks: four spatial merge candidates and one temporal merge candidate.

[0124] A method for constructing a reference picture list for the aforementioned inter prediction will now be described.

[0125] High Level Syntax (HLS) exists in video coding systems to control the general sharing of common information. The purpose of HLS is to manage frames in order to efficiently reduce the number of bits that can be used in memory. HLS uses a reference picture management system, which invokes a process called reference picture marking. This process labels each frame according to its usability for decoding the current frame. In designing and implementing HLS, the goal was to have a simple and effective means for identifying pictures required for decoding the current picture.

[0126] Example 1

[0127] In this embodiment, a reference picture list (RPL) management system is introduced, which is directly signaled and derived. Two lists (i.e., Reference Picture List0 and Reference Picture List1) are used. Information for deriving the two reference picture lists is signaled as syntax elements in the Sequence Parameter Set (SPS), Picture Parameter Set (PPS), and slice headers. Two lists are generated for all types of slices. However, I-slices do not use any information from the lists, while P-slices use only information from List0 and B-slices use information from all the lists. Pictures are identified using a picture order count (POC) number. Each picture can be classified as a "long-term reference picture (LTRP)" or a "short-term reference picture (STRP)." A picture classified as LTRP or STRP is an active picture, meaning that it can be used during decoding, while an inactive picture is one that is not used during the decoding process.

[0128] 11 shows an example of a flowchart for constructing a reference picture list according to an embodiment of the present disclosure. The operations of FIG.

[0129] FIG. 11 shows the overall process of constructing a reference picture list. First, in step S1105, the decoding apparatus 200 parses the total number of entries in the list. Then, for each entry, a picture is marked as an LTRP or STRP through a reference picture marking process (S1120). In this document, marking a picture refers to the operation of adding or updating information (parameters) about the picture. Specifically, the decoding apparatus 200 initializes an entry index (i) (S1110), and if the entry index (i) is less than or equal to the total number of entries (S1115), marks the picture as a reference picture (S1120). Furthermore, once the reference picture marking is completed, the decoding apparatus updates the entry index (i) (S1125). An embodiment of the present invention proposes a new structure for improving the reference picture management system.

[0130] 12 and 13 show an example of a flowchart for marking reference pictures according to an embodiment of the present disclosure.

[0131] [Table 1]

[0132] Table 1 is an example of syntax for constructing a reference picture list. This embodiment provides a possible structure for managing reference pictures, where the ref_pic_list_struct function receives, as its first input, listIdx, indicated by the syntax element rpl1_same_as_rpl0_flag. That is, if the flag (rpl1_same_as_rpl0_flag) is true, this indicates that the lists are identical and one list (L0 or L1) can be inferred from another (L1 or L0). The second input (rplsIdx) of the ref_pic_list_struct function indicates the number of reference picture lists used in the coded video sequence specified in the SPS. num_ref_entries indicates the number of entries in the syntax structure ref_pic_list_struct(listIdx,rpIsIdx). For each entry, if an LTRP frame is used, a flag is parsed to read whether the frame in the list is treated as an LTRP or a STRP. If the entry is a STRP and is the first STRP entry in the syntax structure ref_pic_list_struct(listIdx,rpIsIdx), it represents the absolute difference between the POC of the current picture and the POC of the picture referenced by the i-th entry; if the i-th entry is a STRP entry but is not the first entry, it represents the absolute difference between the POC of the picture referenced by the i-th entry and the POC of the picture pointed to by the previous STRP.(If the entry is a STRP, it is the first STRP entry in the ref_pic_list_struc(listIdx,rpIsIdx) syntax structure and it specifies the absolute difference in POC of the current picture and the picture referred to by the i-th entry, or when the i-th entry is an STRP entry but not the first entry, then it specifies the absolute difference between the POC of the picture referred to by the i-th entry and by the previous STRP). If abs_delta_poc for the i-th entry is greater than 0, a flag marking the sign information of the STRP is signaled. However, if st_ref_pic_flag is 0, it indicates that the i-th entry is a long-term reference picture. The LTRP is signaled by the POC LSB (least significant bit).

[0133] The syntax elements for Table 1 can be defined as follows:

[0134] num_ref_entries[listIdx][rplsIdx] represents the number of entries in the structure of the ref_pic_list_struct(listIdx,rplsIdx) syntax. The value of num_ref_entries[listIdx][rplsIdx] can be between 0 and sps_max_dec_pic_buffering_minus1+14.

[0135] If rpl1_same_as_rpl0_flag is 1, the syntax structures num_ref_pic_lists_in_sps[1] and ref_pic_list_struct(1,rplsIdx) do not exist and the following applies:

[0136] - The value of num_ref_pic_lists_in_sps[1] is inferred to be the same as the value of num_ref_pic_lists_in_sps[0].

[0137] - The value of each syntax element in ref_pic_list_struct(1,rplsIdx) is inferred to be the same as the value of the corresponding syntax element in ref_pic_list_struct(0,rplsIdx) for rplsIdx, which can be a value between 0 and num_ref_pic_lists_in_sps[0]-1.

[0138] long_term_ref_pics_flag is 0, which indicates that the LTRP is not used for inter prediction of a coded picture in a CVS (coded video sequence).long_term_ref_pics_flag is 1, which indicates that the LTRP is used for inter prediction of one or more coded pictures in a CVS.

[0139] If st_ref_pic_flag[listIdx][rplsIdx][i] is 1, it indicates that the i-th entry in the syntax structure ref_pic_list_struct(listIdx,rplsIdx) is a STRP entry. If st_ref_pic_flag[listIdx][rplsIdx][i] is 0, it indicates that the i-th entry in the syntax structure ref_pic_list_struct(listIdx,rplsIdx) is an LTRP entry. If not present, the value of st_ref_pic_flag[listIdx][rplsIdx][i] is inferred to be 1.

[0140] abs_delta_poc_st[listIdx][rplsIdx][i] represents the absolute difference between the picture order count values ​​of the current picture and the picture referenced by the i-th entry if the i-th entry is the first STRP in the syntax structure ref_pic_list_struct(listIdx,rplsIdx), or represents the absolute difference between the picture order count values ​​of the picture indicated by the i-th entry and the previous STRP in the syntax structure ref_pic_list_struct(listIdx,rplsIdx) if the i-th entry is a STRP entry but is not the first STRP entry in the syntax structure ref_pic_list_struct(listIdx,rplsIdx).

[0141] The value of abs_delta_poc_st[listIdx][rplsIdx][i] is between 0 and 215-1.

[0142] If strp_entry_sign_flag[listIdx][rplsIdx][i] is 1, it indicates that the i-th entry in the syntax structure ref_pic_list_struct(listIdx,rplsIdx) has a value greater than or equal to 0. If strp_entry_sign_flag[listIdx][rplsIdx][i] is 0, it indicates that the i-th entry in the syntax structure ref_pic_list_struct(listIdx,rplsIdx) has a value less than 0. If not present, the value of strp_entry_sign_flag[i][j] is inferred to be 1.

[0143] poc_lsb_lt[listIdx][rplsIdx][i] represents the picture order count modulo MaxPicOrderCntLsb of the picture referenced by the i-th entry in the syntax structure ref_pic_list_struct(listIdx,rplsIdx). The length of the syntax element poc_lsb_lt[listIdx][rplsIdx][i] is log2_max_pic_order_cnt_lsb_minus4+4 bits.

[0144] The above syntax element descriptions can be organized as shown in Table 2 below.

[0145] [Table 2]

[0146] 12, the decoding apparatus 200 determines whether an LTRP is used in a corresponding reference picture list entry via long_term_ref_pics_flag (S1205). If an LTRP is used (long_term_ref_pics_flag is 1), the decoding apparatus 200 parses st_ref_pic_flag (S1210) and determines whether st_ref_pic_flag is true (S1215). If st_ref_pic_flag is true (1) (i.e., the current entry corresponds to an STRP), the decoding apparatus 200 parses abs_delta_poc_st. That is, if the current entry corresponds to an STRP entry, the decoding apparatus 200 obtains a difference value between the POC of the picture related to the current entry and the previous picture.

[0147] If long_term_ref_pics_flag is false in step 1205, the decoding device immediately parses abs_delta_poc_st. Thereafter, the decoding device 200 checks whether abs_delta_poc_st is greater than 0 (S1225), and if abs_delta_poc_st is greater than 0, parses strp_entry_sign_flag (S1230).

[0148] If st_ref_pic_flag is false (0) at step S1215, the decoding apparatus 200 parses poc_lsb_lt (S1235). That is, if the current entry corresponds to an LTRP entry, the decoding apparatus 200 obtains the POC modulo value of the picture related to the current entry. The POC modulo value of the picture is a value used to derive the POC value of the corresponding picture, and the POC value of the picture can be derived in combination with the POC most significant bit (MSB).

[0149] The above-described reference picture list construction process can be expressed by two blocks as shown in Figure 13. The operations of each block can be implemented by the same hardware or different hardware. The decoding apparatus 200 acquires the number of entries (num_ref_entries) and a flag (long_term_ref_pic_flag) indicating whether a long-term reference picture is used (1310), and determines whether each entry is an LTRP or STRP (1320).

[0150] Example 2

[0151] In this embodiment, we propose to limit the parameter num_ref_entries described in Table 1. Typically, num_ref_entries[listIdx][rpIsIdx] represents the number of entries in ref_pic_list_struct(listIdx, rpIsIdx). It is noted that the value of num_ref_entries can range from 0 to sps_max_dec_pic_buffering_minus1+14. A limit on the range of num_ref_entries can be introduced so that fewer frames are used. The worst-case scenario is to require 14 frames more than the value indicated by sps_max_dec_pic_buffering_minus1. This means that the decoded picture buffer must be able to have sufficient memory to accommodate that many frames. Alternatively, for hierarchical groups of structures (GOPs), a limit can be introduced taking into account the layer's temporal ID.

[0152] Further syntax can be defined as shown in Table 3. sps_max_dec_pic_buffering_minus1 plus 1 represents the maximum required size of one decoded picture buffer in CVS in units of picture storage buffer. The value of sps_max_dec_pic_buffering_minus1 can have a value between 0 and MaxDpbSize-1, and MaxDpbSize can be defined separately.

[0153] [Table 3]

[0154] That is, the number of entries in the reference picture list may have a maximum value equal to the maximum required size of the decoding picture buffer plus 14. Also, in this embodiment (embodiment 2), the number of entries in the reference picture list may have a maximum value equal to the maximum required size of the decoding picture buffer.

[0155] Example 3

[0156] In this embodiment, a method is proposed in which the syntax element long_term_ref_pics_flag is removed from ref_pic_list_struct(listIdx, rpIsIdx). When this syntax element (long_term_ref_pics_flag) is 0, it indicates that the LTRP is no longer used for inter-prediction of a coded picture in CVS. When long_term_ref_pics_flag is 1, it indicates that the LTRP is used for inter-prediction of one or more coded filters in CVS. By removing the syntax element long_term_ref_pics_flag, it can be inferred that the LTRP is basically used in CVS. This is useful because the LTRP is usually a highly reliable picture in DPB.

[0157] Example 4

[0158] FIG. 14 illustrates another example of a flowchart for marking reference pictures according to an embodiment of the present disclosure.

[0159] Another embodiment for ref_pic_list_struct(listIdx, rpIsIdx) is provided. In particular, ref_pic_list_struct(listIdx, rpIsIdx) can be set taking into account the case where LTRP and STRP are signaled separately. This can be expressed as shown in Table 4 below. Here, the embodiment of the present specification provides a method of limiting num_strp_entries[listIdx][rpIsIdx], which represents the total number of STRPs used, and num_ltrp_entries[listIdx][rpIsIdx], which represents the total number of LTRPs.

[0160] Figure 14 shows an overview of the proposed structure in which the number of LTRPs and STRPs are signaled separately. Here, the number of STRPs and LTRPs is first parsed using the LTRP via long_term_ref_pics_flag. Then, for each entry, lt_ref_pic_flag is parsed, which indicates whether the picture is an LTRP or not. If lt_ref_pic_flag is true, the counter for the LTRP is incremented. Otherwise, the counter for the STRP is incremented. Furthermore, if lt_ref_pic_flag is false, the STRP is identified by parsing delta_poc_st and its sign information (if delta_poc_st is greater than 0). Otherwise, the LTRP is identified by its LSB POC.

[0161] Table 4 below is an example of syntax for constructing a reference picture list according to this embodiment.

[0162] [Table 4]

[0163] num_strp_entries[listIdx][rplsIdx] represents the number of STRP entries in ref_pic_list_struct(listIdx,rplsIdx).

[0164] num_ltrp_entries[listIdx][rplsIdx] represents the number of LTRP entries in ref_pic_list_struct(listIdx,rplsIdx).

[0165] If lt_ref_pic_flag[listIdx][rplsIdx] is not present, the value of lt_ref_pic_flag[listIdx][rplsIdx] is inferred as follows:

[0166] If num_ltrp_entries[listIdx][rplsIdx] is 0, the value (lt_ref_pic_flag[listIdx][rplsIdx]) is inferred to be 0. Otherwise, if numLtrps (number of LTRPs) is like num_ltrp_entries[listIdx][rplsIdx], it is inferred to be 0, and if numStrps (number of STRPs) is like num_strp_entries[listIdx][rplsIdx], it is inferred to be 0.

[0167] [Table 5]

[0168] 14, the decoding apparatus 200 starts constructing a reference list (S1405) and parses the number of STRP entries (num_strp_entries) (S1410). The decoding apparatus 200 checks whether an LTRP is used (S1415), and if an LTRP is used, parses the number of LTRP entries (num_ltrp_entries) (S1420). The decoding apparatus 200 initializes variables i, numLtrps, and numStrps to 0 (S1425), determines whether the variable i is equal to or less than the number of entries (num_ref_entires) (S1430), and if i is equal to or less than the number of entries (num_ref_entires), performs a reference picture marking operation (S1435), and updates i (S1440).

[0169] 15 shows an example of a flowchart for processing a video signal according to an embodiment of the present invention. The operations of FIG.

[0170] In step S1510, the decoding apparatus 200 constructs a reference picture list for a current picture in a video signal. Here, if a first entry in the reference picture list corresponds to an STRP entry, the decoding apparatus 200 obtains a POC difference value (e.g., abs_delta_poc_st) between the picture corresponding to the first entry and the previous picture. If a second entry in the reference picture list corresponds to an LTRP entry, the decoding apparatus 200 obtains a POC modulo value (e.g., poc_lsb_lt) of the picture corresponding to the second entry.

[0171] In one embodiment, in the process of constructing a reference picture list, the decoding device 200 obtains the number of entries (num_ref_entries) of the reference picture list, and can mark the reference picture in the LTRP or STRP for each entry of the reference picture list by the number of entries.

[0172] In one embodiment, the number of entries may have a maximum value equal to the maximum required size of the decoded picture buffer plus 14 (sps_max_dec_pic_buffering_minus1+14) or the maximum required size of the decoded picture buffer (sps_max_dec_pic_buffering_minus1).

[0173] In one embodiment, in the process of marking reference pictures, the decoding device 200 acquires the number of STRP entries (e.g., num_strp_entries) and the number of LTRP entries (e.g., num_ltrp_entries) as shown in Table 4, and can mark STRPs as many times as the number of STRP entries and mark LTRPs as many times as the number of LTRP entries.

[0174] In one embodiment, in the process of constructing a reference picture list, decoding apparatus 200 may acquire a first flag and determine whether an entry in the reference picture list corresponds to a STRP entry or an LTRP entry based on the first flag. For example, decoding apparatus 200 may determine whether the current entry is a STRP entry or an LTRP entry using a flag (e.g., st_ref_pic_flag) indicating whether a picture related to the current entry corresponds to a STRP.

[0175] In one embodiment, the POC difference value represents the absolute value of the POC of the picture related to the first entry minus the POC of the previous picture. Also, if the absolute value (absolute difference of POC) (e.g., abs_delta_poc_st) is greater than 0, the decoding apparatus 200 may acquire a second flag (e.g., strp_entry_sign_flag) indicating the sign of the POC difference value.

[0176] At step S1520, the decoding apparatus 200 performs prediction for the current picture using the reference picture list. For example, the decoding apparatus 200 may generate prediction samples using sample values ​​indicated by motion vectors in reference pictures included in the first reference picture list L0 or the second reference picture list L1.

[0177] The embodiments of the present specification provide a method and an apparatus for encoding information necessary for constructing a reference picture list. The encoding apparatus 100 constructs the reference picture list and can perform prediction using the reference pictures in the reference picture list.

[0178] In addition, the encoding apparatus 100 may code information for constructing a reference picture list for each entry of the reference picture list. For example, the information for constructing the reference picture list may include a difference value (e.g., abs_delta_poc_st) between the POC of a picture corresponding to the STRP and a previous picture, and a modulo value (e.g., poc_lsb_lt) of the POC of a picture corresponding to the LTRP.

[0179] Furthermore, encoding apparatus 100 may include a first flag (e.g., st_ref_pic_flag) indicating whether a picture of an entry in the reference picture list is an STRP or an LTRP in the information for constructing the reference picture list and may code the information. Furthermore, if the difference value (e.g., abs_delta_poc_st) of the POC is greater than 0, the information may include a second flag (e.g., strp_entry_sign_flag) indicating the sign of the difference value of the POC. The information for constructing the reference picture list may be signaled by being included in an SPS, PPS, or slice header.

[0180] The processing method to which the present specification applies may be produced in the form of a computer-executable program and stored in a computer-readable recording medium. Multimedia data having a data structure according to the present specification may also be stored in a computer-readable recording medium. The computer-readable recording medium includes all types of storage devices and distributed storage devices in which computer-readable data is stored. The computer-readable recording medium may include, for example, a Blu-ray Disc (BD), a Universal Serial Bus (USB), a ROM, a PROM, an EPROM, an EEPROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, and an optical data storage device. The computer-readable recording medium may also include media realized in the form of a carrier wave (e.g., transmission via the Internet). The bitstream generated by the encoding method may be stored in a computer-readable recording medium or transmitted via a wired or wireless communication network.

[0181] Furthermore, the embodiments of the present specification may be realized in a computer program product by program code, which may be executed by a computer according to the embodiments of the present specification. The program code may be stored on a computer-readable carrier.

[0182] The decoding device and encoding device to which this specification applies may be included in a digital device. The term "digital device" includes, for example, all digital devices capable of performing at least one of transmitting, receiving, processing, and outputting data, content, services, etc. Here, processing data, content, services, etc. by a digital device includes encoding and / or decoding the data, content, services, etc. Such digital devices pair or connect (hereinafter referred to as "pairing") with other digital devices, external servers, etc. via a wired / wireless network to transmit and receive data and convert data as needed.

[0183] Digital devices include, for example, standing devices such as network TVs, hybrid broadcast broadband TVs (HBTVs), smart TVs, internet protocol televisions (IPTVs), and personal computers (PCs), as well as mobile or handheld devices such as personal digital assistants (PDAs), smartphones, tablet PCs, and laptops.

[0184] Meanwhile, the term "wired / wireless network" used herein refers to a communication network that supports various communication standards or protocols for interconnection and / or data transmission and reception between digital devices or digital devices and external servers. Such wired / wireless networks may include all communication networks and communication protocols currently or in the future supported by standards, for example, communication standards or protocols for wired connections such as Universal Serial Bus (USB), Composite Video Banking Sync (CVBS), component, S-Video (analog), Digital Visual Interface (DVI), High Definition Multimedia Interface (HDMI), RGB, and D-SUB, and communication standards for wireless connections such as Bluetooth, Radio Frequency Identification (RFID), infrared Data Association (IrDA), Ultra Wideband (UWB), ZigBee, Digital Living Network Alliance (DLNA), Wireless LAN (WLAN) (Wi-Fi), Wireless broadband (Wibro), World Interoperability for Microwave Access (Wimax), High Speed ​​Downlink Packet Access (HSDPA), Long Term Evolution (LTE), and Wi-Fi Direct.

[0185] The above-described embodiments are combinations of the elements and features of the present specification in a predetermined form. Each element or feature should be considered optional unless otherwise explicitly stated. Each element or feature may be implemented without being combined with other elements or features. It is also possible to combine some elements and / or features to form embodiments of the present specification. The order of operations described in the embodiments of the present specification may be changed. Some components or features of any embodiment may be included in other embodiments, or may be replaced with corresponding components or features of other embodiments. It is obvious that claims that do not have an explicit reference relationship in the claims may be combined to form embodiments, or may be included in new claims by amendment after filing.

[0186] In the case of implementation by firmware or software, an embodiment of this specification may be implemented in the form of a module, procedure, function, etc. that performs the functions or operations described above. The software code may be stored in a memory and driven by a processor. The memory may be located inside or outside the processor and may exchange data with the processor by various means that are already known.

[0187] It is obvious to those skilled in the art that the present invention can be embodied in other specific forms without departing from the essential features of the present invention. Therefore, the above detailed description should not be construed as limiting in all respects, but should be considered as illustrative. The scope of the present invention should be determined by reasonable interpretation of the appended claims, and all modifications within the scope of the present invention that fall within the equivalent scope of the present invention are included in the scope of the present invention. [Industrial Applicability]

[0188] The above-described preferred embodiments of the present invention have been disclosed for illustrative purposes, and those skilled in the art may improve, modify, substitute, or add various other embodiments within the technical idea and scope of the present invention disclosed in the appended claims.

Claims

1. 1. A method for an apparatus to decode a video signal, comprising: obtaining the number of entries contained in the reference picture list of the current picture; marking each of the reference picture list entries as an STRP or an LTRP, the number of entries being equal to the number of entries; deriving prediction samples for the current picture based on the reference picture list; The number of entries has a maximum value of sps_max_dec_pic_buffering_minus1+14.

2. The step of marking each of the entries comprises: obtaining a difference value of POC between the reference picture associated with the entry marked as STRP and other pictures in the reference picture list; 2. The method of claim 1, further comprising the step of: obtaining a modulo value of a POC of a picture associated with the entry marked as an LTRP.

3. The method of claim 2 , wherein the POC difference value is the absolute value of the POC of the other picture minus the POC of the reference picture associated with the entry marked as the STRP.

4. 4. The method of claim 3, wherein the step of marking each of the entries further comprises the step of acquiring a flag indicating a sign of the POC difference value based on the absolute value being greater than zero.

5. 3. The method of claim 2, wherein, in the marking of each of the reference picture entries, the entry marked as the STRP is the first STRP entry, and the other picture is the current picture; or, in the marking of each of the reference picture entries, the entry marked as the STRP is not the first STRP, and the other picture is a previous STRP in the reference picture list.

6. The step of marking each of the entries comprises: obtaining a flag containing information about whether the entry in the reference picture list corresponds to a STRP entry or a LTRP entry; and determining, based on the flag, whether the entry in the reference picture list corresponds to the STRP entry or the LTRP entry.

7. 1. A method for encoding a video signal by an apparatus, comprising: constructing a reference picture list for the current picture; generating code information for constructing the reference picture list; the code information includes the number of entries in the reference picture list; The number of entries has a maximum value of sps_max_dec_pic_buffering_minus1+14.

8. The code information is a POC difference value between the reference picture associated with the entry marked as STRP and the other pictures in the reference picture list; and a modulo value of a POC of a picture associated with an entry marked as an LTRP in the reference picture list.

9. The method of claim 8 , wherein the POC difference value is the absolute value of the POC of the other picture minus the POC of the reference picture associated with the entry marked as the STRP.

10. The method of claim 9 , wherein the code information further includes a flag indicating a sign of the POC difference value for an entry whose absolute value of the POC difference value is greater than zero.

11. 9. The method of claim 8, wherein the other picture is the current picture based on the entry marked as the STRP being the first STRP entry in the code information, or the other picture is a previous STRP in the reference picture list based on the entry marked as the STRP not being the first STRP in the code information.

12. The method of claim 7 , wherein the code information further includes a flag containing information about whether the entry in the reference picture list corresponds to an STRP entry or an LTRP entry.

13. 8. The method of claim 7, further comprising loading the code information into a sequence parameter set (SPS), a picture parameter set (PPS), or a slice header in a bitstream associated with the video signal.

14. 1. A method for transmitting data including a bitstream associated with a video signal, comprising: obtaining the bitstream containing code information for constructing a reference picture list of a current picture; transmitting the data including the bitstream containing the code information; The bitstream comprises: constructing the reference picture list for the current picture in the video signal; encoding the code information to construct the reference picture list; the code information for configuring the reference picture list includes the number of entries in the reference picture list; The number of entries has a maximum value of sps_max_dec_pic_buffering_minus1+14.

Images