Method and apparatus for processing a video signal using inter-picture prediction - Patents.com

Abstract

To provide methods and devices for decoding video signals using inter prediction.SOLUTION: A method comprises constructing a reference picture list of a current picture in the video signal, and performing a prediction for a current picture by using the reference picture list. The step of constructing the reference picture list comprises, if a first entry of the reference picture list corresponds to an STRP entry, obtaining a POC difference value between a picture related to the first entry and another picture, and if a second entry of the reference picture list corresponds to an LTRP entry, obtaining a POC modulo value of a picture related to the second entry. A reference picture list for identifying a picture may be generated in a simple and effective manner, and thus, the compression performance may be increased and the computation time may be decreased.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] SUMMARY OF THE DISCLOSURE The embodiments herein relate to methods and apparatus for processing a video signal using inter prediction, and more particularly, to methods and apparatus for constructing a reference picture list for prediction of a current picture. [Background technology]

[0002] Compression coding refers to a series of signal processing techniques for transmitting digitized information over a communication line or storing it in a suitable form on a storage medium. Media such as pictures, images, and audio can be subject to compression coding, and the technology for compressing and coding images in particular is called video image compression.

[0003] Next-generation video content will be characterized by high spatial resolution, high frame rate, and high dimensionality of scene representation. Processing such content will bring about a significant increase in memory storage, memory access rate, and processing power.

[0004] Therefore, there is a need to design coding tools to process next-generation image content more efficiently. In particular, video codec standards after the high efficiency video coding (HEVC) standard require prediction techniques that can generate accurate prediction samples while using resources more efficiently. Summary of the Invention [Problem to be solved by the invention]

[0005] The embodiments herein provide a method and apparatus for constructing a reference picture list required for prediction of a current picture.

[0006] Additionally, embodiments herein provide methods and apparatus for obtaining information required to construct a reference picture list.

[0007] The embodiments herein also provide a method and apparatus for defining the number of entries required for constructing a reference picture list.

[0008] Additionally, embodiments herein provide methods and apparatus for reducing the amount of syntax elements required to construct a reference picture list.

[0009] The embodiments herein also provide a method and apparatus for counting the number of times a reference picture is marked in the process of constructing a reference picture list.

[0010] The technical problems to be solved by the present invention are not limited to the technical problems mentioned above, and other technical problems not mentioned should be clearly understood by a person having ordinary skill in the art to which this specification pertains from the following description. [Means for solving the problem]

[0011] An embodiment of the present specification provides a method and apparatus for decoding a video signal using inter prediction. A method for processing a video signal according to an embodiment of the present specification includes: constructing a reference picture list of a current picture in the video signal; and performing prediction for the current picture using the reference picture list, and the constructing the reference picture list includes, when a first entry of the reference picture list corresponds to a short-term reference picture (STRP) entry, acquiring a POC difference value between a picture related to the first entry and another picture; and, when a second entry of the reference picture list corresponds to a long-term reference picture (LTRP) entry, acquiring a POC modulo value of a picture related to the second entry.

[0012] In one embodiment, the step of constructing the reference picture list may include a step of obtaining the number of entries in the reference picture list, and a step of marking the LTRP or STRP as a reference picture for each entry in the reference picture list by the number of entries.

[0013] In one embodiment, the number of entries may have a maximum value of 14 plus the maximum required size of the decoded picture buffer.

[0014] In one embodiment, the step of marking the reference picture may include the steps of obtaining the number of STRP entries and the number of LTRP entries, marking STRPs for the number of STRP entries, and marking LTRPs for the number of LTRP entries.

[0015] In one embodiment, the step of constructing the reference picture list may include the steps of acquiring a first flag and determining whether an entry of the reference picture list corresponds to the STRP entry or the LTRP entry based on the first flag.

[0016] In one embodiment, the POC difference value may be determined as the absolute value of the POC of the other picture minus the POC of the picture related to the first entry.

[0017] In one embodiment, constructing the reference picture list may include obtaining a second flag indicating a sign of the POC difference value if the absolute value is greater than zero.

[0018] A method for encoding a video signal according to another embodiment of the present specification includes constructing a reference picture list for a current picture in the video signal, performing prediction for the current picture using the reference picture list, and coding information for constructing the reference picture list. The information for constructing the reference picture list may include a POC difference between a picture related to a short-term reference picture (STRP) entry in the reference picture list and another picture, and a modulo value of a POC of a picture related to a long-term reference picture (LTRP) entry in a second entry in the reference picture list.

[0019] In one embodiment, the information for constructing the reference picture list further includes the number of entries in the reference picture list, and the number of entries may have a maximum value equal to the maximum required size of the decoded picture buffer plus 14.

[0020] In one embodiment, the information for constructing the reference picture list may further include a first flag indicating whether an entry of the reference picture list corresponds to the STRP entry.

[0021] In one embodiment, the information for constructing the reference picture list may further include a second flag indicating a sign of the POC difference value for an entry whose POC difference value is greater than zero.

[0022] In one embodiment, information for constructing the reference picture list may be included in a sequence parameter set (SPS), a picture parameter set (PPS), or a slice header in a bitstream for the video signal.

[0023] According to another embodiment of the present specification, an apparatus for decoding a video signal includes a memory for storing the video signal, and a processor coupled to the memory. The processor is configured to construct a reference picture list for a current picture in the video signal, and to perform prediction for the current picture using the reference picture list. The processor is also configured to obtain, in order to construct the reference picture list, a difference value of a POC between a picture related to the first entry and another picture when a first entry of the reference picture list corresponds to a STRP entry, and obtain a modulo value of a POC of a picture related to the second entry when a second entry of the reference picture list corresponds to a LTRP entry.

[0024] According to another embodiment of the present specification, an apparatus for encoding a video signal includes a memory for storing the video signal, and a processor coupled to the memory, the processor being configured to construct a reference picture list for a current picture in the video signal, perform prediction for the current picture using the reference picture list, and code information for constructing the reference picture list, the information for constructing the reference picture list including a POC difference between a picture related to a short-term reference picture (STRP) entry in the reference picture list and another picture, and a modulo value of a POC of a picture related to a long-term reference picture (LTRP) entry in a second entry in the reference picture list.

[0025] Another embodiment of the present disclosure provides a non-transitory computer-readable medium storing one or more commands, the one or more commands being executed by one or more processors to control a video signal processing device to configure a reference picture list for a current picture in a video signal and to perform prediction for the current picture using the reference picture list. The one or more commands also control the video signal processing device to obtain a POC difference between a picture related to a first entry of the reference picture list and another picture when a first entry of the reference picture list corresponds to a short-term reference picture (STRP) entry, and to obtain a modulo value of a POC of a picture related to the second entry when a second entry of the reference picture list corresponds to a long-term reference picture (LTRP) entry, in order to configure the reference picture list. Effect of the Invention

[0026] According to an embodiment of the present specification, a list of reference pictures necessary for prediction of a current picture can be constructed through the marking of reference pictures.

[0027] Furthermore, according to the embodiments of the present specification, appropriate information can be obtained for each type of reference picture.

[0028] Furthermore, according to the embodiments of the present specification, by defining the number of entries required for constructing a reference picture list, the memory required for storing pictures can be efficiently managed.

[0029] Additionally, according to the embodiments of the present specification, the amount of data required to construct a reference picture list can be reduced by omitting the parsing of some syntax elements.

[0030] Furthermore, according to the embodiment of the present specification, by counting the number of times a reference picture is marked, it is possible to efficiently realize a syntax for constructing a reference picture list.

[0031] The effects obtained by the present invention are not limited to those mentioned above, and other effects not mentioned should be clearly understood by those having ordinary skill in the art to which the present invention pertains from the following description. [Brief description of the drawings]

[0032] The accompanying drawings, which are included as part of the detailed description to aid in understanding the present specification, provide examples for the present specification and, together with the detailed description, explain the technical features of the present specification.

[0033] [Figure 1] 1 illustrates an example of a video coding system according to an embodiment of the present disclosure. [Diagram 2]1 shows a schematic block diagram of an encoding device for encoding a video / image signal according to an embodiment of the present specification; [Diagram 3] As an example embodiment of the present specification, a schematic block diagram of a decoding device for decoding an image signal is shown. [Figure 4] 1 illustrates an example structural diagram of a content streaming system according to an embodiment of the present disclosure; [Diagram 5] 1 shows an example of a block diagram of an apparatus for processing a video signal according to an embodiment of the present specification. [Figure 6a] 1 shows an example of a block division structure of an image according to an embodiment of the present specification, which is a block division structure based on a QuadTree (QT). [Figure 6b] 1 shows an example of a block division structure of an image according to an embodiment of the present specification, which is an example of a block division structure based on a Binary Tree (BT). [Figure 6c] 1 shows an example of a block division structure of an image according to an embodiment of the present specification, which is an example of a block division structure based on a TT (Ternary Tree, TT). [Figure 6d] 1 shows an example of a block division structure of an image according to an embodiment of the present specification, which is an example of a block division structure based on an Asymmetric Tree (AT). [Figure 7] 2 illustrates a video / image encoding procedure based on inter prediction and an inter prediction unit in an encoding device according to an embodiment of the present specification; [Figure 8] 2 illustrates a video / image encoding procedure based on inter prediction and an inter prediction unit in an encoding device according to an embodiment of the present specification; [Figure 9] 2 illustrates a video / image decoding procedure based on inter prediction and an inter prediction unit in a decoding device according to an embodiment of the present specification; [Figure 10] 2 illustrates a video / image decoding procedure based on inter prediction and an inter prediction unit in a decoding device according to an embodiment of the present specification; [Figure 11] 1 illustrates an example of a flowchart for constructing a reference picture list according to an embodiment of the present disclosure. [Figure 12] 1 illustrates an example of a flow chart for marking reference pictures according to an embodiment of the present disclosure. [Figure 13] 1 illustrates an example of a flow chart for marking reference pictures according to an embodiment of the present disclosure. [Figure 14] 13 illustrates another example of a flow chart for marking reference pictures according to an embodiment of the present disclosure. [Figure 15] 1 shows an example of a flow chart for processing a video signal according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0034] Hereinafter, preferred embodiments of the present specification will be described in detail with reference to the accompanying drawings. The detailed description disclosed below together with the accompanying drawings is intended to describe exemplary embodiments of the present specification, and is not intended to show the only embodiments in which the present specification can be implemented. The following detailed description includes specific details to provide a complete understanding of the present specification. However, those skilled in the art will recognize that the present specification can be implemented without such specific details.

[0035] In some cases, in order to avoid obscuring the concepts of the present specification, well-known structures and devices may be omitted or shown in block diagram form focusing on the core functions of each structure and device.

[0036] In addition, the terms used in this specification are generally those currently in wide use, but in certain cases, terms arbitrarily selected by the applicant are used for explanation. In such cases, the meanings will be clearly described in the detailed explanation of the relevant part, so it is made clear that the terms used in this specification should not be interpreted simply based on their names, but should be interpreted by understanding the meanings of the relevant terms.

[0037] The specific terms used in the following description are provided to facilitate understanding of the present specification, and the use of such specific terms may be changed to other forms without departing from the technical spirit of the present specification. For example, in the case of a signal, data, sample, picture, slice, tile, frame, and block, they may be appropriately replaced and interpreted in each coding process.

[0038] Hereinafter, the term "processing unit" as used herein refers to a unit in which encoding / decoding processes such as prediction, transformation, and / or quantization are performed. In addition, the processing unit may be interpreted as including a unit for a luma component and a unit for a chroma component. For example, the processing unit may correspond to a block, a coding unit (CU), a prediction unit (PU), or a transform unit (TU).

[0039] Also, the processing unit may be interpreted as a unit for a luminance component or a unit for a chrominance component. For example, the processing unit may correspond to a coding tree block (CTB), a coding block (CB), a PU, or a transform block (TB) for a luminance component. Alternatively, the processing unit may correspond to a CTB, a CB, a PU, or a TB for a chrominance component. Also, without being limited thereto, the processing unit may be interpreted to include a unit for a luminance component and a unit for a chrominance component.

[0040] Furthermore, the processing units are not necessarily limited to square blocks, but may also be configured in the form of polygons having three or more vertices.

[0041] In addition, hereinafter, pixels, picture elements, or coefficients (transform coefficients or transform coefficients that have undergone a linear transformation) are commonly referred to as samples. Using a sample may mean using a pixel value, picture element value, or coefficient (transform coefficients or transform coefficients that have undergone a linear transformation), etc.

[0042] FIG. 1 illustrates an example of a video coding system according to an embodiment of the present disclosure.

[0043] The video coding system may include a source device 10 and a receiving device 20. The source device 10 may transmit encoded video / image information or data to the receiving device 20 in the form of a file or streaming via a digital storage medium or a network.

[0044] The source device 10 may include a video source 11, an encoding device 12, and a transmitter 13. The receiving device 20 may include a receiver 21, a decoding device 22, and a renderer 23. The encoding device 10 is referred to as a video / image encoding device, and the decoding device 20 is referred to as a video / image decoding device. The transmitter 13 may be included in the encoding device 12. The receiver 21 may be included in the decoding device 22. The renderer 23 may include a display unit, which may be configured as a separate device or an external component.

[0045] A video source may acquire video / images through a video / image capture, synthesis or generation process, etc. A video source may include a video / image capture device and / or a video / image generation device. A video / image capture device may include, for example, one or more cameras, a video / image archive containing previously captured videos / images, etc. A video / image generation device may include, for example, a computer, a tablet, a smartphone, etc., and may (electronically) generate video / images. For example, a virtual video / image may be generated through a computer, etc., in which case the video / image capture process may be replaced by a process in which the associated data is generated.

[0046] The encoding device 12 can encode the input video / image. The encoding device 12 can perform a series of procedures such as prediction, transformation, quantization, etc. for compression and coding efficiency. The encoded data (encoded video / image information) can be output in the form of a bitstream.

[0047] The transmitting unit 13 may transmit the encoded video / image information or data output in the form of a bitstream to a receiving unit of a receiving device via a digital storage medium or a network in the form of a file or streaming. The digital storage medium may include various storage media such as a universal serial bus (USB), a secure digital (SD), a compact disk (CD), a digital video disk (DVD), a Bluray, a hard disk drive (HDD), and a solid state drive (SSD). The transmitting unit 13 may include elements for generating a media file through a predetermined file format and may include elements for transmission via a broadcasting / communication network. The receiver 21 may extract the bitstream and transmit it to a decoding device 22.

[0048] The decoding device 22 can decode the video / image by performing a series of procedures such as inverse quantization, inverse transformation, prediction, etc., corresponding to the operations of the encoding device 12.

[0049] The renderer 23 can render the decoded video / images. The rendered video / images can be displayed via a display unit.

[0050] FIG. 2 shows a schematic block diagram of an encoding device for encoding a video / image signal according to an embodiment of the present disclosure.

[0051] 2, the encoding device 100 may include an image division unit 110, a subtraction unit 115, a transform unit 120, a quantization unit 130, an inverse quantization unit 140, an inverse transform unit 150, an addition unit 155, a filtering unit 160, a memory 170, an inter prediction unit 180, an intra prediction unit 185, and an entropy encoding unit 190. The inter prediction unit 180 and the intra prediction unit 185 may be commonly referred to as a prediction unit. That is, the prediction unit may include the inter prediction unit 180 and the intra prediction unit 185. The transform unit 120, the quantization unit 130, the inverse quantization unit 140, and the inverse transform unit 150 may be included in a residual processing unit. The residual processing unit may further include a subtraction unit 115. The image division unit 110, subtraction unit 115, transformation unit 120, quantization unit 130, inverse quantization unit 140, inverse transformation unit 150, addition unit 155, filtering unit 160, inter prediction unit 180, intra prediction unit 185, and entropy encoding unit 190 may be configured by one hardware component (e.g., an encoder or processor) according to an embodiment. In addition, the memory 170 may be configured by one hardware component (e.g., a memory or digital storage medium) according to an embodiment, and the memory 170 may include a decoded picture buffer (DPB) 175.

[0052] The image division unit 110 may divide an input image (or picture, frame) input to the encoding device 100 into one or more processing units. As an example, the processing units are called coding units (CUs). In this case, the coding units may be divided recursively from a coding tree unit (CTU) or a largest coding unit (LCU) according to a quad-tree binary-tree (QTBT) structure. For example, one coding unit may be divided into multiple coding units of deeper depths based on a quad-tree structure and / or a binary tree structure. In this case, for example, the quad-tree structure may be applied first, and then the binary tree structure may be applied. Alternatively, the binary tree structure may be applied first. The coding procedure according to this specification is performed based on the final coding unit that is not further divided. In this case, the largest coding unit may be used as the final coding unit immediately based on coding efficiency according to image characteristics, or the coding unit may be recursively divided into coding units of lower depths as necessary, and the coding unit of the optimal size may be used as the final coding unit. Here, the coding procedure may include procedures such as prediction, transformation, and restoration, which will be described later. As another example, the processing unit may further include a prediction unit (PU) or a transform unit (TU). In this case, the prediction unit and the transform unit may be divided or partitioned from the final coding unit described above, respectively. The prediction unit may be a unit of sample prediction, and the transform unit may be a unit for deriving transform coefficients and / or a unit for deriving a residual signal from the transform coefficients.

[0053] The unit may be used interchangeably with terms such as block or area. In the general case, an MxN block may refer to a set of samples or transform coefficients consisting of M columns and N rows. A sample may generally refer to a pixel or a pixel value, or may refer to only a pixel / pixel value of a luma component, or may refer to only a pixel / pixel value of a chroma component. A sample may be used as a term corresponding to a pixel or pel of one picture (or image).

[0054] The encoding apparatus 100 may generate a residual signal (residual block, residual sample array) by subtracting a prediction signal (predicted block, prediction sample array) output from the inter prediction unit 180 or the intra prediction unit 185 from an input image signal (original block, original sample array), and the generated residual signal is transmitted to the conversion unit 120. In this case, as shown, a unit that subtracts a prediction signal (predicted block, prediction sample array) from an input image signal (original block, original sample array) in the encoding apparatus 100 is called a subtraction unit 115. The prediction unit may perform prediction on a block to be processed (hereinafter, referred to as a current block) and generate a predicted block including prediction samples for the current block. The prediction unit may determine whether intra prediction or inter prediction is applied on a block or CU basis. The prediction unit may generate various information related to prediction, such as prediction mode information, and transmit the information to the entropy encoding unit 190, as described below in the description of each prediction mode. The prediction-related information can be encoded in the entropy encoding unit 190 and output in the form of a bitstream.

[0055] The intra prediction unit 185 may predict the current block by referring to samples in the current picture. The referenced samples may be located in the neighborhood of the current block or may be located away from the current block depending on the prediction mode. In the intra prediction, the prediction mode may include a plurality of non-directional modes and a plurality of directional modes. The non-directional modes may include, for example, a DC mode and a planar mode. The directional modes may include, for example, 33 directional prediction modes or 65 directional prediction modes depending on the fineness of the prediction direction. However, this is merely an example, and more or less directional prediction modes may be used depending on the settings. The intra prediction unit 185 may also determine the prediction mode to be applied to the current block using the prediction modes applied to the neighboring blocks.

[0056] The inter prediction unit 180 may derive a predicted block for the current block based on a reference block (reference sample array) identified by a motion vector on a reference picture. In this case, in order to reduce the amount of motion information transmitted in the inter prediction mode, the motion information may be predicted in units of blocks, sub-blocks, or samples based on the correlation of the motion information between the neighboring blocks and the current block. The motion information may include a motion vector and a reference picture index. The motion information may further include information on an inter prediction direction (L0 prediction, L1 prediction, Bi prediction, etc.). In the case of inter prediction, the neighboring blocks may include a spatial neighboring block present in the current picture and a temporal neighboring block present in the reference picture. The reference picture including the reference block and the reference picture including the temporal neighboring block may be the same or different. The temporal neighboring block is called a collocated reference block, a collocated CU (colCU), etc., and the reference picture including the temporal neighboring block is also called a collocated picture (colPic). For example, the inter prediction unit 180 may generate information indicating which candidate is used to construct a candidate list of motion information based on neighboring blocks and derive a motion vector and / or a reference picture index of the current block. Inter prediction is performed based on various prediction modes, and for example, in the case of skip mode and merge mode, the inter prediction unit 180 may use the motion information of neighboring blocks as the motion information of the current block. In the case of skip mode, unlike the merge mode, a residual signal may not be transmitted.In the case of motion vector prediction (MVP) mode, the motion vector of the surrounding blocks can be used as a motion vector predictor, and the motion vector of the current block can be indicated by signaling the motion vector difference.

[0057] The prediction signal generated via the inter prediction unit 180 or the intra prediction unit 185 may be used to generate a reconstructed signal or may be used to generate a residual signal.

[0058] The transform unit 120 may generate transform coefficients by applying a transform technique to the residual signal. For example, the transform technique may include at least one of a discrete cosine transform (DCT), a discrete sine transform (DST), a Karhunen-Loeve transform (KLT), a graph-based transform (GBT), or a conditionally non-linear transform (CNT). Here, the GBT refers to a transform obtained from a graph when the relationship information between pixels is expressed as a graph. The CNT refers to a transform obtained based on a predicted signal generated using all previously reconstructed pixels. In addition, the transform process may be applied to pixel blocks having the same square size, or may be applied to blocks of variable size that are not square.

[0059] The quantization unit 130 quantizes the transform coefficients and transmits the quantized transform coefficients to the entropy encoding unit 190, which may encode the quantized signal (information on the quantized transform coefficients) and output the quantized signal to a bitstream. The information on the quantized transform coefficients is called residual information. The quantization unit 130 may rearrange the quantized transform coefficients in the form of a block into the form of a one-dimensional vector based on a scan order of the coefficients, and may generate information on the quantized transform coefficients based on the quantized transform coefficients in the form of a one-dimensional vector. The entropy encoding unit 190 may perform various encoding methods, such as exponential Golomb, context-adaptive variable length coding (CAVLC), context-adaptive binary arithmetic coding (CABAC), etc. The entropy encoding unit 190 may also encode information required for video / image restoration other than the quantized transform coefficients (e.g., values ​​of syntax elements, etc.) together with or separately from the quantized transform coefficients. The encoded information (e.g., video / image information) may be transmitted or stored in the form of a bitstream in network abstraction layer (NAL) units. The bitstream may be transmitted via a network or stored in a digital storage medium. Here, the network may include a broadcast network and / or a communication network, and the digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. A transmitting unit (not shown) for transmitting the signal output from the entropy encoding unit 190 and / or a storing unit (not shown) for storing the signal may be configured as an internal / external element of the encoding apparatus 100, or the transmitting unit may be a component of the entropy encoding unit 190.

[0060] The quantized transform coefficients output from the quantizer 130 may be used to generate a prediction signal. For example, a residual signal may be restored by applying inverse quantization and inverse transform to the quantized transform coefficients via the inverse quantizer 140 and the inverse transformer 150 in a loop. The adder 155 may generate a reconstructed signal (reconstructed picture, reconstructed block, reconstructed sample array) by adding the reconstructed residual signal to the prediction signal output from the inter prediction unit 180 or the intra prediction unit 185. When there is no residual for the block to be processed, such as when the skip mode is applied, the predicted block may be used as the reconstructed block. The adder 155 may be referred to as a reconstruction unit or a reconstructed block generator. The reconstructed signal may be used for intra prediction of the next block to be processed in the current picture, and may also be used for inter prediction of the next picture after filtering as described below.

[0061] The filtering unit 160 may apply filtering to the reconstructed signal to improve subjective / objective image quality. For example, the filtering unit 160 may apply various filtering methods to the reconstructed picture to generate a modified reconstructed picture, and may transmit the modified reconstructed picture to the decoded picture buffer 170. The various filtering methods may include, for example, deblocking filtering, sample adaptive offset, an adaptive loop filter, and a bilateral filter. The filtering unit 160 may generate various information related to filtering and transmit the information to the entropy encoding unit 190, as will be described later in relation to each filtering method. The information related to filtering may be encoded by the entropy encoding unit 190 and output in the form of a bitstream.

[0062] The modified reconstructed picture transmitted to the decoded picture buffer 170 can be used as a reference picture in the inter predictor 180. Through this, the encoding apparatus 100 can avoid prediction mismatch between the encoding apparatus 100 and the decoding apparatus 200 when inter prediction is applied, and can also improve encoding efficiency.

[0063] The decoded picture buffer 170 may store the modified reconstructed picture for use as a reference picture by the inter predictor 180.

[0064] FIG. 3 shows a schematic block diagram of a decoding device for decoding an image signal as an embodiment of the present invention.

[0065] Referring to FIG. 3, the decoding device 200 may include an entropy decoding unit 210, an inverse quantization unit 220, an inverse transform unit 230, an adder 235, a filtering unit 240, a memory 250, an inter prediction unit 260, and an intra prediction unit 265. The inter prediction unit 260 and the intra prediction unit 265 may be commonly referred to as a prediction unit. That is, the prediction unit may include an inter prediction unit 180 and an intra prediction unit 185. The inverse quantization unit 220 and the inverse transform unit 230 may be commonly referred to as a residual processing unit. That is, the residual processing unit may include an inverse quantization unit 220 and an inverse transform unit 230. The entropy decoding unit 210, the inverse quantization unit 220, the inverse transform unit 230, the adder 235, the filtering unit 240, the inter prediction unit 260, and the intra prediction unit 265 may be configured by one hardware component (e.g., a decoder or a processor) depending on the embodiment. Also, the decoded picture buffer 250 may be implemented by a single hardware component (e.g., a memory or a digital storage medium) depending on the embodiment. Also, the memory 250 may include the DPB 175 and may be implemented by a digital storage medium.

[0066] When a bitstream including video / image information is input, the decoding apparatus 200 can restore an image corresponding to the process in which the video / image information is processed by the encoding apparatus 100 of Fig. 2. For example, the decoding apparatus 200 can perform decoding using a processing unit applied in the encoding apparatus 100. Thus, the processing unit in decoding may be, for example, a coding unit, and the coding unit may be divided according to a coding tree unit or a quad tree structure and / or a binary tree structure from the maximum coding unit. Also, the restored image signal decoded and output by the decoding apparatus 200 can be reproduced through a reproduction device.

[0067] The decoding apparatus 200 may receive a signal output from the encoding apparatus 100 of Fig. 2 in the form of a bitstream, and the received signal may be decoded via the entropy decoding unit 210. For example, the entropy decoding unit 210 may parse the bitstream to derive information (e.g., video / image information) required for image restoration (or picture restoration). For example, the entropy decoding unit 210 may decode information in the bitstream based on a coding method such as Exponential Golomb coding, CAVLC, or CABAC, and output values ​​of syntax elements required for image restoration and quantized values ​​of transform coefficients related to residuals. More specifically, the CABAC entropy decoding method receives bins corresponding to each syntax element from a bitstream, determines a context model using information on the syntax element to be decoded and decoding information on the neighboring and blocks to be decoded, or information on symbols / bins decoded in a previous step, predicts the occurrence probability of bins according to the determined context model, and performs arithmetic decoding of the bins to generate symbols corresponding to the values ​​of each syntax element. In this case, the CABAC entropy decoding method may update the context model using information on the decoded symbols / bins for the context model of the next symbol / bin after determining the context model. Prediction information among the information decoded by the entropy decoding unit 210 is provided to a prediction unit (inter prediction unit 260 and intra prediction unit 265), and residual values ​​entropy-decoded by the entropy decoding unit 210, i.e., quantized transform coefficients and related parameter information, may be input to the inverse quantization unit 220. In addition, information regarding filtering among the information decoded by the entropy decoding unit 210 can be provided to the filtering unit 240 .Meanwhile, a receiver (not shown) for receiving a signal output from the encoding apparatus 100 may be further configured as an internal / external element of the decoding apparatus 200, or the receiver may be a component of the entropy decoding unit 210.

[0068] The inverse quantization unit 220 may output transform coefficients by inverse quantizing the quantized transform coefficients. The inverse quantization unit 220 may rearrange the quantized transform coefficients in the form of a two-dimensional block. In this case, the rearrangement may be performed based on the coefficient scanning order performed in the encoding apparatus 100. The inverse quantization unit 220 may perform inverse quantization on the quantized transform coefficients using a quantization parameter (e.g., quantization step size information) to obtain transform coefficients.

[0069] The inverse transform unit 230 may output a residual signal (residual block, residual sample array) by applying an inverse transform to the transform coefficients.

[0070] The prediction unit may perform prediction on the current block and generate a predicted block including prediction samples for the current block. The prediction unit may determine whether intra prediction or inter prediction is applied to the current block based on the prediction information output from the entropy decoding unit 210, and may determine a specific intra / inter prediction mode.

[0071] The intra prediction unit 265 may predict the current block by referring to samples in the current picture. The referenced samples may be located in the neighborhood of the current block or may be located far away from the current block depending on the prediction mode. Prediction modes in intra prediction may include a plurality of non-directional modes and a plurality of directional modes. The intra prediction unit 265 may also determine a prediction mode to be applied to the current block using prediction modes applied to neighboring blocks.

[0072] The inter prediction unit 260 may derive a predicted block for the current block based on a reference block (reference sample array) identified by a motion vector on a reference picture. In this case, in order to reduce the amount of motion information transmitted in the inter prediction mode, the motion information may be predicted in units of blocks, sub-blocks, or samples based on the correlation of motion information between neighboring blocks and the current block. The motion information may include a motion vector and a reference picture index. The motion information may further include information on an inter prediction direction (L0 prediction, L1 prediction, Bi prediction, etc.). In the case of inter prediction, the neighboring blocks may include spatial neighboring blocks present in the current picture and temporal neighboring blocks present in the reference picture. For example, the inter prediction unit 260 may configure a candidate list of motion information based on the neighboring blocks, and derive a motion vector and / or a reference picture index for the current block based on the received candidate selection information. Inter prediction is performed based on various prediction modes, and the information on the prediction may include information indicating a mode of inter prediction for the current block.

[0073] The adder 235 may generate a reconstructed signal (reconstructed picture, reconstructed block, reconstructed sample array) by adding the acquired residual signal to a prediction signal (predicted block, predicted sample array) output from the inter prediction unit 260 or the intra prediction unit 265. When there is no residual for a block to be processed, such as when a skip mode is applied, the predicted block can be used as a reconstructed block.

[0074] The adder 235 is called a reconstruction unit or a reconstruction block generator. The generated reconstruction signal can be used for intra prediction of a next block to be processed in the current picture, and can also be used for inter prediction of a next picture after filtering, as described below.

[0075] The filtering unit 240 may improve subjective / objective picture quality by applying filtering to the reconstructed signal. For example, the filtering unit 240 may apply various filtering methods to the reconstructed picture to generate a modified reconstructed picture, and may transmit the modified reconstructed picture to the decoded picture buffer 250. The various filtering methods may include, for example, deblocking filtering, sample adaptive offset (SAO), an adaptive loop filter (ALF), a bilateral filter, etc.

[0076] The modified reconstructed picture sent to the decoded picture buffer 250 can be used as a reference picture by the inter predictor 260 .

[0077] In this specification, the embodiments described for the filtering unit 160, inter prediction unit 180, and intra prediction unit 185 of the encoding device 100 can also be applied identically or correspondingly to the filtering unit 240, inter prediction unit 260, and intra prediction unit 265 of the decoding device, respectively.

[0078] FIG. 4 illustrates an example of a structural diagram of a content streaming system according to an embodiment of the present disclosure.

[0079] The content streaming system to which this specification is applied can broadly include an encoding server 410 , a streaming server 420 , a web server 430 , a media storage 440 , a user device 450 , and a multimedia input device 460 .

[0080] The encoding server 410 can compress content input from a multimedia input device such as a smartphone, camera, camcorder, etc. into digital data to generate a bitstream and transmit the bitstream to the streaming server 420. As another example, when a multimedia input device 460 such as a smartphone, camera, camcorder, etc. directly generates a bitstream, the encoding server 410 can be omitted.

[0081] The bitstream may be generated by an encoding method or a bitstream generation method to which this specification applies, and the streaming server 420 may temporarily store the bitstream in the process of transmitting or receiving the bitstream.

[0082] The streaming server 420 transmits multimedia data to the user device 450 based on a user request via the web server 430, and the web server 430 acts as a medium to inform the user of available services. When a user requests a desired service from the web server 430, the web server 430 transfers it to the streaming server 420, and the streaming server 420 transmits the multimedia data to the user. In this case, the content streaming system may include a separate control server, and in this case, the control server plays a role of controlling commands / responses between each device in the content streaming system.

[0083] The streaming server 420 may receive content from the media storage 440 and / or the encoding server 410. For example, the streaming server 420 may receive content in real time from the encoding server 410. In this case, in order to provide a smooth streaming service, the streaming server 420 may store the bitstream for a certain period of time.

[0084] For example, the user device 450 may include a mobile phone, a smart phone, a laptop computer, a digital broadcasting terminal, a personal digital assistant (PDA), a portable multimedia player (PMP), a navigation system, a slate PC, a tablet PC, an ultrabook, a wearable device such as a smartwatch, a smart glass, a head mounted display (HMD), a digital TV, a desktop computer, and a digital signage.

[0085] Each server in the content streaming system can be operated as a distributed server, in which case data received by each server can be processed in a distributed manner.

[0086] 5 shows an example of a block diagram of an apparatus for processing a video signal according to an embodiment of the present disclosure. The video signal processing apparatus of FIG. 5 may correspond to the encoding apparatus 100 of FIG. 2 or the decoding apparatus 200 of FIG.

[0087] A video signal processing device 500 according to an embodiment of the present disclosure may include a memory 520 for storing a video signal, and a processor 510 coupled to the memory for processing the video signal.

[0088] The processor 510 according to the embodiment of the present specification may be configured with at least one processing circuit for processing a video signal, and may process an image signal by executing a command for encoding or decoding a video signal, i.e., the processor 510 may encode an original video signal or decode an encoded video signal by executing an encoding or decoding method described below.

[0089] The processor 510 according to the embodiment of the present invention may be configured with at least one processing circuit for processing an image signal, and may process the image signal by executing a command for encoding or decoding the image signal. That is, the processor 510 may encode original image data or decode an encoded image signal by executing the encoding or decoding method described above.

[0090] Figure 6 shows examples of block division structures of an image according to an embodiment of this specification, with Figure 6a showing an example of a block division structure using QT (QuadTree, QT), Figure 6b showing an example of a block division structure using BT (Binary Tree, BT), Figure 6c showing an example of a block division structure using TT (Ternary Tree, TT), and Figure 6d showing an example of a block division structure using AT (Asymmetric Tree, AT).

[0091] In video coding, a block may be divided based on QT. Also, a subblock divided by QT may be further divided recursively using QT. A leaf block that is not further divided by QT may be divided by at least one of BT, TT, and AT. BT may have two types of division, horizontal BT (2NxN, 2NxN) and vertical BT (Nx2N, Nx2N). TT may have two types of division, horizontal TT (2Nx1 / 2N, 2NxN, 2Nx1 / 2N) and vertical TT (1 / 2Nx2N, Nx2N, 1 / 2Nx2N). AT can have four forms of partitioning: horizontal-up AT (2Nx1 / 2N, 2Nx3 / 2N), horizontal-down AT (2Nx3 / 2N, 2Nx1 / 2N), vertical-left AT (1 / 2Nx2N, 3 / 2Nx2N), vertical-right AT (3 / 2Nx2N, 1 / 2Nx2N). Each BT, TT, and AT can be further partitioned recursively using BT, TT, and AT.

[0092] Figure 6a shows an example of QT division. Block A can be divided into four sub-blocks (A0, A1, A2, A3) by QT. Sub-block A1 can be divided again into four sub-blocks (B0, B1, B2, B3) by QT.

[0093] Figure 6b shows an example of division of BT. Block B3, which is not further divided by QT, can be divided into vertical BT (C0, C1) or horizontal BT (D0, D1). Each subblock, such as block C0, can be further divided recursively, such as into horizontal BT (E0, E1) or vertical BT (F0, F1) forms.

[0094] Figure 6c shows an example of a TT division. Block B3, which is not further divided by QT, can be divided into vertical TTs (C0, C1, C2) or horizontal TTs (D0, D1, D2). Each subblock, such as block C1, can be further divided recursively, such as into horizontal TTs (E0, E1, E2) or vertical TTs (F0, F1, F2).

[0095] Figure 6d shows an example of the division of an AT. Block B3, which is not further divided by QT, can be divided into vertical AT(C0,C1) or horizontal AT(D0,D1). Each subblock, such as block C1, can be further divided recursively, such as in the form of horizontal AT(E0,E1) or vertical TT(F0,F1).

[0096] Meanwhile, division can be performed using all of BT, TT, and AT divisions. For example, a subblock divided by BT can be divided by TT or AT. Also, a subblock divided by TT can be divided by BT or AT. A subblock divided by AT can be divided by BT or TT. For example, after horizontal BT division, each subblock can be divided into vertical BT, or after vertical BT division, each subblock can be divided into horizontal BT. In this case, the order of division is different, but the final divided shape is the same.

[0097] In addition, when a block is divided, the order of searching the block can be defined in various ways. In general, searching a block from left to right and from top to bottom can refer to an order for determining whether each divided sub-block can be further divided into blocks, or can refer to an encoding order of each sub-block when the block is not further divided, or can refer to a search order when referring to information of other adjacent blocks from a sub-block.

[0098] 7 and 8 show a video / image encoding procedure based on inter prediction and an inter prediction unit in an encoding device according to an embodiment of the present specification.

[0099] The encoding apparatus 100 performs inter prediction on the current block (S710). The encoding apparatus 100 may derive an inter prediction mode and motion information of the current block and generate a prediction sample of the current block. Here, the procedures of determining the inter prediction mode, deriving the motion information, and generating the prediction sample may be performed simultaneously, or one procedure may be performed before the other procedure. For example, the inter prediction unit 180 of the encoding apparatus 100 may include a prediction mode determination unit 181, a motion information derivation unit 182, and a prediction sample derivation unit 183, and the prediction mode determination unit 181 may determine a prediction mode for the current block, the motion information derivation unit 182 may derive motion information of the current block, and the prediction sample derivation unit 183 may derive a prediction sample of the current block. For example, the inter prediction unit 180 of the encoding apparatus 100 may search for a block similar to the current block within a certain region (search region) of a reference picture through motion estimation, and derive a reference block whose difference from the current block is minimum or equal to or less than a certain criterion. Based on this, a reference picture index indicating a reference picture in which the reference block is located may be derived, and a motion vector may be derived based on the difference between the positions of the reference block and the current block. The encoding apparatus 100 may determine a mode to be applied to the current block from among various prediction modes. The encoding apparatus 100 may compare RD costs for various prediction modes to determine an optimal prediction mode for the current block.

[0100] For example, when a skip mode or a merge mode is applied to the current block, the encoding apparatus 100 may construct a merge candidate list (to be described later) and derive a reference block whose difference from the current block is a minimum or equal to a certain criterion among reference blocks indicated by merge candidates included in the merge candidate list. In this case, a merge candidate associated with the derived reference block may be selected, and merge index information indicating the selected merge candidate may be generated and signaled to the decoding apparatus 200. Motion information of the current block may be derived using motion information of the selected merge candidate.

[0101] As another example, when the (A)MVP mode is applied to the current block, the encoding apparatus 100 may configure an (A)MVP candidate list, which will be described later, and may use a motion vector of an MVP candidate selected from among the MVP (motion vector predictor) candidates included in the (A)MVP candidate list as the MVP of the current block. In this case, for example, a motion vector indicating a reference block derived by the above-mentioned motion estimation may be used as the motion vector of the current block, and an MVP candidate having a motion vector with the smallest difference from the motion vector of the current block among the MVP candidates may become the selected MVP candidate. A motion vector difference (MVD), which is a difference obtained by subtracting the MVP from the motion vector of the current block, may be derived. In this case, information regarding the MVD may be signaled to the decoding apparatus 200. In addition, when the (A)MVP mode is applied, a value of a reference picture index may be configured as reference picture index information and signaled to the decoding apparatus 200 separately.

[0102] The encoding apparatus 100 may derive a residual sample based on the predicted sample (S720). The encoding apparatus 100 may derive the residual sample through a comparison between an original sample of a current block and a predicted sample.

[0103] The encoding apparatus 100 encodes image information including prediction information and residual information (S730). The encoding apparatus 100 may output the encoded image information in the form of a bitstream. The prediction information may include prediction mode information (e.g., a skip flag, a merge flag, or a mode index) and motion information as information related to a prediction procedure. The motion information may include candidate selection information (e.g., a merge index, an mvp flag, or an mvp index) which is information for deriving a motion vector. In addition, the motion information may include information related to the above-mentioned MVD and / or reference picture index information. In addition, the motion information may include information indicating whether L0 prediction, L1 prediction, or bi-prediction is applied. The residual information is information related to a residual sample. The residual information may include information related to a quantized transform coefficient for the residual sample.

[0104] The output bitstream can be stored in a (digital) storage medium and then transmitted to the decoding device, or can be transmitted to the decoding device via a network.

[0105] Meanwhile, as described above, the encoding apparatus may generate a reconstructed picture (including reconstructed samples and reconstructed blocks) based on the reference samples and the residual samples. This is because the encoding apparatus 100 derives the same prediction result as that performed by the decoding apparatus 200, thereby improving coding efficiency. Therefore, the encoding apparatus 100 may store the reconstructed picture (or reconstructed samples, reconstructed blocks) in a memory and use it as a reference picture for inter prediction. As described above, an in-loop filtering procedure, etc. may be further applied to the reconstructed picture.

[0106] 9 and 10 show an inter prediction unit in a video / image decoding procedure and a decoding device based on inter prediction according to an embodiment of the present specification.

[0107] The decoding apparatus 200 may perform operations corresponding to those performed by the encoding apparatus 100. The decoding apparatus 200 may perform prediction on a current block based on received prediction information and derive a prediction sample.

[0108] Specifically, the decoding apparatus 200 may determine a prediction mode for a current block based on the received prediction information (S910). The decoding apparatus 200 may determine which inter prediction mode is applied to the current block based on prediction mode information in the prediction information.

[0109] For example, the decoding apparatus 200 may determine whether a merge mode is applied to the current block or whether an (A)MVP mode is determined based on a merge flag. Alternatively, the decoding apparatus 200 may select one of various inter prediction mode candidates based on a mode index. The inter prediction mode candidates may include a skip mode, a merge mode, and / or an (A)MVP mode, or may include various inter prediction modes described below.

[0110] The decoding apparatus 200 derives motion information of the current block based on the determined inter prediction mode (S920). For example, when a skip mode or a merge mode is applied to the current block, the decoding apparatus 200 may form a merge candidate list (to be described later) and select one merge candidate from among the merge candidates included in the merge candidate list. The merge candidate is selected based on a merge index. Motion information of the current block may be derived from motion information of the selected merge candidate. The motion information of the selected merge candidate may be used as motion information of the current block.

[0111] As another example, when the (A)MVP mode is applied to the current block, the decoding apparatus 200 may construct an (A)MVP candidate list described later, and use a motion vector of an MVP candidate selected from the MVP candidates included in the (A)MVP candidate list as the MVP of the current block. The MVP is selected based on the above-mentioned selection information (MVP flag or MVP index). In this case, the decoding apparatus 200 may derive the MVD of the current block based on information on the MVD, and may derive a motion vector of the current block based on the MVP and MVD of the current block. Also, the decoding apparatus 200 may derive a reference picture index of the current block based on reference picture index information. In the reference picture list for the current block, a picture pointed to by a reference picture index may be derived as a reference picture referenced for inter prediction of the current block.

[0112] On the other hand, as described below, the motion information of the current block can be derived without constructing a candidate list, in which case the motion information of the current block can be derived by a procedure started in a prediction mode, as described below. In this case, the construction of the candidate list as described above can be omitted.

[0113] The decoding apparatus 200 may generate a prediction sample for the current block based on the motion information of the current block (S930). In this case, the decoding apparatus 200 may derive a reference picture based on a reference picture index of the current block, and derive a prediction sample for the current block using a sample of a reference block to which a motion vector of the current block points on the reference picture. In this case, as described below, a prediction sample filtering procedure may be further performed on all or some of the prediction samples of the current block, depending on the circumstances.

[0114] For example, the inter prediction unit 260 of the decoding device 200 may include a prediction mode determination unit 261, a motion information derivation unit 262, and a prediction sample derivation unit 263, and may determine a prediction mode for the current block based on prediction mode information received by the prediction mode determination unit 181, derive motion information (such as a motion vector and / or a reference picture index) of the current block based on information regarding the motion information received by the motion information derivation unit 182, and derive a prediction sample of the current block from the prediction sample derivation unit 183.

[0115] The decoding apparatus 200 generates a residual sample for the current block based on the received residual information (S940). The decoding apparatus 200 generates a reconstructed sample for the current block based on the prediction sample and the residual sample, and may generate a reconstructed picture based on the reconstructed sample (S950). As described above, an in-loop filtering procedure may then be further applied to the reconstructed picture.

[0116] As described above, the inter prediction procedure may include an inter prediction mode determination step, a motion information derivation step based on the determined prediction mode, and a prediction execution step (generation of a prediction sample) based on the derived motion information.

[0117] Various inter prediction modes can be used for predicting the current block in a picture. For example, various modes such as merge mode, skip mode, MVP mode, and affine mode can be used. Decoder side motion vector refinement mode (DMVR), adaptive motion vector resolution mode (AMVR), etc. can be further used as an additional mode. The affine mode is also called affine motion prediction mode. The MVP mode is also called advanced motion vector prediction mode (AMVP).

[0118] Prediction mode information indicating an inter prediction mode of a current block may be signaled from the encoding apparatus to the decoding apparatus 200. The prediction mode information may be included in a bitstream and received by the decoding apparatus 200. The prediction mode information may include index information indicating one of a number of candidate modes. Alternatively, the inter prediction mode may be indicated through hierarchical signaling of flag information. In this case, the prediction mode information may include one or more flags. For example, the encoding apparatus 100 may signal a skip flag to indicate whether or not the skip mode is applied, and when the skip mode is not applied, may signal a merge flag to indicate whether or not the merge mode is applied, and when the merge mode is not applied, may indicate that the MVP mode is applied, or may further signal a flag for further classification. The affine mode may be signaled as an independent mode, or may be signaled as a mode dependent on the merge mode, MVP mode, etc. For example, the affine mode may be configured as one candidate of a merge candidate list or an MVP candidate list, as described below.

[0119] The encoding apparatus 100 or the decoding apparatus 200 may perform inter prediction using motion information of a current block. The encoding apparatus 100 may derive optimal motion information for the current block through a motion estimation procedure. For example, the encoding apparatus 100 may use an original block in an original picture for the current block to search for a similar reference block with high correlation in a fractional pixel unit within a determined search range in the reference picture, and may derive motion information through this. The similarity of the block may be derived based on a difference in phase-based sample values. For example, the similarity of the block may be calculated based on a sum of absolute difference (SAD) between the current block (or a template of the current block) and the reference block (or a template of the reference block). In this case, the motion information may be derived based on a reference block with the smallest SAD in the search area. The derived motion information may be signaled to the decoding apparatus by various methods based on the inter prediction mode.

[0120] When a merge mode is applied, the motion information of the current prediction block is not directly transmitted, but is derived using the motion information of the neighboring prediction blocks. Therefore, the encoding apparatus 100 can indicate the motion information of the current prediction block by transmitting flag information indicating that the merge mode is used and a merge index indicating which neighboring prediction block is used.

[0121] To perform the merge mode, the encoding apparatus 100 should search for a merge candidate block to be used to derive motion information of the current prediction block. For example, up to five merge candidate blocks may be used, but the present specification is not limited thereto. The maximum number of merge candidate blocks may be transmitted in a slice header, but the present specification is not limited thereto. After finding the merge candidate blocks, the encoding apparatus 100 may generate a merge candidate list, and may select a merge candidate block having the smallest cost among them as a final merge candidate block.

[0122] This specification provides various examples for the merge candidate blocks that make up the merge candidate list.

[0123] The merge candidate list may, for example, utilize five merge candidate blocks, e.g., four spatial merge candidates and one temporal merge candidate.

[0124] A method for constructing a reference picture list for the above-mentioned inter prediction will now be described.

[0125] High level syntax (HLS) exists in video coding systems to control the general sharing of common information. The purpose of HLS is to manage frames in order to reduce the number of bits that can be efficiently used in memory. HLS uses a reference picture management system, which invokes a process called reference picture marking. This process labels each frame according to its usability for decoding the current frame. In designing and implementing HLS, the goal is to have a simple and effective means for identification of pictures required for decoding the current picture.

[0126] Example 1

[0127] In this embodiment, a reference picture list (RPL) management system is introduced, which is directly signaled and derived. Two lists are used, namely Reference Picture List0 and Reference Picture List1. Information for deriving the two reference picture lists is signaled as syntax elements in the Sequence Parameter Set (SPS), Picture Parameter Set (PPS) and slice headers. Two lists are generated for all types of slices. However, I-slices do not use any information from the lists, whereas P-slices only use information from List0 and B-slices use information from all the lists. Pictures are identified using a picture order count (POC) number. Each picture can be classified as a "long-term-reference-picture (LTRP)" or a "short-term-reference-picture (STRP)". A picture classified as LTRP or STRP is an active picture, indicating that it can be used during decoding, while an inactive picture is one that is not used during the decoding process.

[0128] 11 shows an example of a flowchart for constructing a reference picture list according to an embodiment of the present disclosure. The operations of FIG.

[0129] FIG. 11 shows an overall process of constructing a reference picture list. First, in step S1105, the decoding apparatus 200 parses the total number of entries in the list. Then, for each entry, a picture is marked as an LTRP or STRP through a reference picture marking process (S1120). In this document, marking a picture means an operation of adding or updating information (parameters) about the picture. Specifically, the decoding apparatus 200 initializes an entry index (i) (S1110), and if the entry index (i) is less than or equal to the total number of entries (S1115), performs reference picture marking (S1120). Also, when the marking of the reference picture is completed, the decoding apparatus updates the entry index (i) (S1125). The embodiment of the present invention proposes a new structure for improving the reference picture management system.

[0130] 12 and 13 show an example of a flow chart for marking reference pictures according to an embodiment of the present disclosure.

[0131] [Table 1]

[0132] Table 1 is an example of a syntax for constructing a reference picture list. Here, this embodiment provides a possible structure for managing reference pictures, where the ref_pic_list_struct function receives listIdx indicated by the syntax element rpl1_same_as_rpl0_flag as a first input. That is, if the flag (rpl1_same_as_rpl0_flag) is true, this indicates that the lists are identical and one list (L0 or L1) can be inferred from another (L1 or L0). The second input (rplsIdx) of the ref_pic_list_struct function indicates the number of reference picture lists used in the coded video sequence specified in the SPS. num_ref_entries indicates the number of entries in the syntax structure ref_pic_list_struct(listIdx,rpIsIdx). For each entry, if an LTRP frame is used, a flag is parsed to read whether the frame in the list is treated as an LTRP or a STRP. If the entry is a STRP and is the first STRP entry in the syntax structure ref_pic_list_struct(listIdx,rpIsIdx), it represents the absolute difference between the POC of the current picture and the POC of the picture referenced by the i-th entry, or if the i-th entry is a STRP entry but is not the first entry, it represents the absolute difference between the POC of the picture referenced by the i-th entry and the POC of the picture pointed to by the previous STRP.(If the entry is a STRP, it is the first STRP entry in the ref_pic_list_struc(listIdx,rpIsIdx) syntax structure and it specifies the absolute difference in POC of the current picture and the picture referred to by the i-th entry or when the i-th entry is an STRP entry but not the first entry, then it specifies the absolute difference between the POC of the pictured referred to by the i-th entry and by the previous STRP). If abs_delta_poc for the i-th entry is greater than 0, then a flag marking the sign information of the STRP is signaled. But if st_ref_pic_flag is 0, it indicates that the i-th entry is a long term reference picture. The LTRP is signaled by the POC least significant bit (LSB).

[0133] The syntax elements for Table 1 can be defined as follows:

[0134] num_ref_entries[listIdx][rplsIdx] represents the number of entries in a structure with the syntax ref_pic_list_struct(listIdx,rplsIdx). The value of num_ref_entries[listIdx][rplsIdx] can be between 0 and sps_max_dec_pic_buffering_minus1+14.

[0135] When rpl1_same_as_rpl0_flag is 1, the syntax structures num_ref_pic_lists_in_sps[1] and ref_pic_list_struct(1,rplsIdx) do not exist and the following applies:

[0136] - The value of num_ref_pic_lists_in_sps[1] is inferred to be identical to the value of num_ref_pic_lists_in_sps[0].

[0137] - The value of each syntax element in ref_pic_list_struct(1,rplsIdx) is inferred to be the same as the value of the corresponding syntax element in ref_pic_list_struct(0,rplsIdx) for rplsIdx, which can be a value between 0 and num_ref_pic_lists_in_sps[0]-1.

[0138] long_term_ref_pics_flag is 0, which indicates that the LTRP is not used for inter prediction of a coded picture in a CVS (coded video sequence).long_term_ref_pics_flag is 1, which indicates that the LTRP is used for inter prediction of one or more coded pictures in a CVS.

[0139] If st_ref_pic_flag[listIdx][rplsIdx][i] is 1, it indicates that the i-th entry in the syntax structure ref_pic_list_struct(listIdx,rplsIdx) is a STRP entry. If st_ref_pic_flag[listIdx][rplsIdx][i] is 0, it indicates that the i-th entry in the syntax structure ref_pic_list_struct(listIdx,rplsIdx) is a LTRP entry. If not present, the value of st_ref_pic_flag[listIdx][rplsIdx][i] is inferred to be 1.

[0140] abs_delta_poc_st[listIdx][rplsIdx][i] represents the absolute difference between the picture order count values ​​of the current picture and the picture referenced by the i-th entry if the i-th entry is the first STRP entry in the syntax structure ref_pic_list_struct(listIdx,rplsIdx), or represents the absolute difference between the picture order count values ​​of the picture indicated by the i-th entry and the previous STRP in the syntax structure ref_pic_list_struct(listIdx,rplsIdx) if the i-th entry is a STRP entry but is not the first STRP entry in the syntax structure ref_pic_list_struct(listIdx,rplsIdx).

[0141] The value of abs_delta_poc_st[listIdx][rplsIdx][i] ranges from 0 to 215-1.

[0142] strp_entry_sign_flag[listIdx][rplsIdx][i] equal to 1 indicates that the i-th entry in the syntax structure ref_pic_list_struct(listIdx,rplsIdx) has a value greater than or equal to 0. strp_entry_sign_flag[listIdx][rplsIdx][i] equal to 0 indicates that the i-th entry in the syntax structure ref_pic_list_struct(listIdx,rplsIdx) has a value less than 0. If not present, the value of strp_entry_sign_flag[i][j] is inferred to be 1.

[0143] poc_lsb_lt[listIdx][rplsIdx][i] represents the picture order count modulo MaxPicOrderCntLsb of the picture referenced by the i-th entry in the syntax structure ref_pic_list_struct(listIdx,rplsIdx). The length of the syntax element poc_lsb_lt[listIdx][rplsIdx][i] is log2_max_pic_order_cnt_lsb_minus4+4 bits.

[0144] The above syntax element descriptions can be summarized as shown in Table 2 below.

[0145] [Table 2]

[0146] 12, the decoding apparatus 200 checks whether the LTRP is used in the corresponding reference picture list entry through long_term_ref_pics_flag (S1205), and if the LTRP is used (long_term_ref_pics_flag is 1), parses st_ref_pic_flag (S1210) and checks whether st_ref_pic_flag is true (S1215). If st_ref_pic_flag is true (1) (i.e., the current entry corresponds to the STRP), the decoding apparatus 200 parses abs_delta_poc_st. That is, if the current entry corresponds to the STRP entry, the decoding apparatus 200 obtains a difference value between the POC of the picture related to the current entry and the previous picture.

[0147] If long_term_ref_pics_flag is false at step 1205, the decoding device immediately parses abs_delta_poc_st. After that, the decoding device 200 checks whether abs_delta_poc_st is greater than 0 (S1225), and if abs_delta_poc_st is greater than 0, parses strp_entry_sign_flag (S1230).

[0148] If st_ref_pic_flag is false (0) in step S1215, the decoding apparatus 200 parses poc_lsb_lt (S1235). That is, if the current entry corresponds to an LTRP entry, the decoding apparatus 200 obtains a POC modulo value of a picture related to the current entry. The POC modulo value of a picture is a value used to derive a POC value of the corresponding picture, and the POC value of the picture can be derived in combination with the POC most significant bit (MSB).

[0149] The above-mentioned reference picture list construction process can be expressed by two blocks as shown in Figure 13. The operation of each block can be realized by the same hardware or different hardware. The decoding apparatus 200 can obtain the number of entries (num_ref_entries) and a flag (long_term_ref_pic_flag) indicating whether a long-term reference picture is used (1310), and determine whether each entry is an LTRP or STRP (1320).

[0150] Example 2

[0151] In this embodiment, we propose to limit the parameter num_ref_entries described in Table 1. Typically, num_ref_entries[listIdx][rpIsIdx] represents the number of entries in ref_pic_list_struct(listIdx, rpIsIdx). It is noted that the value of num_ref_entries can have a value ranging from 0 to sps_max_dec_pic_buffering_minus1+14. In order to use fewer frames, a limit on the range of num_ref_entries can be introduced. The worst case scenario is to request 14 frames more than the value indicated by sps_max_dec_pic_buffering_minus1. This means that the decoded picture buffer must be able to have enough memory to accommodate so many frames. Alternatively, for hierarchical groups of structres (GOPs), a limit can be introduced taking into account the temporalId of the layer.

[0152] Further syntax can be defined as shown in Table 3. sps_max_dec_pic_buffering_minus1 plus 1 represents the maximum required size of one decoded picture buffer in CVS in units of picture storage buffer. The value of sps_max_dec_pic_buffering_minus1 can have a value between 0 and MaxDpbSize-1, where MaxDpbSize can be defined separately.

[0153] [Table 3]

[0154] That is, the number of entries in the reference picture list may have a maximum value equal to the maximum required size of the decoding picture buffer plus 14. In addition, in this embodiment (embodiment 2), the number of entries in the reference picture list may have a maximum value equal to the maximum required size of the decoding picture buffer.

[0155] Example 3

[0156] In this embodiment, a method is proposed to remove the syntax element long_term_ref_pics_flag from ref_pic_list_struct(listIdx,rpIsIdx). When this syntax element (long_term_ref_pics_flag) is 0, it indicates that the LTRP is no longer used for inter prediction of a coded picture in the CVS. When long_term_ref_pics_flag is 1, it indicates that the LTRP is used for inter prediction of one or more coded filters in the CVS. By removing the syntax element long_term_ref_pics_flag, it can be inferred that the LTRP is basically used in the CVS. This is useful because the LTRP is usually a highly reliable picture in the DPB.

[0157] Example 4

[0158] FIG. 14 illustrates another example of a flow chart for marking reference pictures according to an embodiment of the present disclosure.

[0159] Another embodiment for ref_pic_list_struct(listIdx,rpIsIdx) is provided. In particular, ref_pic_list_struct(listIdx,rpIsIdx) can be set in consideration of the case where LTRP and STRP are signaled separately. This can be expressed as shown in Table 4 below. Here, the embodiment of the present specification provides a method of limiting num_strp_entries[listIdx][rpIsIdx] representing the total number of STRPs used and num_ltrp_entries[listIdx][rpIsIdx] representing the total number of LTRPs.

[0160] Figure 14 shows an overview of the proposed structure where the number of LTRPs and STRPs are signaled separately. Here, the number of STRPs and LTRPs is first parsed using the LTRP via long_term_ref_pics_flag. Then, for each entry, lt_ref_pic_flag is parsed, which indicates whether the picture is a LTRP or not. If lt_ref_pic_flag is true, the counter for the LTRP is incremented. Otherwise, the counter for the STRP is incremented. Furthermore, if lt_ref_pic_flag is false, the STRP is identified by parsing delta_poc_st and parsing its sign information (if delta_poc_st is greater than 0). Otherwise, the LTRP is identified by its LSB POC.

[0161] Table 4 below is an example of syntax for constructing a reference picture list according to this embodiment.

[0162] [Table 4]

[0163] num_strp_entries[listIdx][rplsIdx] represents the number of STRP entries in ref_pic_list_struct(listIdx,rplsIdx).

[0164] num_ltrp_entries[listIdx][rplsIdx] represents the number of LTRP entries in ref_pic_list_struct(listIdx,rplsIdx).

[0165] If lt_ref_pic_flag[listIdx][rplsIdx] is not present, the value of lt_ref_pic_flag[listIdx][rplsIdx] is inferred as follows:

[0166] If num_ltrp_entries[listIdx][rplsIdx] is 0, the value (lt_ref_pic_flag[listIdx][rplsIdx]) is inferred to be 0. Otherwise, if numLtrps (number of LTRPs) is like num_ltrp_entries[listIdx][rplsIdx], it is inferred to be 0, and if numStrps (number of STRPs) is like num_strp_entries[listIdx][rplsIdx], it is inferred to be 0.

[0167] [Table 5]

[0168] Referring to FIG. 14, the decoding apparatus 200 starts constructing a reference list (S1405) and parses the number of STRP entries (num_strp_entries) (S1410). The decoding apparatus 200 checks whether an LTRP is used (S1415), and if an LTRP is used, parses the number of LTRPs (num_ltrp_entries) (S1420). The decoding apparatus 200 initializes variables i, numLtrps, and numStrps to 0 (S1425), determines whether the variable i is equal to or less than the number of entries (num_ref_entires) (S1430), and if i is equal to or less than the number of entries (num_ref_entires), performs a reference picture marking operation (S1435) and updates i (S1440).

[0169] 15 shows an example of a flow chart for processing a video signal according to an embodiment of the present invention. The operations of FIG.

[0170] In step S1510, the decoding apparatus 200 constructs a reference picture list for a current picture in a video signal. Here, if a first entry of the reference picture list corresponds to a STRP entry, the decoding apparatus 200 obtains a POC difference value (e.g., abs_delta_poc_st) between a picture related to the first entry and a previous picture, and if a second entry of the reference picture list corresponds to a LTRP entry, the decoding apparatus 200 obtains a POC modulo value (e.g., poc_lsb_lt) of a picture related to the second entry.

[0171] In one embodiment, in the process of constructing a reference picture list, the decoding apparatus 200 may obtain the number of entries (num_ref_entries) of the reference picture list, and may mark the reference picture in the LTRP or STRP for each entry of the reference picture list as many times as the number of entries.

[0172] In one embodiment, the number of entries may have a maximum value equal to the maximum required size of the decoded picture buffer plus 14 (sps_max_dec_pic_buffering_minus1+14) or the maximum required size of the decoded picture buffer (sps_max_dec_pic_buffering_minus1).

[0173] In one embodiment, in the process of marking a reference picture, the decoding device 200 obtains the number of STRP entries (e.g., num_strp_entries) and the number of LTRP entries (e.g., num_ltrp_entries) as shown in Table 4, and can perform STRP marking as many times as the number of STRP entries and LTRP marking as many times as the number of LTRP entries.

[0174] In one embodiment, in the process of constructing a reference picture list, the decoding apparatus 200 may obtain a first flag and determine whether an entry of the reference picture list corresponds to a STRP entry or an LTRP entry based on the first flag. For example, the decoding apparatus 200 may determine whether the current entry is a STRP entry or an LTRP entry using a flag (e.g., st_ref_pic_flag) indicating whether a picture related to the current entry corresponds to a STRP.

[0175] In one embodiment, the POC difference value represents the absolute value of the POC of the previous picture minus the POC of the picture related to the first entry. Also, if the absolute value (absolute difference of POC) (e.g., abs_delta_poc_st) is greater than 0, the decoding apparatus 200 may obtain a second flag (e.g., strp_entry_sign_flag) indicating the sign of the POC difference value.

[0176] In operation S1520, the decoding apparatus 200 performs prediction for the current picture using the reference picture list. For example, the decoding apparatus 200 may generate a prediction sample using a sample value indicated by a motion vector in a reference picture included in the first reference picture list L0 or the second reference picture list L1.

[0177] The embodiments of the present specification provide a method and an apparatus for encoding information required for constructing a reference picture list. The encoding apparatus 100 can construct a reference picture list and perform prediction using the reference pictures in the reference picture list.

[0178] In addition, the encoding apparatus 100 may code information for constructing a reference picture list for each entry of the reference picture list. For example, the information for constructing the reference picture list may include a difference value (e.g., abs_delta_poc_st) between the POC of a picture corresponding to the STRP and a previous picture, and a modulo value (e.g., poc_lsb_lt) of the POC of a picture corresponding to the LTRP.

[0179] Furthermore, the encoding apparatus 100 may code a first flag (e.g., st_ref_pic_flag) for indicating whether a picture of an entry in the reference picture list is a STRP or an LTRP, in the information for constructing the reference picture list. Furthermore, if the difference value (e.g., abs_delta_poc_st) of the POC is greater than 0, a second flag (e.g., strp_entry_sign_flag) for indicating the sign of the difference value of the POC may be included. The information for constructing the reference picture list may be signaled by being included in the SPS, PPS, or slice header.

[0180] The processing method to which the present specification is applied may be produced in the form of a program executed by a computer and may be stored in a computer-readable recording medium. Multimedia data having a data structure according to the present specification may also be stored in a computer-readable recording medium. The computer-readable recording medium includes all kinds of storage devices and distributed storage devices in which computer-readable data is stored. The computer-readable recording medium may include, for example, a Blu-ray Disc (BD), a Universal Serial Bus (USB), a ROM, a PROM, an EPROM, an EEPROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, and an optical data storage device. The computer-readable recording medium may also include a medium realized in the form of a carrier wave (e.g., transmission via the Internet). Also, a bit stream generated by the encoding method may be stored in a computer-readable recording medium or transmitted via a wired or wireless communication network.

[0181] Also, the embodiments of the present specification may be realized in a computer program product by a program code, the program code being executable by a computer according to the embodiments of the present specification. The program code may be stored on a carrier readable by a computer.

[0182] The decoding device and encoding device to which the present specification applies may be included in a digital device. The term "digital device" includes all digital devices capable of performing at least one of transmitting, receiving, processing, and outputting data, content, services, etc. Here, the processing of data, content, services, etc. by a digital device includes the operation of encoding and / or decoding the data, content, services, etc. Such digital devices are paired or connected (hereinafter referred to as "paired") with other digital devices, external servers, etc. via a wire / wireless network to transmit and receive data and convert data as necessary.

[0183] Digital devices include, for example, standing devices such as network TVs, hybrid broadcast broadband TVs (HBTVs), smart TVs, internet protocol televisions (IPTVs), and personal computers (PCs), as well as mobile devices or handheld devices such as personal digital assistants (PDAs), smart phones, tablet PCs, and notebook computers.

[0184] Meanwhile, the term "wired / wireless network" as used herein generally refers to a communication network that supports various communication standards or protocols for interconnection and / or data transmission and reception between digital devices or between digital devices and external servers. Such wired / wireless networks may include all communication networks and communication protocols currently or in the future supported by standards, for example, communication standards or protocols for wired connections such as Universal Serial Bus (USB), Composite Video Banking Sync (CVBS), component, S-Video (analog), Digital Visual Interface (DVI), High Definition Multimedia Interface (HDMI), RGB, and D-SUB, and communication standards for wireless connections such as Bluetooth, Radio Frequency Identification (RFID), infrared Data Association (IrDA), Ultra Wideband (UWB), ZigBee, Digital Living Network Alliance (DLNA), Wireless LAN (WLAN) (Wi-Fi), Wireless broadband (Wibro), World Interoperability for Microwave Access (Wimax), High Speed ​​Downlink Packet Access (HSDPA), Long Term Evolution (LTE), and Wi-Fi Direct.

[0185] The above-described embodiments are combinations of the components and features of the present specification in a predetermined form. Each component or feature should be considered as optional unless otherwise expressly stated. Each component or feature may be implemented in a form not combined with other components or features. It is also possible to combine some components and / or features to constitute an embodiment of the present specification. The order of operations described in the embodiments of the present specification may be changed. Some configurations or features of any embodiment may be included in other embodiments, or may be replaced with corresponding configurations or features of other embodiments. It is obvious that claims that do not have an explicit reference relationship in the claims may be combined to constitute an embodiment, or may be included in a new claim by amendment after filing.

[0186] In the case of implementation by firmware or software, an embodiment of this specification may be implemented in the form of a module, procedure, function, etc. that performs the functions or operations described above. The software code may be stored in a memory and driven by a processor. The memory may be located inside or outside the processor and may exchange data with the processor by various means that are already known in the art.

[0187] It is obvious to those skilled in the art that the present specification can be embodied in other specific forms without departing from the essential features of the present specification. Therefore, the above detailed description should not be interpreted as limiting in all respects, but should be considered as illustrative. The scope of the present specification should be determined by a reasonable interpretation of the attached claims, and all modifications within the equivalent scope of the present specification are included in the scope of the present specification. [Industrial Applicability]

[0188] The above-described preferred embodiments of the present invention have been disclosed for illustrative purposes, and those skilled in the art may improve, modify, substitute or add various other embodiments within the technical idea and technical scope of the present invention disclosed in the appended claims.

Claims

1. 1. A method for an apparatus decoding a video signal, comprising: obtaining the number of entries in a reference picture list of the current picture; marking each of the reference picture list entries as a STRP or an LTRP, the number of entries being equal to the number of entries; and deriving prediction samples for the current picture based on the reference picture list, The step of marking each of the entries comprises: obtaining a first flag indicating whether an entry of the reference picture list is an STRP entry or an LTRP entry based on information on whether the LTRP is used for inter prediction; obtaining a difference value of a POC between the reference picture associated with the STRP entry and other pictures based on the entry of the reference picture list being the STRP entry; obtaining a second flag indicating a sign of the POC difference value based on the POC difference value; and obtaining a modulo value of a POC of a reference picture associated with the LTRP entry based on the reference picture list entry being the LTRP entry; the number of entries has a maximum value equal to the maximum required size of the decoded picture buffer plus 14; The second flag is obtained based on the POC difference value being greater than zero.

2. The method of claim 1 , wherein the POC difference value is an absolute value of the POC of the other picture minus the POC of the reference picture associated with the STRP entry.

3. 2. The method of claim 1, wherein, in the marking of each of the reference picture entries, the entry marked as the STRP is a first STRP entry, and the other picture is the current picture, or, in the marking of each of the reference picture entries, the entry marked as the STRP is not the first STRP, and the other picture is a previous STRP in the reference picture list.

4. The step of marking each of the entries comprises: obtaining a number of STRP entries and a number of LTRP entries; performing STRP marking for the number of said STRP entries, and performing LTRP marking for the number of said LTRP entries.

5. 1. A method for an apparatus encoding a video signal, comprising: constructing a reference picture list for the current picture; generating code information for constructing the reference picture list; The code information includes: a number of entries in the reference picture list; a first flag indicating whether the entry in the reference picture list is a STRP entry or an LTRP entry; a difference value of a POC between the reference picture associated with the STRP entry and another picture; a second flag indicating a sign of the difference value of the POC; and a modulo value of a POC of the reference picture associated with the LTRP entry; the number of entries has a maximum value equal to the maximum required size of the decoded picture buffer plus 14; The method of claim 1, wherein the second flag is generated based on the POC difference value being greater than zero.

6. The method of claim 5 , wherein the POC difference value is the absolute value of the POC of the other picture minus the POC of the reference picture associated with the STRP entry.

7. 6. The method of claim 5, wherein the other picture is the current picture based on the entry marked as a STRP being the first STRP entry in the code information, or the other picture is a previous STRP in the reference picture list based on the entry marked as a STRP not being the first STRP entry in the code information.

8. The method of claim 5 , wherein the at least one coded information is included in a sequence parameter set (SPS), a picture parameter set (PPS), or a slice header associated with the video signal.

9. 1. A method for transmitting data including a bitstream associated with a video signal, comprising the steps of: obtaining the bitstream associated with the video signal; transmitting the data including the bitstream; The bitstream comprises: constructing a reference picture list for the current picture; generating code information for constructing the reference picture list; The code information includes: a number of entries in the reference picture list; a first flag indicating whether the entry in the reference picture list is a STRP entry or an LTRP entry; a difference value of a POC between the reference picture associated with the STRP entry and another picture; a second flag indicating a sign of the difference value of the POC; and a modulo value of a POC of the reference picture associated with the LTRP entry; the number of entries has a maximum value equal to the maximum required size of the decoded picture buffer plus 14; The method of claim 1, wherein the second flag is generated based on the POC difference value being greater than zero.

Images