Method and apparatus for processing video signals using inter-screen prediction.

Abstract

The present invention provides a method and apparatus for decoding a video signal using inter-screen prediction. [Solution] The method includes the steps of constructing a reference picture list of the current picture using the video signal, and making a prediction for the current picture using the reference picture list, wherein the step of constructing the reference picture list includes, if the first entry of the reference picture list corresponds to a STRP entry, obtaining the difference value of the POC between the picture relating to the first entry and other pictures, and if the second entry of the reference picture list corresponds to an LTRP entry, obtaining the modulo value of the POC of the picture relating to the second entry. By generating a reference picture list for picture identification in a simple and effective manner, compression performance can be increased and computation time can be reduced.

Description

Technical Field

[0005] , ,

[0001] The embodiments of this specification relate to a method and apparatus for processing video signals using inter prediction, and more specifically, to a method and apparatus for constructing a reference picture list for prediction of a current picture.

Background Art

[0002] Compression encoding refers to a series of signal processing techniques for transmitting digitized information via a communication line or storing it in a form suitable for a storage medium. Media such as images, images, and audio can be the target of compression encoding, and in particular, a technique for performing compression encoding on an image is called video image compression.

[0003] Next-generation video content will come to have characteristics such as high spatial resolution, high frame rate, and high dimensionality of scene representation. To process such content, it will bring about a significant increase in terms of memory storage, memory access rate, and processing power.

[0004] Therefore, it is necessary to design coding tools for processing next-generation image content more efficiently. In particular, video codec standards after the HEVC (High Efficiency Video Coding) standard require prediction techniques that can accurately generate prediction samples while using resources more efficiently.

Summary of the Invention

Problems to be Solved by the Invention

[0005] ​The embodiments herein provide a method and apparatus for constructing a reference picture list necessary for predicting the current picture.

[0006] Furthermore, the embodiments described herein provide methods and apparatus for obtaining information necessary to construct a reference picture list.

[0007] Furthermore, the embodiments herein provide a method and apparatus for defining the number of entries required for the configuration of a reference picture list.

[0008] Furthermore, the embodiments herein provide methods and apparatus for reducing the amount of syntax elements required to construct a reference picture list.

[0009] Furthermore, the embodiments of this specification provide a method and apparatus for counting the number of times a reference picture is marked during the process of constructing a reference picture list.

[0010] The technical problems that this invention aims to solve are not limited to those mentioned above, and other technical problems not mentioned should be clearly understood by those skilled in the art to which this specification belongs from the following description. [Means for solving the problem]

[0011] Embodiments of this specification provide a method and apparatus for decoding a video signal using inter-prediction. A video signal processing method according to one embodiment of this specification includes the steps of: constructing a reference picture list of the current picture with the video signal; and making a prediction for the current picture using the reference picture list, wherein the step of constructing the reference picture list includes, if the first entry of the reference picture list is a short-term reference picture (STRP) entry, obtaining a picture order count (POC) difference between the picture relating to the first entry and other pictures; and if the second entry of the reference picture list is a long-term reference picture (LTRP) entry, obtaining a POC modulo value of the picture relating to the second entry.

[0012] In one embodiment, the step of constructing the reference picture list may include the step of obtaining the number of entries in the reference picture list, and the step of marking each of the entries in the reference picture list with a reference picture in the LTRP or STRP, for the number of entries.

[0013] In one embodiment, the number of entries may have a maximum value obtained by adding 14 to the maximum requested size of the decoded picture buffer.

[0014] In one embodiment, the step of marking the reference picture may include the steps of obtaining the number of STRP entries and the number of LTRP entries, and the steps of marking STRPs for the number of STRP entries and marking LTRPs for the number of LTRP entries.

[0015] In one embodiment, the steps of constructing the reference picture list may include acquiring a first flag and determining, based on the first flag, whether an entry in the reference picture list corresponds to a STRP entry or an LTRP entry.

[0016] In one embodiment, the difference value of the POC can be determined as the absolute value obtained by subtracting the POC of the picture relating to the first entry from the POC of the other picture.

[0017] In one embodiment, the step of constructing the reference picture list may include a step of acquiring a second flag that indicates the sign of the difference value of the POC if the absolute value is greater than 0.

[0018] A video signal encoding method according to another embodiment of this specification includes the steps of: constructing a reference picture list of the current picture with the video signal; making a prediction for the current picture using the reference picture list; and coding information for constructing the reference picture list. The information for constructing the reference picture list may include the difference in picture order count (POC) between the picture relating to a short-term reference picture (STRP) entry in the reference picture list and other pictures, and the modulo value of the POC of the picture relating to a long-term reference picture (LTRP) entry in the reference picture list.

[0019] In one embodiment, the information for configuring the reference picture list further includes the number of entries in the reference picture list, the number of entries may have a maximum value of the maximum requested size of the decoded picture buffer plus 14.

[0020] In one embodiment, the information for configuring the reference picture list may further include a first flag indicating whether or not an entry in the reference picture list corresponds to the STRP entry.

[0021] In one embodiment, the information for constructing the reference picture list may further include a second flag indicating the sign of the difference value of the POC for entries where the difference value of the POC is greater than 0.

[0022] In one embodiment, the information for configuring the reference picture list may be included in the bitstream relating to the video signal, specifically in an SPS (sequence parameter set), PPS (picture parameter set), or slice header.

[0023] A video signal decoding device according to another embodiment of this specification includes a memory for storing the video signal and a processor coupled to the memory. The processor is configured to construct a reference picture list of the current picture with the video signal and to use the reference picture list to make predictions for the current picture. The processor is also configured to, in order to construct the reference picture list, obtain a difference value of the POC between the picture and other pictures for the first entry of the reference picture list if the first entry of the reference picture list is a STRP entry, and obtain a modulo value of the POC of the picture for the second entry if the second entry of the reference picture list is an LTRP entry.

[0024] A video signal encoding device according to another embodiment of this specification includes a memory for storing the video signal and a processor coupled to the memory. The processor is configured to construct a reference picture list of the current picture with the video signal, make predictions for the current picture using the reference picture list, and code information for constructing the reference picture list, wherein the information for constructing the reference picture list includes the difference in picture order count (POC) between a picture relating to a short-term reference picture (STRP) entry in the reference picture list and other pictures, and the modulo value of the POC of a picture relating to a long-term reference picture (LTRP) entry in the reference picture list.

[0025] Another embodiment of this specification provides a non-transitory computer-readable medium for storing one or more commands. The one or more commands, executed by one or more processors, construct a reference picture list for the current picture in a video signal and control a video signal processing device to use the reference picture list to make predictions for the current picture. The one or more commands also control the video signal processing device to construct the reference picture list by obtaining a picture order count (POC) difference between the picture with respect to the first entry and other pictures if the first entry in the reference picture list is a short-term reference picture (STRP) entry, and by obtaining a modulo value of the picture's POC with respect to the second entry if the second entry in the reference picture list is a long-term reference picture (LTRP) entry. [Effects of the Invention]

[0026] According to the embodiments of this specification, a reference picture list necessary for predicting the current picture can be constructed through the marking of reference pictures.

[0027] Also, according to the embodiments of this specification, appropriate information can be obtained for each type of reference picture.

[0028] Also, according to the embodiments of this specification, the memory required for picture storage can be efficiently managed by defining the number of entries necessary for constructing the reference picture list.

[0029] Also, according to the embodiments of this specification, the amount of data required to construct the reference picture list can be reduced by omitting the parsing of some syntax elements.

[0030] Also, according to the embodiments of this specification, the syntax for constructing the reference picture list can be efficiently realized by counting the number of markings of the reference pictures.

[0031] The effects obtained by the present invention are not limited to the effects mentioned above, and other effects not mentioned should be clearly understood by those with ordinary knowledge in the technical field to which the present invention pertains from the following description.

Brief Description of the Drawings

[0032] The accompanying drawings, which are included in a part of the detailed description to assist in understanding this specification, provide examples of the embodiments of this specification and explain the technical features of this specification together with the detailed description.

[0033] [Figure 1] An example of a video coding system according to the embodiments of this specification is shown. [Figure 2]A schematic block diagram of an encoding device for encoding video / image signals according to the embodiments described herein is shown. [Figure 3] As an embodiment of this specification, a schematic block diagram of a decoding device for decoding image signals is shown. [Figure 4] An example of a structural diagram of a content streaming system according to the embodiments described herein is shown. [Figure 5] An example block diagram of an apparatus for processing video signals according to the embodiments of this specification is shown. [Figure 6a] This example shows a block division structure of an image according to the embodiments of this specification, and illustrates an example of a block division structure using QT (QuadTree). [Figure 6b] This document provides an example of an image block partitioning structure according to the embodiments of this specification, showing an example of a block partitioning structure using BT (Binary Tree). [Figure 6c] This example shows a block division structure of an image according to the embodiments of this specification, and illustrates an example of a block division structure using a Ternary Tree (TT). [Figure 6d] This document provides an example of an image block partitioning structure according to the embodiments of this specification, showing an example of a block partitioning structure using an AT (Asymmetric Tree). [Figure 7] This specification shows an interpretation-based video / image encoding procedure and an interpretation unit within an encoding device according to embodiments of this specification. [Figure 8] This specification shows an interpretation-based video / image encoding procedure and an interpretation unit within an encoding device according to embodiments of this specification. [Figure 9] This specification shows an interpretation-based video / image decoding procedure and an interpretation unit within a decoding apparatus according to embodiments of this specification. [Figure 10] This specification shows an interpretation-based video / image decoding procedure and an interpretation unit within a decoding apparatus according to embodiments of this specification. [Figure 11] An example flowchart for constructing a reference picture list according to the embodiments of this specification is shown. [Figure 12] An example flowchart for marking reference pictures in the embodiments of this specification is shown. [Figure 13] An example flowchart for marking reference pictures in the embodiments of this specification is shown. [Figure 14] Another example of a flowchart for marking reference pictures in the embodiments described herein is shown. [Figure 15] An example flowchart for processing a video signal according to the embodiments of this specification is shown. [Modes for carrying out the invention]

[0034] Preferred embodiments of this specification will be described in detail below with reference to the accompanying drawings. The detailed description disclosed below, together with the accompanying drawings, is intended to illustrate exemplary embodiments of this specification and not to show the only embodiments in which this specification may be implemented. The following detailed description includes specific details to provide a complete understanding of this specification. However, those skilled in the art will know that this specification may be implemented without such specific details.

[0035] In some cases, known structures and devices may be omitted or shown in the form of block diagrams focusing on the core function of each structure and device, in order to avoid ambiguity of the concepts described herein.

[0036] Furthermore, while the terminology used in this specification has been selected to be as widely used and general as possible, in certain cases, the applicant may use terminology of their own choosing. In such cases, the meaning will be clearly stated in the detailed description of the relevant section, so it should be made clear that the terminology used in this specification should not be interpreted simply by its name alone, but rather by understanding the meaning of the relevant term.

[0037] The specific terms used in the following description are provided to aid in understanding this specification, and the use of such specific terms may be modified to other forms without deviating from the technical ideas of this specification. For example, the terms signals, data, samples, pictures, slices, tiles, frames, and blocks may be appropriately substituted and interpreted in each coding process.

[0038] Hereinafter, in this specification, "processing unit" means a unit on which encoding / decoding processes such as prediction, transformation, and / or quantization are performed. Furthermore, "processing unit" may be interpreted to include units for luminance (luma) components and units for chroma components. For example, a processing unit may correspond to a block, a coding unit (CU), a prediction unit (PU), or a transform unit (TU).

[0039] Furthermore, a processing unit can be interpreted as a unit for the luminance component or a unit for the chrominance component. For example, a processing unit may correspond to a CTB (coding tree block), CB (coding block), PU, ​​or TB (transform block) for the luminance component. Alternatively, a processing unit may correspond to a CTB, CB, PU, ​​or TB for the chrominance component. Moreover, it is not limited to these interpretations, and a processing unit may also be interpreted as encompassing units for both the luminance and chrominance components.

[0040] Furthermore, processing units are not necessarily limited to square blocks, but may also consist of polygonal shapes with three or more vertices.

[0041] Furthermore, in this specification, pixels, picture elements, or coefficients (conversion coefficients or conversion coefficients obtained through linear transformation) will be commonly referred to as samples. Also, using samples may mean using pixel values, picture element values, or coefficients (conversion coefficients or conversion coefficients obtained through linear transformation), etc.

[0042] Figure 1 shows an example of a video coding system according to the embodiments of this specification.

[0043] The video coding system may include a source device 10 and a receiving device 20. The source device 10 can transmit encoded video / image information or data to the receiving device 20 in the form of a file or streaming via a digital storage medium or a network.

[0044] The source device 10 may include a video source 11, an encoding device 12, and a transmitter 13. The receiving device 20 may include a receiver 21, a decoding device 22, and a renderer 23. The encoding device 10 is referred to as a video / image encoding device, and the decoding device 20 is referred to as a video / image decoding device. The transmitter 13 may be included in the encoding device 12. The receiver 21 may be included in the decoding device 22. The renderer 23 may include a display unit, which may consist of another device or external component.

[0045] A video source can acquire video / images through processes such as video / image capture, synthesis, or generation. A video source may include video / image capture devices and / or video / image generation devices. Video / image capture devices may include, for example, one or more cameras, or video / image archives containing previously captured video / images. Video / image generation devices may include, for example, computers, tablets, and smartphones, and can generate video / images (electronically). For example, virtual video / images can be generated via a computer, in which case the process of generating the associated data can replace the video / image capture process.

[0046] The encoding device 12 can encode input video / images. The encoding device 12 can perform a series of procedures such as prediction, transformation, and quantization for the efficiency of compression and coding. The encoded data (encoded video / image information) can be output in the form of a bitstream.

[0047] The transmitter 13 can transmit encoded video / image information or data output in bitstream form to the receiver of a receiving device via a digital storage medium or network in file or streaming form. The digital storage medium can include a variety of storage media such as USB (universal serial bus), SD (secure digital), CD (compact disk), DVD (digital video disk), Blu-ray, HDD (hard disk drive), and SSD (solid state drive). The transmitter 13 may include elements for generating media files via a predetermined file format and may include elements for transmission via a broadcast / communication network. The receiver 21 can extract the bitstream and transmit it to the decoding device 22.

[0048] The decoding device 22 can decode video / images by performing a series of procedures such as inverse quantization, inverse transformation, and prediction corresponding to the operation of the encoding device 12.

[0049] Renderer 23 can render the decoded video / image. The rendered video / image can be displayed via the display unit.

[0050] Figure 2 shows a schematic block diagram of an encoding apparatus for encoding video / image signals according to the embodiments described herein.

[0051] Referring to Figure 2, the encoding device 100 may include an image splitting unit 110, a subtraction unit 115, a conversion unit 120, a quantization unit 130, an inverse quantization unit 140, an inverse conversion unit 150, an addition unit 155, a filtering unit 160, a memory 170, an inter-prediction unit 180, an intra-prediction unit 185, and an entropy encoding unit 190. The inter-prediction unit 180 and the intra-prediction unit 185 may be commonly referred to as the prediction unit. That is, the prediction unit may include the inter-prediction unit 180 and the intra-prediction unit 185. The conversion unit 120, quantization unit 130, inverse quantization unit 140, and inverse conversion unit 150 may be included in the residual processing unit. The residual processing unit may further include a subtraction unit 115. The aforementioned image segmentation unit 110, subtraction unit 115, transformation unit 120, quantization unit 130, inverse quantization unit 140, inverse transformation unit 150, addition unit 155, filtering unit 160, inter-prediction unit 180, intra-prediction unit 185, and entropy encoding unit 190 can be configured by a single hardware component (e.g., an encoder or processor) according to the embodiment. Furthermore, the memory 170 can be configured by a single hardware component (e.g., a memory or digital storage medium) according to the embodiment, and the memory 170 may include a DPB (decoded picture buffer) 175.

[0052] The image splitting unit 110 can split an input image (or picture, frame) input to the encoding device 100 into one or more processing units. For example, a processing unit is called a coding unit (CU). In this case, a coding unit can be recursively split from a coding tree unit (CTU) or the largest coding unit (LCU) using a quad-tree binary-tree (QTBT) structure. For example, a single coding unit can be split into multiple coding units of deeper depth based on a quad-tree structure and / or a binary tree structure. In this case, for example, the quad-tree structure may be applied first and the binary tree structure later. Alternatively, the binary tree structure may be applied first. The coding procedure described herein is performed based on the final coding unit that cannot be further split. In this case, based on the coding efficiency due to the characteristics of the image, the largest coding unit can be immediately used as the final coding unit, or, if necessary, the coding unit can be recursively divided into lower-depth coding units, and the coding unit of optimal size can be used as the final coding unit. Here, the coding procedure may include procedures such as prediction, transformation, and reconstruction, which will be described later. As another example, the processing unit may further include a prediction unit (PU) or a transformation unit (TU). In this case, the prediction unit and the transformation unit can be divided or partitioned from the final coding unit described above. The prediction unit may be a unit of sample prediction, and the transformation unit may be a unit that derives transformation coefficients and / or a unit that derives a residual signal from the transformation coefficients.

[0053] The term "unit" may, as in some cases, be used interchangeably with terms such as "block" or "area." Generally, an MxN block can represent a set of samples or transform coefficients consisting of M columns and N rows. A sample can generally represent a pixel or a pixel value, and may represent only the luminance (luma) component pixel / pixel value, or only the chroma component pixel / pixel value. A sample can be used as the term corresponding to a single picture (or image) as a pixel or pel.

[0054] The encoding device 100 can generate a residual signal (residual block, residual sample array) by subtracting the prediction signal (predicted block, predicted sample array) output from the inter-prediction unit 180 or intra-prediction unit 185 from the input image signal (original block, original sample array), and the generated residual signal is transmitted to the conversion unit 120. In this case, as shown, the unit that subtracts the prediction signal (predicted block, predicted sample array) from the input image signal (original block, original sample array) within the encoding device 100 is called the subtraction unit 115. The prediction unit can make predictions for the block to be processed (hereinafter referred to as the current block) and generate a predicted block that includes the predicted sample for the current block. The prediction unit can determine whether intra-prediction or inter-prediction is applied on a block or CU basis. As will be described later in the explanation of each prediction mode, the prediction unit can generate various types of prediction information, such as prediction mode information, and transmit it to the entropy encoding unit 190. Information regarding the prediction can be encoded by the entropy encoding unit 190 and output in the form of a bitstream.

[0055] The intra-prediction unit 185 can predict the current block by referring to a sample in the current picture. The referenced sample may be located in the vicinity (neighbor) of the current block or at a distance, depending on the prediction mode. The prediction mode in intra-prediction can include multiple non-directional modes and multiple directional modes. Non-directional modes may include, for example, DC mode and planar mode. Directional modes may include, for example, 33 directional prediction modes or 65 directional prediction modes, depending on the degree of detail of the prediction direction. However, this is an example, and more or fewer directional prediction modes may be used depending on the settings. The intra-prediction unit 185 can also determine the prediction mode to be applied to the current block using the prediction modes applied to the surrounding blocks.

[0056] The interprediction unit 180 can guide the predicted block relative to the current block based on the reference block (reference sample array) identified by the motion vector on the reference picture. In this case, in order to reduce the amount of motion information transmitted in interprediction mode, motion information can be predicted in units of blocks, subblocks, or samples based on the correlation of motion information between the surrounding block and the current block. The motion information can include the motion vector and the reference picture index. The motion information can further include information on the interprediction direction (L0 prediction, L1 prediction, Bi prediction, etc.). In the case of interprediction, the surrounding block can include spatial neighboring blocks that exist in the current picture and temporal neighboring blocks that exist in the reference picture. The reference picture containing the reference block and the reference picture containing the temporal neighboring block may be the same or different. The temporal neighboring block is called a collocated reference block, colCU, etc., and the reference picture containing the temporal neighboring block is also called a collocated picture (colPic). For example, the interpretation unit 180 can construct a list of candidate motion information based on surrounding blocks and generate information indicating which candidate is used to derive the motion vector and / or reference picture index of the current block. Interpretation is performed based on various prediction modes; for example, in skip mode and merge mode, the interpretation unit 180 can use the motion information of surrounding blocks as the motion information of the current block. In skip mode, unlike merge mode, a residual signal does not need to be transmitted.In motion vector prediction (MVP) mode, the motion vectors of surrounding blocks are used as motion vector predictors, and the motion vector difference is signaled to indicate the motion vector of the current block.

[0057] The prediction signal generated via the interpretation unit 180 or the intrapretation unit 185 can be used to generate a reconstruction signal or a residual signal.

[0058] The transformation unit 120 can generate transformation coefficients by applying a transformation technique to the residual signal. For example, the transformation technique may include at least one of the following: DCT (discrete cosine transform), DST (discrete sine transform), KLT (Karhunen-Loeve transform), GBT (graph-based transform), or CNT (conditionally non-linear transform). Here, GBT refers to a transformation obtained from a graph in which the relationship information between pixels is represented by a graph. CNT refers to a transformation obtained by generating a prediction signal using all previously reconstructed pixels. Furthermore, the transformation process may be applied to pixel blocks of the same size that are square, or to blocks of variable size that are not square.

[0059] The quantization unit 130 quantizes the conversion coefficients and transmits them to the entropy encoding unit 190, which can encode the quantized signal (information about the quantized conversion coefficients) and output it as a bitstream. The information about the quantized conversion coefficients is called residual information. The quantization unit 130 can rearrange the quantized conversion coefficients in block form into a one-dimensional vector form based on the scan order of the coefficients, and can also generate information about the quantized conversion coefficients based on the quantized conversion coefficients in the form of a one-dimensional vector. The entropy encoding unit 190 can perform various encoding methods, such as exponential Golomb, CAVLC (context-adaptive variable length coding), and CABAC (context-adaptive binary arithmetic coding). The entropy encoding unit 190 can also encode information necessary for video / image reconstruction other than the quantized conversion coefficients (e.g., values ​​of syntax elements) together with or separately. The encoded information (e.g., video / image information) can be transmitted or stored in the form of a bitstream in units of NAL (network abstraction layer) units. The bitstream can be transmitted over a network or stored on a digital storage medium. Here, the network may include broadcast networks and / or communication networks, and the digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. The signal output from the entropy encoding unit 190 may be transmitted by a transmitting unit (not shown) and / or stored by a storage unit (not shown) which are configured as internal / external elements of the encoding device 100, or the transmitting unit may be a component of the entropy encoding unit 190.

[0060] The quantized conversion coefficients output from the quantization unit 130 can be used to generate a prediction signal. For example, the residual signal can be reconstructed by applying inverse quantization and inverse transformation to the quantized conversion coefficients via the inverse quantization unit 140 and inverse transformation unit 150 in the loop. The adder 155 can generate a reconstructed signal (reconstructed picture, reconstructed block, reconstructed sample array) by adding the reconstructed residual signal to the prediction signal output from the inter-prediction unit 180 or intra-prediction unit 185. If there are no residuals for the block to be processed, such as when skip mode is applied, the predicted block can be used as the reconstructed block. The adder 155 is referred to as the reconstruction unit or reconstructed block generation unit. The reconstructed signal can be used for intra-prediction of the next block to be processed in the current picture, or, after filtering as described later, it can be used for inter-prediction of the next picture.

[0061] The filtering unit 160 can apply filtering to the restored signal to improve subjective / objective image quality. For example, the filtering unit 160 can apply various filtering methods to the restored picture to generate a modified restored picture, and can transmit the modified restored picture to the decoded picture buffer 170. Various filtering methods can include, for example, deblocking filtering, sample adaptive offset, adaptive loop filter, and bilateral filter. As will be described later in the explanation of each filtering method, the filtering unit 160 can generate various filtering-related information and transmit it to the entropy encoding unit 190. The filtering-related information can be encoded by the entropy encoding unit 190 and output in the form of a bitstream.

[0062] The corrected restored picture sent to the decoded picture buffer 170 can be used as a reference picture in the interpretation unit 180. Through this, the encoding device 100 can avoid prediction mismatches between the encoding device 100 and the decoding device 200 when interpretation is applied, and the encoding efficiency can also be improved.

[0063] The decoded picture buffer 170 can store the corrected restored picture for use as a reference picture in the interpretation unit 180.

[0064] Figure 3 shows a schematic block diagram of a decoding device for decoding an image signal, as an embodiment of this specification.

[0065] Referring to Figure 3, the decoding device 200 may be configured to include an entropy decoding unit 210, an inverse quantization unit 220, an inverse transformation unit 230, an addition unit 235, a filtering unit 240, a memory 250, an inter-prediction unit 260, and an intra-prediction unit 265. The inter-prediction unit 260 and the intra-prediction unit 265 may be commonly referred to as the prediction unit. That is, the prediction unit may include an inter-prediction unit 180 and an intra-prediction unit 185. The inverse quantization unit 220 and the inverse transformation unit 230 may be commonly referred to as the residual processing unit. That is, the residual processing unit may include an inverse quantization unit 220 and an inverse transformation unit 230. The entropy decoding unit 210, the inverse quantization unit 220, the inverse transformation unit 230, the addition unit 235, the filtering unit 240, the inter-prediction unit 260, and the intra-prediction unit 265 may be configured by a single hardware component (e.g., a decoder or processor) according to the embodiment. Furthermore, the decoded picture buffer 250 can be implemented by a single hardware component (e.g., memory or a digital storage medium) according to the embodiment. The memory 250 may include the DPB 175 and may also be comprised of a digital storage medium.

[0066] When a bitstream containing video / image information is input, the decoding device 200 can reconstruct the image corresponding to the process by which the video / image information was processed in the encoding device 100 in Figure 2. For example, the decoding device 200 can perform decoding using the processing units applied in the encoding device 100. Therefore, the processing unit during decoding may be, for example, a coding unit, which can be divided from a coding tree unit or the largest coding unit according to a quad-tree structure and / or a binary tree structure. Furthermore, the reconstructed image signal decoded and output via the decoding device 200 can be reproduced via a playback device.

[0067] The decoding device 200 can receive the signal output from the encoding device 100 in Figure 2 in the form of a bitstream, and the received signal can be decoded via the entropy decoding unit 210. For example, the entropy decoding unit 210 can parse the bitstream to derive the information necessary for image restoration (or picture restoration) (e.g., video / image information). For example, the entropy decoding unit 210 can decode the information in the bitstream based on a coding method such as exponential Golomb coding, CAVLC, or CABAC, and output the values ​​of syntax elements necessary for image restoration and the quantized values ​​of conversion coefficients related to residuals. More specifically, the CABAC entropy decoding method receives bins corresponding to each syntactic element in a bitstream, determines a context model using the syntactic element information to be decoded, the decoding information of the surrounding and decoded blocks, or symbol / bin information decoded in a previous stage, predicts the probability of bin occurrence based on the determined context model, performs arithmetic decoding of the bins, and generates symbols corresponding to the values ​​of each syntactic element. In this process, after determining the context model, the CABAC entropy decoding method can update the context model using the decoded symbol / bin information for the context model of the next symbol / bin. Information related to predictions from the information decoded by the entropy decoding unit 210 is provided to the prediction unit (inter-prediction unit 260 and intra-prediction unit 265), and residual values ​​from entropy decoding performed by the entropy decoding unit 210, i.e., quantized conversion coefficients and related parameter information, can be input to the inverse quantization unit 220. Furthermore, information related to filtering among the information decoded by the entropy decoding unit 210 can be provided to the filtering unit 240.On the other hand, a receiving unit (not shown) that receives the signal output from the encoding device 100 may be further configured as an internal / external element of the decoding device 200, or the receiving unit may be a component of the entropy decoding unit 210.

[0068] The inverse quantization unit 220 can output transformation coefficients by inverse quantization of the quantized transformation coefficients. The inverse quantization unit 220 can rearrange the quantized transformation coefficients in the form of two-dimensional blocks. In this case, the rearrangement can be performed based on the coefficient scan order performed by the encoding device 100. The inverse quantization unit 220 can perform inverse quantization on the quantized transformation coefficients using quantization parameters (e.g., quantization step size information) to obtain the transformation coefficients.

[0069] The inverse transform unit 230 can output residual signals (residual blocks, residual sample arrays) by applying an inverse transform to the transformation coefficients.

[0070] The prediction unit can make predictions for the current block and generate a predicted block containing prediction samples for the current block. Based on the prediction information output from the entropy decoding unit 210, the prediction unit can determine whether intra-prediction or inter-prediction is applied to the current block, and can determine a specific intra / inter-prediction mode.

[0071] The intra-prediction unit 265 can predict the current block by referencing a sample within the current picture. The referenced sample may be located in the vicinity (neighbor) of the current block or at a distance from it, depending on the prediction mode. The prediction mode in intra-prediction can include multiple non-directional modes and multiple directional modes. The intra-prediction unit 265 can also determine the prediction mode to be applied to the current block by utilizing the prediction modes applied to the surrounding blocks.

[0072] The interprediction unit 260 can derive a predicted block for the current block based on a reference block (reference sample array) identified by motion vectors on a reference picture. In this case, in order to reduce the amount of motion information transmitted in interprediction mode, motion information can be predicted in units of blocks, subblocks, or samples based on the correlation of motion information between surrounding blocks and the current block. Motion information can include motion vectors and reference picture indices. Motion information can further include information regarding the interprediction direction (L0 prediction, L1 prediction, Bi prediction, etc.). In the case of interprediction, surrounding blocks can include spatial neighboring blocks that exist in the current picture and temporal neighboring blocks that exist in the reference picture. For example, the interprediction unit 260 can construct a candidate list of motion information based on surrounding blocks and derive the motion vector and / or reference picture index of the current block based on the received candidate selection information. Interprediction is performed based on various prediction modes, and the prediction information can include information indicating the mode of interprediction for the current block.

[0073] The summing unit 235 can generate a reconstructed signal (reconstructed picture, reconstructed block, reconstructed sample array) by adding the acquired residual signal to the predicted signal (predicted block, predicted sample array) output from the inter-prediction unit 260 or intra-prediction unit 265. If there is no residual for the block to be processed, such as when skip mode is applied, the predicted block can be used as the reconstructed block.

[0074] The addition unit 235 is called the restoration unit or restoration block generation unit. The generated restoration signal can be used for intra-prediction of the next block to be processed in the current picture, or, as described later, can be filtered and used for intra-prediction of the next picture.

[0075] The filtering unit 240 can improve subjective / objective image quality by applying filtering to the restored signal. For example, the filtering unit 240 can apply various filtering methods to the restored picture to generate a modified restored picture, and can transmit the modified restored picture to the decoded picture buffer 250. Various filtering methods can include, for example, deblocking filtering, sample adaptive offset (SAO), adaptive loop filter (ALF), bilateral filter, etc.

[0076] The corrected restored picture sent to the decoded picture buffer 250 can be used as a reference picture by the interpretation unit 260.

[0077] The embodiments described herein for the filtering unit 160, inter-prediction unit 180, and intra-prediction unit 185 of the encoding device 100 can also be applied identically or in a corresponding manner to the filtering unit 240, inter-prediction unit 260, and intra-prediction unit 265 of the decoding device, respectively.

[0078] Figure 4 shows an example of a structural diagram of a content streaming system according to the embodiments described herein.

[0079] The content streaming system to which this specification applies may broadly include an encoding server 410, a streaming server 420, a web server 430, media storage 440, a user device 450, and a multimedia input device 460.

[0080] The encoding server 410 can compress content input from multimedia input devices such as smartphones, cameras, and camcorders into digital data to generate a bitstream, which it can then transmit to the streaming server 420. In another example, if the multimedia input device 460, such as a smartphone, camera, or camcorder, generates the bitstream directly, the encoding server 410 may be omitted.

[0081] The bitstream can be generated by an encoding method or bitstream generation method to which this specification applies, and the streaming server 420 can temporarily store the bitstream in the process of transmitting or receiving the bitstream.

[0082] The streaming server 420 transmits multimedia data to the user device 450 based on a user request via the web server 430, with the web server 430 acting as an intermediary to inform the user of available services. When a user requests a desired service from the web server 430, the web server 430 transmits this to the streaming server 420, which then transmits the multimedia data to the user. In this case, the content streaming system may include a separate control server, in which case the control server controls the commands and responses between the devices within the content streaming system.

[0083] The streaming server 420 can receive content from the media storage 440 and / or the encoding server 410. For example, the streaming server 420 can receive content from the encoding server 410 in real time. In this case, to provide a smooth streaming service, the streaming server 420 can store the bitstream for a certain period of time.

[0084] For example, user devices 450 may include mobile phones, smartphones, laptop computers, digital broadcasting terminals, PDAs (personal digital assistants), PMPs (portable multimedia players), navigation systems, slate PCs, tablet PCs, ultrabooks, wearable devices such as smartwatches, smart glass, HMDs (head-mounted displays), digital TVs, desktop computers, and digital signage.

[0085] Each server within a content streaming system can be operated as a distributed server, in which case the data received by each server can be processed in a distributed manner.

[0086] Figure 5 shows an example block diagram of an apparatus for processing video signals according to an embodiment of this specification. The video signal processing apparatus in Figure 5 may correspond to the encoding apparatus 100 in Figure 2 or the decoding apparatus 200 in Figure 3.

[0087] The video signal processing device 500 according to the embodiments of this specification may include a memory 520 for storing video signals and a processor 510 that processes video signals while being coupled to the memory.

[0088] The processor 510 according to the embodiments of this specification may consist of at least one processing circuit for processing video signals and can process image signals by executing commands for encoding or decoding video signals. That is, the processor 510 can encode the original video signal or decode the encoded video signal by executing the encoding or decoding methods described below.

[0089] The processor 510 according to an embodiment of the present invention may consist of at least one processing circuit for processing an image signal, and can process an image signal by executing commands for encoding or decoding the image signal. That is, the processor 510 can encode the original image data or decode the encoded image signal by executing the encoding or decoding method described above.

[0090] Figure 6 shows examples of block division structures for images according to the embodiments of this specification, with Figure 6a showing an example of a block division structure using a Quad Tree (QT), Figure 6b showing an example of a Binary Tree (BT), Figure 6c showing an Asymmetric Tree (TT), and Figure 6d showing an Asymmetric Tree (AT).

[0091] In video coding, a single block can be partitioned using QT. A subblock partitioned by QT can then be recursively partitioned further using QT. A leaf block that is not further partitioned by QT can be partitioned by at least one of the following methods: BT, TT, or AT. BT can have two forms of partitioning: horizontal BT (2NxN, 2NxN) and vertical BT (Nx2N, Nx2N). TT can have two forms of partitioning: horizontal TT (2Nx1 / 2N, 2NxN, 2Nx1 / 2N) and vertical TT (1 / 2Nx2N, Nx2N, 1 / 2Nx2N). AT can have four forms of partitioning: horizontal-up AT (2Nx1 / 2N, 2Nx3 / 2N), horizontal-down AT (2Nx3 / 2N, 2Nx1 / 2N), vertical-left AT (1 / 2Nx2N, 3 / 2Nx2N), and vertical-right AT (3 / 2Nx2N, 1 / 2Nx2N). Each BT, TT, and AT can be further recursively partitioned using BT, TT, and AT.

[0092] Figure 6a shows an example of QT partitioning. Block A can be partitioned by QT into four subblocks (A0, A1, A2, A3). Subblock A1 can then be partitioned again by QT into four subblocks (B0, B1, B2, B3).

[0093] Figure 6b shows an example of BT partitioning. Block B3, which cannot be further partitioned by QT, can be partitioned into vertical BT(C0, C1) or horizontal BT(D0, D1). Each subblock, like block C0, can be recursively partitioned further into the form of horizontal BT(E0, E1) or vertical BT(F0, F1).

[0094] Figure 6c shows an example of TT partitioning. Block B3, which cannot be further partitioned by QT, can be partitioned into vertical TT(C0, C1, C2) or horizontal TT(D0, D1, D2). Each subblock, like block C1, can be recursively partitioned further into the form of horizontal TT(E0, E1, E2) or vertical TT(F0, F1, F2).

[0095] Figure 6d shows an example of AT partitioning. Block B3, which cannot be further partitioned by QT, can be partitioned into vertical AT(C0, C1) or horizontal AT(D0, D1). Each subblock, like block C1, can be recursively partitioned further into the form of horizontal AT(E0, E1) or vertical TT(F0, F1).

[0096] On the other hand, BT, TT, and AT divisions can all be used together for division. For example, a subblock divided by BT can be divided by TT or AT. Similarly, a subblock divided by TT can be divided by BT or AT. A subblock divided by AT can be divided by BT or TT. For example, after a horizontal BT division, each subblock can be divided into vertical BTs, or after a vertical BT division, each subblock can be divided into horizontal BTs. In this case, although the order of division is different, the final divided shape is the same.

[0097] Furthermore, once a block is divided, the order in which the block is searched can be defined in various ways. Generally, searching a block from left to right and from top to bottom can mean the order in which each divided subblock is determined to be further divided, or, if the block cannot be divided any further, the coding order of each subblock, or the search order when referencing information from other adjacent blocks from a subblock.

[0098] Figures 7 and 8 show an interpretation-based video / image encoding procedure and an interpretation unit in an encoding device according to an embodiment of this specification.

[0099] The encoding device 100 performs interpretation for the current block (S710). The encoding device 100 can derive the interpretation mode and motion information of the current block and generate prediction samples for the current block. Here, the procedures for determining the interpretation mode, deriving motion information, and generating prediction samples may be performed simultaneously, or any of the procedures may be performed before the others. For example, the interpretation unit 180 of the encoding device 100 may include a prediction mode determination unit 181, a motion information derivation unit 182, and a prediction sample derivation unit 183. The prediction mode determination unit 181 determines the prediction mode for the current block, the motion information derivation unit 182 derives motion information for the current block, and the prediction sample derivation unit 183 derives prediction samples for the current block. For example, the interpretation unit 180 of the encoding device 100 can search for blocks similar to the current block within a certain area (search area) of the reference picture via motion estimation, and derive a reference block whose difference from the current block is minimal or below a certain standard. Based on this, it can derive a reference picture index that points to the reference picture where the reference block is located, and derive a motion vector based on the difference in position between the reference block and the current block. The encoding device 100 can determine which of the various prediction modes is applied to the current block. The encoding device 100 can compare the RD costs (cost) of the various prediction modes and determine the optimal prediction mode for the current block.

[0100] For example, if skip mode or merge mode is applied to the current block, the encoding device 100 can configure a merge candidate list (described later) and derive a reference block from among the reference blocks pointed to by the merge candidates included in the merge candidate list whose difference from the current block is the minimum or below a certain standard. In this case, a merge candidate associated with the derived reference block is selected, merge index information pointing to the selected merge candidate is generated and can be signaled to the decoding device 200. The movement information of the current block can be derived using the movement information of the selected merge candidate.

[0101] As another example, when the (A)MVP mode is applied to the current block, the encoding device 100 can configure the (A)MVP candidate list described later, and use the motion vector of the selected MVP candidate from among the MVP (motion vector predictor) candidates included in the (A)MVP candidate list as the MVP of the current block. In this case, for example, the motion vector pointing to the reference block derived by the motion estimation described above can be used as the motion vector of the current block, and the MVP candidate with the smallest difference between the motion vector of the current block and the selected MVP candidate can be derived. The MVD (motion vector difference), which is the difference obtained by subtracting the MVP from the motion vector of the current block, can be derived. In this case, information regarding the MVD can be signaled to the decoding device 200. Also, when the (A)MVP mode is applied, the value of the reference picture index is configured as reference picture index information and can be separately signaled to the decoding device 200.

[0102] The encoding device 100 can derive the residual sample based on the predicted sample (S720). The encoding device 100 can derive the residual sample by comparing the original sample and the predicted sample of the current block.

[0103] The encoding device 100 encodes image information including prediction information and residual information (S730). The encoding device 100 can output the encoded image information in the form of a bitstream. The prediction information may include prediction mode information (e.g., skip flag, merge flag, or mode index) and motion information as information related to the prediction procedure. The motion information may include candidate selection information (e.g., merge index, mvp flag, or mvp index), which is information for deriving the motion vector. The motion information may also include the aforementioned MVD information and / or reference picture index information. Furthermore, the motion information may include information indicating whether L0 prediction, L1 prediction, or bi prediction is applied. The residual information is information related to the residual sample. The residual information may include information related to the quantized transformation coefficients for the residual sample.

[0104] The output bitstream can be stored on a (digital) storage medium and transmitted to a decoding device, or it can be transmitted to a decoding device via a network.

[0105] On the other hand, as mentioned above, the encoding device can generate a reconstructed picture (including the reconstructed sample and the reconstructed block) based on the reference sample and the residual sample. This is because the encoding device 100 can derive the same prediction results from the decoding device 200, thereby improving the efficiency of coding. Therefore, the encoding device 100 can store the reconstructed picture (or reconstructed sample, reconstructed block) in memory and use it as a reference picture for interpretation. As mentioned above, in-loop filtering procedures and the like can be further applied to the reconstructed picture.

[0106] Figures 9 and 10 show an interpretation-based video / image decoding procedure and an interpretation unit in a decoding apparatus according to an embodiment of this specification.

[0107] The decoding device 200 can perform operations corresponding to those performed by the encoding device 100. Based on the received prediction information, the decoding device 200 can make predictions for the current block and derive prediction samples.

[0108] Specifically, the decoding device 200 can determine the prediction mode for the current block based on the received prediction information (S910). Based on the prediction mode information in the prediction information, the decoding device 200 can determine which inter-prediction mode is applied to the current block.

[0109] For example, the decoding device 200 can determine, based on the merge flag, whether a merge mode is applied to the current block or whether the (A)MVP mode is determined. Alternatively, the decoding device 200 can select one of several candidate interprediction modes based on the mode index. The candidate interprediction modes may include skip mode, merge mode and / or (A)MVP mode, or may include several interprediction modes as described later.

[0110] The decoding device 200 derives motion information for the current block based on the determined interpretation mode (S920). For example, if a skip mode or merge mode is applied to the current block, the decoding device 200 can configure a merge candidate list (described later) and select one merge candidate from among those included in the merge candidate list. The selection of a merge candidate is made based on a merge index. Motion information for the current block can be derived from the motion information of the selected merge candidate. Motion information for the selected merge candidate can be used as the motion information for the current block.

[0111] As another example, when the (A)MVP mode is applied to the current block, the decoding device 200 can configure the (A)MVP candidate list described later, and use the motion vector of the selected MVP candidate from among the MVP candidates included in the (A)MVP candidate list as the MVP of the current block. The selection of the MVP is made based on the selection information (MVP flag or MVP index) described above. In this case, the decoding device 200 can derive the MVD of the current block based on the information regarding the MVD, and can derive the motion vector of the current block based on the MVP and MVD of the current block. The decoding device 200 can also derive the reference picture index of the current block based on the reference picture index information. Within the reference picture list for the current block, the picture pointed to by the reference picture index can be derived as the reference picture referenced for interpretation of the current block.

[0112] On the other hand, as will be described later, the movement information of the current block can be derived without constructing a candidate list, in which case the movement information of the current block can be derived by a procedure initiated in the prediction mode described later. In this case, the construction of the candidate list as described above can be omitted.

[0113] The decoding device 200 can generate predicted samples for the current block based on the motion information of the current block (S930). In this case, the decoding device 200 can derive a reference picture based on the reference picture index of the current block, and use the sample of the reference block pointed to by the motion vector of the current block on the reference picture to derive predicted samples for the current block. In this case, as will be described later, a filtering procedure of all or some of the predicted samples for the current block may be further performed.

[0114] For example, the interpretation unit 260 of the decoding device 200 may include a prediction mode determination unit 261, a motion information derivation unit 262, and a prediction sample derivation unit 263. The prediction mode determination unit 181 determines the prediction mode for the current block based on the prediction mode information received, the motion information derivation unit 182 derives motion information (motion vector and / or reference picture index, etc.) for the current block based on the motion information information received, and the prediction sample derivation unit 183 derives a prediction sample for the current block.

[0115] The decoding device 200 generates a residual sample for the current block based on the received residual information (S940). The decoding device 200 can generate a reconstructed sample for the current block based on the predicted sample and the residual sample, and generate a reconstructed picture based on this (S950). As previously mentioned, in-loop filtering procedures and the like can be further applied to the reconstructed picture thereafter.

[0116] As mentioned above, the interpretation procedure may include an interpretation mode determination stage, a motion information derivation stage based on the determined prediction mode, and a prediction execution (prediction sample generation) stage based on the derived motion information.

[0117] A variety of interpretation modes can be used to predict the current block within a picture. For example, various modes such as merge mode, skip mode, MVP mode, and affine mode can be used. Additional modes such as DMVR (Decoder side motion vector refinement) mode and AMVR (adaptive motion vector resolution) mode can be used. Affine mode is also called affine motion prediction mode. MVP mode is also called AMVP (advanced motion vector prediction) mode.

[0118] Prediction mode information indicating the inter-prediction mode of the current block can be signaled from the encoding device to the decoding device 200. The prediction mode information can be included in the bitstream and received by the decoding device 200. The prediction mode information may include index information indicating one of a number of candidate modes. Alternatively, the inter-prediction mode can be indicated via hierarchical signaling of flag information. In this case, the prediction mode information may include one or more flags. For example, the encoding device 100 may signal a skip flag to indicate whether a skip mode is applicable, and if the skip mode is not applicable, it may signal a merge flag to indicate whether a merge mode is applicable, and if the merge mode is not applicable, it may indicate that the MVP mode is applicable, or it may further signal flags for further distinctions. An affine mode may be signaled to an independent mode, or to a mode dependent on a merge mode or MVP mode, etc. For example, an affine mode may consist of one candidate from a merge candidate list or an MVP candidate list, as described later.

[0119] The encoding device 100 or the decoding device 200 can perform interpretation using motion information of the current block. The encoding device 100 can derive optimal motion information for the current block through a motion estimation procedure. For example, the encoding device 100 can use the original block in the original picture for the current block to search for a highly correlated similar reference block in fractional pixel units within a defined search range in the reference picture, thereby deriving motion information. Block similarity can be derived based on the difference in phase-based sample values. For example, block similarity can be calculated based on the sum of absolute difference (SAD) between the current block (or template of the current block) and the reference block (or template of the reference block). In this case, motion information can be derived based on the reference block with the smallest SAD in the search area. The derived motion information can be signaled to the decoding device in various ways based on the interpretation mode.

[0120] When merge mode is applied, the movement information of the current predicted block is not transmitted directly. Instead, the movement information of surrounding predicted blocks is used to guide the movement information of the current predicted block. Therefore, the encoding device 100 can indicate the movement information of the current predicted block by transmitting flag information indicating that merge mode has been used and a merge index indicating which surrounding predicted block was used.

[0121] The encoding device 100 should search for merge candidate blocks to be used to guide the motion information of the current predicted block in order to perform merge mode. For example, up to five merge candidate blocks may be used, but this specification is not limited thereto. The maximum number of merge candidate blocks can be transmitted in the slice header, but this specification is not limited thereto. After finding merge candidate blocks, the encoding device 100 can generate a merge candidate list and select the merge candidate block with the lowest cost among them as the final merge candidate block.

[0122] This specification provides various examples of merge candidate blocks that constitute a merge candidate list.

[0123] A merge candidate list can, for example, utilize five merge candidate blocks. For instance, it can utilize four spatial merge candidates and one temporal merge candidate.

[0124] The following describes how to construct a reference picture list for the interpretation mentioned above.

[0125] High-level syntax (HLS) exists in video coding systems to control the general sharing of common information. HLS aims to manage frames in order to efficiently reduce the number of bits used for memory utilization. HLS uses a reference picture management system, which invokes a process called reference picture marking. This process labels each frame according to its usability for decoding the current frame. The goal in designing and implementing HLS is to have a simple and effective means for identifying the pictures necessary for decoding the current picture.

[0126] Example 1

[0127] In this embodiment, a reference picture list (RPL) management system is introduced, which is directly signaled and derived. Two lists (i.e., Reference Picture List0 and Reference Picture List1) are used. The information for deriving the two reference picture lists is signaled as syntax elements in the Sequence Parameter Set (SPS), Picture Parameter Set (PPS), and slice headers. Two lists are generated for all types of slices. However, I-slices do not use any information from the lists, P-slices use only information from List0, and B-slices use information from all of the lists. Pictures are identified using a picture order count (POC) number. Each picture can be classified as either an "LTRP (long-term-reference-picture)" or a "STRP (short-term-reference-picture)". Pictures classified as LTRP or STRP are active pictures, meaning they can be used during decoding, while inactive pictures are not used during the decoding process.

[0128] Figure 11 shows an example flowchart for configuring a reference picture list according to the embodiments of this specification. The operation in Figure 11 is performed by the decoding device 200.

[0129] Figure 11 shows the overall process of constructing a reference picture list. First, at step S1105, the decoding device 200 parses the total number of entries in the list. Thereafter, for each entry, the picture is marked as LTRP or STRP through the reference picture marking process (S1120). In this document, marking a picture means adding or updating information (parameters) about the picture. Specifically, the decoding device 200 initializes the entry index (i) (S1110), and if the entry index (i) is less than or equal to the total number of entries (S1115), it performs reference picture marking (S1120). The decoding device also updates the entry index (i) (S1125) once the reference picture marking is complete. Embodiments of the present invention propose a new structure for improving the management system of reference pictures.

[0130] Figures 12 and 13 show examples of flowcharts for marking reference pictures according to the embodiments of this specification.

[0131] [Table 1]

[0132] Table 1 shows an example of syntax for constructing a reference picture list. Here, this embodiment provides a possible structure for managing reference pictures, where the function ref_pic_list_struct takes listIdx as its first input, indicated by the syntax element rpl1_same_as_rpl0_flag. That is, if the flag (rpl1_same_as_rpl0_flag) is true, this indicates that the lists are identical and one list (L0 or L1) can be inferred from the other (L1 or L0). The second input (rplsIdx) of the ref_pic_list_struct function represents the number of reference picture lists used in the coded video sequence explicitly specified in the SPS. num_ref_entries represents the number of entries in the syntax structure ref_pic_list_struct(listIdx, rpIsIdx). For each entry, if an LTRP frame is used, the flags are parsed to read whether the frame is treated as LTRP or STRP in the list. If the entry is STRP and is the first STRP entry in the syntax structure ref_pic_list_struct(listIdx,rpIsIdx), it represents the absolute difference between the POC of the current picture and the POC of the picture referenced by the i-th entry. Alternatively, if the i-th entry is a STRP entry but is not the first entry, it represents the absolute difference between the POC of the picture referenced by the i-th entry and the POC of the picture indicated by the previous STRP.(If the entry is a STRP, it is the first STRP entry in the ref_pic_list_struc(listIdx,rpIsIdx) syntax structure and it specifies the absolute difference in POC of the current picture and the picture referred to by the i-th entry. Or, if the i-th entry is an STRP entry but not the first entry, then it specifies the absolute difference between the POC of the picture referred to by the i-th entry and by the previous STRP.) If abs_delta_poc for the i-th entry is greater than 0, a flag marking information about the STRP's sign is signaled. However, if st_ref_pic_flag is 0, it indicates that the i-th entry is a long-term reference picture. LTRPs are signaled by the POC LSB (least significant bit).

[0133] The syntax elements related to Table 1 can be defined as follows:

[0134] num_ref_entries[listIdx][rplsIdx] represents the number of entries in the structure of the ref_pic_list_struct(listIdx,rplsIdx) syntax. The value of num_ref_entries[listIdx][rplsIdx] can be between 0 and sps_max_dec_pic_buffering_minus1+14.

[0135] If rpl1_same_as_rpl0_flag is 1, the syntax structures num_ref_pic_lists_in_sps[1] and ref_pic_list_struct(1,rplsIdx) do not exist, and the following applies.

[0136] - The value of num_ref_pic_lists_in_sps[1] is inferred to be the same as the value of num_ref_pic_lists_in_sps[0].

[0137] - The values ​​of each syntax element in ref_pic_list_struct(1,rplsIdx) can be between 0 and num_ref_pic_lists_in_sps[0]-1, and are inferred to be the same as the values ​​of the corresponding syntax elements in ref_pic_list_struct(0,rplsIdx).

[0138] A long_term_ref_pics_flag of 0 indicates that LTRP is not used for interpretation of a coded picture in a CVS (coded video sequence). A long_term_ref_pics_flag of 1 indicates that LTRP is used for interpretation of one or more coded pictures in a CVS.

[0139] If st_ref_pic_flag[listIdx][rplsIdx][i] is 1, it indicates that the i-th entry in the syntax structure ref_pic_list_struct(listIdx,rplsIdx) is a STRP entry. If st_ref_pic_flag[listIdx][rplsIdx][i] is 0, it indicates that the i-th entry in the syntax structure ref_pic_list_struct(listIdx,rplsIdx) is an LTRP entry. If it does not exist, the value of st_ref_pic_flag[listIdx][rplsIdx][i] is inferred to be 1.

[0140] abs_delta_poc_st[listIdx][rplsIdx][i] represents the absolute difference between the picture order count values ​​of the current picture and the picture referenced by the i-th entry, if the i-th entry is the first STRP in the syntax structure ref_pic_list_struct(listIdx,rplsIdx), or if the i-th entry is a STRP entry but is not the first STRP entry in the syntax structure ref_pic_list_struct(listIdx,rplsIdx), it represents the absolute difference between the picture order count values ​​of the i-th entry and the picture indicated by the previous STRP in the syntax structure ref_pic_list_struct(listIdx,rplsIdx).

[0141] The value of abs_delta_poc_st[listIdx][rplsIdx][i] can range from 0 to 215-1.

[0142] If strp_entry_sign_flag[listIdx][rplsIdx][i] is 1, it means that the i-th entry in the syntax structure ref_pic_list_struct(listIdx,rplsIdx) has a value greater than or equal to 0. If strp_entry_sign_flag[listIdx][rplsIdx][i] is 0, it means that the i-th entry in the syntax structure ref_pic_list_struct(listIdx,rplsIdx) has a value less than 0. If it does not exist, the value of strp_entry_sign_flag[i][j] is inferred to be 1.

[0143] poc_lsb_lt[listIdx][rplsIdx][i] represents the modulo MaxPicOrderCntLsb of the picture order count of the picture referenced by the i-th entry in the syntax structure ref_pic_list_struct(listIdx,rplsIdx). The length of the syntax element poc_lsb_lt[listIdx][rplsIdx][i] is log2_max_pic_order_cnt_lsb_minus4+4 bits.

[0144] The explanation of syntax elements mentioned above can be summarized as shown in Table 2 below.

[0145] [Table 2]

[0146] Referring to Figure 12, the decoding device 200 checks whether LTRP is used in the entry of the reference picture list via long_term_ref_pics_flag (S1205). If LTRP is used (long_term_ref_pics_flag is 1), it parses st_ref_pic_flag (S1210) and checks whether st_ref_pic_flag is true (S1215). If st_ref_pic_flag is true (1) (i.e., the current entry is a STRP), the decoding device 200 parses abs_delta_poc_st. That is, if the current entry is a STRP entry, the decoding device 200 obtains the difference value of the POC between the picture and the previous picture for the current entry.

[0147] If long_term_ref_pics_flag is false at step 1205, the decoding device immediately parses abs_delta_poc_st. Subsequently, the decoding device 200 checks whether abs_delta_poc_st is greater than 0 (S1225), and if abs_delta_poc_st is greater than 0, it parses strp_entry_sign_flag (S1230).

[0148] If st_ref_pic_flag is false (0) at step S1215, the decoding device 200 parses poc_lsb_lt (S1235). That is, if the current entry is an LTRP entry, the decoding device 200 obtains the POC modulo value of the picture for the current entry. The POC modulo value of a picture is a value used to derive the POC value of the picture, and can be used in combination with the POC MSB (most significant bit) to derive the POC value of the picture.

[0149] The process of constructing the reference picture list described above can be represented by two blocks, as shown in Figure 13. The operation of each block can be implemented by the same hardware or by different hardware. The decoding device 200 obtains the number of entries (num_ref_entries) and a flag (long_term_ref_pic_flag) indicating whether or not a long-term reference picture is used (1310), and can determine whether or not each entry is LTRP or STRP (1320).

[0150] Example 2

[0151] In this embodiment, we propose limiting the parameter num_ref_entries, as described in Table 1. Typically, num_ref_entries[listIdx][rpIsIdx] represents the number of entries in ref_pic_list_struct(listIdx,rpIsIdx). It is observed that the value of num_ref_entries can range from 0 to sps_max_dec_pic_buffering_minus1+14. A limit on the range of num_ref_entries may be introduced to ensure that fewer frames are used. The worst-case scenario is requiring 14 more frames than the value indicated by sps_max_dec_pic_buffering_minus1. This means that the decoded picture buffer must have sufficient memory to accommodate so many frames. Alternatively, a limit may be introduced for hierarchical groups of structres (GOPs), taking into account the layer's temporalId.

[0152] Further syntax can be defined as shown in Table 3. sps_max_dec_pic_buffering_minus1 plus 1 represents the maximum requested size of one decoded picture buffer in CVS in units of picture storage buffers. The value of sps_max_dec_pic_buffering_minus1 can be between 0 and MaxDpbSize-1, where MaxDpbSize can be defined separately.

[0153] [Table 3]

[0154] In other words, the number of entries in the reference picture list can have a maximum value equal to the maximum requested size of the decoded picture buffer plus 14. In addition, according to this embodiment (Embodiment 2), the number of entries in the reference picture list can have a maximum value equal to the maximum requested size of the decoded picture buffer.

[0155] Example 3

[0156] In this embodiment, a method is proposed to remove the syntax element long_term_ref_pics_flag from ref_pic_list_struct(listIdx,rpIsIdx). A value of 0 for this syntax element (long_term_ref_pics_flag) indicates that LTRP will no longer be used for inter-prediction of a coded picture in CVS. A value of 1 for long_term_ref_pics_flag indicates that LTRP will be used for inter-prediction of one or more coded filters in CVS. By removing the syntax element long_term_ref_pics_flag, it can be inferred that LTRP is essentially used in CVS. This is useful because LTRP is typically a high-confidence picture in DPB.

[0157] Example 4

[0158] Figure 14 shows another example of a flowchart for marking reference pictures according to the embodiments of this specification.

[0159] Another embodiment of ref_pic_list_struct(listIdx,rpIsIdx) is provided. In particular, ref_pic_list_struct(listIdx,rpIsIdx) can be configured to take into account the case where LTRP and STRP are signaled separately. This can be expressed as shown in Table 4 below. Herein, the embodiments herein provide a method for limiting num_strp_entries[listIdx][rpIsIdx] representing the total number of STRPs used and num_ltrp_entries[listIdx][rpIsIdx] representing the total number of LTRPs.

[0160] Figure 14 outlines the proposed structure in which the number of LTRPs and STRPs are signaled separately. Here, the number of STRPs and LTRPs are first parsed using the LTRPs indicated by the long_term_ref_pics_flag. Subsequently, for each entry, the lt_ref_pic_flag is parsed, which indicates whether the picture is an LTRP or not. If lt_ref_pic_flag is true, the counter for LTRPs is incremented. Otherwise, the counter for STRPs is incremented. Furthermore, if lt_ref_pic_flag is false, STRPs are confirmed by parsing the delta_poc_st and its sign information (if delta_poc_st is greater than 0). Otherwise, LTRPs are confirmed by their LSB POC.

[0161] Table 4 below shows an example of syntax for configuring the reference picture list according to this embodiment.

[0162] [Table 4]

[0163] num_strp_entries[listIdx][rplsIdx] represents the number of STRP entries in ref_pic_list_struct(listIdx,rplsIdx).

[0164] num_ltrp_entries[listIdx][rplsIdx] represents the number of LTRP entries using ref_pic_list_struct(listIdx,rplsIdx).

[0165] If lt_ref_pic_flag[listIdx][rplsIdx] does not exist, the value of lt_ref_pic_flag[listIdx][rplsIdx] is inferred as follows:

[0166] If num_ltrp_entries[listIdx][rplsIdx] is 0, then its value (lt_ref_pic_flag[listIdx][rplsIdx]) is inferred to be 0. Otherwise, numLtrps (number of LTRPs) is inferred to be 0 when it is like num_ltrp_entries[listIdx][rplsIdx], and numStrps (number of STRPs) is inferred to be 0 when it is like num_strp_entries[listIdx][rplsIdx].

[0167] [Table 5]

[0168] Referring to Figure 14, the decoding device 200 starts constructing the reference list (S1405) and parses the number of STRP entries (num_strp_entries) (S1410). The decoding device 200 checks whether LTRP is used (S1415), and if LTRP is used, parses the number of LTRPs (num_ltrp_entries) (S1420). The decoding device 200 initializes the variables i, numLtrps, and numStrps to 0 (S1425), determines whether the variable i is equal to or less than the number of entries (num_ref_entires) (S1430), and if i is equal to or less than the number of entries (num_ref_entires), performs the reference picture marking operation (S1435) and updates i (S1440).

[0169] Figure 15 shows an example flowchart for processing a video signal according to the implementation of this specification. The operation in Figure 15 is performed by the decoding device 200.

[0170] At step S1510, the decoding device 200 constructs a reference picture list of the current picture using the video signal. Here, if the first entry in the reference picture list corresponds to a STRP entry, the decoding device 200 obtains the difference value of the POC between the picture of the first entry and the previous picture (e.g., abs_delta_poc_st), and if the second entry in the reference picture list corresponds to an LTRP entry, it obtains the modulo value of the POC of the picture of the second entry (e.g., poc_lsb_lt).

[0171] In one embodiment, during the process of constructing the reference picture list, the decoding device 200 obtains the number of entries in the reference picture list (num_ref_entries), and can mark the reference picture in the LTRP or STRP for each of the entries in the reference picture list, up to the number of entries.

[0172] In one embodiment, the number of entries may have a maximum value of either the maximum requested size of the decoded picture buffer plus 14 (sps_max_dec_pic_buffering_minus1+14) or the maximum requested size of the decoded picture buffer (sps_max_dec_pic_buffering_minus1).

[0173] In one embodiment, during the process of marking a reference picture, the decoding device 200 obtains the number of STRP entries (e.g., num_strp_entries) and the number of LTRP entries (e.g., num_ltrp_entries) as shown in Table 4, and can perform STRP marking for the number of STRP entries and LTRP marking for the number of LTRP entries.

[0174] In one embodiment, during the process of constructing the reference picture list, the decoding device 200 can acquire a first flag and determine whether an entry in the reference picture list is a STRP entry or an LTRP entry based on the first flag. For example, the decoding device 200 can use a flag (e.g., st_ref_pic_flag) that indicates whether the picture relating to the current entry is a STRP entry to determine whether the current entry is a STRP entry or an LTRP entry.

[0175] In one embodiment, the difference value of the POC represents the absolute value obtained by subtracting the POC of the picture relating to the first entry from the POC of the previous picture. Furthermore, if the absolute value (absolute difference of POCs) (e.g., abs_delta_poc_st) is greater than 0, the decoding device 200 can obtain a second flag (e.g., strp_entry_sign_flag) that indicates the sign of the difference value of the POC.

[0176] At step S1520, the decoding device 200 makes a prediction for the current picture using the reference picture list. For example, the decoding device 200 can generate a prediction sample using the sample values ​​indicated by the motion vector in the reference picture contained in the first reference picture list L0 or the second reference picture list L1.

[0177] Embodiments of this specification provide a method and apparatus for encoding information necessary to constitute a reference picture list. The encoding apparatus 100 can constitute a reference picture list and make predictions using the reference pictures in the reference picture list.

[0178] Furthermore, the encoding device 100 can encode information for each entry in the reference picture list to constitute the reference picture list. For example, the information for structuring the reference picture list includes the difference value of the POC between the picture corresponding to STRP and the previous picture (e.g., abs_delta_poc_st), and the modulo value of the POC of the picture corresponding to LTRP (e.g., poc_lsb_lt).

[0179] Furthermore, the encoding device 100 can encode a first flag (e.g., st_ref_pic_flag) in the information for constructing the reference picture list to indicate whether the picture of an entry in the reference picture list is STRP or LTRP. In addition, if the difference value of the POC (e.g., abs_delta_poc_st) is greater than 0, a second flag (e.g., strp_entry_sign_flag) can be included to indicate the sign of the difference value of the POC. The information for constructing the reference picture list can be signaled by being included in the SPS, PPS, or slice header.

[0180] The processing methods to which this specification applies can be produced in the form of programs executed on a computer and stored on a computer-readable recording medium. Multimedia data having the data structures described herein can also be stored on a computer-readable recording medium. The computer-readable recording medium includes all types of storage devices and distributed storage devices on which computer-readable data is stored. The computer-readable recording medium can include, for example, Blu-ray discs (BDs), general-purpose serial buses (USBs), ROMs, PROMs, EPROMs, EEPROMs, RAMs, CD-ROMs, magnetic tapes, floppy disks, and optical data storage devices. The computer-readable recording medium also includes media implemented in the form of carrier waves (e.g., transmission over the Internet). Furthermore, a bitstream generated by an encoding method can be stored on a computer-readable recording medium or transmitted over a wireless network.

[0181] Furthermore, the embodiments of this specification can be implemented as computer program products comprising program code, the program code being executed on a computer according to the embodiments of this specification. The program code being stored on a computer-readable carrier.

[0182] Decoding and encoding devices to which this specification applies may be included in digital devices. A "digital device" includes, for example, all digital devices capable of transmitting, receiving, processing, and outputting data, content, services, etc. Processing data, content, services, etc. by a digital device includes encoding and / or decoding data, content, services, etc. Such digital devices may pair or connect (hereinafter referred to as "pairing") with other digital devices, external servers, etc. via a wire / wireless network to send and receive data and convert it as necessary.

[0183] Digital devices include both standing devices such as network TVs, HBB TVs (Hybrid Broadcast Broadband TVs), smart TVs, IPTVs (Internet Protocol Televisions), and PCs (Personal Computers), and mobile devices (or handheld devices) such as PDAs (Personal Digital Assistants), smartphones, tablet PCs, and laptop computers.

[0184] On the other hand, as used herein, “wired / wireless network” refers to a communication network that supports various communication standards or protocols for interconnection and / or transmission and reception of data between digital devices or between digital devices and external servers. Such wired / wireless networks can include all communication networks and communication protocols currently or in the future supported by standards, but can be formed by communication standards or protocols for wired connections such as USB (Universal Serial Bus), CVBS (Composite Video Banking Sync), Component, S-Video (analog), DVI (Digital Visual Interface), HDMI (High Definition Multimedia Interface), RGB, and D-SUB, and communication standards for wireless connections such as Bluetooth, RFID (Radio Frequency Identification), infrared communication (IrDA, infrared Data Association), UWB (Ultra Wideband), ZigBee, DLNA (Digital Living Network Alliance), WLAN (Wireless LAN) (Wi-Fi), Wibro (Wireless broadband), WiMAX (World Interoperability for Microwave Access), HSDPA (High Speed ​​Downlink Packet Access), LTE (Long Term Evolution), and Wi-Fi Direct.

[0185] The embodiments described above are combinations of the components and features of this specification in a predetermined form. Each component or feature should be considered optional unless otherwise explicitly mentioned. Each component or feature may be implemented in a form that is not combined with other components or features. It is also possible to combine some components and / or features to constitute the embodiments described herein. The order of operations described in the embodiments described herein may be changed. Some components or features of one embodiment may be included in other embodiments, or may be substituted for corresponding components or features of other embodiments. It is obvious that claims that are not explicitly referenced in the claims may be combined to constitute embodiments, or may be included in new claims by amendment after filing.

[0186] In the case of implementation using firmware or software, one embodiment of this specification may be implemented in the form of a module, procedure, function, etc., that performs the functions or operations described above. The software code may be stored in memory and driven by a processor. The memory may be located inside or outside the processor and may exchange data with the processor by various already known means.

[0187] It will be obvious to those skilled in the art that this specification can be embodied in other specific forms, without deviating from its essential features. Therefore, the detailed description set forth herein should not be constrained in any way, but should be considered illustrative. The scope of this specification shall be determined by a reasonable interpretation of the appended claims, and all modifications within the equivalent scope of this specification are included within its scope. [Industrial applicability]

[0188] The preferred embodiments of the present invention described above are disclosed for illustrative purposes only, and those skilled in the art can improve, modify, substitute, or add various other embodiments within the technical concept and scope of the present invention disclosed in the appended claims.

Claims

1. A device for decoding video signals, The device described above, A memory for storing the aforementioned video signal, Includes a processor connected to the aforementioned memory, The aforementioned processor, To obtain the number of entries included in the current picture's reference picture list, The number of entries in the aforementioned reference picture list is to be marked as STRP or LTRP for each of the aforementioned entries, It is configured to derive a predicted sample for the current picture based on the aforementioned reference picture list, The device has a maximum number of entries equal to the maximum requested size of the decoded picture buffer plus 13.

2. A processor configured to mark each of the aforementioned entries, Obtain the difference value of POC between the reference picture associated with the entry marked as STRP and other pictures in the reference picture list, The device according to claim 1, further configured to obtain the modulo value of the POC of a picture associated with the entry marked as LTRP.

3. The device according to claim 2, wherein the difference value of the POC is the absolute value of the POC of the other picture obtained by subtracting the POC of the reference picture associated with the entry marked as STRP.

4. The device according to claim 3, wherein the processor configured to mark each of the entries is further configured to acquire a flag indicating the sign of the difference value of the POC, based on the absolute value being greater than zero.

5. The device according to claim 2, wherein, in each marking of the entry in the reference picture, the other picture is the current picture based on the fact that the entry marked as STRP is the first STRP entry, or, in each marking of the entry in the reference picture, the other picture is a previous STRP in the reference picture list based on the fact that the entry marked as STRP is not the first STRP.

6. A processor configured to mark each of the aforementioned entries, Obtain a flag containing information about whether the entry in the aforementioned reference picture list corresponds to an STRP entry or an LTRP entry, The device according to claim 1, further configured to determine, based on the flag, whether the entry in the reference picture list corresponds to the STRP entry or the LTRP entry.

7. A device for encoding video signals, The device described above, A memory for storing the aforementioned video signal, Includes a processor connected to the aforementioned memory, The aforementioned processor, To configure the current picture's reference picture list, It is configured to generate code information for constructing the aforementioned reference picture list, The code information includes the number of entries in the reference picture list. The device has a maximum number of entries equal to the maximum requested size of the decoded picture buffer plus 13.

8. The aforementioned code information is, The difference in POC between the reference picture associated with the entry marked as STRP and other pictures in the reference picture list, The device according to claim 7, further comprising: the modulo value of the POC of the picture associated with the entry marked as LTRP in the aforementioned reference picture list.

9. The device according to claim 8, wherein the difference value of the POC is the absolute value of the POC of the other picture obtained by subtracting the POC of the reference picture associated with the entry marked as STRP.

10. The device according to claim 9, wherein the code information further includes a flag indicating the sign of the difference value of the POC for entries where the absolute value of the difference value of the POC is greater than 0.

11. The device according to claim 8, wherein the other picture is the current picture based on the fact that the entry marked as STRP is the first STRP entry in the code information, or the other picture is the previous STRP in the reference picture list based on the fact that the entry marked as STRP is not the first STRP in the code information.

12. The device according to claim 7, wherein the code information further includes a flag containing information on whether or not an entry in the reference picture list corresponds to an STRP entry or an LTRP entry.

13. The aforementioned processor, The device according to claim 7, further configured to load the code information into a sequence parameter set (SPS), picture parameter set (PPS), or slice header in a bitstream related to the video signal.

14. A computer-readable storage medium in which computer programs and bitstreams are stored, When the computer program is executed by the processor, the following steps are performed to generate the bitstream: The above step involves configuring the reference picture list for the current picture, This includes generating code information for constructing the aforementioned reference picture list, The code information includes the number of entries in the reference picture list. A computer-readable storage medium having a maximum number of entries equal to the maximum requested size of the decoded picture buffer plus 13.

Images