Video decoding method and apparatus

Abstract

The present invention relates to a video decoding method performed by a decoding device. [Solution] The video decoding method performed by the decoding device according to this document is characterized by including the steps of: deriving a value of a variable based on whether the current picture is the first picture of the current AU (Access Unit) which is a CVSS AU (Coded Video Sequence Start Access Unit) other than AU 0, wherein the variable indicates whether all picture storage buffers in the DPB (Decoded Picture Buffer) are empty without output; updating the DPB based on the variable; and decoding the current picture based on the updated DPB.

Description

Technical Field

[0001] This document relates to video coding technology, and more particularly, to a video decoding method and apparatus for performing a DPB management process in a video coding system.

Background Art

[0002] In recent years, the demand for high-resolution and high-quality videos such as HD (High Definition) videos and UHD (Ultra High Definition) videos has been increasing in various fields. As video data becomes higher in resolution and quality, the amount of information or bits to be transmitted increases compared to existing video data. Therefore, when transmitting video data using a medium such as an existing wired or wireless broadband line or storing video data using an existing storage medium, the costs associated with transmission and storage increase.

[0003] Therefore, in order to effectively transmit, store, and reproduce information of high-resolution and high-quality videos, a highly efficient video compression technology is required.

Summary of the Invention

Problems to be Solved by the Invention

[0004] The technical problem of this document is to provide a method and apparatus for increasing video coding efficiency.

[0005] Another technical problem of this document is to provide a method and apparatus for performing a DPB management process.

Means for Solving the Problems

[0006] According to one embodiment of this document, a video decoding method performed by a decoding device is provided. The method is characterized by including the steps of: deriving a variable value based on whether the current picture is the first picture of the current AU (Access Unit) which is a CVSS AU (Coded Video Sequence Start Access Unit) other than AU 0, the variable indicating whether all picture storage buffers in the DPB (Decoded Picture Buffer) are empty without output; updating the DPB based on the variable; and decoding the current picture based on the updated DPB.

[0007] According to other embodiments of this document, a decoding device for video decoding is provided. The decoding device derives a variable value based on whether the current picture is the first picture of the current AU (Access Unit) which is a CVSS AU (Coded Video Sequence Start Access Unit) other than AU 0, the variable indicates whether all picture storage buffers in the DPB (Decoded Picture Buffer) are empty without output, and includes a DPB that updates the DPB based on the variable, and a prediction unit that decodes the current picture based on the updated DPB.

[0008] Further embodiments of this document provide a video encoding method performed by an encoding device. The method is characterized by including the steps of: deriving a value of a variable based on whether the current picture is the first picture of the current AU (Access Unit) which is a CVSS AU (Coded Video Sequence Start Access Unit) other than AU 0, the variable indicating whether all picture storage buffers in the DPB (Decoded Picture Buffer) are empty without output; updating the DPB based on the variable; and encoding video information for the current picture.

[0009] Further embodiments of this document provide a video encoding device. The encoding device derives a variable value based on whether the current picture is the first picture of the current AU (Access Unit) which is a CVSS AU (Coded Video Sequence Start Access Unit) other than AU 0, the variable indicates whether all picture storage buffers in the DPB (Decoded Picture Buffer) are empty without output, and includes a DPB that updates the DPB based on the variable, and an entropy encoding unit that encodes video information for the current picture.

[0010] Further embodiments of this document provide a computer-readable digital storage medium storing a bitstream containing video information for performing a video decoding method. In the computer-readable digital storage medium, the video decoding method is characterized by deriving a value of a variable based on whether the current picture is the first picture of the current AU (Access Unit) which is a CVSS AU (Coded Video Sequence Start Access Unit) other than AU 0, the variable indicating whether all picture storage buffers in the DPB (Decoded Picture Buffer) are empty without output, updating the DPB based on the variable, and decoding the current picture based on the updated DPB. [Effects of the Invention]

[0011] According to this document, instead of deciding whether or not to remove a picture in the DPB without outputting it before decoding all pictures in CVSS AUs other than AU 0, the decision can be made only before decoding the first picture in a CVSS AU other than AU 0. This eliminates the need to change the DPB state for each picture, which affects all layers in the CVS, thereby improving coding efficiency.

[0012] According to this document, the variable indicating whether or not to remove a picture in the DPB without outputting it can be derived only before decoding the first picture in a CVSS AU other than AU 0, rather than before decoding all pictures in CVSS AUs other than AU 0. This eliminates the need to change the DPB state, which affects all layers in CVS, for each picture, thereby improving coding efficiency. [Brief explanation of the drawing]

[0013] [Figure 1]This diagram schematically illustrates an example of a video / image coding system to which the embodiments described in this document can be applied. [Figure 2] This figure schematically illustrates the configuration of a video / image encoding device to which the embodiments described in this document can be applied. [Figure 3] This figure schematically illustrates the configuration of a video / image decoding device to which the embodiments described in this document can be applied. [Figure 4] This figure illustrates the encoding procedure according to the embodiment described in this document. [Figure 5] This figure illustrates the decoding procedure according to the embodiment described in this document. [Figure 6] This diagram schematically illustrates the video encoding method using the encoding device described in this document. [Figure 7] This diagram schematically shows an encoding device that performs the video encoding method described in this document. [Figure 8] This diagram schematically illustrates the video decoding method using the decoding device described in this document. [Figure 9] This diagram schematically shows a decoding device that performs the video decoding method described in this document. [Figure 10] This is a structural diagram of a content streaming system to which the examples in this document are applied. [Modes for carrying out the invention]

[0014] This document can be modified in various ways and can have various embodiments. Specific embodiments are illustrated in the drawings and described in detail. However, this is not intended to limit the embodiments of this document to specific embodiments. The terms commonly used in this specification are only used to describe specific embodiments and are not intended to limit the technical idea of this document. Singular expressions include plural expressions unless otherwise specified in the context. In this specification, terms such as "including" or "having" are used to specify the existence of features, numbers, steps, operations, components, parts, or combinations thereof described in the specification, and should be understood not to preclude in advance the possibility of the existence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

[0015] On the other hand, each configuration on the drawings described in this document is shown independently for the convenience of explaining different characteristic functions, and does not mean that each configuration is implemented as separate hardware or separate software. For example, two or more of the configurations may be combined to form one configuration, or one configuration may be divided into multiple configurations. Embodiments in which each configuration is integrated and / or separated are also included in the scope of rights of this document as long as they do not depart from the essence of this document.

[0016] Hereinafter, with reference to the accompanying drawings, preferred embodiments of this document will be described in more detail. Hereinafter, the same reference numerals are assigned to the same components on the drawings, and duplicate descriptions of the same components are omitted.

[0017] FIG. 1 schematically shows an example of a video / video coding system to which embodiments of this document are applicable.

[0018] Referring to FIG. 1, a video / image coding system can include a first device (source device) and a second device (receiving device). The source device can transmit encoded video / image information or data to the receiving device via a digital storage medium or a network in the form of a file or a stream.

[0019] The source device can include a video source, an encoding device, and a transmitting unit. The receiving device can include a receiving unit, a decoding device, and a renderer. The encoding device may be referred to as a video / image encoding device, and the decoding device may be referred to as a video / image decoding device. A transmitter may be included in the encoding device. A receiver may be included in the decoding device. The renderer can also include a display unit, and the display unit may be configured as a separate device or an external component.

[0020] The video source can obtain video / images through processes such as video / image capture, synthesis, or generation. The video source can include a video / image capture device and / or a video / image generation device. The video / image capture device can include, for example, one or more cameras, a video / image archive including previously captured video / images, etc. The video / image generation device can include, for example, a computer, a tablet, and a smartphone, etc., and can (electronically) generate video / images. For example, virtual video / images may be generated by a computer or the like, and in this case, the video / image capture process may be replaced by the process of generating related data.

[0021] An encoding device can encode input video / image data. The encoding device can perform a series of steps, such as prediction, transformation, and quantization, for compression and coding efficiency. The encoded data (encoded video / image information) may be output in the form of a bitstream.

[0022] The transmitting unit can transmit encoded video / image information or data output in the form of a bitstream to the receiving unit of a receiving device via a digital storage medium or network in the form of a file or streaming. The digital storage medium can include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, and SSD. The transmitting unit may include elements for generating media files in a predetermined file format and may include elements for transmission via a broadcast / communication network. The receiving unit can receive / extract the bitstream and transmit it to a decoding device.

[0023] A decoding device can decode video / images by performing a series of steps, such as inverse quantization, inverse transformation, and prediction, corresponding to the operation of an encoding device.

[0024] The renderer can render the decoded video / image. The rendered video / image may be displayed on the display unit.

[0025] This document relates to video / image coding. For example, the methods / examples disclosed in this document may be applied to methods disclosed in the VVC (versatile video coding) standard, EVC (essential video coding) standard, AV1 (AOMedia Video1) standard, AVS2 (2nd generation of audio video coding standard), or next-generation video / image coding standards (e.g., H.267 or H.268).

[0026] This document presents various embodiments of video / image coding, and unless otherwise specified, these embodiments may be combined with each other.

[0027] In this document, "video" can mean a collection of images over time. "Picture" generally refers to a unit representing a single image at a specific time point, while "subpicture," "slice," and "tile" are units that constitute part of a picture in coding. A subpicture, slice, or tile can contain one or more coding tree units (CTUs). A single picture may consist of one or more subpictures, slices, or tiles. A single picture may consist of one or more tile groups. A tile group can contain one or more tiles. A brick can represent a rectangular area of ​​a CTU row within a picture tile. A tile may be partitioned into multiple bricks, each brick may consist of one or more CTU rows. A tile that is not partitioned into multiple bricks can also be called a brick. A brick scan can represent a specific sequential ordering of CTUs that partition a picture, where these CTUs may be aligned in a CTU raster scan within a brick, where bricks in a tile may be aligned consecutively in a raster scan of the bricks in the tile, and where tiles in a picture may be aligned consecutively in a raster scan of the tiles in the picture. A subpicture can represent a rectangular area of ​​one or more slices within a picture; that is, a subpicture may include one or more slices that collectively cover a rectangular area of ​​the picture. A tile is a rectangular area of ​​CTUs within a specific tile row and a specific tile column in a picture. The tile column is a rectangular area of ​​CTUs, where the rectangular area has the same height as the height of the picture, and its width may be specified by syntax elements in the picture parameter set. The tile row is a rectangular area of ​​CTUs, where the rectangular area has the width specified by syntax elements in the picture parameter set, and its height may be the same as the height of the picture.A tile scan can represent a specific sequential ordering of CTUs that partition a picture, wherein the CTUs may be aligned sequentially within the tile by a CTU raster scan, and the tiles within the picture may be aligned sequentially by a raster scan of the tiles in the picture. A slice may contain an integer number of bricks of a picture, wherein the integer number of bricks may be contained in a single NAL unit. A slice may consist of multiple complete tiles, or a sequential sequence of complete bricks of a single tile. In this document, tile group and slice may be used interchangeably. For example, in this document, tile group / tile group header may be referred to as slice / slice header.

[0028] A pixel or PEL (pel) can refer to the smallest unit that makes up a picture (or image). Alternatively, the term "sample" may be used as a counterpart to "pixel." A sample can generally represent a pixel or a pixel value, and may represent only the luma component pixel / pixel value, or only the chroma component pixel / pixel value.

[0029] A unit can represent a basic unit of image processing. A unit can contain at least one of the following: a specific region of a picture and information associated with that region. A unit can contain one luma block and two chroma (e.g., cb, cr) blocks. The term unit may be used interchangeably with terms such as block or area. In general, an MxN block can contain a sample (or sample array) or a set (or array) of transform coefficients consisting of M columns and N rows.

[0030] In this specification, "A or B" can mean "A only," "B only," or "both A and B." In other words, in this specification, "A or B" can be interpreted as "A and / or B." For example, in this specification, "A, B or C" can mean "A only," "B only," "C only," or "any combination of A, B and C."

[0031] As used herein, slashes ( / ) and commas can mean "and / or". For example, "A / B" can mean "A and / or B". Therefore, "A / B" can mean "A only", "B only", or "both A and B". For example, "A, B, C" can mean "A, B or C".

[0032] In this specification, "at least one of A and B" can mean "A only," "B only," or "both A and B." Furthermore, in this specification, the expressions "at least one of A or B" and "at least one of A and / or B" may be interpreted as equivalent to "at least one of A and B."

[0033] Furthermore, in this specification, "at least one of A, B and C" can mean "A only," "B only," "C only," or "any combination of A, B and C." Also, "at least one of A, B or C" or "at least one of A, B and / or C" can mean "at least one of A, B and C."

[0034] Furthermore, the parentheses used in this specification can mean "for example." Specifically, when "prediction (intra prediction)" is displayed, "intra prediction" may be proposed as an example of "prediction." In other words, "prediction" in this specification is not limited to "intra prediction," and "intra prediction" may be proposed as an example of "prediction." Similarly, when "prediction (i.e., intra prediction)" is displayed, "intra prediction" may be proposed as an example of "prediction."

[0035] In this specification, technical features described individually in a single drawing may be embodied individually or simultaneously.

[0036] The following drawings are provided to illustrate a specific example of this specification. The names of specific devices or signals / messages / fields shown in the drawings are presented illustratively, and the technical features of this specification are not limited to the specific names shown in the following drawings.

[0037] Figure 2 is a schematic diagram illustrating the configuration of a video / image encoding device to which the embodiments described in this document can be applied. Hereinafter, the term "video encoding device" may include an image encoding device.

[0038] Referring to Figure 2, the encoding device 200 may be configured to include an image partitioner (210), a predictor (220), a residual processor (230), an entropy encoder (240), an adder (250), a filter (260), and a memory (270). The predictor (220) may include an inter-predictor (221) and an intra-predictor (222). The residual processor (230) may include a transformer (232), a quantizer (233), a dequantizer (234), and an inverse transformer (235). The residual processor (230) may further include a subtractor (231). The addition unit 250 may also be called a reconstructor or a reconstructed block generator. The video splitting unit 210, prediction unit 220, residual processing unit 230, entropy encoding unit 240, addition unit 250, and filtering unit 260 described above may be composed of one or more hardware components (e.g., an encoder chipset or processor) depending on the embodiment. The memory 270 may also include a DPB (decoded picture buffer) and may be composed of a digital storage medium. The hardware components may further include the memory 270 as an internal / external component.

[0039] The video splitting unit 210 can split the input video (or picture, frame) input to the encoding device 200 into one or more processing units. For example, the processing units can be called coding units (CUs). In this case, the coding units may be recursively split from a coding tree unit (CTU) or the largest coding unit (LCU) using a QTBTTT (Quad-tree binary-tree ternary-tree) structure. For example, one coding unit may be split into multiple coding units of deeper depth based on a quad-tree structure, a binary tree structure, and / or a Tannery structure. In this case, for example, the quad-tree structure may be applied first, followed by the binary tree structure and / or the Tannery structure. Alternatively, the binary tree structure may be applied first. The coding procedure according to this document may be performed based on the final coding unit that is not further split. In this case, based on coding efficiency due to video characteristics, the largest coding unit may be immediately used as the final coding unit, or, if necessary, the coding unit may be recursively divided into coding units of lower depth, and the coding unit of optimal size may be used as the final coding unit. Here, the coding procedure may include procedures such as prediction, transformation, and reconstruction, which will be described later. As another example, the processing unit may further include a prediction unit (PU) or a transformation unit (TU). In this case, the prediction unit and the transformation unit may be divided or partitioned from the final coding unit described above. The prediction unit may be a unit of sample prediction, and the transformation unit may be a unit that derives transformation coefficients and / or a unit that derives a residual signal from transformation coefficients.

[0040] The term "unit" may, depending on the context, be used interchangeably with terms such as "block" or "area." In general, an MxN block can represent a set of samples or transform coefficients consisting of M columns and N rows. A sample can generally represent a pixel or a pixel value, and may represent only the pixel / pixel value of the lumen component, or only the pixel / pixel value of the chroma component. The term "sample" may be used as a term corresponding to a single picture (or image) for a pixel or pel.

[0041] The encoding device 200 can generate a residual signal (residual block, residual sample array) by subtracting the prediction signal (prediction block, prediction sample array) output from the inter-prediction unit 221 or intra-prediction unit 222 from the input video signal (original block, original sample array), and the generated residual signal is transmitted to the conversion unit 232. In this case, as shown in the figure, the unit in the encoding device 200 that subtracts the prediction signal (prediction block, prediction sample array) from the input video signal (original block, original sample array) may be called the subtraction unit 231. The prediction unit can make predictions for the block to be processed (hereinafter referred to as the current block) and generate a predicted block that includes predicted samples for the current block. The prediction unit can determine whether intra-prediction or inter-prediction is applied on a current block or CU basis. As will be described later in the explanation of each prediction mode, the prediction unit can generate various prediction-related information such as prediction mode information and transmit it to the entropy encoding unit 240. Information regarding the prediction may be encoded by the entropy encoding unit 240 and output in the form of a bitstream.

[0042] The intra-prediction unit 222 can predict the current block by referring to a sample in the current picture. The referenced sample may be located in the vicinity (neighbor) or at a distance from the current block, depending on the prediction mode. In intra-prediction, the prediction mode can include multiple non-directional modes and multiple directional modes. Non-directional modes can include, for example, DC mode and planar mode. Directional modes can include, for example, 33 directional prediction modes or 65 directional prediction modes, depending on the degree of detail of the prediction direction. However, this is an example, and more or fewer directional prediction modes may be used depending on the settings. The intra-prediction unit 222 can also determine the prediction mode to be applied to the current block using the prediction modes applied to the surrounding blocks.

[0043] The interprediction unit 221 can guide a predicted block for the current block based on a reference block (reference sample array) identified by a motion vector on the reference picture. At this time, in order to reduce the amount of motion information transmitted in interprediction mode, motion information can be predicted in units of blocks, subblocks, or samples based on the correlation of motion information between the surrounding block and the current block. The motion information may include a motion vector and a reference picture index. The motion information may further include interprediction direction information (L0 prediction, L1 prediction, Bi prediction, etc.). In interprediction, the surrounding block may include a spatial neighboring block existing in the current picture and a temporal neighboring block existing in the reference picture. The reference picture containing the reference block and the reference picture containing the temporal neighboring block may be the same or different. The temporal neighboring block may also be called a collocated reference block, colCU, etc., and the reference picture containing the temporal neighboring block may also be called a collocated picture (colPic). For example, the interpretation unit 221 can construct a motion information candidate list based on surrounding blocks and generate information indicating which candidate is used to derive the motion vector and / or reference picture index of the current block. Interpretation may be performed based on various prediction modes; for example, in skip mode and merge mode, the interpretation unit 221 can use the motion information of surrounding blocks as the motion information of the current block. In skip mode, unlike merge mode, the residual signal does not need to be transmitted.In motion vector prediction (MVP) mode, the motion vectors of surrounding blocks are used as motion vector predictors, and the motion vector difference is signaled to indicate the motion vector of the current block.

[0044] The prediction unit 220 can generate prediction signals based on various prediction methods described later. For example, the prediction unit can apply intra-prediction or inter-prediction for prediction of a single block, or it can apply intra-prediction and inter-prediction simultaneously. This can be called CIIP (combined inter and intra prediction). The prediction unit may also be based on an intra-block copy (IBC) prediction mode or a palette mode for prediction of a block. The IBC prediction mode or palette mode may be used for coding content video / moving images such as games, for example, in SCC (screen content coding). IBC basically performs prediction within the current picture, but it may be performed similarly to inter-prediction in that it derives a reference block within the current picture. That is, IBC can use at least one of the inter-prediction methods described in this document. Palette mode can be considered an example of intra-coding or intra-prediction. When palette mode is applied, in-picture sample values ​​can be signaled based on information about the palette table and palette index.

[0045] The prediction signal generated by the prediction unit (including the inter-prediction unit 221 and / or the intra-prediction unit 222) may be used to generate a reconstructed signal or a residual signal. The transformation unit 232 can generate transformation coefficients by applying a transformation method to the residual signal. For example, the transformation method may include at least one of DCT (Discrete Cosine Transform), DST (Discrete Sine Transform), KLT (Karhunen-Loeve Transform), GBT (Graph-Based Transform), or CNT (Conditionally Non-linear Transform). Here, GBT means a transformation obtained from a graph when the relationship information between pixels is represented by this graph. CNT means a transformation obtained by generating a prediction signal using all previously reconstructed pixels and based on that. Furthermore, the transformation process may be applied to pixel blocks of the same size that are square, or to blocks of variable size other than squares.

[0046] The quantization unit 233 quantizes the conversion coefficients and transmits them to the entropy encoding unit 240, which can encode the quantized signal (information about the quantized conversion coefficients) and output it as a bitstream. The information about the quantized conversion coefficients can be called residual information. The quantization unit 233 can rearrange the block-shaped quantized conversion coefficients into a one-dimensional vector based on the coefficient scan order, and can also generate information about the quantized conversion coefficients based on the one-dimensional vector-shaped quantized conversion coefficients. The entropy encoding unit 240 can perform various encoding methods, such as exponential Golomb, CAVLC (context-adaptive variable length coding), and CABAC (context-adaptive binary arithmetic coding). In addition to the quantized conversion coefficients, the entropy encoding unit 240 can also encode information necessary for video / image restoration (e.g., the values ​​of syntax elements) together with or separately from the quantized conversion coefficients. The encoded information (e.g., encoded video / image information) may be transmitted or stored in the form of a bitstream in units of network abstraction layer (NAL) units. The video / image information may further include information about various parameter sets, such as an adaptation parameter set (APS), picture parameter set (PPS), sequence parameter set (SPS), or video parameter set (VPS). The video / image information may also further include general constraint information. In this document, information and / or syntax elements transmitted / signaled by the encoding device to the decoding device may be included in the video / image information. The video / image information may be encoded by the encoding procedure described above and included in the bitstream.The bitstream may be transmitted over a network or stored in a digital storage medium. Here, the network may include broadcasting networks and / or communication networks, and the digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, and SSD. A transmitting unit (not shown) that transmits the signal output from the entropy encoding unit 240 and / or a storage unit (not shown) that stores it may be configured as an internal / external element of the encoding device 200, or the transmitting unit may be included in the entropy encoding unit 240.

[0047] The quantized conversion coefficients output from the quantization unit 233 may be used to generate a prediction signal. For example, the residual signal (residual block or residual sample) can be reconstructed by applying inverse quantization and inverse transformation to the quantized conversion coefficients in the inverse quantization unit 234 and the inverse transformation unit 235. The adder unit 250 can generate a reconstructed signal (reconstructed picture, reconstructed block, reconstructed sample array) by adding the reconstructed residual signal to the prediction signal output from the inter-prediction unit 221 or the intra-prediction unit 222. When there is no residual for the block to be processed, such as when skip mode is applied, the predicted block may be used as the reconstructed block. The adder unit 250 may be called the reconstruction unit or the reconstructed block generation unit. The generated reconstructed signal may be used for intra-prediction of the next block to be processed in the current picture, or, as described later, may be used for inter-prediction of the next picture after filtering.

[0048] On the other hand, LMCS (luma mapping with chroma scaling) may be applied during the picture encoding and / or restoration process.

[0049] The filtering unit 260 can improve subjective / objective image quality by applying filtering to the restored signal. For example, the filtering unit 260 can apply various filtering methods to the restored picture to generate a modified restored picture, and the modified restored picture can be stored in the memory 270, specifically in the DPB of the memory 270. The various filtering methods can include, for example, deblocking filtering, sample adaptive offset, adaptive loop filter, and bilateral filter. The filtering unit 260 can generate various filtering-related information and transmit it to the entropy encoding unit 240, as will be described later in the description of each filtering method. The filtering-related information may be encoded by the entropy encoding unit 240 and output in the form of a bitstream.

[0050] The corrected restored picture transmitted to memory 270 may be used as a reference picture in the interpretation unit 221. This allows the encoding device to avoid prediction mismatches between the encoding device 200 and the decoding device 300 when interpretation is applied, and also improves encoding efficiency.

[0051] The DPB in memory 270 can store the corrected restored picture for use as a reference picture in the inter-prediction unit 221. Memory 270 can store motion information of blocks from which motion information in the picture has been derived (or encoded) and / or motion information of blocks in the picture that have already been restored. The stored motion information can be transmitted to the inter-prediction unit 221 for use as motion information of spatially surrounding blocks or motion information of temporally surrounding blocks. Memory 270 can store restored samples of blocks that have been restored in the picture and transmit them to the intra-prediction unit 222.

[0052] Figure 3 is a schematic diagram illustrating the configuration of a video / image decoding device to which the embodiments described in this document can be applied.

[0053] Referring to Figure 3, the decoding device 300 may include an entropy decoder (310), a residual processor (320), a predictor (330), an adder (340), a filter (350), and a memory (360). The predictor (330) may include an inter-predictor (331) and an intra-predictor (332). The residual processor (320) may include a dequantizer (321) and an inverse transformer (322). The entropy decoder (310), residual processor (320), predictor (330), adder (340), and filter (350) described above may be configured by a single hardware component (e.g., a decoder chipset or processor) depending on the embodiment. The memory (360) may include a decoded picture buffer (DPB) and may be configured by a digital storage medium. The aforementioned hardware component may further include memory 360 as an internal / external component.

[0054] When a bitstream containing video / image information is input, the decoding device 300 can reconstruct the image in a manner corresponding to the process by which the video / image information was processed in the encoding device shown in Figure 2. For example, the decoding device 300 can derive units / blocks based on block division-related information obtained from the bitstream. The decoding device 300 can perform decoding using the processing units applied in the encoding device. Therefore, the decoding processing unit may be, for example, a coding unit, which may be divided into a quad-tree structure, a binary tree structure, and / or a Tannerly tree structure from a coding tree unit or a maximum coding unit. One or more conversion units may be derived from the coding unit. The reconstructed video signal decoded and output by the decoding device 300 may then be played back by a playback device.

[0055] The decoding device 300 can receive the signal output from the encoding device shown in Figure 2 in the form of a bitstream, and the received signal may be decoded by the entropy decoding unit 310. For example, the entropy decoding unit 310 can parse the bitstream and derive information necessary for video restoration (or picture restoration) (e.g., video / image information). The video / image information may further include information about various parameter sets such as the adaptation parameter set (APS), picture parameter set (PPS), sequence parameter set (SPS), or video parameter set (VPS). The video / image information may also further include general constraint information. The decoding device can decode the picture based on the parameter set information and / or the general constraint information. The signaling / received information and / or syntax elements described later in this document may be decoded by the decoding procedure and obtained from the bitstream. For example, the entropy decoding unit 310 can decode information within a bitstream based on a coding method such as exponential Golomb coding, CAVLC, or CABAC, and output the values ​​of syntax elements necessary for image restoration and the quantized values ​​of conversion coefficients related to the residual. More specifically, the CABAC entropy decoding method receives bins corresponding to each syntax element from the bitstream, determines a context model using the syntax element information to be decoded, the decoding information of the surrounding and decoded blocks, or the symbol / bin information decoded in a previous stage, predicts the probability of bin occurrence based on the determined context model, performs arithmetic decoding of the bins, and generates symbols corresponding to the values ​​of each syntax element. At this time, after determining the context model, the CABAC entropy decoding method can update the context model using the symbol / bin information decoded for the context model of the next symbol / bin.Information related to prediction from the information decoded by the entropy decoding unit 310 is provided to the prediction unit 330 (inter-prediction unit 332 and intra-prediction unit 331), and residual values ​​that have been entropy decoded by the entropy decoding unit 310, i.e., quantized conversion coefficients and related parameter information, may be input to the residual processing unit 320. The residual processing unit 320 can derive residual signals (residual blocks, residual samples, residual sample arrays). In addition, information related to filtering from the information decoded by the entropy decoding unit 310 may be provided to the filtering unit 350. On the other hand, a receiving unit (not shown) that receives signals output from the encoding device may be further configured as an internal / external element of the decoding device 300, or the receiving unit may be a component of the entropy decoding unit 310. On the other hand, the decoding device relating to this document can be called a video / image / picture decoding device, and the decoding device can be distinguished into an information decoder (video / image / picture information decoder) and a sample decoder (video / image / picture sample decoder). The information decoder may include the entropy decoding unit 310, and the sample decoder may include at least one of the inverse quantization unit 321, inverse transformation unit 322, addition unit 340, filtering unit 350, memory 360, inter-prediction unit 332, and intra-prediction unit 331.

[0056] The inverse quantization unit 321 can inverse quantize the quantized transformation coefficients and output the transformation coefficients. The inverse quantization unit 321 can rearrange the quantized transformation coefficients in the form of two-dimensional blocks. In this case, the rearrangement can be performed based on the coefficient scan order performed by the encoding device. The inverse quantization unit 321 can perform inverse quantization on the quantized transformation coefficients using quantization parameters (e.g., quantization step size information) and obtain the transformation coefficients.

[0057] In the inverse conversion unit 322, the conversion coefficients are inversely converted to obtain residual signals (residual blocks, residual sample arrays).

[0058] The prediction unit can make predictions for the current block and generate a predicted block containing prediction samples for the current block. Based on the prediction information output from the entropy decoding unit 310, the prediction unit can determine whether intra-prediction or inter-prediction is applied to the current block and determine a specific intra / inter-prediction mode.

[0059] The prediction unit 320 can generate prediction signals based on various prediction methods described later. For example, the prediction unit can apply intra-prediction or inter-prediction for prediction of a single block, or it can apply intra-prediction and inter-prediction simultaneously. This can be called CIIP (combined inter and intra prediction) mode. The prediction unit may also be based on intra-block copy (IBC) prediction mode or palette mode for prediction of a block. The IBC prediction mode or palette mode may be used for coding content video / moving images such as games, for example, as in SCC (screen content coding). IBC basically performs prediction within the current picture, but it may be performed similarly to inter-prediction in that it derives a reference block within the current picture. That is, IBC can use at least one of the inter-prediction methods described in this document. Palette mode can be considered an example of intra-coding or intra-prediction. When palette mode is applied, information regarding the palette table and palette index may be included in the video / image information and signaled.

[0060] The intra-prediction unit 331 can predict the current block by referring to a sample in the current picture. Depending on the prediction mode, the referenced sample may be located in the vicinity (neighbor) of the current block or at a distance. In intra-prediction, the prediction mode may include multiple non-directional modes and multiple directional modes. The intra-prediction unit 331 can also determine the prediction mode to be applied to the current block using the prediction modes applied to the surrounding blocks.

[0061] The interprediction unit 332 can derive a predicted block relative to the current block based on a reference block (reference sample array) identified by motion vectors on the reference picture. In this case, in order to reduce the amount of motion information transmitted in interprediction mode, motion information can be predicted in units of blocks, subblocks, or samples based on the correlation of motion information between surrounding blocks and the current block. The motion information may include motion vectors and reference picture indices. The motion information may further include interprediction direction information (L0 prediction, L1 prediction, Bi prediction, etc.). In interprediction, surrounding blocks may include spatial neighboring blocks present in the current picture and temporal neighboring blocks present in the reference picture. For example, the interprediction unit 332 can construct a motion information candidate list based on surrounding blocks and derive the motion vector and / or reference picture index of the current block based on the received candidate selection information. Interprediction may be performed based on various prediction modes, and the prediction information may include information indicating the interprediction mode for the current block.

[0062] The adder 340 can generate a restored signal (restored picture, restored block, restored sample array) by adding the acquired residual signal to the predicted signal (predicted block, predicted sample array) output from the prediction unit (including the inter-prediction unit 332 and / or intra-prediction unit 331). When there is no residual for the block to be processed, such as when skip mode is applied, the predicted block may be used as the restored block.

[0063] The summing unit 340 can be called a restoration unit or a restoration block generation unit. The generated restoration signal may be used for intra-prediction of the next block to be processed in the current picture, and may be output after filtering as described later, or may be used for intra-prediction of the next picture.

[0064] On the other hand, LMCS (luma mapping with chroma scaling) may be applied during the picture decoding process.

[0065] The filtering unit 350 can apply filtering to the restored signal to improve subjective / objective image quality. For example, the filtering unit 350 can apply various filtering methods to the restored picture to generate a modified restored picture, and transmit the modified restored picture to the memory 360, specifically to the DPB of the memory 360. The various filtering methods may include, for example, deblocking filtering, sample adaptive offset, adaptive loop filter, and bilateral filter.

[0066] The restored picture stored (modified) in the DPB of memory 360 may be used as a reference picture in the inter-prediction unit 332. Memory 360 can store motion information of blocks from which motion information in the current picture has been derived (or decoded), and / or motion information of blocks in the picture that have already been restored. This stored motion information can be transmitted to the inter-prediction unit 260 for use as motion information of spatially surrounding blocks or motion information of temporally surrounding blocks. Memory 360 can store restored samples of restored blocks in the current picture and transmit them to the intra-prediction unit 331.

[0067] In this specification, the embodiments described for the filtering unit 260, inter-prediction unit 221, and intra-prediction unit 222 of the encoding device 200 may be applied identically or in a corresponding manner to the filtering unit 350, inter-prediction unit 332, and intra-prediction unit 331 of the decoding device 300, respectively.

[0068] In this document, at least one of quantization / inverse quantization and / or transformation / inverse transformation may be omitted. When quantization / inverse quantization is omitted, the quantized transformation coefficient may be called a transformation coefficient. When transformation / inverse transformation is omitted, the transformation coefficient may also be called a coefficient or residual coefficient, or may continue to be called a transformation coefficient for consistency of expression.

[0069] Furthermore, in this document, quantized transformation coefficients and transformation coefficients can be referred to as transformation coefficients and scaled transformation coefficients, respectively. In this case, residual information may include information about the transformation coefficients, and such information about the transformation coefficients may be signaled by residual coding syntax. Transformation coefficients may be derived based on the residual information (or information about the transformation coefficients), and scaled transformation coefficients may be derived by an inverse transformation (scaling) of the transformation coefficients. Residual samples may be derived based on an inverse transformation (transformation) of the scaled transformation coefficients. This may be applied / expressed identically in other parts of this document.

[0070] On the other hand, picture output and removal processes may be performed using a DPB (Decoded Picture Buffer). In existing VVC standards for video / image coding systems, the picture output and removal processes using the DPB (Decoded Picture Buffer) may be as shown in the table below.

[0071] [Table 1-1]

[0072] [Table 1-2]

[0073] [Table 1-3]

[0074] For example, according to the VVC standard for video / image coding systems, the picture output process may be invoked once per picture before decoding the current picture (but after parsing the slice header of the first slice of the current picture), as disclosed in the table above.

[0075] Furthermore, as shown in Table 1, if the current AU (Access Unit) is a CVSS AU (Coded Video Sequence Start AU) other than AU 0, the following steps may be applied in this order.

[0076] - Firstly, the variable NoOutputOfPriorPicsFlag may be derived for the decoder under test as follows:

[0077] - If the values ​​of PicWidthMaxInSamplesY, PicHeightMaxInSamplesY, MaxChromaFormat, MaxBitDepthMinus8, or max_dec_pic_buffering_minus1[Htid] derived for the current AU differ from the values ​​of PicWidthMaxInSamplesY, PicHeightMaxInSamplesY, MaxChromaFormat, MaxBitDepthMinus8, or max_dec_pic_buffering_minus1[Htid] derived for a preceding AU, NoOutputOfPriorPicsFlag may be set to equal to 1 by the decoder under test, regardless of the current AU's ph_no_output_of_prior_pics_flag value.

[0078] - Otherwise, NoOutputOfPriorPicsFlag may be set to the same value as the AU's ph_no_output_of_prior_pics_flag.

[0079] - Secondly, the variable NoOutputOfPriorPicsFlag derived for the decoder under test may be applied to the HRD (Hypothetical Reference Decoder). Therefore, when the value of NoOutputOfPriorPicsFlag is 1, all picture storage buffers of the DPB may be empty with no output for the pictures they contain, and the DPB fullness may be set to 0.

[0080] Furthermore, as shown in Table 1, all pictures k in a DPB may be removed from the DPB if all of the following conditions are true for picture k in the DPB.

[0081] - Picture k is displayed as "unused for reference".

[0082] - Picture k has a PictureOutputFlag equal to 0, or the DPB output time of picture k is currently less than or equal to the CPB removal time of the first DU (Decoding Unit) (represented as DU m) of picture n; that is, DpbOutputTime[k] is less than or equal to DuCpbRemovalTime[m].

[0083] Furthermore, as shown in Table 1, if the current AU (Access Unit) is a CVSS AU (Coded Video Sequence Start AU) other than AU 0, the following steps may be applied in this order.

[0084] - Firstly, the variable NoOutputOfPriorPicsFlag may be derived for the decoder under test as follows:

[0085] - If the values ​​of PicWidthMaxInSamplesY, PicHeightMaxInSamplesY, MaxChromaFormat, MaxBitDepthMinus8, or max_dec_pic_buffering_minus1[Htid] derived for the current AU differ from the values ​​of PicWidthMaxInSamplesY, PicHeightMaxInSamplesY, MaxChromaFormat, MaxBitDepthMinus8, or max_dec_pic_buffering_minus1[Htid] derived for a preceding AU, NoOutputOfPriorPicsFlag may be set to equal to 1 by the decoder under test, regardless of the current AU's ph_no_output_of_prior_pics_flag value.

[0086] - Otherwise, NoOutputOfPriorPicsFlag may be set to the same value as the AU's ph_no_output_of_prior_pics_flag.

[0087] - Secondly, the variable NoOutputOfPriorPicsFlag derived for the decoder under test may be applied to the HRD (Hypothetical Reference Decoder) as follows:

[0088] - For example, if the value of NoOutputOfPriorPicsFlag is 1, all picture storage buffers in the DPB may be empty with no output for the pictures they contain, and the DPB fullness may be set to 0.

[0089] - If not (i.e., if the value of NoOutputOfPriorPicsFlag is 0), all picture storage buffers, including those containing pictures marked as "not needed for output" and "unused for reference" in the DPB, may be emptied (without output), and all non-empty picture storage buffers in the DPB may be emptied by repeatedly calling the "bumping" process specified in section C.5.2.4 (of the VVC standard), and the DPB fullness may be set to 0.

[0090] On the other hand, the bumping process may consist of the following steps in this order.

[0091] 1. First, as the picture to be output, the picture with the smallest PicOrderCntVal value among all the pictures in the DPB that are marked as "needed for output" may be selected.

[0092] 2. Each of the pictures may be cropped in ascending order of nuh_layer_id using a conformance cropping window for the picture, the cropped picture may be output, and the picture may be displayed as "not needed for output".

[0093] 3. If "unused for reference" is displayed, each picture memory buffer containing one of the cropped and output pictures will be emptied, and the DPB fullness may decrease by 1.

[0094] Furthermore, as shown in Table 1, if the AU is not currently a CVSS AU, all picture storage buffers, including those containing pictures marked as "not needed for output" and "unused for reference," may be emptied (without output). For each emptied picture storage buffer, the DPB fullness may be decreased by 1. Also, if one or more of the conditions described below are true, the "bumping" process specified in Section C.5.2.4 (of the VVC standard) may be repeatedly called, further decreasing the DPB fullness by 1 for each additional emptied picture storage buffer, until all of the conditions described below are no longer true.

[0095] - The number of pictures in the DPB that are marked "needed for output" is greater than max_num_reorder_pics[Htid].

[0096] - max_latency_increase_plus1[Htid] is not 0, and there is at least one picture in the DPB where the related variable PicLatencyCount, which is marked as "needed for output", is greater than or equal to MaxLatencyPictures[Htid].

[0097] - The number of pictures in the DPB is greater than or equal to max_dec_pic_buffering_minus1[Htid]+1.

[0098] On the other hand, existing VVC standards for the picture output and removal processes described above may have the following problems:

[0099] For example, firstly, after all slices of a picture have been decoded, the picture may be marked as "used for short-term reference." This means that when the picture is decoded, its status in the DPB may not be clear. Consequently, this can affect the number of picture storages in the DPB.

[0100] Secondly, the assignment of a picture's output status (i.e., need for output) may occur during the bumping process. According to existing VVC standards, this process may not be called for pictures in an AU that is a coded video sequence start AU. As a result, the PicLatencyCount value associated with that picture may not be initialized.

[0101] As mentioned above, the process of outputting and removing pictures in the DPB may be invoked only once per picture, but this process may affect the state of the DPB (i.e., the state of pictures stored in the DPB) which is shared by all layers of the CVS. Given the above facts, the process of deriving NoOutputOfPriorPicsFlag and removing pictures from the DPB based on the value of NoOutputOfPriorPicsFlag may be problematic. According to existing video / image standards, a process may be invoked in the DPB to induce and remove pictures based on the value of the flag (i.e., NoOutputOfPriorPicsFlag) for all pictures in CVSS AUs other than AU 0. The process described above may work without problems only for the first picture. The process starting from the second picture may remove previous pictures (i.e., pictures in earlier order in the decoding order) from the DPB before they are output. Such behavior cannot be considered correct decoder behavior.

[0102] Therefore, this document proposes solutions to the problems described above. The proposed embodiments may be applied individually or in combination.

[0103] As an example, the process of deriving the value of a flag or variable indicating whether or not to remove the DPB reference picture from the DPB without outputting it may be called only once per AU (access unit). That is, for example, a method may be proposed in which the process of deriving the value of a flag or variable indicating whether or not to remove the DPB reference picture from the DPB without outputting it may be called only once per AU (access unit). Here, the variable may be NoOutputOfPriorPicsFlag.

[0104] Furthermore, as an example, the process of deriving the value of NoOutputOfPriorPicsFlag may be called in the Coded Video Sequence Start AU (CVSS AU) before the decoding process of the first picture, but after the slice header of the first slice of the current picture has been parsed. That is, for example, it may be proposed that the process of deriving the value of NoOutputOfPriorPicsFlag be performed before the decoding process of the first picture of the CVSS AU, but after the slice header of the first slice of the current picture has been parsed.

[0105] Furthermore, as an example, if NoOutputOfPriorPicsFlag is 1, the process of removing the picture stored in the DPB without outputting it may be called only once per AU. That is, for example, a method may be proposed in which the process of removing the picture stored in the DPB without outputting it may be called only once per AU when NoOutputOfPriorPicsFlag is 1.

[0106] Furthermore, as an example, the process of removing the picture stored in the DPB without outputting it when NoOutputOfPriorPicsFlag is 1 may be called before the decoding process of the first picture in the CVSS AU, but after parsing the slice header of the first slice of the current picture. That is, for example, it may be proposed that the process of removing the picture stored in the DPB without outputting it when NoOutputOfPriorPicsFlag is 1 may be called before the decoding process of the first picture in the CVSS AU, but after parsing the slice header of the first slice of the current picture.

[0107] Furthermore, as an example, the picture removal in the above-described embodiment does not necessarily include the removal of the current picture from the DPB. That is, for example, a method may be proposed in which the picture removal in the above embodiment does not include the removal of the current picture from the DPB.

[0108] The above-described embodiment may be implemented as follows. For example, the above-described embodiment can be shown based on the VVC specification, as will be described later.

[0109] [Table 2-1]

[0110] [Table 2-2]

[0111] [Table 2-3]

[0112] For example, referring to Table 2, if the current picture is the first picture and the current AU (i.e., the AU containing the current picture) is a CVSS AU other than AU 0 (Coded Video Sequence Start AU), the following sequence of steps may be applied.

[0113] - Firstly, the variable NoOutputOfPriorPicsFlag may be derived for the decoder under test as follows:

[0114] - If the values ​​of PicWidthMaxInSamplesY, PicHeightMaxInSamplesY, MaxChromaFormat, MaxBitDepthMinus8, or max_dec_pic_buffering_minus1[Htid] derived for the current AU differ from the values ​​of PicWidthMaxInSamplesY, PicHeightMaxInSamplesY, MaxChromaFormat, MaxBitDepthMinus8, or max_dec_pic_buffering_minus1[Htid] derived for a preceding AU, NoOutputOfPriorPicsFlag may be set to equal to 1 by the decoder under test, regardless of the current AU's ph_no_output_of_prior_pics_flag value.

[0115] - Otherwise, NoOutputOfPriorPicsFlag may be set to the same value as the AU's ph_no_output_of_prior_pics_flag.

[0116] - Secondly, the variable NoOutputOfPriorPicsFlag derived for the decoder under test may be applied to the HRD (Hypothetical Reference Decoder). Therefore, when the value of NoOutputOfPriorPicsFlag is 1, all picture storage buffers of the DPB may be empty with no output for the pictures they contain, and the DPB fullness may be set to 0.

[0117] Furthermore, as shown in Table 2, if the current AU is not a CVSS AU, or if the current AU is a CVSS AU other than AU 0, but the current picture is not the first picture in the current AU, and all of the following conditions are true for picture k of the DPB, then all pictures k of the DPB may be removed from the DPB.

[0118] - Picture k is displayed as "unused for reference".

[0119] - Picture k has a PictureOutputFlag of 0, or the DPB output time of picture k is less than or equal to the CPB removal time of the first DU (represented as DU m) of picture n; that is, DpbOutputTime[k] is less than or equal to DuCpbRemovalTime[m].

[0120] Furthermore, as shown in Table 2, for example, if the current picture is the first picture and the current AU (i.e., the AU containing the current picture) is a CVSS AU (Coded Video Sequence Start AU) other than AU 0, the following sequence of steps may be applied.

[0121] - Firstly, the variable NoOutputOfPriorPicsFlag may be derived for the decoder under test as follows:

[0122] - If any of the values ​​of PicWidthMaxInSamplesY, PicHeightMaxInSamplesY, MaxChromaFormat, MaxBitDepthMinus8, or max_dec_pic_buffering_minus1[Htid] derived for the current AU differ from the values ​​of PicWidthMaxInSamplesY, PicHeightMaxInSamplesY, MaxChromaFormat, MaxBitDepthMinus8, or max_dec_pic_buffering_minus1[Htid] derived for a preceding AU, NoOutputOfPriorPicsFlag may be set to equal to 1 by the decoder under test, regardless of the current AU's ph_no_output_of_prior_pics_flag value.

[0123] - Otherwise, NoOutputOfPriorPicsFlag may be set to the same value as the AU's ph_no_output_of_prior_pics_flag.

[0124] - Secondly, the variable NoOutputOfPriorPicsFlag derived for the decoder under test may be applied to the HRD (Hypothetical Reference Decoder) as follows:

[0125] - For example, if the value of NoOutputOfPriorPicsFlag is 1, all picture storage buffers in the DPB may be empty with no output for the pictures they contain, and the DPB fullness may be set to 0.

[0126] - If not (i.e., if the value of NoOutputOfPriorPicsFlag is 0), all picture storage buffers, including those containing pictures marked as "not needed for output" and "unused for reference" in the DPB, may be emptied (without output), and all non-empty picture storage buffers in the DPB may be emptied by repeatedly calling the "bumping" process specified in section C.5.2.4 (of the VVC standard), and the DPB fullness may be set to 0.

[0127] Furthermore, as shown in Table 2, for example, if the current AU is not a CVSS AU, or if the current AU is a CVSS AU other than AU 0, but the current picture is not the first picture of the current AU, all picture storage buffers, including those containing pictures marked "not needed for output" and "unused for reference," may be emptied (without output). For each emptied picture storage buffer, the DPB fullness may be decreased by 1. Also, if one or more of the conditions described later are true, the "bumping" process specified in Section C.5.2.4 (of the VVC standard) may be repeatedly called, further decreasing the DPB fullness by 1 for each additional emptied picture storage buffer, until all of the conditions described later are no longer true.

[0128] - The number of pictures in the DPB that are marked "needed for output" is greater than max_num_reorder_pics[Htid].

[0129] - max_latency_increase_plus1[Htid] is not 0, and there is at least one picture in the DPB where the related variable PicLatencyCount, which is marked as "needed for output", is greater than or equal to MaxLatencyPictures[Htid].

[0130] - The number of pictures in the DPB is greater than or equal to max_dec_pic_buffering_minus1[Htid]+1.

[0131] Alternatively, the above-described embodiments may be implemented as follows. For example, the above-described embodiments can be shown based on the VVC specification, as will be described later.

[0132] [Table 3-1]

[0133] [Table 3-2]

[0134] [Table 3-3]

[0135] For example, referring to Table 3, if the current picture is the first picture in the current AU and the current AU is a CVSS AU other than AU 0 (Coded Video Sequence Start AU), the following sequence of steps may be applied.

[0136] - Firstly, the variable NoOutputOfPriorPicsFlag may be derived for the decoder under test as follows:

[0137] - If any of the values ​​of PicWidthMaxInSamplesY, PicHeightMaxInSamplesY, MaxChromaFormat, MaxBitDepthMinus8, or max_dec_pic_buffering_minus1[Htid] derived for the current AU differ from the values ​​of PicWidthMaxInSamplesY, PicHeightMaxInSamplesY, MaxChromaFormat, MaxBitDepthMinus8, or max_dec_pic_buffering_minus1[Htid] derived for a preceding AU, NoOutputOfPriorPicsFlag may be set to equal to 1 by the decoder under test, regardless of the current AU's ph_no_output_of_prior_pics_flag value.

[0138] - Otherwise, NoOutputOfPriorPicsFlag may be set to the same value as the AU's ph_no_output_of_prior_pics_flag.

[0139] - Secondly, the variable NoOutputOfPriorPicsFlag derived for the decoder under test may be applied to the HRD (Hypothetical Reference Decoder). Therefore, when the value of NoOutputOfPriorPicsFlag is 1, all picture storage buffers of the DPB may be empty with no output for the pictures they contain, and the DPB fullness may be set to 0.

[0140] Furthermore, as shown in Table 3, for example, if the current AU is not a CVSS AU, or if the current AU is a CVSS AU other than AU 0, but the current picture is not the first picture of the current AU, and all of the following conditions are true for picture k of the DPB, then all pictures k of the DPB may be removed from the DPB.

[0141] - Picture k is displayed as "unused for reference".

[0142] - Picture k has a PictureOutputFlag of 0, or the DPB output time of picture k is less than or equal to the CPB removal time of the first DU (represented as DU m) of picture n; that is, DpbOutputTime[k] is less than or equal to DuCpbRemovalTime[m].

[0143] Furthermore, as shown in Table 3, for example, if the current picture is the first picture in the current AU, and the current AU is a CVSS AU other than AU 0 (Coded Video Sequence Start AU), the following sequence of steps may be applied.

[0144] - Firstly, the variable NoOutputOfPriorPicsFlag may be derived for the decoder under test as follows:

[0145] - If any of the values ​​of PicWidthMaxInSamplesY, PicHeightMaxInSamplesY, MaxChromaFormat, MaxBitDepthMinus8, or max_dec_pic_buffering_minus1[Htid] derived for the current AU differ from the values ​​of PicWidthMaxInSamplesY, PicHeightMaxInSamplesY, MaxChromaFormat, MaxBitDepthMinus8, or max_dec_pic_buffering_minus1[Htid] derived for a preceding AU, NoOutputOfPriorPicsFlag may be set to equal to 1 by the decoder under test, regardless of the current AU's ph_no_output_of_prior_pics_flag value.

[0146] - Otherwise, NoOutputOfPriorPicsFlag may be set to the same value as the AU's ph_no_output_of_prior_pics_flag.

[0147] - Secondly, the variable NoOutputOfPriorPicsFlag derived for the decoder under test may be applied to the HRD (Hypothetical Reference Decoder) as follows:

[0148] - For example, if the value of NoOutputOfPriorPicsFlag is 1, all picture storage buffers in the DPB may be empty with no output for the pictures they contain, and the DPB fullness may be set to 0.

[0149] - If not (i.e., if the value of NoOutputOfPriorPicsFlag is 0), all picture storage buffers, including those containing pictures marked as "not needed for output" and "unused for reference" in the DPB, may be emptied (without output), and all non-empty picture storage buffers in the DPB may be emptied by repeatedly calling the "bumping" process specified in section C.5.2.4 (of the VVC standard), and the DPB fullness may be set to 0.

[0150] Furthermore, as shown in Table 3, for example, if the current AU is not a CVSS AU, or if the current AU is a CVSS AU other than AU 0, but the current picture is not the first picture of the current AU, all picture storage buffers, including those containing pictures marked "not needed for output" and "unused for reference," may be emptied (without output). For each emptied picture storage buffer, the DPB fullness may be decreased by 1. Also, if one or more of the conditions described later are true, the "bumping" process specified in Section C.5.2.4 (of the VVC standard) may be repeatedly called, further decreasing the DPB fullness by 1 for each additional emptied picture storage buffer, until all of the conditions described later are no longer true.

[0151] - The number of pictures in the DPB that are marked "needed for output" is greater than max_num_reorder_pics[Htid].

[0152] - max_latency_increase_plus1[Htid] is not 0, and there is at least one picture in the DPB where the related variable PicLatencyCount, which is marked as "needed for output", is greater than or equal to MaxLatencyPictures[Htid].

[0153] - The number of pictures in the DPB is greater than or equal to max_dec_pic_buffering_minus1[Htid]+1.

[0154] On the other hand, for example, the embodiment may be applied by the following procedure. One or more of the steps in the procedure described later may be omitted.

[0155] Figure 4 illustrates the encoding procedure according to the embodiment described in this document.

[0156] Referring to Figure 4, the encoding device decodes (restores) the picture (S400). The encoding device can now decode the picture of AU.

[0157] The encoding device manages the DPB based on the DPB parameters (S410). Here, DPB management can also be called DPB updating. The DPB management process may include marking and / or removal processes of the pictures decoded in the DPB. The decoded pictures may be used as references for interpretation of subsequent pictures. That is, the decoded pictures may be used as reference pictures for interpretation of pictures that follow in the decoding order. Each decoded picture may be inserted into the DPB in principle. Also, the DPB may generally be updated before decoding the current picture. If the layer associated with the DPB is not an output layer (or the DPB parameters are not associated with an output layer) and is a reference layer, the decoded pictures in the DPB do not need to be output. If the layer associated with the DPB (or DPB parameters) is an output layer, the decoded pictures in the DPB may be output based on the DPB and / or DPB parameters. DPB management may include outputting a picture decoded from the DPB.

[0158] The encoding device encodes video information including information related to DPB parameters (S420). The information related to DPB parameters may include the information / syntax elements disclosed in the embodiments described above and / or the syntax elements disclosed in the table below.

[0159] [Table 4]

[0160] For example, Table 4 above can show a Video Parameter Set (VPS) containing syntax elements for the signaled DPB parameters.

[0161] The semantics for the syntax elements shown in Table 4 above can be as follows:

[0162] [Table 5-1]

[0163] [Table 5-2]

[0164] For example, the syntax element vps_num_dpb_params can indicate the number of dpb_parameters() syntax structures in a VPS. For example, the value of vps_num_dpb_params can be in the range of 0 to 16. Also, if the syntax element vps_num_dpb_params does not exist, its value may be considered equal to 0.

[0165] Furthermore, for example, the syntax element same_dpb_size_output_or_nonoutput_flag can indicate whether or not the syntax element layer_nonoutput_dpb_params_idx[i] exists in the VPS. For example, if the value of the syntax element same_dpb_size_output_or_nonoutput_flag is 1, it indicates that the syntax element layer_nonoutput_dpb_params_idx[i] does not exist in the VPS, and if the value of the syntax element same_dpb_size_output_or_nonoutput_flag is 0, it indicates that the syntax element layer_nonoutput_dpb_params_idx[i] may exist in the VPS.

[0166] Furthermore, for example, the syntax element vps_sublayer_dpb_params_present_flag may be used to control the existence of the syntax elements max_dec_pic_buffering_minus1[], max_num_reorder_pics[], and max_latency_increase_plus1[] within the VPS's dpb_parameters() syntax structure. Also, if the syntax element vps_sublayer_dpb_params_present_flag does not exist, its value may be considered equal to 0.

[0167] Furthermore, for example, the syntax element dpb_size_only_flag[i] can indicate whether the syntax elements max_num_reorder_pics[] and max_latency_increase_plus1[] exist in the i-th dpb_parameters() syntax structure of the VPS. For example, if the value of the syntax element dpb_size_only_flag[i] is 1, the syntax element dpb_size_only_flag[i] can indicate that the syntax elements max_num_reorder_pics[] and max_latency_increase_plus1[] do not exist in the i-th dpb_parameters() syntax structure of the VPS. If the value of the syntax element dpb_size_only_flag[i] is 0, the syntax element dpb_size_only_flag[i] can indicate that the syntax elements max_num_reorder_pics[] and max_latency_increase_plus1[] may exist in the i-th dpb_parameters() syntax structure of the VPS.

[0168] Furthermore, for example, the syntax element dpb_max_temporal_id[i] can indicate the TemporalId of the highest sublayer representation in which a DPB parameter may exist in the i-th dpb_parameters() syntax structure in the VPS. The value of dpb_max_temporal_id[i] may be in the range of 0 to vps_max_sublayers_minus1. For example, if the value of vps_max_sublayers_minus1 is 0, the value of dpb_max_temporal_id[i] may be considered 0. Also, for example, if the value of vps_max_sublayers_minus1 is greater than 0 and vps_all_layers_same_num_sublayers_flag is 1, the value of dpb_max_temporal_id[i] may be considered identical to vps_max_sublayers_minus1.

[0169] Additionally, for example, the syntax element layer_output_dpb_params_idx[i] can specify the index of the dpb_parameters() syntax structure applied to the i-th layer, which is the output layer of the OLS, in the list of dpb_parameters() syntax structures in the VPS. If the syntax element layer_output_dpb_params_idx[i] exists, its value may be in the range of 0 to vps_num_dpb_params-1.

[0170] For example, if vps_independent_layer_flag[i] is 1, the dpb_parameters() syntax structure applied to the i-th layer, which is the output layer, may be a dpb_parameters() syntax structure that exists in the SPS referenced by the layer.

[0171] Alternatively, for example, if vps_independent_layer_flag[i] is 0, the following may apply:

[0172] - If vps_num_dpb_params is 1, the value of layer_output_dpb_params_idx[i] may be considered 0.

[0173] - The bitstream conformance requirement may be that the value of layer_output_dpb_params_idx[i] should be such that the value of dpb_size_only_flag[layer_output_dpb_params_idx[i]] is 0.

[0174] Furthermore, for example, the syntax element layer_nonoutput_dpb_params_idx[i] can specify the index of the dpb_parameters() syntax structure applied to the i-th layer, which is a non-output layer of the OLS, in the list of dpb_parameters() syntax structures in the VPS. If the syntax element layer_nonoutput_dpb_params_idx[i] exists, its value may be in the range of 0 to vps_num_dpb_params-1.

[0175] For example, if same_dpb_size_output_or_nonoutput_flag is 1, the following may apply.

[0176] - If vps_independent_layer_flag[i] is 1, the dpb_parameters() syntax structure applied to the i-th layer, which is a non-output layer, may be the dpb_parameters() syntax structure in the SPS referenced by the layer.

[0177] - If vps_independent_layer_flag[i] is 0, the value of layer_nonoutput_dpb_params_idx[i] may be considered identical to layer_output_dpb_params_idx[i].

[0178] Alternatively, for example, if same_dpb_size_output_or_nonoutput_flag is 0 and vps_num_dpb_params is 1, then the value of layer_output_dpb_params_idx[i] may be considered 0.

[0179] On the other hand, for example, the dpb_parameters() syntax structure, which is the DPB parameter syntax structure disclosed in Table 4 above, may be as follows:

[0180] [Table 6]

[0181] Referring to Table 6, the dpb_parameters() syntax structure can provide information regarding the DPB size, maximum picture reorder number, and maximum latency for each CLVS in the CVS. The dpb_parameters() syntax structure can also be expressed as information regarding DPB parameters or DPB parameter information.

[0182] If the VPS includes the dpb_parameters() syntax structure, the OLS to which the dpb_parameters() syntax structure applies may be specified by the VPS. Also, if the dpb_parameters() syntax structure is included in the SPS, the dpb_parameters() syntax structure may be applied to an OLS that includes only the lowest layer among the layers referencing the SPS, where the lowest layer may be an independent layer.

[0183] The semantics for the syntax elements shown in Table 6 above can be as follows:

[0184] [Table 7]

[0185] For example, the value obtained by adding 1 to the syntax element max_dec_pic_buffering_minus1[i] can indicate the maximum required size of the DPB in picture storage buffer units for each CLVS of the CVS, when Htid is the same as i. For example, max_dec_pic_buffering_minus1[i] may be information about the DPB size. For example, the value of the syntax element max_dec_pic_buffering_minus1[i] may be in the range of 0 to MaxDpbSize-1. Also, for example, when i is greater than 0, max_dec_pic_buffering_minus1[i] may be greater than or equal to max_dec_pic_buffering_minus1[i-1]. Furthermore, for example, if there is no max_dec_pic_buffering_minus1[i] for i within the range of 0 to maxSubLayersMinus1-1, the subLayerInfoFlag is 0, so the value of the syntax element max_dec_pic_buffering_minus1[i] may be considered identical to max_dec_pic_buffering_minus1[maxSubLayersMinus1].

[0186] Furthermore, for example, the syntax element max_num_reorder_pics[i] may precede all pictures in the decoding order for each CLVS in CVS, and when Htid is the same as i, it can indicate the maximum allowed number of pictures in the CLVS that can follow that picture in the output order. For example, max_num_reorder_pics[i] may be information about the maximum number of picture reorders in the DPB. The value of max_num_reorder_pics[i] may be in the range of 0 to max_dec_pic_buffering_minus1[i]. Also, for example, when i is greater than 0, max_num_reorder_pics[i] may be greater than or equal to max_num_reorder_pics[i-1]. Furthermore, for example, if there is no max_num_reorder_pics[i] for i within the range of 0 to maxSubLayersMinus1-1, the subLayerInfoFlag is 0, so the syntax element max_num_reorder_pics[i] may be considered identical to max_num_reorder_pics[maxSubLayersMinus1].

[0187] Furthermore, for example, a syntax element max_latency_increase_plus1[i] with a non-zero value may be used to calculate the value of MaxLatencyPictures[i]. MaxLatencyPictures[i] can indicate the maximum number of pictures in a CLVS that may precede all pictures in the output order for each CLVS in a CVS, and that can follow the picture in the decoding order when Htid is equal to i. For example, max_latency_increase_plus1[i] may be information regarding the maximum latency of the DPB.

[0188] For example, if max_latency_increase_plus1[i] is not 0, the value of MaxLatencyPictures[i] may be derived as follows:

[0189]

number

[0190] On the other hand, if max_latency_increase_plus1[i] is 0, for example, the corresponding limit does not need to be displayed. The value of max_latency_increase_plus1[i] is between 0 and 2. 32 It may be in the range of -2. Also, for example, if there is no max_latency_increase_plus1[i] for i in the range of 0 to maxSubLayersMinus1-1, then the subLayerInfoFlag is 0, and the syntax element max_latency_increase_plus1[i] may be considered identical to max_latency_increase_plus1[maxSubLayersMinus1].

[0191] Based on the information / syntax elements related to the DPB parameters described above, the DPB management described above may be performed. Different DPB parameters may be signaled depending on whether the current layer is an output layer or a reference layer, or different DPB parameters may be signaled depending on whether the DPB (or DPB parameters) is for an OLS (is OLS-mapped) or not.

[0192] On the other hand, although not shown in Figure 4, the encoding device can decode the current picture based on the updated / managed DPB. The decoded current picture may also be inserted into the DPB, and the DPB containing the decoded current picture may be updated based on DPB parameters before decoding the next picture following the current picture in the decoding sequence.

[0193] Figure 5 illustrates the decoding procedure according to the embodiment described in this document.

[0194] The decoding device acquires video information from the bitstream, including information related to the DPB parameters (S500). The decoding device can acquire video information including information related to the DPB parameters. The information / syntax elements related to the DPB parameters may be as described above.

[0195] The decoding device manages the DPB based on the DPB parameters (S510). Here, DPB management may also be called DPB updating. The DPB management process may include marking and / or removal processes of the picture decoded by the DPB. The decoding device can derive the DPB parameters based on information related to the DPB parameters and can perform the DPB management process based on the derived DPB parameters.

[0196] The decoding device decodes / outputs the current picture based on the DPB (S520). The decoding device can decode the current picture based on an updated / managed DPB. For example, blocks / slices in the current picture may be decoded based on interpretation using a previously decoded picture in the DPB as a reference picture.

[0197] Figure 6 schematically shows a video encoding method using the encoding device relating to this document. The method disclosed in Figure 6 may be performed using the encoding device disclosed in Figure 2. Specifically, for example, steps S600 to S610 in Figure 6 may be performed by the DPB of the encoding device, and step S620 may be performed by the entropy encoding unit of the encoding device. Furthermore, although not shown, the process of decoding the current picture may be performed by the prediction unit and residual processing unit of the encoding device.

[0198] The encoding device derives the value of the variable based on whether the current picture is the first picture of the current AU (Access Unit) that is a CVSS AU (Coded Video Sequence Start Access Unit) other than AU 0 (S600). The encoding device may derive the value of the variable in order to update the DPB before decoding the current picture and after generating / encoding a slice header for the current picture. The DPB may include pictures decoded before the current picture.

[0199] For example, the encoding device can derive a variable value based on whether the current picture is the first picture in the current AU (Access Unit) that is a CVSS AU other than AU 0. Here, the variable can indicate whether all picture storage buffers in the DPB (Decoded Picture Buffer) become empty without output. The current AU may be the AU that contains the current picture. Also, for example, AU 0 may be the first AU in the bitstream in the decoding order. That is, for example, AU 0 may be the first AU in the bitstream to be decoded. On the other hand, the encoding device can generate / encode a slice header for the current picture and then derive a variable value based on whether the current picture is the first picture in the current AU (Access Unit) that is a CVSS AU other than AU 0.

[0200] For example, the encoding device can determine whether the current picture is the first picture of the current AU (Access Unit) which is a CVSS AU other than AU 0. If the current AU is a CVSS AU other than AU 0, and the current picture is the first picture of the current AU, the encoding device can derive the value of the variable.

[0201] For example, if the current AU is a CVSS AU other than AU 0, and the current picture is the first picture of the current AU, the encoding device can determine whether at least one of the parameters for the current AU is different from the parameters for the preceding AU of the current AU in the decoding order. If at least one of the parameters for the current AU is different from the parameters for the preceding AU, the value of the variable may be set to the same value as 1, and if the parameter for the current AU is the same as the parameter for the preceding AU, the value of the variable may be set to the same value as the syntax element for the variable. The encoding device can generate / encode video information for the current picture, and the video information may include the syntax element. The syntax element may be the ph_no_output_of_prior_pics_flag described above. Furthermore, the parameters for the current AU may include parameters for the maximum picture width, maximum picture height, available chroma format, maximum bit depth, and maximum DPB size for the current AU. The parameter for the maximum picture width may be PicWidthMaxInSamplesY as described above, the parameter for the maximum picture height may be PicHeightMaxInSamplesY as described above, the parameter for the available chroma format may be MaxChromaFormat as described above, the parameter for the maximum bit depth may be MaxBitDepthMinus8 as described above, and the parameter for the maximum DPB size may be max_dec_pic_buffering_minus1[Htid] as described above.

[0202] On the other hand, for example, if the current AU is not a CVSS AU, or if the current picture is not the first picture of a current AU (Access Unit) that is a CVSS AU other than AU 0, the encoding device does not need to derive the value of the variable.

[0203] As a result, the variable may be derived only before decoding the current picture, which is the first picture of the current AU, rather than before decoding all pictures of the current AU, which is a CVSS AU other than AU 0, and the process of emptying all picture storage buffers in the DPB (Decoded Picture Buffer) without output may occur only before decoding the current picture, which is the first picture of the current AU.

[0204] The encoding device updates the DPB based on the variable (S610). For example, the encoding device can update the DPB based on the variable.

[0205] For example, if the value of the variable is 1, all picture storage buffers in the DPB may be empty with no output, and the DPB fullness may be set to the same as 0. Also, for example, if the value of the variable is 0, the picture storage buffer containing a specific picture in the DPB may be empty with no output, and a bumping process may be performed on the unused picture storage buffers in the DPB. The DPB fullness may also be set to 0. Here, for example, the specific picture may be a picture marked as "not needed for output" and "unused for reference". The bumping process may be as described above.

[0206] Furthermore, for example, if the current picture is not the first picture of the current AU which is a CVSS AU other than AU 0, the encoding device may remove a specific picture from the DPB that satisfies the first and second conditions. Here, the first condition is that the specific picture is a picture marked "Not used as reference," and the second condition is that the specific picture has a picture output flag equal to 0, or the DPB output time of the specific picture is less than or equal to the CPB removal time of the first DU (Decoding Unit) of the current picture. Here, the picture output flag may be the PictureOutputFlag described above.

[0207] Furthermore, for example, if the current picture is not the first picture of the current AU which is a CVSS AU other than AU 0, the picture storage buffer containing the specific picture in the DPB may become empty without output. Here, the specific picture may be a picture that is marked as "not needed for output" and "unused for reference". Also, the DPB fullness may decrease by 1 for each empty picture storage buffer. That is, for example, the DPB fullness may decrease by 1 each time the picture storage buffer becomes empty. Furthermore, if at least one of the conditions described later is true, the bumping process described above may be repeated, further decreasing the DPB fullness by 1 for each additional empty picture storage buffer until all of the above conditions are no longer true.

[0208] For example, the first condition may be that the number of pictures in the DPB marked "needed for output" is greater than the syntax element max_num_reorder_pics[Htid] for the current AU; the second condition may be that the syntax element max_latency_increase_plus1[Htid] for the current AU is not 0, and there is at least one picture in the DPB where the related variable PicLatencyCount, marked "needed for output," is greater than or equal to MaxLatencyPictures[Htid]; and the third condition may be that the number of pictures in the DPB is greater than or equal to the value obtained by adding 1 to the syntax element max_dec_pic_buffering_minus1[Htid] for the current AU. The video information may include syntax elements for the current AU.

[0209] The encoding device encodes video information for the current picture (S620). The encoding device can encode video information including syntax elements for updating the DPB. The video information may also include a slice header for the current picture.

[0210] On the other hand, although not illustrated, the encoding device can decode the current picture based on the updated DPB. For example, the encoding device can perform interpretation on blocks in the current picture based on a reference picture in the DPB to derive predicted samples, and generate restored samples and / or restored pictures for the current picture based on the predicted samples. Alternatively, for example, the encoding device can derive residual samples for blocks in the current picture, and generate restored samples and / or restored pictures by adding the predicted samples and the residual samples. Subsequently, as described above, in-loop filtering procedures such as deblocking filtering, SAO and / or ALF procedures may be applied to the restored samples to improve subjective / objective image quality as needed. The encoding device can generate / encode prediction-related information and / or residual information for the blocks, and the video information may include the prediction-related information and / or the residual information. The encoding device can also insert the decoded current picture into the DPB. Furthermore, for example, the encoding device can derive DPB parameters for the current AU and generate DBP-related information for the DPB parameters. The video information may include the DBP-related information.

[0211] On the other hand, the bitstream containing the video information may be transmitted to a decoding device via a network or a (digital) storage medium. Here, the network may include broadcast networks and / or communication networks, and the digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc.

[0212] Figure 7 schematically shows an encoding device that performs the video encoding method relating to this document. The method disclosed in Figure 7 may be performed by the encoding device disclosed in Figure 6. Specifically, for example, the DPB of the encoding device in Figure 7 can perform S600 to S610, and the entropy encoding unit of the encoding device in Figure 7 can perform S620. Furthermore, although not shown, the process of decoding the current picture may be performed by the prediction unit and residual processing unit of the encoding device.

[0213] Figure 8 schematically shows a video decoding method using a decoding device relating to this document. The method disclosed in Figure 8 may be performed using the decoding device disclosed in Figure 3. Specifically, for example, steps S800 to S810 in Figure 8 may be performed by the DPB of the decoding device, and step S820 in Figure 8 may be performed by the prediction unit and residual processing unit of the decoding device.

[0214] The decoding device derives the value of a variable based on whether the current picture is the first picture of the current AU (Access Unit) that is a CVSS AU (Coded Video Sequence Start Access Unit) other than AU 0 (S800). The decoding device can derive the value of a variable based on whether the current picture is the first picture of the current AU (Access Unit) that is a CVSS AU other than AU 0. Here, the variable can indicate whether all picture storage buffers in the DPB (Decoded Picture Buffer) become empty without output. The current AU may be the AU that contains the current picture. Also, for example, AU 0 may be the first AU of the bitstream in the decoding order. That is, for example, AU 0 may be the first AU of the bitstream that the decoding device decodes. On the other hand, the decoding device can parse the slice header for the current picture and then derive the value of a variable based on whether the current picture is the first picture of the current AU (Access Unit) that is a CVSS AU other than AU 0.

[0215] For example, the decoding device can determine whether the current picture is the first picture of the current AU (Access Unit) which is a CVSS AU other than AU 0. If the current AU is a CVSS AU other than AU 0, and the current picture is the first picture of the current AU, the decoding device can derive the value of the variable.

[0216] For example, if the current AU is a CVSS AU other than AU 0, and the current picture is the first picture of the current AU, the decoding device can determine whether at least one of the parameters for the current AU is different from the parameter for the preceding AU of the current AU in the decoding order. If at least one of the parameters for the current AU is different from the parameter for the preceding AU, the value of the variable may be set to the same as 1, and if the parameter for the current AU is the same as the parameter for the preceding AU, the value of the variable may be set to the same as the value of the syntax element signaled to the variable. The decoding device can acquire video information for the current picture, and the video information may include the syntax element. The syntax element may be the ph_no_output_of_prior_pics_flag described above. Furthermore, the parameters for the current AU may include parameters for the maximum picture width, maximum picture height, available chroma format, maximum bit depth, and maximum DPB size for the current AU. The parameter for the maximum picture width may be PicWidthMaxInSamplesY as described above, the parameter for the maximum picture height may be PicHeightMaxInSamplesY as described above, the parameter for the available chroma format may be MaxChromaFormat as described above, the parameter for the maximum bit depth may be MaxBitDepthMinus8 as described above, and the parameter for the maximum DPB size may be max_dec_pic_buffering_minus1[Htid] as described above.

[0217] On the other hand, for example, if the current AU is not a CVSS AU, or if the current picture is not the first picture of a current AU (Access Unit) that is a CVSS AU other than AU 0, the decoding device does not need to derive the value of the variable.

[0218] As a result, the variable may be derived only before decoding the current picture, which is the first picture of the current AU, rather than before decoding all pictures of the current AU, which is a CVSS AU other than AU 0, and the process of emptying all picture storage buffers in the DPB (Decoded Picture Buffer) without output may occur only before decoding the current picture, which is the first picture of the current AU.

[0219] The decoding device updates the DPB based on the variable (S810). For example, the decoding device can update the DPB based on the variable. Before being updated, the DPB may include pictures that were decoded before the current picture.

[0220] For example, if the value of the variable is 1, all picture storage buffers in the DPB may be empty with no output, and the DPB fullness may be set to the same as 0. Also, for example, if the value of the variable is 0, the picture storage buffer containing a specific picture in the DPB may be empty with no output, and a bumping process may be performed on the unused picture storage buffers in the DPB. The DPB fullness may also be set to 0. Here, for example, the specific picture may be a picture marked as "not needed for output" and "unused for reference". The bumping process may be as described above.

[0221] Furthermore, for example, if the current picture is not the first picture of the current AU which is a CVSS AU other than AU 0, the decoding device can remove a specific picture from the DPB that satisfies the first and second conditions in the DPB. Here, the first condition is that the specific picture is a picture that is marked as "not used as a reference," and the second condition may be that the specific picture has a picture output flag equal to 0, or that the DPB output time of the specific picture is less than or equal to the CPB removal time of the first DU (Decoding Unit) of the current picture. Here, the picture output flag may be the PictureOutputFlag described above.

[0222] Furthermore, for example, if the current picture is not the first picture of the current AU which is a CVSS AU other than AU 0, the picture storage buffer containing the specific picture in the DPB may become empty without output. Here, the specific picture may be a picture that is marked as "not needed for output" and "unused for reference". Also, the DPB fullness may decrease by 1 for each empty picture storage buffer. That is, for example, the DPB fullness may decrease by 1 each time the picture storage buffer becomes empty. Furthermore, if at least one of the conditions described later is true, the bumping process described above may be repeated, further decreasing the DPB fullness by 1 for each additional empty picture storage buffer until all of the above conditions are no longer true.

[0223] For example, the first condition may be that the number of pictures in the DPB marked "needed for output" is greater than the syntax element max_num_reorder_pics[Htid] for the current AU; the second condition may be that the syntax element max_latency_increase_plus1[Htid] for the current AU is not 0, and there is at least one picture in the DPB where the related variable PicLatencyCount, marked "needed for output," is greater than or equal to MaxLatencyPictures[Htid]; and the third condition may be that the number of pictures in the DPB is greater than or equal to the value of the syntax element max_dec_pic_buffering_minus1[Htid] for the current AU plus 1. The video information may include syntax elements for the current AU.

[0224] The decoding device decodes the current picture based on the updated DPB (S820). For example, the decoding device can decode the current picture based on the updated DPB. For example, the decoding device can perform interpretation on blocks in the current picture based on a reference picture in the DPB to derive predicted samples, and can generate restored samples and / or restored pictures for the current picture based on the predicted samples. On the other hand, for example, the decoding device can derive residual samples for blocks in the current picture based on residual information for the current picture received using a bitstream, and can generate restored samples and / or restored pictures by adding the predicted samples and the residual samples. The video information may include the residual information. The decoding device can also insert the decoded current picture into the DPB.

[0225] As mentioned above, subsequently, in-loop filtering procedures such as deblocking filtering, SAO, and / or ALF procedures may be applied to the restored sample to improve subjective / objective image quality as needed.

[0226] Figure 9 schematically shows a decoding device that performs the video decoding method relating to this document. The method disclosed in Figure 8 may be performed by the decoding device disclosed in Figure 9. Specifically, for example, the DPB of the decoding device in Figure 9 can perform steps S800 to S810 in Figure 8, and the prediction unit and residual processing unit of the decoding device in Figure 9 can perform step S820 in Figure 8.

[0227] According to the document mentioned above, instead of deciding whether or not to remove a picture in the DPB without outputting it before decoding all pictures in CVSS AUs other than AU 0, the decision can be made only before decoding the first picture in a CVSS AU other than AU 0. This eliminates the need to change the DPB state for each picture, which affects all layers in the CVS, thereby improving coding efficiency.

[0228] Furthermore, according to this document, the variable indicating whether or not to remove a picture in the DPB without outputting it can be derived only before decoding the first picture in a CVSS AU other than AU 0, rather than before decoding all pictures in CVSS AUs other than AU 0. This eliminates the need to change the DPB state, which affects all layers in CVS, for each picture, thereby improving coding efficiency.

[0229] In the embodiments described above, the method is a series of steps or blocks and is explained based on a sequence diagram. However, this document is not limited to the order of the steps, and some steps may occur in a different order or simultaneously with other steps than those described above. Furthermore, those skilled in the art will understand that the steps shown in the sequence diagram are not exclusive, and other steps may be included, or one or more steps in the sequence diagram may be omitted without affecting the scope of this document.

[0230] The embodiments described herein may be implemented and performed on a processor, microprocessor, controller, or chip. For example, the functional units shown in each figure may be implemented and performed on a computer, processor, microprocessor, controller, or chip. In this case, information on instructions or algorithms for implementation may be stored on a digital storage medium.

[0231] Furthermore, the decoding and encoding devices to which the embodiments of this document apply may include multimedia broadcasting transceivers, mobile communication terminals, home cinema video equipment, digital cinema video equipment, surveillance cameras, video conferencing equipment, real-time communication equipment such as video communications, mobile streaming equipment, storage media, CAM coders, video-on-demand (VoD) service providers, OTT video (Over the Top) video equipment, internet streaming service providers, 3D video equipment, image-phone video equipment, transportation terminals (e.g., vehicle terminals, airplane terminals, ship terminals, etc.), and medical video equipment, and may be used to process video signals or data signals. For example, OTT video (Over the Top) equipment may include game consoles, Blu-ray players, internet-connected TVs, home theater systems, smartphones, tablet PCs, DVRs (Digital Video Recorders), etc.

[0232] Furthermore, the processing methods to which the embodiments of this document apply may be produced in the form of programs executed on a computer and stored on a computer-readable recording medium. Multimedia data having the data structure according to this document may also be stored on a computer-readable recording medium. The computer-readable recording medium includes all kinds of storage devices and distributed storage devices that store computer-readable data. The computer-readable recording medium may include, for example, Blu-ray discs (BDs), general-purpose serial buses (USBs), ROMs, PROMs, EPROMs, EEPROMs, RAMs, CD-ROMs, magnetic tapes, floppy disks, and optical data storage devices. The computer-readable recording medium also includes media embodied in the form of carrier waves (e.g., transmission over the Internet). Furthermore, a bitstream generated by an encoding method may be stored on a computer-readable recording medium and transmitted over a wireless communication network.

[0233] Furthermore, the embodiments described in this document may be embodied as computer program products in the form of program code, and the program code may be executed on a computer according to the embodiments described in this document. The program code may be stored on a computer-readable carrier.

[0234] Figure 10 is a structural diagram illustrating a content streaming system to which the embodiments described in this document are applied.

[0235] The content streaming system to which the embodiments described herein apply may broadly include an encoding server, a streaming server, a web server, media storage, user equipment, and multimedia input devices.

[0236] The encoding server is responsible for compressing content input from multimedia input devices such as smartphones, cameras, and CAM coders into digital data, generating a bitstream, and transmitting it to the streaming server. In other cases, the encoding server may be omitted if the multimedia input device, such as a smartphone, camera, or CAM coder, directly generates the bitstream.

[0237] The bitstream may be generated by an encoding method or bitstream generation method to which the embodiments of this document apply, and the streaming server may temporarily store the bitstream in the process of transmitting or receiving the bitstream.

[0238] The streaming server transmits multimedia data to user devices based on user requests via a web server, and the web server acts as an intermediary to inform users about available services. When a user requests a desired service from the web server, the web server transmits it to the streaming server, and the streaming server transmits multimedia data to the user. In this case, the content streaming system may include a separate control server, in which case the control server is responsible for controlling the commands and responses between each device in the content streaming system.

[0239] The streaming server can receive content from media storage and / or encoding servers. For example, when receiving content from the encoding server, the content can be received in real time. In this case, in order to provide a smooth streaming service, the streaming server can store the bitstream for a certain period of time.

[0240] Examples of user devices include mobile phones, smartphones, laptop computers, digital broadcasting terminals, PDAs (personal digital assistants), PMPs (portable multimedia players), navigation systems, slate PCs, tablet PCs, ultrabooks, wearable devices (e.g., smartwatches, smart glasses, HMDs (head-mounted displays)), digital TVs, desktop computers, and digital signage. Each server in the content streaming system may be operated as a distributed server, in which case the data received by each server may be processed in a distributed manner.

[0241] The claims described herein may be combined in various ways. For example, the technical features of the method claims herein may be combined to embody an apparatus, and the technical features of the apparatus claims herein may be combined to embody a method. Furthermore, the technical features of the method claims and the technical features of the apparatus claims herein may be combined to embody an apparatus, and the technical features of the method claims and the technical features of the apparatus claims herein may be combined to embody a method.

Claims

1. A video decoding method performed by a decoding device, Currently, we are at the stage of deriving the variable value based on whether the current picture is the first picture in the Access Unit (AU), The aforementioned current AU is a CVSS AU (Coded Video Sequence Start AU) other than AU 0. The aforementioned AU 0 is the first AU in the bitstream, The aforementioned variable indicates whether all picture storage buffers within the DPB (Decoded Picture Buffer) become empty without output, and represents a step. A step of updating the DPB based on the aforementioned variables, The step includes decoding the current picture based on the updated DPB, The value of the aforementioned variable is derived after parsing the slice header for the current picture, by a method.

2. If the value of the aforementioned variable is 1, All of the picture storage buffers in the DPB become empty without output. The method according to claim 1, wherein the DPB filling level is set to equal to 0.

3. If the value of the aforementioned variable is 0, The picture storage buffer containing the specific picture in the aforementioned DPB becomes empty without output. A bumping process is performed on the unused picture storage buffers within the DPB. The method according to claim 2, wherein the specified picture is a picture marked "not needed for output" and "not used as a reference".

4. The step of deriving the value of the variable includes, if the current picture is the first picture of the current AU which is a CVSS AU other than AU 0, the step of determining whether at least one of the parameters for the current AU is different from the parameters for the preceding AU of the current AU in the decoding order. If at least one of the parameters for the current AU is different from the parameter for the preceding AU, the value of the variable is set to equal 1. The method according to claim 1, wherein, if the parameter for the current AU is the same as the parameter for the preceding AU, the value of the variable is set to be equal to the value of the syntax element signaled to the variable.

5. The method according to claim 4, wherein the parameters for the current AU include parameters for the current AU for the maximum picture width, the maximum picture height, the available chroma format, the maximum bit depth, and the maximum DPB size.

6. The step of updating the DPB includes, if the current picture is not the first picture of the current AU which is a CVSS AU other than AU 0, the step of removing a specific picture that satisfies the first and second conditions from the DPB. The first condition is that the specific picture is a picture that is marked as "not used as a reference," The method according to claim 1, wherein the second condition is that the specific picture has a picture output flag equal to 0, or the DPB output time of the specific picture is less than or equal to the CPB removal time of the first DU (Decoding Unit) of the current picture.

7. If the current picture is not the first picture of the current AU, which is a CVSS AU other than AU 0, the picture storage buffer containing the specific picture in the DPB becomes empty without output. The method according to claim 1, wherein the specified picture is a picture marked "not needed for output" and "not used as a reference".

8. A video encoding method performed by an encoding device, Currently, we are at the stage of deriving the variable value based on whether the current picture is the first picture in the Access Unit (AU), The aforementioned current AU is a CVSS AU (Coded Video Sequence Start AU) other than AU 0. The aforementioned AU 0 is the first AU in the bitstream, The aforementioned variable indicates whether all picture storage buffers within the DPB (Decoded Picture Buffer) become empty without output, and represents a step. A step of updating the DPB based on the aforementioned variables, The step includes encoding video information for the aforementioned picture, The value of the aforementioned variable is derived after encoding the slice header for the current picture, by a method.

9. If the value of the aforementioned variable is 1, All of the picture storage buffers in the DPB become empty without output. The method according to claim 8, wherein the DPB filling level is set to equal to 0.

10. If the value of the aforementioned variable is 0, The picture storage buffer containing the specific picture in the aforementioned DPB becomes empty without output. A bumping process is performed on the unused picture storage buffers within the DPB. The method according to claim 9, wherein the specified picture is a picture marked "not needed for output" and "not used as a reference".

11. The step of deriving the value of the variable includes, if the current picture is the first picture of the current AU which is a CVSS AU other than AU 0, the step of determining whether at least one of the parameters for the current AU is different from the parameters for the preceding AU of the current AU in the decoding order. If at least one of the parameters for the current AU is different from the parameter for the preceding AU, the value of the variable is set to equal 1. If the parameter for the current AU is the same as the parameter for the preceding AU, the value of the variable is set to be equal to the value of the syntax element for the variable. The method according to claim 8, wherein the video information includes the syntax element.

12. The method according to claim 11, wherein the parameters for the current AU include parameters for the maximum picture width, parameters for the maximum picture height, parameters for the available chroma format, parameters for the maximum bit depth, and parameters for the maximum DPB size for the current AU.

13. The step of updating the DPB includes, if the current picture is not the first picture of the current AU which is a CVSS AU other than AU 0, the step of removing a specific picture that satisfies the first and second conditions from the DPB. The first condition is that the specific picture is a picture that is marked as "not used as a reference," The method according to claim 8, wherein the second condition is that the specific picture has a picture output flag equal to 0, or the DPB output time of the specific picture is less than or equal to the CPB removal time of the first DU (Decoding Unit) of the current picture.

14. If the current picture is not the first picture of the current AU, which is a CVSS AU other than AU 0, the picture storage buffer containing the specific picture in the DPB becomes empty without output. The method according to claim 8, wherein the specified picture is a picture marked "not needed for output" and "not used as a reference".

15. A method for transmitting data related to video information, Currently, we are at the stage of deriving the variable value based on whether the current picture is the first picture in the Access Unit (AU), The aforementioned current AU is a CVSS AU (Coded Video Sequence Start AU) other than AU 0. The aforementioned AU 0 is the first AU in the bitstream, The aforementioned variable indicates whether all picture storage buffers within the DPB (Decoded Picture Buffer) become empty without output. The value of the aforementioned variable is derived after encoding the slice header for the current picture, and is a step. A step of updating the DPB based on the aforementioned variables, The steps include: encoding video information for the current picture to generate a bitstream, A method comprising the step of transmitting the data, which includes the bitstream.

Images