Image decoding method, image encoding method, and image data transmission method

Abstract

The invention provides an image decoding method, an image encoding method, and an image data transmission method. An image decoding method performed by a decoding device according to the present document comprises the steps of: deriving a value of a variable based on whether a current picture is a first picture of a current AU that is a coded video sequence start access unit (CVSS) AU that is not an access unit AU 0, the variable indicates whether all picture storage buffers in a decoded picture buffer (DPB) are emptied without output; updating the DPB based on the variable; and decoding the current picture on the basis of the updated DPB.

Description

[0001] This application is a divisional application of the original patent application No. 202180035728.6 (International Application No.: PCT / KR2021 / 005601, Application Date: May 4, 2021, Invention Title: Image Decoding Method and Apparatus for the Image Decoding Method). Technical Field

[0002] This disclosure relates to image coding technology, and more specifically, to an image decoding method and apparatus for performing a DPB management process in an image coding system. Background Technology

[0003] Recently, the demand for high-resolution, high-quality images, such as HD (high-definition) and UHD (ultra-high-definition) images, has been growing across various fields. Because image data is high-resolution and high-quality, the amount of information or bits to be transmitted increases compared to traditional image data. Therefore, transmission and storage costs increase when using media such as traditional wired / wireless broadband lines to send image data or when storing image data using existing storage media.

[0004] Therefore, there is a need for efficient image compression technology to effectively transmit, store, and reproduce information from high-resolution, high-quality images. Summary of the Invention

[0005] Technical issues

[0006] This disclosure provides methods and apparatus for improving image coding efficiency.

[0007] Another technical challenge of this disclosure is to provide methods and apparatus for performing DPB management processes.

[0008] Technical solution

[0009] According to embodiments of this disclosure, an image decoding method performed by a decoding device is provided. The method includes the following steps: deriving the value of a variable based on whether the current image is the first image of the current AU of a coded video sequence start access unit (CVSS AU) other than access unit (AU) 0; updating the DPB based on the variable; and decoding the current image based on the updated DPB. The variable indicates whether all image storage buffers in the decoded image buffer (DPB) have been cleared and have no output.

[0010] According to another embodiment of this disclosure, a decoding apparatus for performing image decoding is provided. The decoding apparatus includes: a DPB (Device Buffer Block), the DPB being configured to derive the value of a variable based on whether the current image is the first image of the current AU of a Coded Video Sequence Start Access Unit (CVSS AU) other than Access Unit (AU) 0, and to update the DPB based on the variable; and a predictor, the predictor being configured to decode the current image based on the updated DPB. The variable indicates whether all image storage buffers in the Decoded Image Buffer (DPB) have been cleared and have no output.

[0011] According to another embodiment of this disclosure, an image encoding method performed by an encoding device is provided. The method includes the following steps: deriving the value of a variable based on whether the current image is the first image of the current AU of the Coded Video Sequence Start Access Unit (CVSS AU) other than Access Unit (AU) 0; updating the DPB based on the variable; and encoding image information of the current image. The variable indicates whether all image storage buffers in the Decoded Image Buffer (DPB) have been cleared and have no output.

[0012] According to another embodiment of this disclosure, a video encoding apparatus is provided. The encoding apparatus includes: a DPB (Decoded Picture Buffer), the DPB being used to derive the value of a variable based on whether the current image is the first image of the current AU of the Coded Video Sequence Start Access Unit (CVSS AU) other than Access Unit (AU) 0, and to update the DPB based on the variable; and an entropy encoder, the entropy encoder being used to encode image information of the current image. The variable indicates whether all image storage buffers in the Decoded Picture Buffer (DPB) have been cleared and have no output.

[0013] According to another embodiment of this disclosure, a computer-readable digital storage medium is provided in which a bitstream including image information causing an image decoding method to be executed is stored. In the computer-readable digital storage medium, the image decoding method includes the steps of: deriving the value of a variable based on whether the current image is the first image of the current AU as the start of the coded video sequence access unit (CVSS AU) other than access unit (AU) 0; updating the DPB based on the variable; and decoding the current image based on the updated DPB. The variable indicates whether all image storage buffers in the decoded image buffer (DPB) have been cleared and have no output.

[0014] Technical effect

[0015] According to this disclosure, whether to perform the process of removing images from the DPB without outputting them can be determined before decoding only the first image of the CVSS AU except AU 0, rather than before decoding all images of the CVSS AU except AU 0. In this way, the DPB state affecting all layers in the CVS can be avoided for each image, and encoding efficiency can be improved.

[0016] According to this disclosure, the variable indicating whether to remove images from the DPB without output can be determined before decoding only the first image of the CVSS AU (excluding AU0), rather than before decoding all images of the CVSS AU (excluding AU0). This avoids changing the DPB state affecting all layers in the CVS for each image and improves encoding efficiency. Attached Figure Description

[0017] Figure 1 Examples of video / image encoding apparatuses to which embodiments of the present disclosure are applied are briefly illustrated.

[0018] Figure 2 This is a schematic diagram illustrating the configuration of a video / image encoding device to which embodiments of the present disclosure can be applied.

[0019] Figure 3 This is a schematic diagram illustrating the configuration of a video / image decoding device to which embodiments of the present disclosure can be applied.

[0020] Figure 4 An exemplary embodiment of the coding process according to an implementation of the present disclosure is illustrated.

[0021] Figure 5 An exemplary decoding process according to an embodiment of the present disclosure is illustrated.

[0022] Figure 6 The image encoding method performed by the encoding device according to this document is illustrated schematically.

[0023] Figure 7 An encoding device for performing an image encoding method according to this document is illustrated schematically.

[0024] Figure 8 The image decoding method performed by the decoding device according to this document is illustrated schematically.

[0025] Figure 9 A decoding device for performing an image decoding method according to this document is illustrated schematically.

[0026] Figure 10 A structural diagram illustrating a content streaming system using this disclosure is provided. Detailed Implementation

[0027] This disclosure may be modified in various forms, and specific embodiments thereof will be described and illustrated in the accompanying drawings. However, these embodiments are not intended to limit this disclosure. The terminology used in the following description is for the purpose of describing specific embodiments only and is not intended to limit this disclosure. Singular expressions include plural expressions, provided that they are clearly read differently. Terms such as “comprising” and “having” are intended to indicate the presence of the features, numbers, steps, operations, elements, components or combinations thereof used in the following description, and therefore it should be understood that the possibility of having or adding one or more different features, numbers, steps, operations, elements, components or combinations thereof is not excluded.

[0028] Furthermore, the elements in the figures described in this disclosure are drawn independently for the purpose of illustrating different specific functions, but this does not mean that these elements are implemented by independent hardware or independent software. For example, two or more of these elements may be combined to form a single element, or a single element may be divided into multiple elements. Embodiments in which elements are combined and / or divided are part of this disclosure without departing from its concept.

[0029] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. Furthermore, throughout the drawings, similar reference numerals are used to indicate similar elements, and identical descriptions of similar elements will be omitted.

[0030] Figure 1 Examples of video / image encoding apparatuses to which embodiments of the present disclosure may be applied are briefly illustrated.

[0031] Reference Figure 1 A video / image encoding system may include a first device (source device) and a second device (receiver). The source device may transmit encoded video / image information or data to the receiving device in the form of a file or stream via a digital storage medium or network.

[0032] The source device may include a video source, an encoding device, and a transmitter. The receiving device may include a receiver, a decoding device, and a renderer. The encoding device may be referred to as a video / image encoding device, and the decoding device may be referred to as a video / image decoding device. The transmitter may be included in the encoding device. The receiver may be included in the decoding device. The renderer may include a display, and the display may be configured as a separate device or an external component.

[0033] Video sources can acquire video / images through processes that capture, synthesize, or generate video / images. Video sources may include video / image capture devices and / or video / image generation devices. Video / image capture devices may include, for example, one or more cameras, video / image archives including previously captured video / images, etc. Video / image generation devices may include, for example, computers, tablets, and smartphones, and can generate video / images (electronically). For example, virtual video / images can be generated by computers, etc. In this case, the video / image capture process can be replaced by a process that generates related data.

[0034] Encoding devices can encode input video / images. They can perform a series of processes such as prediction, transformation, and quantization to achieve compression and encoding efficiency. The encoded data (encoded video / image information) can be output as a bitstream.

[0035] The transmitter can send encoded images / image information or data, output as a bitstream, to the receiver of the receiving device in the form of a file or stream via a digital storage medium or network. The digital storage medium can include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. The transmitter can include elements for generating media files according to a predetermined file format and may include elements for transmission via a broadcast / communication network. The receiver can receive / extract the bitstream and send the received bitstream to a decoding device.

[0036] Decoding devices can decode video / images by performing a series of processes such as inverse quantization, inverse transform, and prediction, which correspond to the operations of encoding devices.

[0037] The renderer can render decoded video / images. The rendered video / images can then be displayed on a monitor.

[0038] This disclosure relates to video / image coding. For example, the methods / implementations disclosed in this disclosure can be applied to methods disclosed in Universal Video Coding (VVC), EVC (Essential Video Coding) standard, AOMedia Video 1 (AV1) standard, second-generation audio-visual coding standard (AVS2), or next-generation video / image coding standards (e.g., H.267 or H.268).

[0039] This disclosure presents various implementations of video / image coding, and unless otherwise mentioned, these implementations can be combined with each other.

[0040] In this disclosure, video can refer to a series of images over time. An image typically refers to a unit representing an image within a specific time region, and sub-images / slices / tiles are units that constitute part of an image during encoding. Sub-images / slices / tiles may include one or more coding tree units (CTUs). An image may consist of one or more sub-images / slices / tiles. An image may consist of one or more groups of tiles. A group of tiles may include one or more tiles. A brick can represent a rectangular area of ​​a row of CTUs within a tile in an image. A tile can be divided into multiple bricks, each of which consists of one or more rows of CTUs within the tile. A tile that is not divided into multiple bricks may also be called a brick. Brick scanning is a specific ordering of the CTUs in a segmented image, where CTUs are ordered by raster scans of CTUs within a brick, bricks within a tile are ordered sequentially by raster scans of bricks within a tile, and tiles within an image are ordered sequentially by raster scans of tiles within an image. Additionally, a sub-image can represent a rectangular region of one or more slices within an image. That is, a sub-image contains one or more slices that collectively cover a rectangular region of the image. A tile is a rectangular region of CTUs within a specific tile column and a specific tile row in an image. A tile column is a rectangular region of CTUs with a height equal to the height of the image and a width specified by a syntax element in the image parameter set. A tile row is a rectangular region of CTUs with a height specified by a syntax element in the image parameter set and a width equal to the width of the image. A tile scan is a specific ordering of the CTUs in a segmented image: CTUs are ordered consecutively by a raster scan of CTUs within a tile, and tiles in the image are ordered consecutively by a raster scan of the tiles in the image. A slice comprises an integer number of tiles of an image that can be exclusively contained within a single NAL unit. A slice can consist of a consecutive sequence of either multiple complete tiles or only one complete tile. In this disclosure, tile groups and slices can be used interchangeably. For example, in this disclosure, a tile group / tile group header can be referred to as a slice / slice header.

[0041] A pixel, or image unit, can refer to the smallest unit that makes up a picture (or image). Additionally, the term "sample" can be used as the counterpart to a pixel. A sample can typically represent a pixel or a pixel value, and can represent only the pixel / pixel value of the luminance component or only the pixel / pixel value of the chrominance component.

[0042] A unit can represent a basic unit of image processing. A unit may include a specific region of an image and at least one of the information associated with that region. A unit may include a luminance block and two chrominance (e.g., cb, cr) blocks. In some cases, the terms "unit" and "block" or "region" may be used interchangeably. Typically, an M×N block may include a set (or array) of samples (or transform coefficients) with M columns and N rows.

[0043] In this specification, “A or B” may mean “A only”, “B only”, or “both A and B”. In other words, in this specification, “A or B” can be interpreted as “A and / or B”. For example, “A, B or C” in this document means “A only”, “B only”, “C only”, or “any one and any combination of A, B and C”.

[0044] In this specification, a forward slash ( / ) or a comma (,) can mean "and / or". For example, "A / B" can mean "A and / or B". Accordingly, "A / B" can mean "A only", "B only", or "both A and B". For example, "A, B, C" can mean "A, B, or C".

[0045] In this specification, "at least one of A and B" may mean "A only", "B only" or "both A and B". Furthermore, in this specification, the expression "at least one of A or B" or "at least one of A and / or B" may be interpreted as the same as "at least one of A and B".

[0046] Additionally, in this specification, "at least one of A, B, and C" means "A only", "B only", "C only" or "any combination of A, B, and C". Furthermore, "at least one of A, B, or C" or "at least one of A, B, and / or C" may mean "at least one of A, B, and C".

[0047] Additionally, the parentheses used in this specification may mean "for example". Specifically, when "prediction (intra-frame prediction)" is indicated, "intra-frame prediction" may be proposed as an example of "prediction". In other words, "prediction" in this specification is not limited to "intra-frame prediction", and "intra-frame prediction" may be proposed as an example of "prediction". Furthermore, even when "prediction (i.e., intra-frame prediction)" is indicated, "intra-frame prediction" may be proposed as an example of "prediction".

[0048] In this specification, a technical feature described individually in a single drawing may be implemented individually or simultaneously.

[0049] The following figures were created to illustrate specific examples of this specification. Since the names of specific devices or signals / messages / fields described in the figures are presented by way of example, the technical features of this specification are not limited to the specific names used in the following figures.

[0050] Figure 2 This is a schematic diagram illustrating the configuration of a video / image encoding apparatus to which embodiments of the present disclosure may be applied. Hereinafter, a video encoding apparatus may include an image encoding apparatus.

[0051] Reference Figure 2 The encoding device 200 includes an image segmenter 210, a predictor 220, a residual processor 230, an entropy encoder 240, an adder 250, a filter 260, and a memory 270. The predictor 220 may include an inter-frame predictor 221 and an intra-frame predictor 222. The residual processor 230 may include a transform 232, a quantizer 233, an inverse quantizer 234, and an inverse transform 235. The residual processor 230 may also include a subtractor 231. The adder 250 may be referred to as a reconstructor or a reconstruction block generator. According to embodiments, the image segmenter 210, predictor 220, residual processor 230, entropy encoder 240, adder 250, and filter 260 may be constituted by at least one hardware component (e.g., an encoder chipset or a processor). Additionally, the memory 270 may include a decoded image buffer (DPB) or may be constituted by a digital storage medium. The hardware component may also include the memory 270 as an internal / external component.

[0052] Image segmenter 210 can segment an input image (or picture or frame) input to encoding device 200 into one or more processors. For example, a processor may be referred to as a coding unit (CU). In this case, coding units can be recursively segmented from coding tree units (CTUs) or maximum coding units (LCUs) according to a quadtree-binary-trinary (QTBTTT) structure. For example, a coding unit can be segmented into multiple deeper coding units based on a quadtree structure, a binary tree structure, and / or a ternary structure. In this case, for example, a quadtree structure can be applied first, followed by a binary tree structure and / or a ternary structure. Alternatively, a binary tree structure can be applied first. The encoding process according to this disclosure can be performed based on the final coding unit that is no longer segmented. In this case, the maximum coding unit can be used as the final coding unit based on encoding efficiency according to image characteristics, or, if necessary, the coding unit can be recursively segmented into deeper coding units, and the coding unit with the optimal size can be used as the final coding unit. Here, the encoding process may include prediction, transformation, and reconstruction processes, which will be described later. As another example, the processor may also include a prediction unit (PU) or a transform unit (TU). In this case, the prediction unit and the transform unit can be separated or divided from the final encoding unit described above. The prediction unit may be a unit for predicting samples, and the transform unit may be a unit for deriving transform coefficients and / or a unit for deriving residual signals from transform coefficients.

[0053] In some cases, a unit can be used interchangeably with terms such as a block or region. Generally, an M×N block can represent a set of samples or transform coefficients consisting of M columns and N rows. A sample can typically represent a pixel or pixel value, and may represent only the pixel / pixel value of the luminance component or only the pixel / pixel value of the chrominance component. A sample can be used as a term corresponding to a picture (or image) of pixels or pictographs.

[0054] In the encoding device 200, the predicted signal (prediction block, prediction sample array) output from the inter-frame predictor 221 or the intra-frame predictor 222 is subtracted from the input image signal (original block, original sample array) to generate a residual signal (residual block, residual sample array), and the generated residual signal is sent to the converter 232. In this case, as shown, the unit in the encoding device 200 used to subtract the predicted signal (prediction block, prediction sample array) from the input image signal (original block, original sample array) can be called the subtractor 231. The predictor can perform prediction on the block to be processed (hereinafter referred to as the current block) and generate a prediction block that includes the prediction samples of the current block. The predictor can determine whether to apply intra-frame prediction or inter-frame prediction based on the current block or CU. As described later in the description of each prediction mode, the predictor can generate various information related to the prediction, such as prediction mode information, and send the generated information to the entropy encoder 240. The information about the prediction can be encoded in the entropy encoder 240 and output as a bitstream.

[0055] Intra-predictor 222 can predict the current block by referencing samples in the current image. Depending on the prediction mode, the referenced samples may be located near or far from the current block. In intra-prediction, the prediction mode can include multiple non-directional modes and multiple directional modes. Non-directional modes can include, for example, DC mode and planar mode. Depending on the level of detail in the prediction direction, the directional modes can include, for example, 33 or 65 directional prediction modes. However, this is just an example, and more or fewer directional prediction modes may be used depending on the settings. Intra-predictor 222 can determine the prediction mode to be applied to the current block by using prediction modes applied to neighboring blocks.

[0056] Inter-frame predictor 221 can deduce the predicted block of the current block based on a reference block (reference sample array) specified by motion vectors on a reference image. Here, to reduce the amount of motion information transmitted in inter-frame prediction mode, motion information can be predicted on a block, sub-block, or sample basis based on the correlation between motion information between neighboring blocks and the current block. Motion information may include motion vectors and reference image indices. Motion information may also include inter-frame prediction direction (L0 prediction, L1 prediction, Bi prediction, etc.) information. In the case of inter-frame prediction, neighboring blocks may include spatially neighboring blocks existing in the current image and temporally neighboring blocks existing in the reference image. The reference image including the reference block and the reference image including the temporally neighboring block may be the same or different. The temporally neighboring block may be called a juxtaposed reference block, a co-located CU (colCU), etc., and the reference image including the temporally neighboring block may be called a juxtaposed image (colPic). For example, inter-frame predictor 221 can configure a motion information candidate list based on neighboring blocks and generate information indicating which candidate to use to deduce the motion vector and / or reference image index of the current block. Inter-frame prediction can be performed based on various prediction modes. For example, in skip mode and merge mode, the inter-frame predictor 221 can use motion information from neighboring blocks as motion information for the current block. In skip mode, unlike merge mode, residual signals may not be sent. In motion vector prediction (MVP) mode, motion vectors from neighboring blocks can be used as motion vector predictors, and the motion vector of the current block can be indicated by signaling the motion vector difference.

[0057] Predictor 220 can generate prediction signals based on various prediction methods described below. For example, the predictor can not only apply intra-frame prediction or inter-frame prediction to predict a block, but can also apply both intra-frame prediction and inter-frame prediction simultaneously. This can be referred to as Inter-intra-frame Combined Prediction (CIIP). Alternatively, the predictor can predict blocks based on an Intra-Block Copy (IBC) prediction mode or a palette mode. IBC prediction modes or palette modes can be used for content image / video coding, such as screen content coding (SCC), for games, etc. IBC essentially performs prediction in the current frame, but can be performed similarly to inter-frame prediction because the reference block is derived in the current frame. That is, IBC can use at least one of the inter-frame prediction techniques described in this disclosure. A palette mode can be considered as an example of intra-frame coding or intra-frame prediction. When a palette mode is applied, sample values ​​within the frame can be signaled based on information about the palette table and palette index.

[0058] The predicted signal generated by the predictor (including inter-frame predictor 221 and / or intra-frame predictor 222) can be used to generate a reconstructed signal or a residual signal. Transformer 232 can generate transform coefficients by applying transform techniques to the residual signal. For example, the transform technique can include at least one of Discrete Cosine Transform (DCT), Discrete Sine Transform (DST), Karhunen-Loève Transform (KLT), Graph-Based Transform (GBT), or Conditional Nonlinear Transform (CNT). Here, GBT refers to a transform obtained from a graph when the relationship information between pixels is represented by a graph. CNT refers to a transform generated based on the predicted signal generated using all previously reconstructed pixels. Furthermore, the transform processing can be applied to square pixel blocks of the same size, or it can be applied to blocks of variable size that are not square.

[0059] The quantizer 233 quantizes the transform coefficients and sends them to the entropy encoder 240, which encodes the quantized signal (information about the quantized transform coefficients) and outputs a bitstream. This information about the quantized transform coefficients can be called residual information. The quantizer 233 can rearrange the block-type quantized transform coefficients into a one-dimensional vector based on the coefficient scan order and generate information about the quantized transform coefficients based on this one-dimensional vector form. The entropy encoder 240 can perform various encoding methods, such as Golomb, Context Adaptive Variable Length Coding (CAVLC), and Context Adaptive Binary Arithmetic Coding (CABAC). The entropy encoder 240 can encode information required for video / image reconstruction other than the quantized transform coefficients (e.g., values ​​of syntax elements) together or separately. It can send or store encoded information (e.g., encoded video / image information) in bitstream form at NAL (Network Abstraction Layer) units. The video / image information may also include information about various parameter sets such as Adaptive Parameter Set (APS), Picture Parameter Set (PPS), Sequence Parameter Set (SPS), or Video Parameter Set (VPS). Additionally, the video / image information may include general constraint information. In this disclosure, information and / or syntax elements that transmit / signal notifications from the encoding device to the decoding device may be included in the video / image information. The video / image information may be encoded by the encoding process described above and included in a bitstream. The bitstream may be transmitted over a network or stored in a digital storage medium. The network may include broadcast networks and / or communication networks, and the digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. A transmitter (not shown) that transmits a signal output from the entropy encoder 240 and / or a storage unit (not shown) that stores the signal may be included as internal / external components of the encoding device 200, and alternatively, the transmitter may be included in the entropy encoder 240.

[0060] The quantization transform coefficients output from quantizer 233 can be used to generate a prediction signal. For example, the residual signal (residual block or residual sample) can be reconstructed by applying inverse quantization and inverse transform to the quantization transform coefficients using inverse quantizer 234 and inverse transformer 235. Adder 250 adds the reconstructed residual signal to the prediction signal output from inter-frame predictor 221 or intra-frame predictor 222 to generate a reconstructed signal (reconstructed image, reconstructed block, reconstructed sample array). If the block to be processed has no residual (such as when a skip mode is applied), the prediction block can be used as a reconstructed block. Adder 250 can be called a reconstructor or reconstructed block generator. The generated reconstructed signal can be used for intra-frame prediction of the next block to be processed in the current image, and can be used for inter-frame prediction of the next image by filtering as described below.

[0061] In addition, Luminance Mapping and Chroma Scaling (LMCS) can be applied during image encoding and / or reconstruction.

[0062] Filter 260 can improve subjective / objective image quality by applying filtering to the reconstructed signal. For example, filter 260 can generate a modified reconstructed image by applying various filtering methods to the reconstructed image and store the modified reconstructed image in memory 270 (specifically, the DPB of memory 270). Various filtering methods may include, for example, deblocking filtering, sample adaptive offset, adaptive loop filtering, bilateral filtering, etc. Filter 260 can generate various filtering-related information and send the generated information to entropy encoder 240, as described later in the description of the various filtering methods. The filtering-related information can be encoded by entropy encoder 240 and output as a bitstream.

[0063] The modified reconstructed image sent to memory 270 can be used as a reference image in inter-frame predictor 221. When inter-frame prediction is applied through the encoding device, prediction mismatch between the encoding device 200 and the decoding device can be avoided, and encoding efficiency can be improved.

[0064] The DPB of memory 270 can store a modified reconstructed image used as a reference image in inter-frame predictor 221. Memory 270 can store motion information of blocks from which motion information in the current image is derived (or encoded) and / or motion information of reconstructed blocks in the image. The stored motion information can be sent to inter-frame predictor 221 and used as motion information for spatially or temporally neighboring blocks. Memory 270 can store reconstructed samples of reconstructed blocks in the current image and can transmit these reconstructed samples to intra-frame predictor 222.

[0065] Figure 3 This is a schematic diagram illustrating the configuration of a video / image decoding device to which embodiments of the present disclosure can be applied.

[0066] Reference Figure 3 The decoding device 300 may include an entropy decoder 310, a residual processor 320, a predictor 330, an adder 340, a filter 350, and a memory 360. The predictor 330 may include an inter-frame predictor 332 and an intra-frame predictor 331. The residual processor 320 may include an inverse quantizer 321 and an inverse transformer 322. According to embodiments, the entropy decoder 310, residual processor 320, predictor 330, adder 340, and filter 350 may be constructed from hardware components (e.g., a decoder chipset or processor). Additionally, the memory 360 may include a decoded image buffer (DPB) or may be constructed from a digital storage medium. The hardware components may also include the memory 360 as an internal / external component.

[0067] When the input includes a bitstream containing video / image information, the decoding device 300 can interact with... Figure 2 The processing of video / image information in the encoding device correspondingly reconstructs the image. For example, the decoding device 300 can deduce units / blocks based on block segmentation information obtained from the bitstream. The decoding device 300 can use a processor applied in the encoding device to perform decoding. Therefore, the decoding processor can be, for example, an encoding unit, and the encoding unit can be segmented from the encoding tree unit or the maximum encoding unit according to a quadtree structure, binary tree structure, and / or ternary tree structure. One or more transform units can be derived from the encoding unit. The reconstructed image signal decoded and output by the decoding device 300 can be reproduced by a reproduction device.

[0068] Decoding device 300 can receive data in bitstream form from... Figure 2The signal output by the encoding device can be decoded by the entropy decoder 310. For example, the entropy decoder 310 can parse the bitstream to derive information (e.g., video / image information) required for image reconstruction (or picture reconstruction). The video / image information may also include information about various parameter sets such as Adaptive Parameter Set (APS), Picture Parameter Set (PPS), Sequence Parameter Set (SPS), or Video Parameter Set (VPS). In addition, the video / image information may also include general constraint information. The decoding device can also decode the picture based on the information about the parameter sets and / or general constraint information. The signaling / receiving information and / or syntax elements described later in this disclosure can be decoded and obtained from the bitstream through the decoding process. For example, the entropy decoder 310 decodes the information in the bitstream based on encoding methods such as exponential Golomb coding, CAVLC, or CABAC, and outputs the quantized values ​​of the syntax elements and transform coefficients of the residuals required for image reconstruction. More specifically, the CABAC entropy decoding method can receive a bin corresponding to each syntax element in the bitstream, determine the context model using information about the target syntax element, decoding information about the target block, or information about symbols / bins decoded in previous stages, and perform arithmetic decoding on the bin by predicting the occurrence probability of the bin based on the determined context model, generating symbols corresponding to the value of each syntax element. In this case, after determining the context model, the CABAC entropy decoding method can update the context model by using the information of the decoded symbols / bins for the context model of the next symbol / bin. Prediction-related information from the information decoded by the entropy decoder 310 can be provided to the predictors (inter-frame predictor 332 and intra-frame predictor 331), and the residual values ​​(i.e., quantization transform coefficients and related parameter information) from which entropy decoding was performed in the entropy decoder 310 can be input to the residual processor 320. The residual processor 320 can derive the residual signals (residual blocks, residual samples, residual sample arrays). Additionally, filtering information from the information decoded by the entropy decoder 310 can be provided to the filter 350. Furthermore, the receiver (not shown) for receiving the signal output from the encoding device can be further configured as an internal / external element of the decoding device 300, or the receiver can be a component of the entropy decoder 310. Additionally, the decoding device according to this disclosure can be referred to as a video / image / picture decoding device, and the decoding device can be classified as an information decoder (video / image / picture information decoder) and a sample decoder (video / image / picture sample decoder). The information decoder may include the entropy decoder 310, and the sample decoder may include at least one of an inverse quantizer 321, an inverse transformer 322, an adder 340, a filter 350, a memory 360, an inter-frame predictor 332, and an intra-frame predictor 331.

[0069] The dequantizer 321 can dequantize the quantized transform coefficients and output the transform coefficients. The dequantizer 321 can rearrange the quantized transform coefficients in the form of two-dimensional blocks. In this case, the rearrangement can be performed based on the coefficient scan order executed in the encoding device. The dequantizer 321 can perform dequantization on the quantized transform coefficients using quantization parameters (e.g., quantization step size information) and obtain the transform coefficients.

[0070] The inverse transformer 322 performs an inverse transformation on the transformation coefficients to obtain the residual signal (residual block, residual sample array).

[0071] The predictor can perform prediction on the current block and generate a prediction block that includes the prediction samples of the current block. The predictor can determine whether to apply intra-frame prediction or inter-frame prediction to the current block based on the prediction information output from the entropy decoder 310, and can determine the specific intra-frame / inter-frame prediction mode.

[0072] The predictor can generate a prediction signal based on various prediction methods described below. For example, the predictor can not only apply intra-frame prediction or inter-frame prediction to predict a block, but can also apply both intra-frame prediction and inter-frame prediction simultaneously. This can be referred to as combined intra-frame and inter-frame prediction (CIIP). Alternatively, the predictor can predict blocks based on an intra-block copy (IBC) prediction mode or a palette mode. The IBC prediction mode or palette mode can be used for content image / video coding, such as screen content coding (SCC), for games, etc. IBC essentially performs prediction within the current image, but can be performed similarly to inter-frame prediction because a reference block is derived within the current image. That is, IBC can use at least one of the inter-frame prediction techniques described in this disclosure. The palette mode can be considered an example of intra-frame coding or intra-frame prediction. When applying a palette mode, sample values ​​within the image can be signaled based on information about the palette table and palette index.

[0073] Intra-predictor 331 can predict the current block by referencing samples in the current image. Depending on the prediction mode, the referenced samples may be located near or far from the current block. In intra-prediction, the prediction mode can include multiple non-directional modes and multiple directional modes. Intra-predictor 331 can determine the prediction mode applied to the current block by using prediction modes applied to neighboring blocks.

[0074] Inter-frame predictor 332 can deduce the predicted block of the current block based on a reference block (reference sample array) specified by a motion vector on a reference image. In this case, to reduce the amount of motion information transmitted in the inter-frame prediction mode, motion information can be predicted on a block, sub-block, or sample basis based on the correlation of motion information between neighboring blocks and the current block. Motion information may include motion vectors and reference image indices. Motion information may also include inter-frame prediction direction (L0 prediction, L1 prediction, Bi prediction, etc.) information. In the case of inter-frame prediction, neighboring blocks may include spatially neighboring blocks existing in the current image and temporally neighboring blocks existing in the reference image. For example, inter-frame predictor 332 can configure a motion information candidate list based on neighboring blocks and deduce the motion vector and / or reference image index of the current block based on the received candidate selection information. Inter-frame prediction can be performed based on various prediction modes, and the information about the prediction may include information indicating the mode of inter-frame prediction for the current block.

[0075] Adder 340 can generate a reconstruction signal (reconstructed image, reconstruction block, reconstruction sample array) by adding the obtained residual signal to the prediction signal (prediction block, prediction sample array) output from the predictor (including inter-frame predictor 332 and / or intra-frame predictor 331). If the block to be processed has no residual (e.g., when a skip mode is applied), the prediction block can be used as the reconstruction block.

[0076] Adder 340 can be called a reconstructor or reconstruction block generator. The generated reconstructed signal can be used for intra-frame prediction of the next block to be processed in the current image, through filtering output as described below, or it can be used for inter-frame prediction of the next image.

[0077] In addition, Luminance Mapping and Chromaticity Scaling (LMCS) can be applied during image decoding.

[0078] Filter 350 can improve subjective / objective image quality by applying filtering to the reconstructed signal. For example, filter 350 can generate a modified reconstructed image by applying various filtering methods to the reconstructed image and store the modified reconstructed image in memory 360 (specifically, the DPB of memory 360). Various filtering methods may include, for example, deblocking filtering, sample adaptive shifting, adaptive loop filtering, bilateral filtering, etc.

[0079] The (modified) reconstructed image stored in the DPB of memory 360 can be used as a reference image in inter-frame predictor 332. Memory 360 can store motion information of blocks from which motion information in the current image is derived (or decoded) and / or motion information of reconstructed blocks in the image. The stored motion information can be sent to inter-frame predictor 332 for use as motion information of spatially or temporally neighboring blocks. Memory 360 can store reconstructed samples of reconstructed blocks in the current image and can transmit the reconstructed samples to intra-frame predictor 331.

[0080] In this disclosure, the embodiments described in the filter 260, inter-frame predictor 221, and intra-frame predictor 222 of the encoding device 200 can be the same as, or applied to, the filter 350, inter-frame predictor 332, and intra-frame predictor 331 of the decoding device 300, respectively. The same content can also be applied to the inter-frame predictor 332 and intra-frame predictor 331.

[0081] In this disclosure, at least one of quantization / inverse quantization and / or transformation / inverse transformation may be omitted. When quantization / inverse quantization is omitted, the transformation coefficients of the quantization may be referred to as transformation coefficients. When transformation / inverse transformation is omitted, the transformation coefficients may be referred to as coefficients or residual coefficients, or for the sake of consistency, they may still be referred to as transformation coefficients.

[0082] In this disclosure, quantization transform coefficients and transform coefficients can be referred to as transform coefficients and scaling transform coefficients, respectively. In this case, residual information can include information about the transform coefficients, and this information can be signaled via residual coding syntax. Transform coefficients can be derived based on residual information (or information about the transform coefficients), and scaling transform coefficients can be derived by inverse transforming (scaling) the transform coefficients. Residual samples can be derived based on the inverse transform (scaling) of the scaling transform coefficients. This can also be applied / expressed in other parts of this disclosure.

[0083] In addition, the image output and removal process in the Decoded Picture Buffer (DPB) can be performed. The image output and removal process in the Decoded Picture Buffer (DPB) of the existing VVC standard for video / image coding systems can be shown in the table below.

[0084] [Table 1]

[0085] For example, according to the VVC standard for video / image coding systems, the image output procedure can be called once for each image before decoding the current image (but after parsing the slice header of the first slice of the current image), as disclosed in the table above.

[0086] Additionally, for example, referring to Table 1, when the current access unit (AU) is the start AU of the encoded video sequence other than AU 0 (CVSS AU), the following steps can be applied in sequence.

[0087] - First, for the decoder to be tested, the variable NoOutputOfPriorPicsFlag can be derived as follows.

[0088] - When the values ​​of PicWidthMaxInSamplesY, PicHeightMaxInSamplesY, MaxChromaFormat, MaxBitDepthMinus8, or max_dec_pic_buffering_minus1[Htid] derived for the current AU are different from the values ​​of PicWidthMaxInSamplesY, PicHeightMaxInSamplesY, MaxChromaFormat, MaxBitDepthMinus8, or max_dec_pic_buffering_minus1[Htid] derived for a previous AU, NoOutputOfPriorPicsFlag can be set to 1 by the decoder under test, regardless of the value of ph_no_output_of_prior_pics_flag of the current AU.

[0089] - Otherwise, NoOutputOfPriorPicsFlag can be set to the value of ph_no_output_of_prior_pics_flag for the current AU.

[0090] Secondly, the variable NoOutputOfPriorPicsFlag, derived for the decoder under test, can be applied to the hypothetical reference decoder (HRD). Therefore, when the value of NoOutputOfPriorPicsFlag is 1, all image storage buffers in the DPB can be cleared without outputting the images they contain, and the DPB fullness can be set to 0.

[0091] Additionally, for example, referring to Table 1, all such images k in DPB can be removed from DPB when all of the following conditions are true for any image k in DPB.

[0092] Image k is marked "not for reference".

[0093] - Image k has a PictureOutputFlag equal to 0 or the DPB output time of image k is less than or equal to the CPB removal time of the first decoding unit (DU) (denoted as DU m) of the current image n; that is, DpbOutputTime[k] is less than or equal to DuCpbRemovalTime[m].

[0094] Additionally, for example, referring to Table 1, when the current access unit (AU) is the start AU of the encoded video sequence other than AU 0 (CVSS AU), the following steps can be applied in sequence.

[0095] - First, for the decoder to be tested, the variable NoOutputOfPriorPicsFlag can be derived as follows.

[0096] - When the values ​​of PicWidthMaxInSamplesY, PicHeightMaxInSamplesY, MaxChromaFormat, MaxBitDepthMinus8, or max_dec_pic_buffering_minus1[Htid] derived for the current AU are different from the values ​​of PicWidthMaxInSamplesY, PicHeightMaxInSamplesY, MaxChromaFormat, MaxBitDepthMinus8, or max_dec_pic_buffering_minus1[Htid] derived for a previous AU, NoOutputOfPriorPicsFlag can be set to 1 by the decoder under test, regardless of the value of ph_no_output_of_prior_pics_flag of the current AU.

[0097] - Otherwise, NoOutputOfPriorPicsFlag can be set to the value of ph_no_output_of_prior_pics_flag for the current AU.

[0098] - Secondly, the variable NoOutputOfPriorPicsFlag derived for the decoder under test can be applied to the HRD (hypothetical reference decoder) as follows.

[0099] - For example, when the value of NoOutputOfPriorPicsFlag is 1, all image storage buffers in the DPB can be cleared without outputting the images they contain, and the DPB fullness can be set to 0.

[0100] - Otherwise (i.e., when the value of NoOutputOfPriorPicsFlag is 0), all picture storage buffers in the DPB that contain pictures marked as "no output required" and "not for reference" can be cleared (without output), and all non-empty picture storage buffers in the DPB can be cleared by repeatedly calling the "bumping" procedure specified in Clause C.5.2.4 of the (VVC standard), and the DPB fill level can be set to 0.

[0101] Furthermore, the concave-convex process can consist of the following sequential steps.

[0102] 1. Among all the images in the DPB marked "Required Output", the first image (or multiple images) to be output can be selected as the one with the smallest PicOrderCntVal value.

[0103] 2. A consistent cropping window for images can be used to crop each image in ascending order of nuh_layer_id, the cropped images can be output, and images can be marked as "not needed for output".

[0104] 3. Each image storage buffer containing an image marked "not for reference" and that is one of the images to be cropped and output can be cleared, and the DPB's fullness can be decremented by 1.

[0105] Additionally, for example, referring to Table 1, when the current AU is not a CVSS AU, all picture storage buffers containing pictures marked "No Output Required" and "Not for Reference" can be cleared (without output). For each picture storage buffer, the DPB fullness can be decremented by 1. Furthermore, the "bump" procedure specified in Clause C.5.2.4 (of the VVC standard) can be repeatedly invoked when one or more of the following conditions are true, further decrementing the DPB fullness by 1 for each additional picture storage buffer cleared, until none of the following conditions are true.

[0106] - The number of images marked as "needs output" in the DPB is greater than max_num_reorder_pics[Htid].

[0107] - max_latency_increase_plus1[Htid] is not equal to 0, and there is at least one picture in the DPB that is marked as "needs output" and whose associated variable PicLatencyCount is greater than or equal to MaxLatencyPictures[Htid].

[0108] The number of images in the DPB is greater than or equal to max_dec_pic_buffering_minus1[Htid]+1.

[0109] On the other hand, the existing VVC standard used for the above image output and removal process may have the following problems.

[0110] For example, after decoding all slices of an image, the image can be marked as "for short-term reference." Therefore, the image cannot be in a cleared state in the DPB when it is decoded. As a result, the number of images stored in the DPB may be affected.

[0111] Secondly, the assignment of the image's output state (i.e., whether it needs to be output) may be performed during the bumping process. According to the existing VVC standard, this process cannot be called for images that are the starting AU of a encoded video sequence. Therefore, the value of PicLatencyCount associated with the corresponding image cannot be initialized.

[0112] As mentioned above, the process of outputting and removing images in the DPB can be called once per image, but this process may affect the state of the DPB shared by all layers of CVS (i.e., the state of the images stored in the DPB). Given this, the process of deriving NoOutputOfPriorPicsFlag from the DPB and removing images based on the value of NoOutputOfPriorPicsFlag may be problematic. According to existing video / image standards, for all images in CVSS AUs except AU 0, the process of deriving and removing images from the DPB based on the value of the flag (i.e., NoOutputOfPriorPicsFlag) can be called. The execution of this process as described above may only be possible for the first image. The process starting from the second image may remove previous images from the DPB before outputting the previous images (i.e., the images in the previous order according to the decoding order). This behavior may not be correct decoder behavior.

[0113] Accordingly, this disclosure proposes a solution to the above-mentioned problems. The proposed implementations can be applied independently or in combination.

[0114] As an example, the procedure for deducing whether to remove the reference image from the DPB without outputting the value of its flag or variable can be called only once per access unit (AU). That is, for example, a method can be proposed such that the procedure for deducing whether to remove the reference image from the DPB without outputting the value of its flag or variable is called only once per access unit (AU). Here, the variable could be NoOutputOfPriorPicsFlag.

[0115] Additionally, as an example, the process of deriving the value of NoOutputOfPriorPicsFlag can be called before the decoding process of the first image in the CVSS AU (CVSS AU), but after parsing the slice header of the first slice of the current image. That is, for example, the following method can be proposed: Perform the process of deriving the value of NoOutputOfPriorPicsFlag before the decoding process of the first image in the CVSS AU, but after parsing the slice header of the first slice of the current image.

[0116] Additionally, as an example, when NoOutputOfPriorPicsFlag is 1, the procedure for removing images stored in the DPB without outputting them can be called only once per AU. That is, for example, when NoOutputOfPriorPicsFlag is 1, the procedure for removing images stored in the DPB without outputting them can be proposed and called only once per AU.

[0117] Additionally, as an example, when NoOutputOfPriorPicsFlag is 1, before the decoding process of the first image in the CVSS AU but after parsing the slice header of the first slice of the current image, a process can be called to remove the image stored in the DPB without outputting it. That is, for example, when NoOutputOfPriorPicsFlag is 1, the following method can be proposed: before the decoding process of the first image in the CVSS AU but after parsing the slice header of the first slice of the current image, a process can be called to remove the image stored in the DPB without outputting it.

[0118] Furthermore, as an example, the image removal in the above embodiments may not include the removal of the current image in the DPB. That is, for example, a method in the above embodiments that excludes the removal of the current image in the DPB can be proposed.

[0119] The above implementation methods can be implemented as follows. For example, the above implementation methods can be represented based on the VVC standard specification, as described below.

[0120] [Table 2]

[0121] For example, referring to Table 2, when the current image is the first image and the current AU (i.e., the AU including the current image) is the coded video sequence start AU (CVSS AU) other than AU 0, the following steps can be applied in sequence.

[0122] - First, for the decoder to be tested, the variable NoOutputOfPriorPicsFlag can be derived as follows.

[0123] - When the values ​​of PicWidthMaxInSamplesY, PicHeightMaxInSamplesY, MaxChromaFormat, MaxBitDepthMinus8, or max_dec_pic_buffering_minus1[Htid] derived for the current AU are different from the values ​​of PicWidthMaxInSamplesY, PicHeightMaxInSamplesY, MaxChromaFormat, MaxBitDepthMinus8, or max_dec_pic_buffering_minus1[Htid] derived for a previous AU, NoOutputOfPriorPicsFlag can be set to 1 by the decoder under test, regardless of the value of ph_no_output_of_prior_pics_flag of the current AU.

[0124] - Otherwise, NoOutputOfPriorPicsFlag can be set to the value of ph_no_output_of_prior_pics_flag for the current AU.

[0125] Secondly, the variable NoOutputOfPriorPicsFlag, derived for the decoder under test, can be applied to the hypothetical reference decoder (HRD). Therefore, when the value of NoOutputOfPriorPicsFlag is 1, all image storage buffers in the DPB can be cleared without outputting the images they contain, and the DPB fullness can be set to 0.

[0126] Additionally, for example, referring to Table 2, all such images k in the DPB can be removed from the DPB when the current AU is not a CVSS AU or the current AU is a CVSS AU other than AU 0 but the current image is not the first image in the current AU, and when all of the following conditions are true for any image k in the DPB.

[0127] Image k is marked "not for reference".

[0128] - Image k has a PictureOutputFlag equal to 0 or the DPB output time of image k is less than or equal to the CPB removal time of the first decoding unit (DU) (denoted as DU m) of the current image n; that is, DpbOutputTime[k] is less than or equal to DuCpbRemovalTime[m].

[0129] Additionally, for example, referring to Table 2, when the current image is the first image and the current AU (i.e., the AU including the current image) is the coded video sequence start AU (CVSS AU) other than AU 0, the following steps can be applied in sequence.

[0130] - First, for the decoder to be tested, the variable NoOutputOfPriorPicsFlag can be derived as follows.

[0131] - When the values ​​of PicWidthMaxInSamplesY, PicHeightMaxInSamplesY, MaxChromaFormat, MaxBitDepthMinus8, or max_dec_pic_buffering_minus1[Htid] derived for the current AU are different from the values ​​of PicWidthMaxInSamplesY, PicHeightMaxInSamplesY, MaxChromaFormat, MaxBitDepthMinus8, or max_dec_pic_buffering_minus1[Htid] derived for a previous AU, NoOutputOfPriorPicsFlag can be set to 1 by the decoder under test, regardless of the value of ph_no_output_of_prior_pics_flag of the current AU.

[0132] - Otherwise, NoOutputOfPriorPicsFlag can be set to the value of ph_no_output_of_prior_pics_flag for the current AU.

[0133] - Secondly, the variable NoOutputOfPriorPicsFlag, which is derived for the decoder under test, can be applied to the hypothetical reference decoder (HRD) as follows.

[0134] - For example, when the value of NoOutputOfPriorPicsFlag is 1, all image storage buffers in the DPB can be cleared without outputting the images they contain, and the DPB fullness can be set to 0.

[0135] - Otherwise (i.e., when the value of NoOutputOfPriorPicsFlag is 0), all picture storage buffers in the DPB that contain pictures marked as "no output required" and "not for reference" can be cleared (without output), and all non-empty picture storage buffers in the DPB can be cleared by repeatedly calling the "bump" procedure specified in Clause C.5.2.4 of the (VVC standard), and the DPB fill level can be set to 0.

[0136] Additionally, for example, referring to Table 2, when the current AU is not a CVSS AU, or when the current AU is a CVSS AU other than AU 0 but the current image is not the first image of the current AU, all image storage buffers containing images marked "No output required" and "Not for reference" can be cleared (without output). For each image storage buffer cleared, the DPB fullness can be decremented by 1. Furthermore, the "bump" procedure specified in Clause C.5.2.4 (of the VVC standard) can be repeatedly invoked when one or more of the following conditions are true, further decrementing the DPB fullness by 1 for each additional image storage buffer cleared, until none of the following conditions are true.

[0137] - The number of images marked as "needs output" in the DPB is greater than max_num_reorder_pics[Htid].

[0138] - max_latency_increase_plus1[Htid] is not equal to 0, and there is at least one picture in the DPB that is marked as "needs output" and whose associated variable PicLatencyCount is greater than or equal to MaxLatencyPictures[Htid].

[0139] - The number of images in the DPB is greater than or equal to max_dec_pic_buffering_minus1[Htid]+1.

[0140] Alternatively, the above implementation can be carried out as follows. For example, the above implementation can be based on the VVC standard specification, as described below.

[0141] [Table 3]

[0142] For example, referring to Table 3, when the current image is the first image of the current AU and the current AU is the start AU of the encoded video sequence other than AU 0 (CVSS AU), the following steps can be applied in sequence.

[0143] - First, for the decoder to be tested, the variable NoOutputOfPriorPicsFlag can be derived as follows.

[0144] - When the values ​​of PicWidthMaxInSamplesY, PicHeightMaxInSamplesY, MaxChromaFormat, MaxBitDepthMinus8, or max_dec_pic_buffering_minus1[Htid] derived for the current AU are different from the values ​​of PicWidthMaxInSamplesY, PicHeightMaxInSamplesY, MaxChromaFormat, MaxBitDepthMinus8, or max_dec_pic_buffering_minus1[Htid] derived for a previous AU, NoOutputOfPriorPicsFlag can be set to 1 by the decoder under test, regardless of the value of ph_no_output_of_prior_pics_flag of the current AU.

[0145] - Otherwise, NoOutputOfPriorPicsFlag can be set to the value of ph_no_output_of_prior_pics_flag for the current AU.

[0146] Secondly, the variable NoOutputOfPriorPicsFlag, derived for the decoder under test, can be applied to the hypothetical reference decoder (HRD). Therefore, when the value of NoOutputOfPriorPicsFlag is 1, all image storage buffers in the DPB can be cleared without outputting the images they contain, and the DPB fullness can be set to 0.

[0147] Additionally, for example, referring to Table 3, all such images k in the DPB can be removed from the DPB when the current AU is not a CVSS AU or the current AU is a CVSS AU other than AU 0 but the current image is not the first image of the current AU, and when all of the following conditions are true for any image k in the DPB.

[0148] Image k is marked "not for reference".

[0149] - Image k has a PictureOutputFlag equal to 0 or the DPB output time of image k is less than or equal to the CPB removal time of the first decoding unit (DU) (denoted as DU m) of the current image n; that is, DpbOutputTime[k] is less than or equal to DuCpbRemovalTime[m].

[0150] Additionally, for example, referring to Table 3, when the current image is the first image and the current access unit (AU) is the coded video sequence start AU (CVSS AU) other than AU 0, the following steps can be applied in sequence.

[0151] - First, for the decoder to be tested, the variable NoOutputOfPriorPicsFlag can be derived as follows.

[0152] - When the values ​​of PicWidthMaxInSamplesY, PicHeightMaxInSamplesY, MaxChromaFormat, MaxBitDepthMinus8, or max_dec_pic_buffering_minus1[Htid] derived for the current AU are different from the values ​​of PicWidthMaxInSamplesY, PicHeightMaxInSamplesY, MaxChromaFormat, MaxBitDepthMinus8, or max_dec_pic_buffering_minus1[Htid] derived for a previous AU, NoOutputOfPriorPicsFlag can be set to 1 by the decoder under test, regardless of the value of ph_no_output_of_prior_pics_flag of the current AU.

[0153] - Otherwise, NoOutputOfPriorPicsFlag can be set to the value of ph_no_output_of_prior_pics_flag for the current AU.

[0154] - Secondly, the variable NoOutputOfPriorPicsFlag, which is derived for the decoder under test, can be applied to the hypothetical reference decoder (HRD) as follows.

[0155] - For example, when the value of NoOutputOfPriorPicsFlag is 1, all image storage buffers in the DPB can be cleared without outputting the images they contain, and the DPB fullness can be set to 0.

[0156] - Otherwise (i.e., when the value of NoOutputOfPriorPicsFlag is 0), all picture storage buffers in the DPB that contain pictures marked as "no output required" and "not for reference" can be cleared (without output), and all non-empty picture storage buffers in the DPB can be cleared by repeatedly calling the "bump" procedure specified in Clause C.5.2.4 of the (VVC standard), and the DPB fill level can be set to 0.

[0157] Additionally, for example, referring to Table 3, when the current AU is not a CVSS AU or the current AU is a CVSS AU other than AU 0 but the current image is not the first image of the current AU, all image storage buffers containing images marked "No output required" and "Not for reference" can be cleared (without output). For each image storage buffer cleared, the DPB fullness can be decremented by 1. Furthermore, the "bump" procedure specified in Clause C.5.2.4 (of the VVC standard) can be repeatedly invoked when one or more of the following conditions are true, further decrementing the DPB fullness by 1 for each additional image storage buffer cleared, until none of the following conditions are true.

[0158] - The number of images marked as "needs output" in the DPB is greater than max_num_reorder_pics[Htid].

[0159] - max_latency_increase_plus1[Htid] is not equal to 0, and there is at least one picture in the DPB that is marked as "needs output" and whose associated variable PicLatencyCount is greater than or equal to MaxLatencyPictures[Htid].

[0160] - The number of images in the DPB is greater than or equal to max_dec_pic_buffering_minus1[Htid]+1.

[0161] Furthermore, for example, the implementation method can be applied according to the following process. One or more steps of the process, which will be described later, may be omitted.

[0162] Figure 4 An exemplary embodiment of the coding process according to an implementation of the present disclosure is illustrated.

[0163] Reference Figure 4 The encoding device decodes (restores) the image (S400). The encoding device can decode the image of the current AU.

[0164] The encoding device manages the DPB based on DPB parameters (S410). Here, DPB management can be referred to as DPB update. The DPB management process may include the process of marking and / or removing decoded images in the DPB. Decoded images can be used as references for inter-frame prediction of subsequent images. That is, decoded images can be used as reference images for inter-frame prediction of images that follow in the decoding order. Each decoded image can be substantially inserted into the DPB. In addition, the DPB can usually be updated before decoding the current image. When the layer associated with the DPB is not an output layer (or the DPB parameters are not associated with the output layer) and is a reference layer, the decoded images in the DPB cannot be output. If the layer associated with the DPB (or DPB parameters) is an output layer, the decoded images in the DPB can be output based on the DPB and / or DPB parameters. DPB management may include outputting decoded images from the DPB.

[0165] The encoding device encodes image information including information related to DPB parameters (S420). The information related to DPB parameters may include the information / syntax elements disclosed in the above embodiments and / or the syntax elements disclosed in the tables described later.

[0166] [Table 4]

[0167] For example, Table 4 above can represent a set of video parameters (VPS) that includes syntax elements for DPB parameters used for signal notification.

[0168] The semantics used for the grammatical elements shown in Table 4 above can be as follows.

[0169] [Table 5]

[0170] For example, the syntax element `vps_num_dpb_params` can indicate the number of `dpb_parameters()` syntax structures in the VPS. The value of `vps_num_dpb_params` can be in the range of 0 to 16. Additionally, when the syntax element `vps_num_dpb_params` is not present, its value can be inferred to be 0.

[0171] Additionally, for example, the syntax element `same_dpb_size_output_or_nonoutput_flag` can indicate whether the syntax element `layer_nonoutput_dpb_params_idx[i]` can exist in the VPS. For instance, when the value of the syntax element `same_dpb_size_output_or_nonoutput_flag` is 1, it indicates that the syntax element `layer_nonoutput_dpb_params_idx[i]` does not exist in the VPS, and when the value of the syntax element `same_dpb_size_output_or_nonoutput_flag` is 0, it indicates that the syntax element `layer_nonoutput_dpb_params_idx[i]` can exist in the VPS.

[0172] Additionally, for example, the syntax element `vps_sublayer_dpb_params_present_flag` can be used to control the existence of the syntax elements `max_dec_pic_buffering_minus1[]`, `max_num_reorder_pics[]`, and `max_latency_increase_plus1[]` in the `dpb_parameters()` syntax structure of the VPS. Furthermore, when the syntax element `vps_sublayer_dpb_params_present_flag` is absent, its value can be inferred to be 0.

[0173] Additionally, for example, the syntax element `dpb_size_only_flag[i]` can indicate whether the syntax elements `max_num_reorder_pics[]` and `max_latency_increase_plus1[]` can exist in the `i`th `dpb_parameters()` syntax structure of the VPS. For instance, when the value of `dpb_size_only_flag[i]` is 1, it indicates that `max_num_reorder_pics[]` and `max_latency_increase_plus1[]` do not exist in the `i`th `dpb_parameters()` syntax structure of the VPS. When the value of `dpb_size_only_flag[i]` is 0, it indicates that `max_num_reorder_pics[]` and `max_latency_increase_plus1[]` can exist in the `i`th `dpb_parameters()` syntax structure of the VPS.

[0174] Additionally, for example, the syntax element `dpb_max_temporal_id[i]` can indicate the `TemporalId` of the highest sublayer representation in the `i`th `dpb_parameters()` syntax structure within the VPS. Furthermore, the value of `dpb_max_temporal_id[i]` can be in the range of 0 to `vps_max_sublayers_minus1`. Additionally, for example, when the value of `vps_max_sublayers_minus1` is 0, the value of `dpb_max_temporal_id[i]` can be inferred to be 0. Additionally, for example, when the value of `vps_max_sublayers_minus1` is greater than 0 and `vps_all_layers_same_num_sublayers_flag` is 1, the value of `dpb_max_temporal_id[i]` can be inferred to be equal to `vps_max_sublayers_minus1`.

[0175] Additionally, for example, the syntax element `layer_output_dpb_params_idx[i]` can specify the index of the `dpb_parameters()` syntax structure applied to the i-th layer, which is the output layer of the OLS, to a list of `dpb_parameters()` syntax structures in the VPS. When the syntax element `layer_output_dpb_params_idx[i]` exists, its value can be in the range of 0 to `vps_num_dpb_params-1`.

[0176] For example, when vps_independent_layer_flag[i] is 1, it can be the dpb_parameters() syntax structure in the SPS that is referenced by the dpb_parameters() syntax structure layer of the i-th layer as the output layer.

[0177] Alternatively, for example, when vps_independent_layer_flag[i] is 0, the following can be applied.

[0178] - When vps_num_dpb_params is 1, the value of layer_output_dpb_params_idx[i] can be inferred to be 0.

[0179] - The requirement for bitstream consistency is that the value of layer_output_dpb_params_idx[i] makes the value of dpb_size_only_flag[layer_output_dpb_params_idx[i]] 0.

[0180] Additionally, for example, the syntax element `layer_nonoutput_dpb_params_idx[i]` can specify the index of the `dpb_parameters()` syntax structure applied to the i-th layer, which is a non-output layer of the OLS, in a list of `dpb_parameters()` syntax structures in the VPS. When the syntax element `layer_nonoutput_dpb_params_idx[i]` exists, its value can be in the range of 0 to `vps_num_dpb_params-1`.

[0181] For example, when same_dpb_size_output_or_nonoutput_flag is 1, the following can be applied.

[0182] - When vps_independent_layer_flag[i] is 1, it can be the dpb_parameters() syntax structure in the SPS that the layer references as the dpb_parameters() syntax structure of the i-th layer which is a non-output layer.

[0183] - When vps_independent_layer_flag[i] is 0, the value of layer_nonoutput_dpb_params_idx[i] can be inferred to be equal to layer_output_dpb_params_idx[i].

[0184] Alternatively, for example, when same_dpb_size_output_or_nonoutput_flag is 0, if vps_num_dpb_params is 1, then the value of layer_output_dpb_params_idx[i] can be inferred to be 0.

[0185] Furthermore, for example, the dpb_parameters() syntax structure, which is the DPB parameter syntax structure disclosed in Table 4 above, can be as follows.

[0186] [Table 6]

[0187] Referring to Table 6, the dpb_parameters() syntax structure can provide information about the DPB size, maximum number of image reorderings, and maximum wait time for each CLVS of CVS. The dpb_parameters() syntax structure can represent information about DPB parameters or DPB parameter information.

[0188] When the dpb_parameters() syntax structure is included in a VPS, the VPS can specify the OLS to apply the dpb_parameters() syntax structure. Alternatively, when the dpb_parameters() syntax structure is included in an SPS, the dpb_parameters() syntax structure can be applied to the OLS of only the lowest layer among the layers referencing the SPS, where the lowest layer can be an independent layer.

[0189] The semantics used for the grammatical elements shown in Table 6 above can be as follows.

[0190] [Table 7]

[0191] For example, the value obtained by adding 1 to the syntax element max_dec_pic_buffering_minus1[i] can represent the maximum required size of the DPB in units of the image storage buffer for each CLVS of CVS when Htid equals i. For example, max_dec_pic_buffering_minus1[i] can be information about the DPB size. For example, the value of the syntax element max_dec_pic_buffering_minus1[i] can be in the range of 0 to MaxDpbSize-1. Additionally, for example, when i is greater than 0, max_dec_pic_buffering_minus1[i] may be greater than or equal to max_dec_pic_buffering_minus1[i-1]. Additionally, for example, when there is no max_dec_pic_buffering_minus1[i] for i in the range from 0 to maxSubLayersMinus1-1, the value of the syntax element max_dec_pic_buffering_minus1[i] can be inferred to be equal to max_dec_pic_buffer_minus1[maxSubLayersMinus1] because subLayerInfoFlag is 0.

[0192] Additionally, for example, the syntax element `max_num_reorder_pics[i]` can represent, for each CLVS of a CVS, the maximum allowed number of CLVS pictures that, when `Htid` equals `i`, can be in decoding order before all pictures in the CLVS and in output order after those pictures. For example, `max_num_reorder_pics[i]` can be information about the maximum picture reordering number of the DPB. The value of `max_num_reorder_pics[i]` can be in the range of 0 to `max_dec_pic_buffering_minus1[i]`. Additionally, for example, when `i` is greater than 0, `max_num_reorder_pics[i]` can be greater than or equal to `max_num_reorder_pics[i-1]`. Additionally, for example, when `max_num_reorder_pics[i]` does not exist for `i` in the range of 0 to `maxSubLayersMinus1-1`, because `subLayerInfoFlag` is 0, the syntax element `max_num_reorder_pics[i]` can be inferred to be equal to `max_num_reorder_pics[maxSubLayersMinus1]`.

[0193] Additionally, for example, the value of MaxLatencyPictures[i] can be calculated using the syntax element max_latency_increase_plus1[i], which has a non-zero value. MaxLatencyPictures[i] can represent, for each CLVS of a CVS, the maximum number of CLVS images that can be output before all images in the CLVS in output order and after those images in decoding order when Htid equals i. For example, max_latency_increase_plus1[i] can be information about the maximum latency of the DPB.

[0194] For example, when max_latency_increase_plus1[i] is not 0, the value of MaxLatencyPictures[i] can be derived as follows.

[0195] [Formula 1]

[0196] On the other hand, for example, if max_latency_increase_plus1[i] is 0, then the corresponding limit may not be expressed. The value of max_latency_increase_plus1[i] can be between 0 and 2. 32 The range is -2. Additionally, for example, when there is no max_latency_increase_plus1[i] for i in the range from 0 to maxSubLayersMinus1-1, the syntax element max_latency_increase_plus1[i] can be inferred to be equal to max_latency_increase_plus1[maxSubLayersMinus1] because subLayerInfoFlag is 0.

[0197] The above DPB management can be performed based on information / syntax elements related to the DPB parameters. Other DPB parameters can be signaled based on whether the current layer is an output layer or a reference layer, or based on whether the DPB (or DPB parameter) is used for OLS (mapped to OLS).

[0198] Furthermore, despite Figure 4 Not illustrated, but the encoding device can decode the current image based on the updated / managed DPB. Furthermore, the decoded current image can be inserted into the DPB, and the DPB including the decoded current image can be updated based on DPB parameters before decoding the next image in the decoding order.

[0199] Figure 5 An exemplary decoding process according to an embodiment of the present disclosure is illustrated.

[0200] The decoding device obtains image information including information related to the DPB parameters from the bitstream (S500). The decoding device can obtain image information including information related to the DPB parameters. The information / syntax elements related to the DPB parameters can be as described above.

[0201] The decoding device manages the DPB based on the DPB parameters (S510). Here, DPB management can be referred to as DPB update. The DPB management process may include the process of marking and / or removing decoded images in the DPB. The decoding device can derive the DPB parameters based on information related to the DPB parameters, and can perform the DPB management process based on the derived DPB parameters.

[0202] The decoding device decodes / outputs the current image based on the DPB (S520). The decoding device can decode the current image based on an updated / managed DPB. For example, it can decode blocks / slices in the current image based on inter-frame predictions using (previously) decoded images in the DPB as reference images.

[0203] Figure 6 An image encoding method performed by an encoding device according to the present disclosure is illustrated schematically. Figure 6 The method disclosed in the article can be derived from Figure 2 The encoding device illustrated in the example performs this. Specifically, for example, Figure 6 S600 to S610 can be executed by the DPB of the encoding device, and S620 can be executed by the entropy encoder of the encoding device. Additionally, although not shown, the process of decoding the current image can be executed by the predictor and residual processor of the encoding device.

[0204] The encoding device derives the value of a variable (S600) based on whether the current image is the first image of the current AU of the coded video sequence start access unit (CVSS AU) other than access unit (AU) 0. The encoding device can derive the value of the variable to update the DPB before decoding the current image and after generating / encoding the slice header of the current image. The DPB may include images decoded before the current image.

[0205] For example, the encoding device can deduce the value of a variable based on whether the current image is the first image of the current AU of a CVSS AU other than Access Unit (AU) 0. Here, the variable can indicate whether all image storage buffers in the Decode Picture Buffer (DPB) have been cleared and there is no output. The current AU can be the AU that includes the current image. Alternatively, for example, AU 0 can be the first AU of the bitstream in decoding order. That is, for example, AU 0 can be the first AU of the bitstream to be decoded. On the other hand, the encoding device can generate / encode a slice header of the current image and then deduce the value of the variable based on whether the current image is the first image of the current AU of a CVSS AU other than Access Unit (AU) 0.

[0206] For example, the encoding device can determine whether the current image is the first image of the current AU other than access unit (AU) 0. When the current AU is a CVSS AU other than AU 0 and the current image is the first image of the current AU, the encoding device can deduce the value of the variable.

[0207] For example, when the current AU is a CVSS AU other than AU 0 and the current image is the first image of the current AU, the encoding device can determine whether at least one of the parameters of the current AU differs in decoding order from the parameters of the previous AUs. When at least one of the parameters of the current AU differs from the parameters of the previous AUs, the value of the variable can be set to 1, and when the parameters of the current AU are the same as the parameters of the previous AUs, the value of the variable can be set to the value of the syntax element used for the variable. The encoding device can generate / encode image information for the current image, and the image information can include syntax elements. The syntax element can be the ph_no_output_of_prior_pics_flag mentioned above. Furthermore, the parameters of the current AU include parameters for maximum image width, maximum image height, available chroma formats, maximum bit depth, and maximum DPB size. The parameters for maximum image width, maximum image height, available chroma format, maximum bit depth, and maximum DPB size can be PicWidthMaxInSamplesY, PicHeightMaxInSamplesY, MaxChromaFormat, MaxBitDepthMinus8, and max_dec_pic_buffering_minus1[Htid], respectively.

[0208] On the other hand, for example, when the current access unit (AU) is not a CVSS AU or the current picture is not the first picture of the current AU as a CVSS AU other than AU0, the encoding device may not deduce the value of the variable.

[0209] In this way, variables can be derived before decoding only the current image that is the first image of the current AU, rather than before decoding all images of the current AU that are CVSS AUs other than AU 0. A process of clearing all image storage buffers in the decoded image buffer (DPB) without output can be performed before decoding only the current image that is the first image of the current AU.

[0210] The encoding device updates the DPB based on variables (S610). For example, the encoding device can update the DPB based on variables.

[0211] For example, when the variable's value is 1, all image storage buffers in the DPB can be cleared without output, and the DPB's fullness can be set to 0. Alternatively, for example, when the variable's value is 0, the image storage buffers in the DPB that include a specific image can be cleared without output, and a bumping process can be performed on non-empty image storage buffers in the DPB. Additionally, the DPB's fullness can be set to 0. Here, for example, the specific image could be an image marked "no output needed" and "not for reference." The bumping process can be performed as described above.

[0212] Additionally, for example, when the current image is not the first image of the current AU of a CVSS AU other than AU 0, the encoding device can remove a specific image from the DPB that satisfies the first and second conditions. Here, the first condition may be that the specific image is marked as "not for reference," and the second condition may be that the specific image has a picture output flag equal to 0 or that the DPB output time (DPB) of the specific image is less than or equal to the CPB removal time of the first decoding unit (DU) of the current image. Here, the picture output flag may be the PictureOutputFlag mentioned above.

[0213] Additionally, for example, when the current image is not the first image of the current AU of CVSS AU other than AU 0, the image storage buffer in the DPB that includes the specific image can be cleared without output. Here, the specific image can be an image marked as "no output required" and "not for reference". Furthermore, for each cleared image storage buffer, the DPB fullness can be decremented by 1. That is, for example, the DPB fullness can be decremented by 1 whenever an image storage buffer is cleared. Moreover, the above convex-concave process can be repeated when one or more of the following conditions are true, while further decrementing the DPB fullness by 1 for each additional image storage buffer that is cleared, until all conditions are no longer true.

[0214] For example, the first condition could be that the number of pictures marked "needs output" in the DPB is greater than the current AU's syntax element `max_num_reorder_pics[Htid]`. The second condition could be that the current AU's syntax element `max_latency_increase_plus1[Htid]` is not equal to 0 and there is at least one picture in the DPB marked "needs output" whose associated variable `PicLatencyCount` is greater than or equal to `MaxLatencyPictures[Htid]`. The third condition could be that the number of pictures in the DPB is greater than or equal to the value obtained by adding 1 to the current AU's syntax element `max_dec_pic_buffering_minus1[Htid]`. Image information can include the current AU's syntax elements.

[0215] The encoding device encodes the image information of the current image (S620). The encoding device can encode image information including syntax elements used to update the DPB. Additionally, the image information may include the slice header of the current image.

[0216] Furthermore, although not illustrated, the encoding device can decode the current image based on the updated DPB. For example, the encoding device can derive prediction samples by performing inter-frame prediction on blocks in the current image using a reference image based on the DPB, and can generate reconstructed samples and / or reconstructed images for the current image based on the prediction samples. Additionally, for example, the encoding device can derive residual samples of blocks in the current image, and can generate reconstructed samples and / or reconstructed images by adding the prediction samples to the residual samples. As described above, loop filtering processes such as deblocking filtering, SAO, and / or ALF processes can be applied to the reconstructed samples to improve subjective / objective image quality. The encoding device can generate / encode prediction-related information and / or residual information for blocks, and the image information can include prediction-related information and / or residual information. Additionally, the encoding device can insert the decoded current image into the DPB. Furthermore, for example, the encoding device can derive the DPB parameters of the current AU, and can generate DBP-related information for the DPB parameters. The image information can include DBP-related information.

[0217] Furthermore, a bitstream containing image information can be sent to the decoding device via a network or (digital) storage medium. Here, the network can include broadcast networks and / or communication networks, and the digital storage medium can include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, and SSD.

[0218] Figure 7 An encoding device for performing an image encoding method according to the present disclosure is illustrated schematically. Figure 7 The method disclosed in the article can be derived from Figure 6 The encoding device disclosed in the document executes the code. Specifically, for example, Figure 7 The DPB of the encoding device can execute S600 to S610, and Figure 7 The entropy encoder of the encoding device can execute S620. Additionally, although not shown, the process of decoding the current image can be performed by the predictor and residual processor of the encoding device.

[0219] Figure 8 An image decoding method performed by a decoding device according to the present disclosure is illustrated schematically. Figure 8 The method disclosed in the article can be derived from Figure 3 The decoding device illustrated in the example performs this. Specifically, for example, Figure 8 S800 to S810 in the process can be executed by the DPB of the decoding device, and Figure 8 The S820 in the decoding device can be executed by the predictor and residual processor.

[0220] The decoding device derives the value of a variable based on whether the current image is the first image of the current AU of the CVSS AU (Current Video Sequence Start Access Unit, excluding Access Unit (AU) 0) other than Access Unit (AU) 0 (S800). The decoding device can derive the value of the variable based on whether the current image is the first image of the current AU of the CVSS AU other than Access Unit (AU) 0. Here, the variable can indicate whether all image storage buffers in the decoded image buffer (DPB) are cleared and there is no output. The current AU can be the AU that includes the current image. Alternatively, for example, AU 0 can be the first AU of the bitstream in decoding order. That is, for example, AU 0 can be the first AU of the bitstream that the decoding device will decode. On the other hand, the decoding device can parse the slice header of the current image, and then derive the value of the variable based on whether the current image is the first image of the current AU of the CVSS AU other than Access Unit (AU) 0.

[0221] For example, the decoding device can determine whether the current image is the first image of the current AU, excluding AU 0 (CVSS AU). When the current AU is a CVSS AU other than AU 0 and the current image is the first image of the current AU, the decoding device can deduce the value of the variable.

[0222] For example, when the current AU is a CVSS AU other than AU 0 and the current image is the first image of the current AU, the decoding device can determine whether at least one of the parameters of the current AU differs in decoding order from the parameters of the previous AUs. When at least one of the parameters of the current AU differs from the parameters of the previous AUs, the value of the variable can be set to 1. When the parameters of the current AU are the same as the parameters of the previous AUs, the value of the variable can be set to the value of the syntax element signaled for that variable. The decoding device can obtain the image information of the current image, and the image information can include syntax elements. The syntax element can be the ph_no_output_of_prior_pics_flag mentioned above. In addition, the parameters of the current AU include parameters for maximum image width, maximum image height, available chroma formats, maximum bit depth, and maximum DPB size. The parameters for maximum image width, maximum image height, available chroma format, maximum bit depth, and maximum DPB size can be PicWidthMaxInSamplesY, PicHeightMaxInSamplesY, MaxChromaFormat, MaxBitDepthMinus8, and max_dec_pic_buffering_minus1[Htid], respectively.

[0223] On the other hand, for example, when the current access unit (AU) is not a CVSS AU or the current picture is not the first picture of the current AU as a CVSS AU other than AU0, the decoding device may not deduce the value of the variable.

[0224] In this way, variables can be derived before decoding only the current image, which is the first image of the current AU, rather than before decoding all images of the current AU, which are CVSS AUs other than AU 0. A process of clearing all image storage buffers in the decoded image buffer (DPB) without output can be performed before decoding only the current image, which is the first image of the current AU.

[0225] The decoding device updates the DPB based on variables (S810). For example, the decoding device can update the DPB based on variables. Before being updated, the DPB may include images decoded before the current image.

[0226] For example, when the variable's value is 1, all image storage buffers in the DPB can be cleared without output, and the DPB's fullness can be set to 0. Alternatively, for example, when the variable's value is 0, the image storage buffers in the DPB that include a specific image can be cleared without output, and a bumping process can be performed on non-empty image storage buffers in the DPB. Additionally, the DPB's fullness can be set to 0. Here, for example, the specific image could be an image marked "no output needed" and "not for reference." The bumping process can be performed as described above.

[0227] Additionally, for example, when the current image is not the first image of the current AU of CVSS AU other than AU 0, the decoding device can remove a specific image from the DPB that satisfies the first and second conditions. Here, the first condition may be that the specific image is marked as "not for reference". The second condition may be that the specific image has a picture output flag equal to 0 or that the DPB output time (DPB) of the specific image is less than or equal to the CPB removal time of the first decoding unit (DU) of the current image. Here, the picture output flag may be the PictureOutputFlag mentioned above.

[0228] Additionally, for example, when the current image is not the first image of the current AU of CVSS AU other than AU 0, the image storage buffer in the DPB that includes the specific image can be cleared without output. Here, the specific image can be an image marked as "no output required" and "not for reference". Furthermore, for each cleared image storage buffer, the DPB fullness can be decremented by 1. That is, for example, the DPB fullness can be decremented by 1 whenever an image storage buffer is cleared. Moreover, the above convex-concave process can be repeated when at least one of the following conditions is true, while further decrementing the DPB fullness by 1 for each additional image storage buffer that is cleared, until all conditions are no longer true.

[0229] For example, the first condition could be that the number of pictures marked "needs output" in the DPB is greater than the current AU's syntax element `max_num_reorder_pics[Htid]`. The second condition could be that the current AU's syntax element `max_latency_increase_plus1[Htid]` is not equal to 0 and there is at least one picture in the DPB marked "needs output" whose associated variable `PicLatencyCount` is greater than or equal to `MaxLatencyPictures[Htid]`. The third condition could be that the number of pictures in the DPB is greater than or equal to the value obtained by adding 1 to the current AU's syntax element `max_dec_pic_buffering_minus1[Htid]`. Image information can include the current AU's syntax elements.

[0230] The decoding device decodes the current image based on the updated DPB (S820). For example, the decoding device can decode the current image based on the updated DPB. For example, the decoding device can derive prediction samples by performing inter-frame prediction on blocks in the current image based on a reference image of the DPB, and can generate reconstructed samples and / or reconstructed images for the current image based on the prediction samples. Furthermore, for example, the decoding device can derive residual samples of blocks in the current image based on residual information of the current image received via a bitstream, and can generate reconstructed samples and / or reconstructed images by adding the prediction samples to the residual samples. Image information may include residual information. Additionally, the decoding device can insert the decoded current image into the DPB.

[0231] As mentioned above, loop filtering processes such as deblocking filtering, SAO, and / or ALF can be applied to the reconstructed samples to improve the subjective / objective image quality as needed thereafter.

[0232] Figure 9 A decoding device for performing an image decoding method according to the present disclosure is illustrated schematically. Figure 8 The method disclosed in the article can be derived from Figure 9 The decoding device illustrated in the example performs this. Specifically, for example, Figure 9 The DPB of the decoding device can perform Figure 8 S800 to S810, and Figure 9 The predictor and residual processor of the decoding device can perform Figure 8 The S820 in the middle.

[0233] According to the present disclosure described above, whether to perform the process of removing images from the DPB without outputting them can be determined before decoding only the first image of the CVSS AU except AU 0, rather than before decoding all images of the CVSS AU except AU 0. In this way, the DPB state affecting all layers in the CVS can be avoided for each image, and encoding efficiency can be improved.

[0234] Furthermore, according to this disclosure, the variable indicating whether to remove images from the DPB without output can be determined before decoding only the first image of the CVSS AU (excluding AU 0), rather than before decoding all images of the CVSS AU (excluding AU 0). In this way, the DPB state affecting all layers in the CVS can be avoided for each image, and encoding efficiency can be improved.

[0235] In the above embodiments, the method is described based on a flowchart having a series of steps or blocks. This disclosure is not limited to the order of the above steps or blocks. Some steps or blocks can be performed in a different order than the other steps or blocks described above, or simultaneously. Furthermore, those skilled in the art will understand that the steps shown in the flowchart are not exclusive, and may include other steps, or one or more steps in the flowchart may be deleted without affecting the scope of this disclosure.

[0236] The embodiments described in this specification can be executed by being implemented on a processor, microprocessor, controller, or chip. For example, the functional unit shown in each figure can be executed by being implemented on a computer, processor, microprocessor, controller, or chip. In this case, the information for implementation (e.g., information about instructions) or algorithm can be stored in a digital storage medium.

[0237] Furthermore, the decoding and encoding devices using this disclosure can be included in the following devices: multimedia broadcasting transmitters / receivers, mobile communication terminals, home theater video devices, digital cinema video devices, surveillance cameras, video chat devices, real-time communication devices such as video communication, mobile streaming devices, storage media, portable video cameras, VoD service providers, over-the-top (OTT) video devices, internet streaming service providers, three-dimensional (3D) video devices, teleconferencing video devices, transportation user devices (e.g., vehicle user devices, airplane user devices, and ship user devices), and medical video devices; and the decoding and encoding devices using this disclosure can be used to process video signals or data signals. For example, over-the-top (OTT) video devices may include game consoles, Blu-ray players, internet-access televisions, home theater systems, smartphones, tablet computers, digital video recorders (DVRs), etc.

[0238] Furthermore, the processing methods of this disclosure can be generated in the form of a computer-executable program and can be stored in a computer-readable recording medium. Multimedia data with data structures according to this disclosure can also be stored in a computer-readable recording medium. A computer-readable recording medium includes all types of storage devices in which computer-readable data is stored. Computer-readable recording media can include, for example, BD, Universal Serial Bus (USB), ROM, PROM, EPROM, EEPROM, RAM, CD-ROM, magnetic tape, floppy disk, and optical data storage devices. Additionally, computer-readable recording media include media implemented in the form of a carrier wave (e.g., transmission via the Internet). Furthermore, bitstreams generated by encoding methods can be stored in a computer-readable recording medium or transmitted via wired / wireless communication networks.

[0239] Furthermore, embodiments of this disclosure can be implemented using computer program products based on program code, and the program code can be executed on a computer using embodiments of this disclosure. The program code can be stored on a computer-readable medium.

[0240] Figure 10 A structural diagram illustrating a content streaming system using this disclosure is provided.

[0241] The content streaming system using the embodiments of this disclosure may mainly include an encoding server, a streaming server, a network server, a media storage device, a user device, and a multimedia input device.

[0242] An encoding server compresses content input from multimedia input devices such as smartphones, cameras, or camcorders into digital data to generate a bitstream, which is then sent to a streaming server. As another example, when multimedia input devices such as smartphones, cameras, or camcorders generate bitstreams directly, the encoding server can be omitted.

[0243] A bit stream can be generated by an encoding method or bit stream generation method that applies the embodiments of this disclosure, and the streaming server can temporarily store the bit stream during the sending or receiving of the bit stream.

[0244] A streaming server sends multimedia data to a user device via a web server based on a user request, and the web server acts as a medium for notifying the user of services. When a user requests a service from the web server, the web server delivers the request to the streaming server, and the streaming server sends the multimedia data to the user. In this scenario, the content streaming system may include a separate control server. In this case, the control server is used to control the commands / responses between devices within the content streaming system.

[0245] A streaming server can receive content from media storage and / or encoding servers. For example, when receiving content from an encoding server, the content can be received in real time. In this case, to provide a smooth streaming service, the streaming server can store the bitstream for a predetermined period of time.

[0246] Examples of user equipment can include mobile phones, smartphones, laptops, digital broadcasting terminals, personal digital assistants (PDAs), portable multimedia players (PMPs), navigators, touchscreen PCs, tablet PCs, ultrabooks, wearable devices (e.g., smartwatches, smart glasses, and head-mounted displays), digital TVs, desktop computers, and digital signage. Each server within the content streaming system can operate as a distributed server, in which case data received from each server can be distributed.

[0247] The claims described in this disclosure can be combined in various ways. For example, the technical features of the method claims of this disclosure can be combined to implement an apparatus, and the technical features of the apparatus claims of this disclosure can be combined to implement a method. Furthermore, the technical features of the method claims and the apparatus claims of this disclosure can be combined to implement an apparatus, and the technical features of the method claims and the apparatus claims of this disclosure can be combined to implement a method.

Claims

1. An image decoding method performed by a decoding device, the image decoding method comprising the following steps: The variables are derived based on the fact that the current image is the first image of the current access unit AU, where the current AU is the AU of the start of the encoded video sequence, i.e., the CVSS AU; Update the decoded image buffer DPB based on the aforementioned variables; as well as The current image is decoded based on the updated DPB.

2. The image decoding method according to claim 1, wherein, With the variable value set to 1, all image storage buffers in the DPB are cleared and no output is made, and the DPB fullness is set to 0.

3. The image decoding method according to claim 2, wherein, Since the value of the variable is 0, the image storage buffer in the DPB containing the specific image is cleared and no output is made, and a bumping process is performed on the non-empty image storage buffer in the DPB. The specific image mentioned above is one that is marked as not needing to be output and not used for reference.

4. The image decoding method according to claim 1, wherein, The steps for deriving the variables include: Based on the fact that the current image is the first image of the current AU, determine whether at least one of the parameters for the current AU differs in decoding order from the parameters for the previous AU for the current AU. Wherein, based on at least one parameter in the parameters for the current AU that differs from the parameters for the previous AU, the value of the variable is set to equal to 1, and Wherein, based on the fact that the parameters for the current AU are the same as the parameters for the previous AU, the value of the variable is set to be equal to the value of the syntax element that signals the variable.

5. The image decoding method according to claim 4, wherein, The parameters for the current AU include parameters for the maximum image width, the maximum image height, the available color format, the maximum bit depth, and the maximum DPB size.

6. The image decoding method according to claim 1, wherein, The steps for updating the DPB include: Based on the fact that the current image is not the first image of the current AU, specific images that satisfy the first and second conditions are removed from the DPB. The first condition is that the specific image is marked as not for reference, and The second condition is that the specific image has an image output flag equal to 0 or the DPB output time of the specific image is less than or equal to the CPB removal time of the first decoding unit DU of the current image.

7. The image decoding method according to claim 1, wherein, Since the current image is not the first image of the current AU, the image storage buffer in the DPB that includes the specific image is cleared and no output is made. The specific image mentioned above is one that is marked as not needing to be output and not used for reference.

8. An image encoding method performed by an encoding device, the image encoding method comprising the following steps: The variables are derived based on the fact that the current image is the first image of the current access unit AU. The current AU is the AU that starts the encoded video sequence other than AU 0, i.e., CVSS AU. Update the decoded image buffer DPB based on the aforementioned variables; and The image information for the current image is encoded.

9. The image encoding method according to claim 8, wherein, With the variable value set to 1, all image storage buffers in the DPB are cleared and no output is made, and the DPB fullness is set to 0.

10. The image encoding method according to claim 9, wherein, Since the value of the variable is 0, the image storage buffer in the DPB containing the specific image is cleared and no output is made, and a bumping process is performed on the non-empty image storage buffer in the DPB. The specific image mentioned above is one that is marked as not needing to be output and not used for reference.

11. The image encoding method according to claim 8, wherein, The steps for deriving the variables include: Based on the fact that the current image is the first image of the current AU, determine whether at least one of the parameters for the current AU differs in encoding order from the parameters of a previous AU for the current AU. Wherein, based on at least one parameter of the current AU that differs from that of the previous AU, the value of the variable is set to equal to 1. Wherein, based on the fact that the parameters for the current AU are the same as the parameters for the previous AU, the value of the variable is set to be equal to the value of the syntax element for the variable, and The image information includes the syntax elements.

12. The image encoding method according to claim 11, wherein, The parameters for the current AU include parameters for the maximum image width, the maximum image height, the available color format, the maximum bit depth, and the maximum DPB size.

13. The image encoding method according to claim 8, wherein, The steps for updating the DPB include: Based on the fact that the current image is not the first image of the current AU, specific images that satisfy the first and second conditions are removed from the DPB. The first condition is that the specific image is marked as not for reference. The second condition is that the specific image has an image output flag equal to 0 or the DPB output time of the specific image is less than or equal to the CPB removal time of the first decoding unit DU of the current image.

14. The image encoding method according to claim 8, wherein, Since the current image is not the first image of the current AU, the image storage buffer in the DPB that includes the specific image is cleared and no output is made. The specific image mentioned above is one that is marked as not needing to be output and not used for reference.

15. A method for transmitting data relating to image information, the method comprising the steps of: The variables are derived based on the fact that the current image is the first image of the current access unit AU. The current AU is the AU that starts the encoded video sequence other than AU 0, i.e., CVSS AU. Update the decoded image buffer DPB based on the aforementioned variables; The image information of the current image is encoded to generate a bitstream; as well as Send the data including the bit stream.

Images