Image encoding / decoding apparatus and apparatus for transmitting data

Abstract

The invention relates to an image encoding / decoding apparatus and an apparatus for transmitting data. According to the disclosure herein, a decoded picture buffer (DPB) may be updated based on DPB-related information. The DPB-related information may include syntax elements related to a maximum desired size of the DPB. When updating the DPB, the trimming process may be invoked based on a case in which a first condition is satisfied in which the number of pictures in the DPB is neither greater nor not equal to a value obtained by adding 1 to a value of a syntax element related to a maximum required size of the DPB.

Description

[0001] This application is a divisional application of the original application No. 202180056984.3 (International Application No. PCT / KR2021 / 007057, filed on June 7, 2021, with the title of Image or Video Coding Based on DPB Operation). TECHNICAL FIELD

[0002] The disclosure relates to a video or image coding technology, and more particularly, to a coding technology related to decoded picture buffer (DPB) operation in a video coding system. BACKGROUND

[0003] Recently, in various fields, the demand for high-resolution and high-quality images and videos such as 4K or 8K or higher ultra-high-definition (UHD) images and videos is increasing. As image and video data become high-resolution and high-quality, the amount of information or the number of bits transmitted is relatively increased compared to existing image and video data. Therefore, if the image data is transmitted using a medium such as an existing wired or wireless broadband line, or the image and video data is stored using an existing storage medium, the transmission cost and the storage cost increase.

[0004] In addition, recently, interest and demand for immersive media such as virtual reality (VR), artificial reality (AR) content, or holograms are increasing. Broadcasting of images and videos having image characteristics different from those of real images, such as game images, is increasing.

[0005] Therefore, an efficient image and video compression technology is required in order to effectively compress and transmit or store and play back information of high-resolution and high-quality images and videos having such various characteristics.

[0006] In addition, a way to improve the efficiency of image / video coding is required, and for this, an efficient coding technology related to decoded picture buffer (DPB) operation is required. SUMMARY

[0007] TECHNICAL PROBLEM

[0008] The disclosure provides a method and apparatus for improving video / image coding efficiency.

[0009] The disclosure also provides a method and apparatus for performing a DPB management process.

[0010] TECHNICAL SOLUTION

[0011] According to embodiments of this disclosure, a decoded picture buffer (DPB) can be updated based on DPB-related information. The DPB-related information can include a syntax element related to a maximum required size of the DPB. When updating the DPB, a bumping process can be invoked based on a case that a first condition that a number of pictures in the DPB is not greater than or equal to a value of the syntax element related to the maximum required size of the DPB plus 1 is satisfied.

[0012] Further, according to embodiments of this disclosure, the DPB-related information can include a syntax element related to a maximum picture reorder number of the DPB or a syntax element related to a maximum latency of the DPB. Invocation of the bumping process is determined based on a second condition that is not based on the syntax element related to the maximum picture reorder number of the DPB or a third condition that is based on the syntax element related to the maximum latency of the DPB. For example, the bumping process can not be invoked when the second condition or the third condition is satisfied but the first condition is not satisfied.

[0013] Further, according to embodiments of this disclosure, during the bumping process invoked based on the case that the first condition is satisfied, the DPB fullness can be decreased by 1 for a picture storage buffer that is emptied in the DPB.

[0014] Further, according to embodiments of this disclosure, after the bumping process invoked based on the case that the first condition is satisfied is performed, the operation of decreasing the DPB fullness by 1 can not be performed for the picture storage buffer that is emptied in the DPB.

[0015] Further, according to embodiments of this disclosure, whether to invoke the bumping process can be determined based on whether a current picture is a first picture of a current access unit (AU) that is a coded video sequence start (CVSS) access unit (AU) other than AU 0.

[0016] According to embodiments of the present document, a video / image decoding method performed by a decoding device is provided. The video / image decoding method can include the methods disclosed in embodiments of the present document.

[0017] According to embodiments of the present document, a decoding device for performing video / image decoding is provided. The decoding device can include the methods disclosed in embodiments of the present document.

[0018] According to embodiments of the present document, a video / image encoding method performed by an encoding device is provided. The video / image encoding method can include the methods disclosed in embodiments of the present document.

[0019] According to an embodiment of the present document, there is provided an encoding apparatus for performing video / image encoding. The encoding apparatus can include the method disclosed in an embodiment of the present document.

[0020] According to an embodiment of the present document, there is provided a computer-readable digital storage medium storing encoded video / image information generated according to the video / image encoding method disclosed in at least one embodiment of the present document.

[0021] According to an embodiment of the present document, there is provided a computer-readable digital storage medium storing encoded information or encoded video / image information, which causes a decoding apparatus to perform the video / image decoding method disclosed in at least one embodiment of the present document.

[0022] Advantageous Effects

[0023] According to the present disclosure, various effects can be provided. For example, according to an embodiment of the present disclosure, overall image / video compression efficiency can be improved. Further, according to an embodiment of the present disclosure, a DPB management process is effectively performed, and DPB operation can be improved. Further, according to an embodiment of the present disclosure, DPB fullness can be decreased only once when a picture buffer is emptied in a trim process, and accuracy of output order operation of the DPB can be improved. Further, the number of check numbers of a calling condition of a trim process is reduced, and complexity can be reduced. Accordingly, accuracy and efficiency can be improved in DPB management, i.e., output and removal operations of pictures in the DPB.

[0024] Effects obtainable through the detailed examples of the present document are not limited to those enumerated above. For example, a person of ordinary skill in the related art can understand or induce various technical effects from the present document. Accordingly, detailed effects of the present document are not limited to those explicitly stated in the present document, but can include various effects that can be understood or induced by those skilled in the art from the technical features of the present document. BRIEF DESCRIPTION OF DRAWINGS

[0025] Figure 1 A brief example of a video / image encoding apparatus to which embodiments of the present document are applicable is illustrated.

[0026] Figure 2 is a schematic diagram illustrating a configuration of a video / image encoding apparatus to which embodiments of the present document are applicable.

[0027] Figure 3 is a schematic diagram illustrating a configuration of a video / image decoding apparatus to which embodiments of the present document are applicable.

[0028] Figure 4 An encoding process according to an embodiment of the present disclosure is illustrated.

[0029] Figure 5 A decoding process according to an embodiment of the disclosure is illustrated.

[0030] Figure 6 and Figure 7 Examples of a video / image encoding method and related components according to an embodiment of the present document are illustratively shown.

[0031] Figure 8 and Figure 9 Examples of a video / image decoding method and related components according to an embodiment of the present document are illustratively shown.

[0032] Figure 10 Examples of a content streaming system to which embodiments disclosed in the present document are applicable are illustrated. DETAILED DESCRIPTION

[0033] The present document can be modified in various ways and can have various embodiments, and a specific embodiment will be illustrated in the drawings and described in detail. However, this is not intended to limit the present document to a specific embodiment. The terms commonly used in the specification are used to describe the specific embodiment, not to limit the technical spirit of the present document. Unless otherwise explicitly indicated in the context, the singular expression includes the plural expression. The terms such as "include" or "have" in the specification should be understood to indicate the presence of the characteristics, numbers, steps, operations, elements, parts, or combinations thereof described in the specification, not to exclude the presence or possibility of adding one or more other characteristics, numbers, steps, operations, elements, parts, or combinations thereof.

[0034] In addition, in order to facilitate the description related to different characteristic functions, the elements in the drawings described in the present document are independently illustrated. This does not mean that each element is implemented as a separate hardware or a separate software. For example, at least two elements can be combined to form a single element, or a single element can be divided into multiple elements. Embodiments of combining and / or separating elements are also included in the scope of rights of the present document unless it deviates from the essence of the present document.

[0035] In the present document, "A or B" can mean "only A", "only B", or "both A and B". In other words, "A or B" in the present document can be interpreted as "A and / or B". For example, "A, B, or C" in the present document means "only A", "only B", "only C", or "any combination of A, B, and C".

[0036] The slash ( / ) or comma (,) used in the present document can mean "and / or". For example, "A / B" can mean "A and / or B". Therefore, "A / B" can mean "only A", "only B", or "both A and B". For example, "A, B, C" can mean "A, B, or C".

[0037] In this document, “at least one of A and B” can mean “only A”, “only B”, or “both A and B”. Also, in this document, the expression “at least one of A or B” or “at least one of A and / or B” can be interpreted to be the same as “at least one of A and B”.

[0038] Also, in this document, “at least one of A, B, and C” means “only A”, “only B”, “only C”, or “any combination of A, B, and C”. Also, “at least one of A, B, or C” or “at least one of A, B, and / or C” can mean “at least one of A, B, and C”.

[0039] Also, the brackets used in this document can mean “for example”. Specifically, when indicating “prediction (intra prediction)”, “intra prediction” can be suggested as an example of “prediction”. In other words, “prediction” in this document is not limited to “intra prediction”, and “intra prediction” can be suggested as an example of “prediction”. Also, even when indicating “prediction (i.e., intra prediction)”, “intra prediction” can be suggested as an example of “prediction”.

[0040] This document relates to video / image encoding. For example, the methods / embodiments disclosed in this document can be applied to methods disclosed in the Versatile Video Coding (VVC) standard. In addition, the methods / embodiments disclosed in this document can be applied to methods disclosed in the Essential Video Coding (EVC) standard, the AOMedia Video 1 (AV1) standard, the second generation Audio Video Coding standard (AVS2), or the next generation video / image encoding standard (e.g., H.267, H.268, etc.).

[0041] Various embodiments of video / image encoding are suggested herein, and the above-described embodiments can also be performed in combination with each other unless otherwise specified.

[0042] In this document, a video can refer to a series of pictures over time. A picture generally refers to a unit representing one picture at a particular time frame, while a slice / tile refers to a unit that constitutes a part of a picture in terms of encoding. A slice / tile can include one or more coding tree units (CTUs). One picture can include one or more slices / tiles. A tile is a rectangular region of CTUs within a particular tile column and a particular tile row in a picture (a tile is a rectangular region of CTUs within a particular tile column and a particular tile row in a picture). A tile column is a rectangular region of CTUs having a height equal to a height of the picture and a width that can be specified by a syntax element in a picture parameter set (a tile column is a rectangular region of CTUs having a height equal to a height of the picture and a width that can be specified by a syntax element in a picture parameter set). A tile row is a rectangular region of CTUs having a height specified by a syntax element in a picture parameter set and a width that can be equal to a width of the picture (a tile row is a rectangular region of CTUs having a height specified by a syntax element in a picture parameter set and a width that can be equal to a width of the picture). A tile scan can represent a particular order of partitioning CTUs of a picture, and the CTUs can be sequentially ordered in a CTU raster scan in a tile, and the tiles in a picture can be sequentially ordered in a raster scan of tiles of the picture (a tile scan is a particular sequential ordering of partitioning CTUs of a picture, where the CTUs are sequentially ordered in a CTU raster scan in a tile, and the tiles in a picture are sequentially ordered in a raster scan of tiles of the picture). A slice includes an integer number of complete tiles or an integer number of consecutive complete CTU rows within tiles of a picture that can be exclusively contained in a single NAL unit.

[0043] Further, one picture can be divided into two or more sub-pictures. A sub-picture can be a rectangular region of one or more slices within a picture.

[0044] A pixel or pel can mean a minimum unit constituting one picture (or image). Also, a “sample” can be used as a term corresponding to a pixel. A sample can generally represent a pixel or a value of a pixel, can represent only a pixel / value of a pixel of a luminance component, or can represent only a pixel / value of a pixel of a chrominance component.

[0045] A unit can represent a basic unit of image processing. A unit can include at least one of a particular region of a picture and information related to the region. One unit can include one luminance block and two chroma (e.g., cb, cr) blocks. In some cases, a unit can be used interchangeably with terms such as a block or a region. In general, an MxN block can include a set (or an array) of M columns and N rows of samples (or sample array) or transform coefficients.

[0046] Furthermore, in this document, at least one of quantization / dequantization and / or transform / inverse transform may be omitted. When quantization / dequantization is omitted, the quantization transform coefficients may be referred to as transform coefficients. When transform / inverse transform is omitted, the transform coefficients may be referred to as coefficients or residual coefficients, or, for consistency, may still be referred to as transform coefficients.

[0047] In this document, quantization transform coefficients and transform coefficients can be referred to as transform coefficients and scaling transform coefficients, respectively. In this case, residual information can include information about the transform coefficients, and this information can be signaled using residual coding syntax. Transform coefficients can be derived based on residual information (or information about the transform coefficients), and scaling transform coefficients can be derived by performing an inverse transform (scaling) on ​​the transform coefficients. Residual samples can be derived based on the inverse transform (scaling) of the scaling transform coefficients. This can also be applied / expressed in other parts of this document.

[0048] The technical features described individually in one of the accompanying figures in this document can be implemented individually or simultaneously.

[0049] In the following, preferred embodiments of this document are described in more detail with reference to the accompanying drawings. In the following drawings, the same reference numerals are used for the same elements, and redundant descriptions of the same elements may be omitted.

[0050] Figure 1 Examples of video / image coding systems to which the implementation methods described in this document are applicable are illustrated.

[0051] Reference Figure 1 A video / image encoding system may include a source device and a receiving device. The source device may transmit encoded video / image information or data to the receiving device in the form of a file or stream via a digital storage medium or network.

[0052] The source device may include a video source, an encoding device, and a transmitter. The receiving device may include a receiver, a decoding device, and a renderer. The encoding device may be referred to as a video / image encoding device, and the decoding device may be referred to as a video / image decoding device. The transmitter may be included in the encoding device. The receiver may be included in the decoding device. The renderer may include a display, and the display may be configured as a separate device or an external component.

[0053] Video sources can acquire video / images through processes that capture, synthesize, or generate video / images. Video sources may include video / image capture devices and / or video / image generation devices. Video / image capture devices may include, for example, one or more cameras, video / image archives including previously captured video / images, etc. Video / image generation devices may include, for example, computers, tablets, and smartphones, and can generate video / images (electronically). For example, virtual video / images can be generated by computers, etc. In this case, video / image capture processing can be replaced by processing that generates related data.

[0054] Encoding devices can encode input video / images. They can perform a series of processes such as prediction, transformation, and quantization for compression and encoding efficiency. The encoded data (encoded video / image information) can be output as a bitstream.

[0055] A transmitter can send encoded video / image information or data, output in bitstream form, to a receiver via a digital storage medium or network, either as a file or a stream. The digital storage medium can include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. The transmitter can include elements for generating media files according to a predetermined file format and may include elements for transmission over a broadcast / communication network. The receiver can receive / extract the bitstream and send the received bitstream to a decoding device.

[0056] Decoding devices can decode video / images by performing a series of processes, such as dequantization, inverse transform, and prediction, that correspond to the operations of encoding devices.

[0057] The renderer can render decoded video / images. Rendered video / images can then be displayed on a monitor.

[0058] Figure 2 This is a diagram schematically illustrating the configuration of a video / image encoding device to which the embodiments described in this document may be applied. In the following text, the term "encoding device" may include image encoding devices and / or video encoding devices.

[0059] Reference Figure 2The encoding device 200 may include and be configured with an image segmenter 210, a predictor 220, a residual processor 230, an entropy encoder 240, an adder 250, a filter 260, and a memory 270. The predictor 220 may include an inter-frame predictor 221 and an intra-frame predictor 222. The residual processor 230 may include a transformer 232, a quantizer 233, a dequantizer 234, and an inverse transformer 235. The residual processor 230 may also include a subtractor 231. The adder 250 may be referred to as a reconstructor or a reconstruction block generator. The image segmenter 210, predictor 220, residual processor 230, entropy encoder 240, adder 250, and filter 260 described above may be configured by one or more hardware components (e.g., an encoder chipset or processor) according to an embodiment. Additionally, the memory 270 may include a decoded picture buffer (DPB) and may also be configured by a digital storage medium. The hardware components may also include the memory 270 as an internal / external component.

[0060] Image segmenter 210 can segment an input image (or picture, frame) input to encoding device 200 into one or more processing units. As an example, a processing unit may be referred to as a coding unit (CU). In this case, the coding unit can be recursively segmented from a coding tree unit (CTU) or a maximum coding unit (LCU) according to a quadtree-binary-tritree (QTBTTT) structure. For example, a coding unit can be segmented into multiple deeper coding units based on a quadtree structure, a binary tree structure, and / or a ternary tree structure. In this case, for example, a quadtree structure can be applied first, followed by a binary tree structure and / or a ternary tree structure. Alternatively, a binary tree structure can be applied first. The encoding process according to this document can be performed based on the final coding unit that is no longer segmented. In this case, based on encoding efficiency according to image characteristics, etc., the maximum coding unit can be directly used as the final coding unit, or, as needed, the coding unit can be recursively segmented into deeper coding units such that the coding unit with the optimal size can be used as the final coding unit. Here, the encoding process may include processes such as prediction, transformation, and reconstruction, as described later. As another example, the processing unit may also include a prediction unit (PU) or a transform unit (TU). In this case, each of the prediction unit and the transform unit can be separated or partitioned from the final encoding unit described above. The prediction unit may be a unit for predicting samples, and the transform unit may be a unit for deriving transform coefficients and / or a unit for deriving residual signals from transform coefficients.

[0061] In some cases, a unit can be used interchangeably with terms such as a block or region. Typically, an M×N block can represent a sample or a set of transform coefficients consisting of M columns and N rows. A sample can typically represent a pixel or pixel value, and can also represent only the pixel / pixel value of the luminance component, and can also represent only the pixel / pixel value of the chrominance component. A sample can be used as a term corresponding to the pixels or picometers that configure a frame (or image).

[0062] Encoding device 200 generates a residual signal (residual block, residual sample array) by subtracting the prediction signal (prediction block, prediction sample array) output from inter-frame predictor 221 or intra-frame predictor 222 from the input image signal (original block, original sample array), and the generated residual signal is sent to converter 232. In this case, as shown, the unit within encoding device 200 for subtracting the prediction signal (prediction block, prediction sample array) from the input image signal (original block, original sample array) can be called subtractor 231. The predictor can perform prediction on the block to be processed (hereinafter referred to as the current block) and generate a prediction block including the prediction samples of the current block. The predictor can determine whether to apply intra-frame prediction or inter-frame prediction on a unit of the current block or CU. As described later in the description of each prediction mode, the predictor can generate various information about the prediction, such as prediction mode information, to transmit the generated information to entropy encoder 240. The information about the prediction can be encoded by entropy encoder 240 and output as a bit stream.

[0063] Intra-predictor 222 can refer to samples within the current frame to predict the current block. Depending on the prediction mode, the reference sample can be located near or far from the current block. The prediction modes in intra-prediction can include multiple non-directional modes and multiple directional modes. Non-directional modes can include, for example, DC mode or planar mode. Depending on the fineness of the prediction direction, the directional modes can include, for example, 33 or 65 directional prediction modes. However, this is exemplary, and more or fewer directional prediction modes can be used depending on the settings. Intra-predictor 222 can also determine the prediction mode applied to the current block by using prediction modes applied to neighboring blocks.

[0064] Inter-frame predictor 221 can deduce the predicted block of the current block based on a reference block (reference sample array) specified by motion vectors on a reference frame. Here, to reduce the amount of motion information transmitted in inter-frame prediction mode, motion information can be predicted on a block, sub-block, or sample basis based on the correlation between motion information between neighboring blocks and the current block. Motion information may include motion vectors and reference frame indices. Motion information may also include inter-frame prediction direction (L0 prediction, L1 prediction, Bi prediction, etc.) information. In the case of inter-frame prediction, neighboring blocks may include spatially neighboring blocks existing in the current frame and temporally neighboring blocks existing in the reference frame. The reference frame including the reference block and the reference frame including the temporally neighboring block may be the same or different. The temporally neighboring block may be called a co-located reference block, a co-located CU (colCU), etc., and the reference frame including the temporally neighboring block may be called a co-located frame (colPic). For example, inter-frame predictor 221 can configure a motion information candidate list based on neighboring blocks and generate information indicating which candidate to use to deduce the motion vector and / or reference frame index of the current block. Inter-frame prediction can be performed based on various prediction modes. For example, in skip mode and merge mode, the inter-frame predictor 221 can use motion information from neighboring blocks as motion information for the current block. In skip mode, unlike merge mode, residual signals may not be sent. In motion vector prediction (MVP) mode, the motion vector of the current block can be indicated by using motion vectors from neighboring blocks as motion vector predictors and signaling the motion vector difference.

[0065] Predictor 220 can generate prediction signals based on various prediction methods described later. For example, the predictor can not only apply intra-frame prediction or inter-frame prediction to predict a block, but can also apply intra-frame prediction and inter-frame prediction simultaneously. This can be referred to as combined intra-frame and inter-frame prediction (CIIP). Furthermore, the predictor can perform prediction on blocks based on an intra-block copy (IBC) prediction mode or a palette mode. The IBC prediction mode or palette mode can be used for content image / video coding (e.g., screen content coding (SCC)) such as games. IBC essentially performs prediction in the current frame, but can be performed similarly to inter-frame prediction because it derives a reference block in the current frame. That is, IBC can use at least one of the inter-frame prediction techniques described in this document. The palette mode can be considered an example of intra-frame coding or intra-frame prediction. When applying a palette mode, sample values ​​in the frame can be signaled based on information about the palette index and palette table.

[0066] The predicted signal generated by the predictor (including inter-frame predictor 221 and / or intra-frame predictor 222) can be used to generate a reconstructed signal or a residual signal. Transformer 232 can generate transform coefficients by applying a transform technique to the residual signal. For example, the transform technique may include at least one of Discrete Cosine Transform (DCT), Discrete Sine Transform (DST), Karhunen-Loève Transform (KLT), Graph-Based Transform (GBT), or Conditional Nonlinear Transform (CNT). Here, GBT refers to a transform obtained from a graph when the relationship information between pixels is represented by a graph. CNT refers to a transform generated based on the predicted signal generated using all previously reconstructed pixels. Furthermore, the transform processing can be applied to square pixel blocks of the same size, or it can be applied to blocks of variable size instead of square.

[0067] Quantizer 233 can quantize the transform coefficients to send the quantized transform coefficients to entropy encoder 240, which can encode the quantized signal (information about the quantized transform coefficients) into a bitstream and output the encoded quantized signal. The information about the quantized transform coefficients can be referred to as residual information. Quantizer 233 can rearrange the block-form quantized transform coefficients into a one-dimensional vector form based on the coefficient scan order, and can also generate information about the quantized transform coefficients based on the one-dimensional vector form. Entropy encoder 240 can perform various encoding methods such as Golomb coding, context-adaptive variable-length coding (CAVLC), and context-adaptive binary arithmetic coding (CABAC). Entropy encoder 240 can also encode information necessary for the video / image other than the quantized transform coefficients (e.g., values ​​of syntax elements, etc.) together or separately. Encoded information (e.g., encoded video / image information) can be sent or stored in bitstream form at the Network Abstraction Layer (NAL) unit level. The video / image information may also include information about various parameter sets such as Adaptive Parameter Set (APS), Picture Parameter Set (PPS), Sequence Parameter Set (SPS), or Video Parameter Set (VPS). Additionally, the video / image information may also include general constraint information. Information and / or syntax elements that are signaled / transmitted, as described later in this document, can be encoded through the aforementioned encoding process and thus included in the bitstream. The bitstream can be transmitted over a network or stored in a digital storage medium. Here, the network may include broadcast networks and / or communication networks, etc., and the digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, and SSD. A transmitter (not illustrated) for transmitting the signal output from the entropy encoder 240 and / or a memory (not illustrated) for storing the signal can be configured as internal / external components of the encoding device 200, or the transmitter may also be included within the entropy encoder 240.

[0068] The quantization transform coefficients output from quantizer 233 can be used to generate a prediction signal. For example, dequantizer 234 and inverse transform 235 apply inverse quantization and inverse transform to the quantization transform coefficients, allowing the residual signal (residual block or residual sample) to be reconstructed. Adder 250 adds the reconstructed residual signal to the prediction signal output from inter-frame predictor 221 or intra-frame predictor 222, thereby generating a reconstructed signal (reconstructed frame, reconstructed block, reconstructed sample array). If no residual exists for the block to be processed, such as when a skip mode is applied, the prediction block can be used as the reconstructed block. Adder 250 can be referred to as a reconstructor or reconstructed block generator. The generated reconstructed signal can be used for intra-frame prediction of the next block to be processed in the current frame, and, as described later, can also be used for inter-frame prediction of the next frame by filtering.

[0069] In addition, luminance mapping with chroma scaling (LMCS) can be applied in screen encoding and / or reconstruction processing.

[0070] Filter 260 can apply filtering to the reconstructed signal to improve subjective / objective image quality. For example, filter 260 can apply various filtering methods to the reconstructed image to generate a modified reconstructed image, and store the modified reconstructed image in memory 270, specifically in the DPB of memory 270. Various filtering methods may include, for example, deblocking filtering, sample adaptive offset, adaptive loop filtering, bilateral filtering, etc. Filter 260 can generate various filtering-related information to transmit the generated information to entropy encoder 240, as described later in the description of each filtering method. The filtering-related information can be encoded by entropy encoder 240 to output as a bitstream.

[0071] The modified reconstructed frame sent to memory 270 can be used as a reference frame in inter-frame predictor 221. If the inter-frame predictor applies inter-frame prediction, the encoding device can avoid prediction mismatch between encoding device 200 and decoding device, and also improve encoding efficiency.

[0072] The DPB of memory 270 can store modified reconstructed frames to be used as reference frames in inter-frame predictor 221. Memory 270 can store motion information of blocks in which motion information within the current frame is derived (or encoded) and / or motion information of blocks in previously reconstructed frames. The stored motion information can be transmitted to inter-frame predictor 221 for use as motion information for spatially or temporally adjacent blocks. Memory 270 can store reconstructed samples of reconstructed blocks within the current frame and can transmit the reconstructed samples to intra-frame predictor 222.

[0073] Figure 3This diagram is an illustrative representation of the configuration of a video / image decoding device to which the embodiments described in this document may be applied. In the following text, the term "decoding device" may include image decoding devices and / or video decoding devices.

[0074] Reference Figure 3 The decoding device 300 may include and be configured with an entropy decoder 310, a residual processor 320, a predictor 330, an adder 340, a filter 350, and a memory 360. The predictor 330 may include an intra-frame predictor 331 and an inter-frame predictor 332. The residual processor 320 may include a dequantizer 321 and an inverse transformer 322. The entropy decoder 310, residual processor 320, predictor 330, adder 340, and filter 350 described above may be configured by one or more hardware components (e.g., a decoder chipset or processor) according to an embodiment. Furthermore, the memory 360 may include a decoded picture buffer (DPB) and may be configured by a digital storage medium. The hardware components may also include the memory 360 as an internal / external component.

[0075] When the input includes a bitstream containing video / image information, the decoding device 300 can respond to... Figure 2 The encoding device shown reconstructs an image by processing video / image information. For example, the decoding device 300 can deduce units / blocks based on block segmentation information obtained from the bitstream. The decoding device 300 can perform decoding using processing units applied to the encoding device. Therefore, the processing unit used for decoding can be, for example, an encoding unit, and the encoding unit can be separated from the encoding tree unit or the maximum encoding unit according to a quadtree structure, a binary tree structure, and / or a ternary tree structure. One or more transform units can be derived from the encoding unit. Furthermore, the reconstructed image signal decoded and output by the decoding device 300 can be reproduced by a reproduction device.

[0076] Decoding device 300 can receive data from... in the form of a bitstream. Figure 2The signal output by the encoding device shown can be decoded by the entropy decoder 310. For example, the entropy decoder 310 can deduce the information (e.g., video / image information) required for image reconstruction (or picture reconstruction) by parsing the bitstream. The video / image information may also include information about various parameter sets such as Adaptive Parameter Set (APS), Picture Parameter Set (PPS), Sequence Parameter Set (SPS), and Video Parameter Set (VPS). In addition, the video / image information may also include general constraint information. The decoding device can also decode the picture based on the information about the parameter sets and / or general constraint information. The information and / or syntax elements notified / received by signals, as described later in this document, can be decoded by the decoding process and obtained from the bitstream. For example, the entropy decoder 310 can decode the information within the bitstream based on encoding methods such as exponential Golomb coding, CAVLC, or CABAC, and output the values ​​of the syntax elements necessary for image reconstruction and the quantized values ​​of the residual correlation transform coefficients. More specifically, the CABAC entropy decoding method can receive bins corresponding to each syntax element from the bitstream, use the information of the syntax element to be decoded and decode information of neighboring blocks and the block to be decoded, or information of symbols / bins decoded in the previous stage, to determine a context model, and generate symbols corresponding to the values ​​of each syntax element by predicting the generation probability of bins according to the determined context model, thereby performing arithmetic decoding of bins. At this time, the CABAC entropy decoding method can determine the context model and then update the context model using the information of decoded symbols / bins for the context model of the next symbol / bin. The prediction information in the information decoded by the entropy decoder 310 can be provided to the predictors (inter-frame predictor 332 and intra-frame predictor 331), and the residual values ​​(i.e., quantization transform coefficients and related parameter information) of the entropy decoding performed by the entropy decoder 310 can be input to the residual processor 320. The residual processor 320 can derive the residual signals (residual blocks, residual samples, and residual sample arrays). In addition, the filtering information in the information decoded by the entropy decoder 310 can be provided to the filter 350. Furthermore, a receiver (not illustrated) for receiving signals output from the encoding device may be configured as an internal / external element of the decoding device 300, or the receiver may be a component of the entropy decoder 310. Additionally, the decoding device according to this document may be referred to as a video / image / picture decoding device, and the decoding device may also be classified as an information decoder (video / image / picture information decoder) and a sample decoder (video / image / picture sample decoder). The information decoder may include the entropy decoder 310, and the sample decoder may include at least one of a dequantizer 321, an inverse transformer 322, an adder 340, a filter 350, a memory 360, an inter-frame predictor 332, and an intra-frame predictor 331.

[0077] Dequantizer 321 can dequantize the quantized transform coefficients to output transform coefficients. Dequantizer 321 can rearrange the quantized transform coefficients in the form of two-dimensional blocks. In this case, the rearrangement can be performed based on the coefficient scan order performed by the encoding device. Dequantizer 321 can perform dequantization on the quantized transform coefficients using quantization parameters (e.g., quantization step size information) and obtain the transform coefficients.

[0078] The inverse transformer 322 performs an inverse transformation on the transformation coefficients to obtain the residual signal (residual block, residual sample array).

[0079] Predictor 330 can perform prediction on the current block and generate a prediction block that includes prediction samples of the current block. The predictor can determine whether to apply intra-frame prediction or inter-frame prediction to the current block based on the prediction information output from entropy decoder 310, and determine the specific intra-frame / inter-frame prediction mode.

[0080] The predictor can generate prediction signals based on various prediction methods described later. For example, the predictor can apply not only intra-frame prediction or inter-frame prediction to the prediction of a block, but also both intra-frame and inter-frame prediction simultaneously. This can be referred to as Combined Inter-Frame and Intra-Frame Prediction (CIIP). Furthermore, the predictor can perform prediction on blocks based on Intra-Block Copy (IBC) prediction mode or palette mode. IBC prediction mode or palette mode can be used for content image / video coding (e.g., Screen Content Coding (SCC)) such as games. IBC essentially performs prediction within the current frame, but can be performed similarly to inter-frame prediction because it derives a reference block in the current frame. That is, IBC can use at least one of the inter-frame prediction techniques described in this document. Palette mode can be considered an example of intra-frame coding or intra-frame prediction. When a palette mode is applied, information about the palette table and palette index can be included in the video / image information and signaled.

[0081] The intra-predictor 331 can refer to samples within the current frame to predict the current block. Depending on the prediction mode, the reference samples can be located near the current block or far from it. The prediction modes in intra-prediction can include multiple non-directional modes and multiple directional modes. The intra-predictor 331 can also use prediction modes applied to neighboring blocks to determine the prediction mode applied to the current block.

[0082] Inter-frame predictor 332 can deduce the predicted block for the current block based on a reference block (reference sample array) specified by motion vectors on a reference frame. In this case, to reduce the amount of motion information transmitted in inter-frame prediction mode, motion information can be predicted on a block, sub-block, or sample basis based on the correlation between motion information between neighboring blocks and the current block. Motion information may include motion vectors and reference frame indices. Motion information may also include inter-frame prediction direction information (L0 prediction, L1 prediction, Bi prediction, etc.). In the case of inter-frame prediction, neighboring blocks may include spatially neighboring blocks existing within the current frame and temporally neighboring blocks existing in the reference frame. For example, inter-frame predictor 332 can configure a motion information candidate list based on neighboring blocks and deduce the motion vector and / or reference frame index for the current block based on received candidate selection information. Inter-frame prediction can be performed based on various prediction modes, and the information about the prediction may include information indicating the mode of inter-frame prediction for the current block.

[0083] Adder 340 can add the acquired residual signal to the prediction signal (prediction block, prediction sample array) output from the predictor (including inter-frame predictor 332 and / or intra-frame predictor 331) to generate a reconstruction signal (reconstructed frame, reconstruction block, reconstruction sample array). If no residual exists for the block to be processed when skip mode is applied, the prediction block can be used as the reconstruction block.

[0084] Adder 340 can be referred to as a reconstructor or reconstructed block generator. The generated reconstructed signal can be used for intra-frame prediction of the next block to be processed in the current frame, and as described later, it can also be filtered out or used for inter-frame prediction of the next frame.

[0085] In addition, luminance mapping with chroma scaling (LMCS) can also be applied in the image decoding process.

[0086] Filter 350 can apply filtering to the reconstructed signal, thereby improving the subjective / objective image quality. For example, filter 350 can apply various filtering methods to the reconstructed image to generate a modified reconstructed image, and send the modified reconstructed image to memory 360, specifically to the DPB in memory 360. Various filtering methods may include, for example, deblocking filtering, adaptive sample shifting, adaptive loop filtering, bidirectional filtering, etc.

[0087] The (modified) reconstructed frame stored in the DPB of memory 360 can be used as a reference frame in inter-frame predictor 332. Memory 360 can store motion information of blocks in which motion information within the current frame is derived (decoded) and / or motion information of blocks within previously reconstructed frames. The stored motion information can be transmitted to inter-frame predictor 260 so that it can be used as motion information for spatially or temporally neighboring blocks. Memory 360 can store reconstructed samples of reconstructed blocks within the current frame and can transmit the stored reconstructed samples to intra-frame predictor 331.

[0088] In this document, the exemplary embodiments described in the encoding device 200 filter 260, inter-frame predictor 221 and intra-frame predictor 222 can be equivalently applied to or correspond to the decoding device 300 filter 350, inter-frame predictor 332 and intra-frame predictor 331, respectively.

[0089] As described above, prediction is performed during video encoding to improve compression efficiency. This generates a prediction block that includes prediction samples of the current block (i.e., the target block) to be encoded. Here, the prediction block includes prediction samples in the spatial domain (or pixel domain). The prediction block is derived in the same manner in both the encoding and decoding devices, and the encoding device can signal information about the residual between the original block and the prediction block (residual information) instead of the original sample values ​​of the original block to the decoding device, thereby improving image encoding efficiency. The decoding device can derive a residual block including residual samples based on the residual information, add the residual block and the prediction block to generate a reconstruction block including reconstructed samples, and generate a reconstructed image including the reconstruction block.

[0090] Residual information can be generated through transformation and quantization processes. For example, the encoding device can derive a residual block between the original block and the prediction block, perform a transformation process on the residual samples (residual sample array) included in the residual block to derive transform coefficients, perform a quantization process on the transform coefficients to derive quantized transform coefficients, and (via a bitstream) signal the relevant residual information to the decoding device. Here, the residual information may include the value information, position information, transform technique, transform kernel, quantization parameters, etc., of the quantized transform coefficients. The decoding device can perform dequantization / inverse transform processes based on the residual information and derive residual samples (or residual blocks). The decoding device can generate a reconstructed frame based on the prediction block and the residual block. Furthermore, for reference in future inter-frame prediction of the frame, the encoding device can also perform dequantization / inverse transform on the quantized transform coefficients to derive residual blocks and generate a reconstructed frame based on these residual blocks.

[0091] In addition, the process of outputting and removing images from the decoded picture buffer (DPB) can be performed. The process of outputting and removing images from the decoded picture buffer (DPB) in the conventional VVC standard used in video / image coding systems can be represented as shown in the following table.

[0092] [Table 1]

[0093]

[0094]

[0095] For example, according to the VVC standard for video / image coding systems, the frame output process can be invoked once per frame, as shown in the table above, before decoding the current frame (however, after parsing the slice header of the first slice of the current frame).

[0096] Additionally, for example, referring to Table 1, when the current access unit (AU) is the Encoded Video Sequence Start (CVSS) AU that is not AU 0, the following sorting steps can be applied.

[0097] First, the variable NoOutputOfPriorPicsFlag of the decoder under test can be derived as follows.

[0098] - If each of the values ​​of PicWidthMaxInSamplesY, PicHeightMaxInSamplesY, MaxChromaFormat, MaxBitDepthMinus8, or max_dec_pic_buffering_minus1[Htid] derived for the current AU is different from the values ​​of PicWidthMaxInSamplesY, PicHeightMaxInSamplesY, MaxChromaFormat, MaxBitDepthMinus8, or max_dec_pic_buffering_minus1[Htid] derived for the previous AU in the decoding order, NoOutputOfPriorPicsFlag may be set to 1 by the decoder under test, independent of the value of ph_no_output_of_prior_pics_flag of the current AU.

[0099] Otherwise, NoOutputOfPriorPicsFlag is set to the value of ph_no_output_of_prior_pics_flag for the current AU.

[0100] - Secondly, the value of NoOutputOfPriorPicsFlag derived for the decoder under test can be applied to the HRD (hypothetical reference decoder) as follows. Therefore, when the value of NoOutputOfPriorPicsFlag is 1, all picture storage buffers in the DPB can be cleared without outputting the pictures they contain, and the DPB fullness can be set to 0.

[0101] Furthermore, for example, referring to Table 1, all frames k in the DPB can be removed from the DPB when both of the following conditions are true for any frame k in the DPB.

[0102] - Screen k is marked as "not used for reference".

[0103] - Picture k has a PictureOutputFlag equal to 0 or the DPB output time of picture k is less than or equal to the CPB removal time of the first DU (denoted as DU m) of the current picture n; that is, DpbOutputTime[k] is less than or equal to DuCpbRemovalTime[m].

[0104] Furthermore, for example, referring to Table 1, the DPB fill factor can be reduced by one for each frame removed from the DPB.

[0105] In addition, for example, referring to Table 1, when the current access unit (AU) is the coded video sequence start (CVSS) AU that is not AU 0, the following sorting steps can be applied.

[0106] First, the variable NoOutputOfPriorPicsFlag of the decoder under test can be derived as follows.

[0107] - If each of the values ​​of PicWidthMaxInSamplesY, PicHeightMaxInSamplesY, MaxChromaFormat, MaxBitDepthMinus8, or max_dec_pic_buffering_minus1[Htid] derived for the current AU is different from the values ​​of PicWidthMaxInSamplesY, PicHeightMaxInSamplesY, MaxChromaFormat, MaxBitDepthMinus8, or max_dec_pic_buffering_minus1[Htid] derived for the previous AU in the decoding order, NoOutputOfPriorPicsFlag may be set to 1 by the decoder under test, independent of the value of ph_no_output_of_prior_pics_flag of the current AU.

[0108] - Otherwise, NoOutputOfPriorPicsFlag can be set to the value of ph_no_output_of_prior_pics_flag for the current AU.

[0109] - Secondly, the variable NoOutputOfPriorPicsFlag for the decoder under test can be applied to the HRD (hypothetical reference decoder) as follows.

[0110] For example, when the value of NoOutputOfPriorPicsFlag is 1, all the picture storage buffers in the DPB can be cleared without outputting the pictures it contains, and the DPB fullness can be set to 0.

[0111] - Otherwise (i.e., when the value of NoOutputOfPriorPicsFlag is 0), all picture storage buffers containing pictures marked as "not needed for output" and "not used for reference" can be cleared (in the absence of output, and all non-empty picture storage buffers in the DPB can be cleared by repeatedly calling the "Trim" procedure specified in Section C.5.2.4 of the (VVC standard), and the DPB fullness can be set to equal 0).

[0112] In addition, the trimming process may include the following sorting steps.

[0113] 1. The first output screen (or screen) can be selected as the screen with the smallest PicOrderCntVal value among all screens marked as "needed for output" in the DPB.

[0114] 2. Each element in the image can be cropped in ascending order of nuh_layer_id using a conformance cropping window, and the cropped image can be output and marked as "not needed for output".

[0115] 3. Each frame storage buffer, including one of the frames marked "not needed for output" and cropped and output, can be cleared, and the DPB fullness can be reduced by 1.

[0116] Furthermore, for example, referring to Table 1, when the current access unit (AU) is a non-AU 0 Encoded Video Sequence Start (CVSS) AU, all picture storage buffers containing frames marked "not needed for output" and "not used for reference" can be cleared (in the absence of output). For each of the picture storage buffers, the DPB fill level can be decreased by 1. Additionally, the "trim" procedure specified in Section C.5.2.4 (of the VVC standard) can be repeatedly invoked while further decreasing the DPB fill level by 1 for each additional picture storage buffer cleared, provided one or more of the following conditions are true, until none of the following conditions are true.

[0117] - The number of frames marked as "needed for output" in the DPB is greater than max_num_reorder_pics[Htid].

[0118] - max_latency_increase_plus1[Htid] is not equal to 0 and there is at least one picture in the DPB that is marked as "needed for output" and whose associated variable PicLatencyCount is greater than or equal to MaxLatencyPictures[Htid].

[0119] - The number of frames in the DPB is greater than or equal to max_dec_pic_buffering_minus1[Htid]+1.

[0120] Furthermore, the conventional VVC standard used for image output and removal processes may have the following problems. Specifically, the conventional VVC standard for the operation of the DPB output order may have the following problems in terms of the correlation process of decoding the current frame and the correlation process after decoding the current frame.

[0121] 1. When the frame buffer is cleared by calling the trimming process, the DPB fill level should decrease only once. However, in the standard VVC, the DPB fill level decreases twice. That is, the DPB fill level decreases once during the trimming process and once after the trimming process is complete.

[0122] 2. There are three conditions for invoking the trimming process when one of the conditions is true. However, checks have already been performed during the additional collision process for all three conditions, and not all three conditions are required. Here, as mentioned above, the three conditions can be i) the number of frames marked "required for output" in the DPB is greater than max_num_reorder_pics[Htid], ii) max_latency_increase_plus1[Htid] is not equal to 0 and there is at least one frame in the DPB marked "required for output" with the associated variable PicLatencyCount greater than or equal to MaxLatencyPictures[Htid], and iii) the number of frames in the DPB is greater than or equal to max_dec_pic_buffering_minus1[Htid]+1.

[0123] The following section provides a detailed description of the syntax elements max_num_reorder_pics[Htid], max_latency_increase_plus1[Htid], and max_dec_pic_buffering_minus1[Htid] used for the three conditions.

[0124] Therefore, this disclosure proposes a solution to the aforementioned problems. The proposed implementations can be applied individually or in combination. That is, this disclosure can be modified / applied to the following methods regarding the output and removal of a screen from a DPB.

[0125] In one implementation, the DPB fullness may be reduced only once for each decoded screen buffer cleared by calling the trimming process. The DPB reduction may be performed during the trimming process (i.e., by the trimming process itself) or after the trimming process is completed.

[0126] Furthermore, in one implementation, the trimming process can be invoked during the initial screen decoding (i.e., before decoding the current screen, but after parsing the slice header of the first slice of the current screen) and / or after screen decoding (i.e., when the last DU containing the current screen's AU n is removed from the CPB).

[0127] Furthermore, in one implementation, after clearing all picture storage buffers containing pictures marked "not needed for output" and "not used for reference" (in the absence of output), a trimming process can be invoked at the start of picture decoding if the DPB's picture buffers for storing the current picture are insufficient. This condition can be expressed as "the number of pictures in the DPB is greater than or equal to max_dec_pic_buffering_minus1[Htid]+1".

[0128] Furthermore, in one implementation, the trimming process may be invoked when screen decoding is terminated under at least one of the following conditions.

[0129] i) The number of frames marked as “required for output” in the DPB is greater than max_num_reorder_pics[Htid].

[0130] ii) max_latency_increase_plus1[Htid] is not equal to 0, and there is at least one picture in the DPB marked as "needed for output" and whose associated variable PicLatencyCount is greater than or equal to MaxLatencyPictures[Htid].

[0131] The above implementation methods can be achieved as shown in Table 2 below. In one example, Table 2 below represents some or all of the VVC standard specifications that provide the above implementation methods.

[0132] [Table 2]

[0133]

[0134] For example, referring to Table 2, when the current frame is the first frame of the current AU and the current AU is the Encoded Video Sequence Start (CVSS) AU (which is not AU 0), the following sorting steps can be applied.

[0135] First, the variable NoOutputOfPriorPicsFlag can be derived for the decoder under test as follows.

[0136] - If each of the values ​​of PicWidthMaxInSamplesY, PicHeightMaxInSamplesY, MaxChromaFormat, MaxBitDepthMinus8, or max_dec_pic_buffering_minus1[Htid] derived for the current AU is different from the values ​​of PicWidthMaxInSamplesY, PicHeightMaxInSamplesY, MaxChromaFormat, MaxBitDepthMinus8, or max_dec_pic_buffering_minus1[Htid] derived for the previous AU in the decoding order, NoOutputOfPriorPicsFlag may be set to equal l by the decoder under test, independent of the value of ph_no_output_of_prior_pics_flag of the current AU.

[0137] - Otherwise, NoOutputOfPriorPicsFlag can be set to equal the current AU's ph_no_output_of_prior_pics_flag.

[0138] Secondly, the value of NoOutputOfPriorPicsFlag derived for the decoder under test can be applied to the HRD (hypothetical reference decoder) as follows.

[0139] - When the value of NoOutputOfPriorPicsFlag is 1, all the picture storage buffers in the DPB can be cleared without outputting the pictures it contains, and the DPB fullness can be set to 0.

[0140] - Otherwise (i.e., when the value of NoOutputOfPriorPicsFlag is 0), all picture storage buffers containing pictures marked as "not needed for output" and "not used for reference" can be cleared (in the absence of output), and all non-empty picture storage buffers in the DPB can be cleared by repeatedly calling the "bumping" procedure specified in Section C.5.2.4, and the DPB fullness can be set to equal to 0.

[0141] Additionally, for example, referring to Table 2, when the current frame is not a CVSS AU or the current AU is a CVSS AU other than AU 0, but the current frame is not the first frame in the current AU, all frame storage buffers containing frames marked as "not needed for output" and "not used for reference" can be cleared (in the absence of output). For each cleared frame storage buffer, the DPB fullness can be reduced by 1. The "trim" procedure specified in Section C.5.2.4 (of the VVC standard) can be repeatedly invoked until the number of frames in the DPB becomes no greater than or equal to max_dec_pic_buffering_minus1[Htid]+1.

[0142] Based on the implementation method in Table 2 above, by removing the operation of further reducing the DPB fullness by 1 for each additional screen storage buffer cleared during the trimming process, the DPB fullness can be reduced only once when outputting from the DPB and removing the screen.

[0143] Furthermore, according to the implementation described in Table 2 above, the retouching process is limited to cases where the condition based on max_dec_pic_buffering_minus1[Htid] is met, and unnecessary procedures for checking overlapping conditions are not required. That is, when the retouching process is called, the existing conditions based on max_num_reorder_pics[Htid] and max_latency_increase_plus1[Htid] can be overlapping conditions. This allows the two conditions to be handled without being limited to being checked as conditions in the retouching process call, and only the condition based on max_dec_pic_buffering_minus1[Htid] according to the implementation of this disclosure needs to be checked. Since the two existing conditions based on max_num_reorder_pics[Htid] and max_latency_increase_plus1[Htid] have already been checked when decoding the previous frame ends (i.e., during the additional retouching process), the two conditions can be overlapping conditions when determining the current frame. When the decoder begins decoding the current frame, there is no change in the DPB related to the number of frames marked "needed for output". Therefore, since both conditions have already returned false values ​​until the additional trimming process for the previous frame is completed, there is no situation where true values ​​are returned. Therefore, it is not necessary to apply the two conditions for invoking the trimming process when decoding the current frame begins.

[0144] In one implementation, when the current frame is not a CVSS AU or the current AU is a CVSS AU other than AU 0, but the current frame is not the first frame in the current AU, all frame storage buffers, including those marked as "not needed for output" and "not used for reference," can be cleared (in the case of no output). For each cleared frame storage buffer, the DPB fullness can be reduced by 1. Furthermore, when the trimming process is invoked, it can be called repeatedly until the number of frames in the DPB becomes less than max_dec_pic_buffering_minus1[Htid]+1. However, if the number of frames in the DPB is less than max_dec_pic_buffering_minus1[Htid]+1 (i.e., if the number of frames in the DPB is not greater than or equal to max_dec_pic_buffering_minus1[Htid]+1), the trimming process may not be invoked even if conditions i) and / or ii) are not true.

[0145] i) The number of frames marked as “required for output” in the DPB is greater than max_num_reorder_pics[Htid].

[0146] ii) max_latency_increase_plus1[Htid] is not equal to 0, and there is at least one picture in the DPB that is marked as "needed for output" and whose associated variable PicLatencyCount is greater than or equal to MaxLatencyPictures[Htid].

[0147] In other words, according to the method proposed in this disclosure as described above, the DPB fullness can be reduced only once when the screen buffer is emptied during the trimming process, thus improving the accuracy of the DPB's output sequence operation. Furthermore, the number of call condition check numbers for the trimming process is reduced, and the complexity can be decreased. Therefore, accuracy and efficiency can be improved in DPB management (i.e., screen output and removal operations in the DPB).

[0148] Figure 4 An example is provided illustrating the coding process according to an embodiment of this disclosure. Figure 4 The method shown can be derived from Figure 2 The encoding device 200 shown performs this operation. Additionally, the following can be omitted: Figure 4 The steps shown may include one or more steps, and specific steps may be added according to the implementation method.

[0149] Reference Figure 4The encoding device decodes the (reconstructed) screen (step S400). The encoding device can decode the screen of the current AU.

[0150] The encoding device manages the DPB based on the DPB parameters (step S410). Here, DPB management can also be referred to as DPB update. The DPB management process may include a marking process and / or removal process for the decoded frames in the DPB. The decoded frames can be used as a reference for inter-frame prediction of subsequent frames. That is, the decoded frames are used as reference frames for inter-frame prediction of subsequent frames in the decoding order. Basically, each decoded frame can be inserted into the DPB. In addition, the DPB can usually be updated before decoding the current frame. If the layer associated with the DPB is not an output layer (or the DPB parameters are not related to the output layer) but a reference layer, the decoded frames in the DPB may not be output. If the layer (or DPB parameters) associated with the DPB is an output layer, the decoded frames in the DPB may be output based on the DPB and / or the DPB parameters. DPB management may include the operation of outputting decoded frames from the DPB.

[0151] The encoding device encodes image information including information related to DPB parameters (step S420). The information related to DPB parameters may include the information / syntax elements disclosed in the embodiments described above and / or the syntax elements shown in the following table.

[0152] [Table 3]

[0153]

[0154] For example, Table 3 above can represent a set of video parameters (VPS) that includes syntax elements for DPB parameters that signal notifications.

[0155] The semantics of the syntactic elements represented in Table 3 above can be as follows.

[0156] [Table 4]

[0157]

[0158]

[0159] For example, the syntax element `vps_num_dpb_params` can represent the number of `dpb_parameters()` syntax structures in the VPS. The value of `vps_num_dpb_params` can be in the range of 0 to 16. Furthermore, if the syntax element `vps_num_dpb_params` does not exist, its value can be considered as 0.

[0160] Additionally, for example, the syntax element `same_dpb_size_output_or_nonoutput_flag` can indicate that the syntax element `layer_nonoutput_dpb_params_idx[i]` can exist in the VPS. For instance, if the value of `same_dpb_size_output_or_nonoutput_flag` is 1, it means that `layer_nonoutput_dpb_params_idx[i]` does not exist in the VPS; if the value of `same_dpb_size_output_or_nonoutput_flag` is 0, it means that `layer_nonoutput_dpb_params_idx[i]` can exist.

[0161] Additionally, for example, the syntax element `vps_sublayer_dpb_params_present_flag` can be used to control the presence of the syntax elements `max_dec_pic_buffering_minus1[]`, `max_num_reorder_pics[]`, and `max_latency_increase_plus1[]` in the `dpb_parameters()` syntax structure of the VPS. Furthermore, if the syntax element `vps_sublayer_dpb_params_present_flag` does not exist, its value can be considered as 0.

[0162] Additionally, for example, the syntax element `dpb_size_only_flag[i]` can indicate whether the syntax elements `max_num_reorder_pics[]` and `max_latency_increase_plus1[]` can exist in the `i`th `dpb_parameters()` syntax structure of the VPS. For instance, if the value of `dpb_size_only_flag[i]` is 1, it means that `max_num_reorder_pics[]` and `max_latency_increase_plus1[]` do not exist in the `i`th `dpb_parameters()` syntax structure of the VPS. Conversely, if the value of `dpb_size_only_flag[i]` is 0, it means that `max_num_reorder_pics[]` and `max_latency_increase_plus1[]` can exist in the `i`th `dpb_parameters()` syntax structure of the VPS.

[0163] Additionally, for example, the syntax element `dpb_max_temporal_id[i]` can represent the TemporalId of the highest sublayer representation in the `i`th `dpb_parameters()` syntax structure of the VPS, where the DPB parameters can exist. Furthermore, the value of `dpb_max_temporal_id[i]` can be in the range of 0 to `vps_max_sublayer_minus1`. Additionally, for example, if the value of `vps_max_sublayer_minus1` is 0, the value of `dpb_max_temporal_id[i]` can be considered 0. Furthermore, for example, if the value of `vps_max_sublayer_minus1` is greater than 0 and the value of `vps_all_layers_same_num_sublayer_flag` is 1, the value of `dpb_max_temporal_id[i]` can be considered equal to `vps_max_sublayer_minus1`.

[0164] Additionally, for example, the syntax element `layer_output_dpb_params_idx[i]` can specify the index of the `dpb_parameters()` syntax structure in the list of `dpb_parameters()` syntax structures of the VPS, which is applied to the `i`th layer as the output layer of the OLS. When the syntax element `layer_output_dpb_params_idx[i]` is present, the value of `layer_output_dpb_params_idx[i]` can be in the range of 0 to `vps_num_dpb_params-1`.

[0165] For example, when vps_independent_layer_flag[i] is 1, it can be the dpb_parameters() syntax structure that exists in the SPS that is referenced in the dpb_parameters() syntax structure layer applied to the i-th layer as the output layer.

[0166] Alternatively, for example, when vps_independent_layer_flag[i] is 0, the following description can be applied.

[0167] - When vps_num_dpb_params is 0, the value of layer_output_dpb_params_idx[i] can be considered as 0.

[0168] The value of layer_output_dpb_params_idx[i] can be determined by the bitstream consistency requirement, so that the value of dpb_size_only_flag[layer_output_dpb_params_idx[i]] becomes 0.

[0169] Additionally, for example, the syntax element `layer_nonoutput_dpb_params_idx[i]` can specify the index of the `dpb_parameters()` syntax structure applied to the i-th layer (a non-output layer of the OLS) in the list of `dpb_parameters()` syntax structures of the VPS. When the syntax element `layer_nonoutput_dpb_params_idx[i]` is present, its value can be in the range of 0 to `vps_num_dpb_params-1`.

[0170] For example, when same_dpb_size_output_or_nonoutput_flag is 1, the following description can be applied.

[0171] - When vps_independent_layer_flag[i] is 1, it can be the dpb_parameters() syntax structure that exists in the SPS that is referenced in the dpb_parameters() syntax structure layer applied to the i-th layer as a non-output layer.

[0172] - When vps_independent_layer_flag[i] is 0, the value of layer_nonoutput_dpb_params_idx[i] can be regarded as equal to layer_output_dpb_params_idx[i].

[0173] Alternatively, for example, when same_dpb_size_output_or_nonoutput_flag is 0, the value of layer_output_dpb_params_idx[i] can be considered as 0 when vps_num_dpb_params is 1.

[0174] On the other hand, for example, the dpb_parameters() syntax structure disclosed in Table 3 as a DPB parameter syntax structure can be as follows.

[0175] [Table 5]

[0176]

[0177] Referring to Table 5, the dpb_parameters() syntax structure can provide information on the DPB size, maximum number of frame reorderings, and maximum latency for each CLVS of the CVS. The dpb_parameters() syntax structure can be represented as DPB parameter information or DPB parameter details.

[0178] When a VPS includes the dpb_parameters() syntax structure, the VPS can specify the OLS to which the dpb_parameters() syntax structure applies. Alternatively, when an SPS includes the dpb_parameters() syntax structure, the dpb_parameters() syntax structure can be applied to the OLS, which only includes the lowest layer among the layers referencing the SPS.

[0179] The semantics of the syntactic elements represented in Table 5 described above can be as follows.

[0180] [Table 6]

[0181]

[0182] For example, for each CLVS of CVS, adding 1 to the value of the syntax element max_dec_pic_buffering_minus1[i] can represent the maximum required size of the DPB in units of the screen storage buffer when Htid equals i. For example, max_dec_pic_buffering_minus1[i] can be information about the DPB size. For example, the value of the syntax element max_dec_pic_buffering_minus1[i] can be in the range of 0 to MaxDpbSize-1. Additionally, for example, when i is greater than 0, max_dec_pic_buffering_minus1[i] can be greater than or equal to max_dec_pic_buffering_minus1[i-1]. Furthermore, for example, if there is no max_dec_pic_buffering_minus1[i] for i in the range from 0 to maxSubLayersMinus1-1, the value of the syntax element max_dec_pic_buffering_minus1[i] can be regarded as equal to max_dec_pic_buffering_minus1[maxSubLayersMinus1] since subLayerInfoFlag is equal to 0.

[0183] Additionally, for example, the syntax element `max_num_reorder_pics[i]` can represent the maximum allowed number of frames for each CLVS in the decoding order before all frames of the CLVS and in the output order after the corresponding frame of the CLVS when `Htid` equals `i`. For example, `max_num_reorder_pics[i]` can be information about the maximum number of frame reorderings for the DPB. The value of `max_num_reorder_pics[i]` can be in the range of 0 to `max_dec_pic_buffering_minus1[i]`. Furthermore, for example, when `i` is greater than 0, `max_num_reorder_pics[i]` can be greater than or equal to `max_num_reorder_pics[i-1]`. Additionally, for example, when `i` is in the range of 0 to `maxSubLayersMinus1-1` and `max_num_reorder_pics[i]` does not exist, since `subLayerInfoFlag` is 0, the syntax element `max_num_reorder_pics[i]` can be considered equal to `max_num_reorder_pics[maxSubLayersMinus1]`.

[0184] Additionally, for example, the non-zero syntax element `max_latency_increase_plus1[i]` can be used to calculate the value of `MaxLatencyPictures[i]`. `MaxLatencyPictures[i]` can be any of the frames preceding the CLVS in the output order for each CLVS of the CVS, and can be any of the frames following the corresponding frames in the decoding order when `Htid` equals `i`. For example, `max_latency_increase_plus1[i]` can be information about the maximum latency of the DPB.

[0185] For example, when max_latency_increase_plus1[i] is not 0, the value of MaxLatencyPictures[i] can be derived as shown in the following formula.

[0186] [Formula 1]

[0187]

[0188] Furthermore, for example, when max_latency_increase_plus1[i] is 0, no limit may be specified. The value of max_latency_increase_plus1[i] can be between 0 and 2. 32The range is -2. Furthermore, for example, in the case where max_latency_increase_plus1[i] does not exist for i in the range of 0 to maxSubLayersMinus1-1, since subLayerInfoFlag is 0, the syntax element max_latency_increase_plus1[i] can be regarded as equal to max_dec_pic_buffering_minus1[maxSubLayersMinus1].

[0189] DPB management can be performed based on information related to DPB parameters / syntax elements. Different DPB parameters can be signaled based on whether the current layer is an output layer or a reference layer, or based on whether the DPB (or DPB parameters) is used for OLS (whether OLS mapping is applied).

[0190] Furthermore, despite Figure 4 Although not shown, the encoding device can decode the current frame based on the updated / managed DPB. Furthermore, the decoded current frame can be inserted into the DPB, and the DPB, including the decoded current frame, can be updated based on DPB parameters before decoding the next frame in the decoding order.

[0191] Figure 5 A decoding process according to an embodiment of the present disclosure is illustrated. Figure 5 The method disclosed in the article can be derived from Figure 3 The decoding device 300 disclosed in the document is used for execution. Additionally, the following can be omitted: Figure 5 The steps shown are one or more, and different steps may be added depending on the implementation.

[0192] Reference Figure 5 The decoding device obtains image information, including information related to the DPB parameters, from the bitstream (step S500). The decoding device can obtain information including information related to the DPB parameters. The information / syntax elements related to the DPB parameters can be as described above.

[0193] The decoding device manages the DPB based on the DPB parameters (step S510). Here, DPB management can also be referred to as DPB update. The DPB management process may include the marking and / or removal process of the decoded frames in the DPB. The decoding device can derive the DPB parameters based on information related to the DPB parameters, and can perform the DPB management process based on the derived DPB parameters.

[0194] The decoding device decodes / outputs the current frame based on the DPB (step S520). The decoding device can decode the current frame based on an updated / managed DPB. For example, based on inter-frame predictions where (previously) decoded frames in the DPB are used as reference frames, blocks / slices in the current frame can be decoded.

[0195] The following figures illustrate detailed examples of this disclosure. The names of specific devices, specific terms, or names (e.g., names of grammars / grammatical elements, etc.) shown in the figures are presented as examples, and the technical features of this disclosure are not limited to the specific names used in the figures below.

[0196] Figure 6 and Figure 7 Examples of video / image encoding methods and related components according to embodiments of the present disclosure are illustrated schematically.

[0197] Figure 6 The method disclosed in the article can be derived from Figure 2 or Figure 7 The publicly disclosed encoding device 200 is executed. Here, Figure 7 The coding device 200 disclosed in the paper is schematically shown. Figure 2 The encoding device 200 is disclosed in the document. Specifically, for example, Figure 6 Steps S600 and S610 shown can be performed by Figure 2 The DPB shown is executed, and step S620 can be performed by... Figure 2 The entropy encoder 240 shown performs this operation. Additionally, although not shown in the figure, the decoding process for the current frame can be performed by the predictor 220, residual processor 230, adder 340, etc. Furthermore, Figure 6 The methods disclosed herein can be performed by embodiments of this disclosure. Therefore, descriptions or brief descriptions are omitted or referenced. Figure 6 The description overlaps with the detailed description.

[0198] Reference Figure 6 The encoding device can generate information related to the decoded screen buffer (DPB) (step S600).

[0199] DPB-related information may include at least one of the following: syntax elements related to the maximum required size of the DPB, syntax elements related to the maximum number of screen reorderings of the DPB, or syntax elements related to the maximum latency of the DPB.

[0200] For example, the syntax element related to the maximum required size of the DPB can be the syntax element max_dec_pic_buffering_minus1[i] described above. In this case, the value of max_dec_pic_buffering_minus1[i] plus 1 can represent the maximum required size of the DPB in units of the picture storage buffer when Htid equals i. The syntax element related to the maximum number of picture reorderings for the DPB can be the syntax element max_num_reorder_pics[i] described above. In this case, max_num_reorder_pics[i] can represent the maximum allowed number of pictures of each CLVS for a CVS that are before all pictures of the CLVS in decoding order and after the corresponding pictures in output order when Htid equals i. The syntax element related to the maximum latency of the DPB can be the syntax element max_latency_increase_plus1[i] described above. In this case, a non-zero value of max_latency_increase_plus1[i] can be used to calculate the value of MaxLatencyPictures[i]. MaxLatencyPictures[i] can represent the maximum allowed number of CLVS frames that are in the output order before all frames of the CLVS and, when Htid equals i, can be in the decoding order after the corresponding frames. For example, the value of MaxLatencyPictures[i] can be derived by (the value of the syntax element related to the maximum number of frame reorderings of the DPB + the value of the syntax element related to the maximum latency of the DPB - 1) and can be calculated as shown in Equation 1 above.

[0201] In addition, DPB-related information may also include various types of information related to the output / removal of the image in the DPB, such as information / syntax elements related to the DPB parameters shown in Table 3 above.

[0202] The encoding device can update the DPB based on DPB-related information (step S610). The encoding device can update the DPB before decoding the current frame and after generating the slice header of the current frame / encoding the slice header of the current frame (i.e., marking / removing / outputting frames in the DPB). The DPB may include frames that have been decoded before the current frame.

[0203] In order to update (i.e., mark / remove / output the picture in the DPB) the DPB, the encoding device can perform a DPB trimming process (i.e., the picture storage buffer in the DPB) based on the DPB-related information and perform an operation to reduce the DPB fullness.

[0204] In one implementation, the encoding device may invoke the trimming process based on a first condition that the number of frames in the DPB is not greater than or equal to the value of the syntax element associated with the maximum required size of the DPB (e.g., max_dec_pic_buffering_minus1[i]) plus 1.

[0205] Alternatively, the triggering procedure can be invoked not based on a second condition related to the maximum number of reordered pictures in the DPB (e.g., max_num_reorder_pics) or a third condition related to the maximum latency in the DPB (e.g., max_latency_increase_plus1). In other words, if the second or third condition is met but the first condition is not met, the encoding device may not invoke the triggering procedure.

[0206] Here, the second condition can be related to whether the number of frames in the DPB marked as "needed for output" is greater than the value of the syntax element related to the maximum number of frame reorderings in the DPB (e.g., max_num_reorder_pics). The third condition can be whether the value of the syntax element related to the maximum latency of the DPB (e.g., max_latency_increase_plus1) is not equal to 0 and whether there exists at least one frame in the DPB marked as "needed for output" whose associated variable PicLatencyCount is greater than or equal to MaxLatencyPictures[Htid]. MaxLatencyPictures[i] can be derived by (the value of the syntax element related to the maximum number of frame reorderings in the DPB + the value of the syntax element related to the maximum latency of the DPB - 1).

[0207] In other words, as mentioned above, when the trimming process is invoked, since the second and third conditions have already been checked at the end of the decoding of the previous frame (i.e., during the additional trimming process), the second and third conditions can be redundant conditions when decoding the current frame. Therefore, the trimming process can be performed simply by determining whether the first condition is met without relying on the second and third conditions as redundant conditions (i.e., excluding the second and third conditions). As a specific example, if the first condition is not met, even if conditions i) and / or ii) are not true, the trimming process may not be invoked.

[0208] i) The number of frames marked as “required for output” in the DPB is greater than max_num_reorder_pics[Htid].

[0209] ii) max_latency_increase_plus1[Htid] is not equal to 0, and there is at least one picture in the DPB marked as "needed for output" and whose associated variable PicLatencyCount is greater than or equal to MaxLatencyPictures[Htid].

[0210] Additionally, in one implementation, for picture storage buffers emptied in the DPB during a trimming process invoked based on the fulfillment of the first condition, the encoding device may reduce the DPB fullness by 1. In other words, after the trimming process invoked based on the fulfillment of the first condition, the encoding device does not perform the operation of additionally reducing the DPB fullness by 1 for picture storage buffers emptied in the DPB during the trimming process. In one example, as described above in Table 2, the additional operation of reducing the DPB fullness by 1 for each additional emptied picture storage buffer can be removed from the trimming process invocation operation. In this case, the trimming process is invoked until the number of pictures in the DPB is not greater than or equal to max_dec_pic_buffering_minus1[Htid]+1 (i.e., the first condition is met), and the DPB fullness can be reduced only within the invoked trimming process operation. Since the detailed operation of the trimming process is as described above, it is omitted here.

[0211] Additionally, in one implementation, the encoding device may determine whether to invoke the trimming process based on the fact that the current frame is the first frame of the current access unit (AU) of a non-AU 0 Encoded Video Sequence Start (CVSS) access unit (AU). Here, AU 0 may specify the first AU of the bitstream; for example, AU 0 may be the first AU of the bitstream in decoding order. In one example, as described above in Table 2, if the current AU is not a CVSS AU or if the current AU is a non-AU 0 CVSS AU but the current frame is not the first frame in the current AU, all frame storage buffers, including those marked "not needed for output" and "not used for reference," may be cleared (in the absence of output). For each cleared frame storage buffer, the DPB fullness may be reduced by 1. The trimming process may be repeatedly invoked until the number of frames in the DPB is no greater than or equal to max_dec_pic_buffering_minus1[Htid]+1.

[0212] In other words, as described above, the encoding device can perform the removal and / or output of images in the DPB based on DPB-related information through operations such as trimming processes, and can update the DPB.

[0213] The encoding device can encode image information including DPB-related information (step S620). In one embodiment, the encoding device can encode image information including at least one of the following: syntax elements related to the maximum required size of the DPB, syntax elements related to the maximum number of frame reorderings in the DPB, or syntax elements related to the maximum latency of the DPB. Furthermore, the encoding device can encode image information including the slice header of the current frame.

[0214] Furthermore, although not shown, the encoding device can decode the current frame based on the updated DPB. For example, the encoding device can derive prediction samples by performing inter-frame prediction on blocks in the current frame using a reference frame based on the DPB, and can generate reconstructed samples and / or reconstructed frames for the current frame based on the prediction samples. Additionally, for example, the encoding device can derive residual samples of blocks in the current frame, and can generate reconstructed samples and / or reconstructed frames by adding the prediction samples and residual samples. Then, depending on scene requirements, to improve subjective / objective image quality, loop filtering processes such as deblocking filtering, SAO, and / or ALF processes can be applied to the reconstructed samples as described above. The encoding device can generate / encode prediction-related information and / or residual information for blocks, and the image information may include prediction-related information and / or residual information. Furthermore, the encoding device can insert the decoded current frame into the DPB. Additionally, for example, the encoding device can derive the DPB parameters of the current AU and generate DPB-related information for the DPB parameters. The image information may include DPB-related information.

[0215] Image / video information, including the various types of information mentioned above, can be encoded and output in bitstream format. The bitstream can be sent to a decoding device via a network or (digital) storage medium. Here, the network can include broadcast networks and / or communication networks, and the digital storage medium can include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc.

[0216] Figure 8 and Figure 9 Examples of video / image decoding methods and related components according to embodiments of the present disclosure are illustrated schematically.

[0217] Figure 8 The method disclosed in the article can be derived from Figure 3 or Figure 9 The publicly disclosed decoding device 300 is executed. Here, Figure 9The decoding device 300 disclosed in the paper schematically illustrates... Figure 3 The publicly disclosed decoding device is 300. Specifically, Figure 8 The step S800 shown can be performed by Figure 3 The entropy decoder 310 shown is executed, and step S810 can be performed by... Figure 3 The DPB is executed as shown. Furthermore, step S820 can be executed by the entropy decoder 310, residual processor 320, predictor 330, adder 340, etc. Figure 8 The methods disclosed herein can be performed by embodiments of this disclosure. Therefore, descriptions or brief descriptions are omitted or referenced. Figure 8 The description overlaps with the detailed description.

[0218] Reference Figure 8 The decoding device can obtain image information, including information related to the decoded picture buffer (DPB), from the bitstream (step S800).

[0219] DPB-related information may include at least one of the following: syntax elements related to the maximum required size of the DPB, syntax elements related to the maximum number of screen reorderings of the DPB, or syntax elements related to the maximum latency of the DPB.

[0220] For example, the syntax element related to the maximum required size of the DPB can be the syntax element max_dec_pic_buffering_minus1[i] described above. In this case, the value of max_dec_pic_buffering_minus1[i] plus 1 can represent the maximum required size of the DPB in units of the picture storage buffer when Htid equals i. The syntax element related to the maximum number of picture reorderings for the DPB can be the syntax element max_num_reorder_pics[i] described above. In this case, max_num_reorder_pics[i] can represent the maximum allowed number of CLVSs for each CVS that are before all pictures of the CLVS in decoding order and after the corresponding pictures in output order when Htid equals i. The syntax element related to the maximum latency of the DPB can be the syntax element max_latency_increase_plus1[i] described above. In this case, a non-zero value of max_latency_increase_plus1[i] can be used to calculate the value of MaxLatencyPictures[i]. MaxLatencyPictures[i] can represent the maximum allowed number of CLVS frames that are in the output order before all frames of the CLVS for each CVS, and that are in the decoding order after the corresponding frame when Htid equals i. For example, the value of MaxLatencyPictures[i] can be derived by (the value of the syntax element related to the maximum number of frame reorderings in DPB + the value of the syntax element related to the maximum latency in DPB - 1) and can be calculated as shown in Equation 1 above.

[0221] In addition, DPB-related information may also include various types of information related to the output / removal of the image in the DPB, such as information / syntax elements related to the DPB parameters shown in Tables 3 to 6 above.

[0222] The decoding device can update the DPB based on DPB-related information (step S810). The decoding device can update the DPB before decoding the current frame and after parsing the slice header of the first slice of the current frame (i.e., marking / removing / outputting frames in the DPB). The DPB may include frames decoded before the current frame.

[0223] In order to update (i.e., mark / remove / output the picture in the DPB) the DPB, the decoding device can perform a trimming process of the DPB (i.e., the picture storage buffer in the DPB) based on the DPB-related information and perform an operation to reduce the DPB fullness.

[0224] In one implementation, the decoding device may invoke the trimming process based on a first condition that the number of frames in the DPB is not greater than or equal to the value of the syntax element associated with the maximum required size of the DPB (e.g., max_dec_pic_buffering_minus1[i]) plus 1.

[0225] Alternatively, the triggering procedure can be invoked not based on a second condition related to the maximum number of reordered pictures in the DPB (e.g., max_num_reorder_pics) or a third condition related to the maximum latency in the DPB (e.g., max_latency_increase_plus1). In other words, if the second or third condition is met but the first condition is not met, the encoding device may not invoke the triggering procedure.

[0226] Here, the second condition can be related to whether the number of frames in the DPB marked as "needed for output" is greater than the value of the syntax element related to the maximum number of frame reorderings in the DPB (e.g., max_num_reorder_pics). The third condition can be whether the value of the syntax element related to the maximum latency of the DPB (e.g., max_latency_increase_plus1) is not equal to 0 and whether there exists at least one frame in the DPB marked as "needed for output" whose associated variable PicLatencyCount is greater than or equal to MaxLatencyPictures[Htid]. MaxLatencyPictures[i] can be derived by (the value of the syntax element related to the maximum number of frame reorderings in the DPB + the value of the syntax element related to the maximum latency of the DPB - 1).

[0227] In other words, as mentioned above, when the trimming process is invoked, since the second and third conditions have already been checked at the end of the decoding of the previous frame (i.e., during the additional trimming process), the second and third conditions can be redundant conditions when decoding the current frame. Therefore, the trimming process can be performed simply by determining whether the first condition is met without relying on the second and third conditions as redundant conditions (i.e., excluding the second and third conditions). As a specific example, if the first condition is not met, even if conditions i) and / or ii) are not true, the trimming process may not be invoked.

[0228] i) The number of frames marked as “required for output” in the DPB is greater than max_num_reorder_pics[Htid].

[0229] ii) max_latency_increase_plus1[Htid] is not equal to 0, and there is at least one picture in the DPB marked as "needed for output" and whose associated variable PicLatencyCount is greater than or equal to MaxLatencyPictures[Htid].

[0230] Additionally, in one implementation, for picture storage buffers emptied in the DPB during a trimming process invoked based on the fulfillment of the first condition, the encoding device may reduce the DPB fullness by 1. In other words, after the trimming process invoked based on the fulfillment of the first condition, the encoding device does not perform the operation of additionally reducing the DPB fullness by 1 for picture storage buffers emptied in the DPB during the trimming process. In one example, as described above in Table 2, the additional operation of reducing the DPB fullness by 1 for each additional emptied picture storage buffer can be removed from the trimming process invocation operation. In this case, the trimming process is invoked until the number of pictures in the DPB is not greater than or equal to max_dec_pic_buffering_minus1[Htid]+1 (i.e., the first condition is met), and the DPB fullness can be reduced only within the invoked trimming process operation. Since the detailed operation of the trimming process is as described above, it is omitted here.

[0231] Additionally, in one implementation, the encoding device may determine whether to invoke the trimming process based on the fact that the current frame is the first frame of the current access unit (AU) of a non-AU 0 Encoded Video Sequence Start (CVSS) access unit (AU). Here, AU 0 may specify the first AU of the bitstream; for example, AU 0 may be the first AU of the bitstream in decoding order. In one example, as described above in Table 2, if the current AU is not a CVSS AU or if the current AU is a non-AU 0 CVSS AU but the current frame is not the first frame in the current AU, all frame storage buffers, including those marked "not needed for output" and "not used for reference," may be cleared (in the absence of output). For each cleared frame storage buffer, the DPB fullness may be reduced by 1. The trimming process may be repeatedly invoked until the number of frames in the DPB is no greater than or equal to max_dec_pic_buffering_minus1[Htid]+1.

[0232] In other words, as described above, the decoding device can perform the removal and / or output of images in the DPB based on DPB-related information through operations such as trimming processes, and can update the DPB.

[0233] The decoding device can decode the current image based on DPB (step S820).

[0234] In one implementation, the decoding device can decode the current frame based on an updated DPB. For example, the decoding device can derive prediction samples by performing inter-frame prediction on blocks in the current frame using a reference frame based on the DPB, and can generate reconstructed samples and / or reconstructed frames for the current frame based on the prediction samples. Furthermore, for example, the decoding device can derive residual samples of blocks in the current frame, and can generate reconstructed samples and / or reconstructed frames by adding the prediction samples and residual samples. Image information may include residual information. Additionally, the decoding device can insert the decoded current frame into the DPB.

[0235] Subsequently, depending on the scenario requirements, in order to improve the subjective / objective image quality, loop filtering processes such as deblocking filtering, SAO and / or ALF processes can be applied to the reconstructed samples as described above.

[0236] Although the method has been described based on a flowchart listing the steps or blocks in the above embodiments, the steps in this document are not limited to a specific order, and specific steps may be performed in different steps or in a different order or simultaneously relative to the steps described above. Furthermore, those skilled in the art will understand that the steps in the flowchart are not exclusive, and one or more steps may be included or removed from the flowchart without affecting the scope of this document.

[0237] The methods described above according to this disclosure may be in the form of software, and the encoding and / or decoding devices according to this document may be included in an apparatus for performing image processing (e.g., TV, computer, smartphone, set-top box, display device, etc.).

[0238] When the embodiments described in this document are implemented in software, the methods described above can be implemented by modules (processes or functions) that perform the functions described above. Modules can be stored in memory and executed by a processor. Memory can be installed inside or outside the processor and can be connected to the processor via various known means. The processor may include application-specific integrated circuits (ASICs), other chipsets, logic circuits, and / or data processing devices. Memory may include read-only memory (ROM), random access memory (RAM), flash memory, memory cards, storage media, and / or other storage devices. In other words, the embodiments described in this document can be implemented and executed on a processor, microprocessor, controller, or chip. For example, the functional units shown in the various figures can be implemented and executed on a computer, processor, microprocessor, controller, or chip. In this case, information about the implementation (e.g., information about instructions) or algorithms can be stored in a digital storage medium.

[0239] Furthermore, the decoding and encoding devices using this disclosure can be included in multimedia broadcasting transmission / receiving devices, mobile communication terminals, home theater video devices, digital cinema video devices, surveillance cameras, video chat devices, real-time communication devices such as video communication, mobile streaming media devices, storage media, cameras, VoD service providers, over-the-top (OTT) video devices, internet streaming service providers, three-dimensional (3D) video devices, virtual reality (VR) devices, augmented reality (AR) devices, remote conferencing video devices, transportation user equipment (e.g., vehicle (including autonomous vehicles) user equipment, aircraft user equipment, ship user equipment, etc.), and medical video devices, and can be used to process video signals and data signals. For example, over-the-top (OTT) video devices can include game consoles, Blu-ray players, internet access TVs, home theater systems, smartphones, tablet PCs, digital video recorders (DVRs), etc.

[0240] Furthermore, the processing methods applying the embodiments of this disclosure can be generated in the form of a program that is executed by a computer and can be stored in a computer-readable recording medium. Multimedia data having data structures according to embodiments of this disclosure can also be stored in a computer-readable recording medium. A computer-readable recording medium includes all types of storage devices in which computer-system readable data is stored. Computer-readable recording media can include, for example, BD, Universal Serial Bus (USB), ROM, PROM, EPROM, EEPROM, RAM, CD-ROM, magnetic tape, floppy disk, and optical data storage devices. Furthermore, computer-readable recording media include media implemented in the form of a carrier wave (e.g., transmission over the Internet). Additionally, bitstreams generated by encoding methods can be stored in a computer-readable recording medium or transmitted via wired / wireless communication networks.

[0241] Furthermore, embodiments of this disclosure can be implemented using a computer program product based on program code, and the program code can be executed in a computer using embodiments of this disclosure. The program code can be stored on a computer-readable medium.

[0242] Figure 10 Examples of content streaming systems to which the implementation methods described in this document can be applied.

[0243] Reference Figure 10 The content streaming system that applies the implementation methods described in this document can typically include an encoding server, a streaming server, a network server, a media storage device, a user device, and a multimedia input device.

[0244] An encoding server is used to compress content input from multimedia input devices such as smartphones, cameras, and camcorders into digital data to generate a bitstream, which is then sent to a streaming server. As another example, in cases where multimedia input devices such as smartphones, cameras, and camcorders directly generate bitstreams, the encoding server can be omitted.

[0245] Bitstreams can be generated using the encoding methods or bitstream generation methods described in this document. Furthermore, the streaming server can temporarily store the bitstream during transmission or reception.

[0246] The streaming server sends multimedia data to the user's device via a web server based on the user's request. The web server acts as a tool to notify the user of available services. When a user requests a desired service, the web server forwards the request to the streaming server, which then delivers the multimedia data to the user. In this context, the content streaming system may include a separate control server, which in this case controls the commands / responses between the various devices within the content streaming system.

[0247] A streaming server can receive content from media storage and / or encoding servers. For example, when receiving content from an encoding server, the content can be received in real time. In this case, the streaming server can store the bitstream for a predetermined period of time to smoothly provide streaming services.

[0248] For example, user equipment may include mobile phones, smartphones, laptops, digital broadcasting terminals, personal digital assistants (PDAs), portable multimedia players (PMPs), navigation devices, tablet PCs, tablet PCs, ultrabooks, wearable devices (e.g., watch-type terminals (smartwatches), glasses-type terminals (smart glasses), head-mounted displays (HMDs)), digital televisions, desktop computers, digital signage, etc.

[0249] Each server in a content streaming system can operate as a distributed server, and in this case, the data received by each server can be processed in a distributed manner.

[0250] The claims in this document can be combined in various ways. For example, technical features in the method claims can be combined to be implemented or performed in a device, and technical features in the device claims can be combined to be implemented or performed in a method. Furthermore, technical features in both the method claims and the device claims can be combined to be implemented or performed in a device.

Claims

1. An image decoding apparatus comprising: a memory; and at least one processor connected to the memory, wherein the at least one processor is configured to: obtain, from a bitstream, image information including decoded picture buffer (DPB) related information; update a DPB based on the DPB related information; and decode a current picture based on the DPB, wherein the DPB related information includes a syntax element related to a maximum required size of the DPB, wherein, for a first picture in a current access unit (AU) that is not an AU 0 and is not an AU that is an access unit of a coded video sequence (CVSS) start (CVSS), a trim process is repeatedly invoked until only a number of pictures in the DPB is less than a value of the syntax element related to the maximum required size of the DPB plus 1, based on only a case where a first condition that the number of pictures in the DPB is greater than or equal to the value of the syntax element related to the maximum required size of the DPB plus 1 is satisfied, wherein, during the trim process invoked based on the case where the first condition is satisfied, a DPB fullness is decreased by 1 for a picture store buffer that is emptied in the DPB, and wherein the AU 0 is a first AU of the bitstream. the DPB related information includes a syntax element related to a maximum picture reorder number of the DPB or a syntax element related to a maximum latency of the DPB, and 2. The image decoding apparatus according to claim 1, wherein wherein the invocation of the trim process is not determined based on a second condition related to the syntax element related to the maximum picture reorder number of the DPB or a third condition related to the syntax element related to the maximum latency of the DPB. the trim process is not invoked based on a case where the second condition or the third condition is satisfied but the first condition is not satisfied.

3. The image decoding apparatus according to claim 2, wherein the second condition is a condition related to whether a number of pictures in the DPB that are marked as "needed for output" is greater than a value of the syntax element related to the maximum picture reorder number of the DPB, 4. The image decoding apparatus according to claim 2, wherein wherein the third condition is a condition related to whether a value of the syntax element related to the maximum latency of the DPB is not equal to 0 and whether there is at least one picture in the DPB that is marked as "needed for output" and an associated variable PicLatencyCount is greater than or equal to MaxLatencyPictures, and wherein the MaxLatencyPictures is derived by a value of the syntax element related to the maximum picture reorder number of the DPB + a value of the syntax element related to the maximum latency of the DPB - 1. after the trim process invoked based on the case where the first condition is satisfied is performed, an operation of additionally decreasing a DPB fullness by 1 is not performed for a picture store buffer that is emptied in the DPB.

5. The image decoding apparatus according to claim 1, wherein 6. An image encoding apparatus comprising: a memory; and at least one processor connected to the memory, wherein the at least one processor is configured to: ​ at least one processor connected to the memory, wherein the at least one processor is configured to: generate decoded picture buffer, DPB, related information; update a DPB based on the DPB related information; and encode picture information including the DPB related information, wherein the DPB related information includes a syntax element related to a maximum required size of the DPB, wherein for a first picture in a current access unit, AU, that is not an AU 0 that is not an access unit, AU, of a coded video sequence start, CVSS, access unit, the trimming process is repeatedly invoked based only on a first condition that a number of pictures in the DPB is greater than or equal to a value of the syntax element related to the maximum required size of the DPB plus 1 until only the number of pictures in the DPB is less than the value of the syntax element related to the maximum required size of the DPB plus 1, wherein during the trimming process invoked based on the condition that the first condition is met, a DPB fullness is decreased by 1 for a picture store buffer that is emptied in the DPB, and wherein the AU 0 is a first AU of a bitstream.

7. The image coding apparatus according to claim 6, wherein the DPB related information includes a syntax element related to a maximum picture reordering number of the DPB or a syntax element related to a maximum latency of the DPB, and wherein the invocation of the trimming process is not determined based on a second condition related to the syntax element related to the maximum picture reordering number of the DPB or a third condition related to the syntax element related to the maximum latency of the DPB.

8. The image coding apparatus according to claim 7, wherein the trimming process is not invoked based on a condition that the second condition or the third condition is met but the first condition is not met.

9. The image coding apparatus according to claim 7, wherein the second condition is a condition related to whether a number of pictures in the DPB that are marked as "needed for output" is greater than a value of the syntax element related to the maximum picture reordering number of the DPB, wherein the third condition is a condition related to whether a value of the syntax element related to the maximum latency of the DPB is not equal to 0 and whether there is at least one picture in the DPB that is marked as "needed for output" and an associated variable PicLatencyCount is greater than or equal to MaxLatencyPictures, and wherein the MaxLatencyPictures is derived by a value of the syntax element related to the maximum picture reordering number of the DPB + a value of the syntax element related to the maximum latency of the DPB - 1.

10. The image coding apparatus according to claim 6, wherein after the trimming process invoked based on the condition that the first condition is met is performed, an operation of decreasing a DPB fullness by 1 additionally is not performed for a picture store buffer that is emptied in the DPB.

11. An apparatus for transmitting data for picture information, the apparatus comprising: at least one processor configured to generate a bitstream for the picture information, wherein the bitstream is generated based on operations of generating decoded picture buffer, DPB, related information, updating a DPB based on the DPB related information, and encoding picture information including the DPB related information, and a transmitter configured to transmit the data including the bitstream, wherein the DPB related information includes a syntax element related to a maximum required size of the DPB, wherein for a first picture in a current access unit, AU, that is not an AU 0 that is not an access unit, AU, of a coded video sequence start, CVSS, the trimming process is repeatedly invoked based only on a first condition that a number of pictures in the DPB is greater than or equal to a value of a syntax element related to the maximum required size of the DPB plus 1 until only the number of pictures in the DPB is less than the value of the syntax element related to the maximum required size of the DPB plus 1, wherein during the trimming process invoked based on the condition that the first condition is met, a DPB fullness is decreased by 1 for a picture store buffer that is emptied in the DPB, and wherein the AU 0 is a first AU of the bitstream.

Images