Image encoding / decoding method and apparatus for signaling HRD parameters, and computer-readable recording medium storing bitstream

Abstract

Provided are an image processing and image encoding method and apparatus for signaling HRD parameters, and a method for transmitting a bitstream. An image decoding method according to one embodiment of the present disclosure comprises the steps of: determining whether a reference sample resampling (RPR) operation is allowed; and performing a hypothetical reference decoder (HRD) operation based on the allowed RPR operation, where the HRD operation includes at least one of an operation of determining suitability of a video coding standard of a bitstream including the encoded video data, an operation of determining suitability of a video coding standard of a video decoder, and an HRD picture output timing signaling operation, and the HRD operation may be performed by updating a scheduling index for a hypothetical flow scheduler (HSS).

Description

Technical Field

[0001] The present disclosure relates to an image encoding / decoding method and apparatus, and more particularly, to an image encoding / decoding method and apparatus for notifying a hypothetical reference decoder (HRD) related parameter by signaling, and a computer-readable recording medium for storing a bit stream generated by the image encoding method / apparatus of the present disclosure. Background Art

[0002] Recently, the demand for high-resolution and high-quality images, such as high-definition (HD) images and ultra-high-definition (UHD) images, is increasing in various fields. As the resolution and quality of image data are improved, the amount of information or bit volume transmitted is relatively increased compared to existing image data. The increase in the amount of information or bit volume transmitted leads to an increase in transmission cost and storage cost.

[0003] Therefore, efficient image compression technology is needed to effectively transmit, store, and reproduce information about high-resolution and high-quality images. Summary of the invention

[0004] Technical issues

[0005] An object of the present disclosure is to provide an image encoding / decoding method and apparatus with improved encoding / decoding efficiency.

[0006] Another object of the present disclosure is to provide an image encoding / decoding method and apparatus for improving encoding / decoding efficiency by efficiently signaling HRD parameters.

[0007] Another object of the present disclosure is to provide a method for transmitting a bit stream generated by the image encoding method or apparatus according to the present disclosure.

[0008] Another object of the present disclosure is to provide a recording medium storing a bit stream generated by the image encoding method or apparatus according to the present disclosure.

[0009] Another object of the present disclosure is to provide a recording medium storing a bit stream received and decoded by the image decoding apparatus according to the present disclosure and used to reconstruct an image.

[0010] The technical problems solved by the present disclosure are not limited to the above-mentioned technical problems, and other technical problems not described here will be clear to those skilled in the art through the following description.

[0011] Technical Solution

[0012] According to an aspect of the present disclosure, an image decoding method performed by an image decoding device may include the following steps: determining whether a reference sample resampling (RPR) operation is allowed; and based on allowing the RPR operation, performing a hypothetical reference decoder (HRD) operation. The HRD operation may include at least one of an operation of determining the consistency of a bitstream including encoded video data with a video coding standard, an operation of determining the consistency of a video decoder with a video coding standard, or an HRD picture output timing signaling operation. The HRD operation may be performed by updating a scheduling index selected for a hypothetical stream scheduler (HSS).

[0013] According to one aspect of the present disclosure, an image encoding method performed by an image encoding device may include the following steps: determining whether a reference sample resampling (RPR) operation is allowed; and based on allowing the RPR operation, encoding reference sample resampling permission information and encoding a hypothetical reference decoder (HRD) parameter for an HRD operation. The HRD operation may include at least one of an operation of determining the consistency of a bitstream including encoded video data with a video coding standard, an operation of determining the consistency of a video decoder with a video coding standard, or an HRD picture output timing signaling operation. The HRD operation may be performed by updating a scheduling index for a hypothetical stream scheduler (HSS).

[0014] A transmission method according to another aspect of the present disclosure may transmit a bit stream generated by the image encoding apparatus or the image encoding method of the present disclosure.

[0015] A computer-readable recording medium according to another aspect of the present disclosure may store a bit stream generated by the image encoding apparatus or the image encoding method of the present disclosure.

[0016] The features described above in brief summary of the present disclosure are merely exemplary aspects of the following detailed description of the present disclosure and do not limit the scope of the present disclosure.

[0017] Beneficial Effects

[0018] According to the present disclosure, it is possible to provide an image encoding / decoding method and apparatus with improved encoding / decoding efficiency.

[0019] Furthermore, according to the present disclosure, it is possible to provide an image encoding / decoding method and apparatus that improves encoding / decoding efficiency by efficiently signaling an HRD parameter.

[0020] Furthermore, according to the present disclosure, it is possible to provide a method of transmitting a bit stream generated by the image encoding method or apparatus according to the present disclosure.

[0021] Furthermore, according to the present disclosure, it is possible to provide a recording medium storing a bit stream generated by the image encoding method or apparatus according to the present disclosure.

[0022] Furthermore, according to the present disclosure, it is possible to provide a recording medium that stores a bit stream received and decoded by the image decoding device according to the present disclosure and used to reconstruct an image.

[0023] Those skilled in the art will understand that the effects that can be achieved through the present disclosure are not limited to what has been specifically described above, and other advantages of the present disclosure will be more clearly understood from the detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 is a diagram schematically showing a video encoding system to which an embodiment of the present disclosure is applicable.

[0025] Figure 2 is a view schematically showing an image encoding device to which an embodiment of the present disclosure is applicable.

[0026] Figure 3 is a view schematically showing an image decoding device to which an embodiment of the present disclosure is applicable.

[0027] Figure 4 is a view showing an example of a schematic picture decoding process to which an embodiment of the present disclosure is applicable.

[0028] Figure 5 is a view showing an example of a schematic picture encoding process to which an embodiment of the present disclosure is applicable.

[0029] Figure 6 is a view showing an example of a layer structure of a coded image / video.

[0030] Figure 7 is a view illustrating a syntax structure of a VPS according to an embodiment of the present disclosure.

[0031] Figure 8 is a view illustrating a syntax structure of an SPS for signaling HRD parameters according to an embodiment of the present disclosure.

[0032] Fig. 9 is a view illustrating a general_hrd_parameters() syntax structure according to an embodiment of the present disclosure.

[0033] Fig.10 is a view illustrating a syntax structure of ols_hrd_parameters() according to an embodiment of the present disclosure.

[0034] Fig.11 is a view illustrating a sublayer_hrd_parameters() syntax structure according to an embodiment of the present disclosure.

[0035] Fig.12 is a view illustrating an example of an image encoding method to which an embodiment of the present disclosure is applicable.

[0036] Fig.13 is a view illustrating an example of an image decoding method to which an embodiment of the present disclosure is applicable.

[0037] Fig.14 is a view illustrating another example of an image decoding method to which an embodiment of the present disclosure is applicable.

[0038] Fig.15 is a view illustrating another example of an image decoding method to which an embodiment of the present disclosure is applicable.

[0039] Fig.16 is a view illustrating an example of an image decoding method to which an embodiment of the present disclosure is applicable.

[0040] Fig.17 is a diagram showing a content streaming system to which an embodiment of the present disclosure is applicable. DETAILED DESCRIPTION

[0041] Hereinafter, the embodiments of the present disclosure will be described in detail with reference to the accompanying drawings to facilitate implementation by those skilled in the art. However, the present disclosure can be implemented in various forms and is not limited to the embodiments described herein.

[0042] When describing the present disclosure, if it is determined that the detailed description of related known functions or configurations makes the scope of the present disclosure unnecessarily ambiguous, the detailed description thereof will be omitted. In the drawings, parts irrelevant to the description of the present disclosure are omitted, and like reference numerals are given to like parts.

[0043] In the present disclosure, when a component is "connected", "coupled" or "linked" to another component, it may include not only a direct connection relationship but also an indirect connection relationship with intermediate components. In addition, when a component "includes" or "has" other components, unless otherwise specified, it means that other components may also be included, rather than excluding other components.

[0044] In the present disclosure, the terms first, second, etc. are used only for the purpose of distinguishing one component from other components, and do not limit the order or importance of the components unless otherwise specified. Accordingly, within the scope of the present disclosure, a first component in one embodiment may be referred to as a second component in another embodiment, and similarly, a second component in one embodiment may be referred to as a first component in another embodiment.

[0045] In the present disclosure, components that are distinguished from each other are intended to clearly describe each feature and do not mean that the components must be separated. That is, multiple components can be integrated and implemented in one hardware or software unit, or one component can be distributed and implemented in multiple hardware or software units. Therefore, even if not specifically stated, implementations in which these components are integrated or distributed are also included in the scope of the present disclosure.

[0046] In the present disclosure, the components described in each embodiment are not necessarily indispensable components, and some components may be optional components. Therefore, the embodiments consisting of a subset of the components described in the embodiments are also included in the scope of the present disclosure. In addition, the embodiments including other components in addition to the components described in the various embodiments are included in the scope of the present disclosure.

[0047] The present disclosure relates to encoding and decoding of images. Unless otherwise defined in the present disclosure, terms used in the present disclosure may have general meanings commonly used in the technical field to which the present disclosure belongs.

[0048] In the present disclosure, a "picture" generally refers to a unit representing an image within a specific time period, and a slice / tile is a coding unit constituting a part of a picture, and a picture may be composed of one or more slices / tiles. In addition, a slice / tile may include one or more coding tree units (CTUs).

[0049] In the present disclosure, "pixel" or "pel" may refer to the smallest unit constituting a picture (or image). In addition, "sample" may be used as a term corresponding to a pixel. A sample may generally represent a pixel or a pixel value, or may represent only a pixel / pixel value of a luminance component or only a pixel / pixel value of a chrominance component.

[0050] In the present disclosure, a "unit" may refer to a basic unit of image processing. The unit may include at least one of a specific area of ​​the picture and information related to the area. In certain cases, the unit may be used interchangeably with terms such as "sample array", "block" or "area". In general, an M×N block may include M columns and N rows of samples (or sample arrays) or a set (or array) of transform coefficients.

[0051] In the present disclosure, "current block" may mean one of "current coding block", "current coding unit", "coding target block", "decoding target block" or "processing target block". When prediction is performed, "current block" may mean "current prediction block" or "prediction target block". When transform (inverse transform) / quantization (dequantization) is performed, "current block" may mean "current transform block" or "transform target block". When filtering is performed, "current block" may mean "filtering target block".

[0052] In addition, in the present disclosure, unless explicitly stated as a chroma block, a “current block” may mean a block including both a luma component block and a chroma component block or a “luma block of the current block”. The “luma block of the current block” may be represented by an explicit description including a luma component block such as “luma block” or “current luma block”. The “chroma block of the current block” may be represented by an explicit description including a chroma component block such as “chroma block” or “current chroma block”.

[0053] In this specification, "A or B" may mean "only A", "only B", or "A and B". In other words, in the present disclosure, "A or B" may be interpreted as "A and / or B". For example, in the present disclosure, "A, B or C" may mean "only A", "only B", "only C", or "any combination of A, B, and C".

[0054] A slash ( / ) or a comma used in the present disclosure may mean "and / or". For example, "A / B" may mean "A and / or B". Thus, "A / B" may mean "only A", "only B", or "A and B". For example, "A,B,C" may mean "A, B, or C".

[0055] In the present disclosure, "at least one of A and B" may mean "only A", "only B", or "both A and B". In addition, in the present disclosure, "at least one of A or B" or "at least one of A and / or B" may be interpreted as being the same as "at least one of A and B".

[0056] In addition, in the present disclosure, “at least one of A, B, and C” may mean “only A”, “only B”, “only C”, or “any combination of A, B, and C”. In addition, in the present disclosure, “at least one of A, B, or C” or “at least one of A, B and / or C” may be interpreted as the same as “at least one of A, B, and C”.

[0057] In addition, brackets used in the present disclosure may mean "for example". Specifically, when describing "prediction (intra-frame prediction)", "intra-frame prediction" may be proposed as an example of "prediction". In other words, "prediction" of the present disclosure is not limited to "intra-frame prediction", and "intra-frame prediction" may be proposed as an example of "prediction". In addition, even when describing "prediction (i.e., intra-frame prediction)", "intra-frame prediction" may be proposed as an example of "prediction".

[0058] In the present disclosure, technical features described separately in one figure may be implemented separately or simultaneously.

[0059] Overview of Video Coding Systems

[0060] Figure 1 is a diagram illustrating a video encoding system according to the present disclosure.

[0061] The video encoding system according to the embodiment may include an encoding device 10 and a decoding device 20. The encoding device 10 may deliver the encoded video and / or image information or data to the decoding device 20 via a digital storage medium or a network in the form of a file or a stream.

[0062] The encoding device 10 according to the embodiment may include a video source generator 11, an encoding unit 12, and a transmitter 13. The decoding device 20 according to the embodiment may include a receiver 21, a decoding unit 22, and a renderer 23. The encoding unit 12 may be called a video / image encoding unit, and the decoding unit 22 may be called a video / image decoding unit. The transmitter 13 may be included in the encoding unit 12. The receiver 21 may be included in the decoding unit 22. The renderer 23 may include a display and the display may be configured as a separate device or an external component.

[0063] The video source generator 11 can obtain the video / image by the process of capturing, synthesizing or generating the video / image. The video source generator 11 may include a video / image capturing device and / or a video / image generating device. The video / image capturing device may include, for example, one or more cameras, a video / image archive including previously captured videos / images, etc. The video / image generating device may include, for example, a computer, a tablet computer, and a smart phone, and may generate the video / image (electronically). For example, a virtual video / image may be generated by a computer, etc. In this case, the video / image capturing process may be replaced by a process of generating relevant data.

[0064] The encoding unit 12 may encode the input video / image. For compression and encoding efficiency, the encoding unit 12 may perform a series of processes such as prediction, transformation, and quantization. The encoding unit 12 may output encoded data (encoded video / image information) in the form of a bitstream.

[0065] The transmitter 13 transmits the encoded video / image information or data output in the form of a bit stream to the receiver 21 of the decoding device 20 in the form of a file or stream through a digital storage medium or a network. The digital storage medium may include various storage media, such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. The transmitter 13 may include an element for generating a media file in a predetermined file format and may include an element for transmission through a broadcast / communication network. The receiver 21 may extract / receive a bit stream from a storage medium or a network and transmit the bit stream to the decoding unit 22.

[0066] The decoding unit 22 may decode a video / image by performing a series of processes corresponding to the operations of the encoding unit 12, such as dequantization, inverse transformation, and prediction.

[0067] The renderer 23 may render the decoded video / image. The rendered video / image may be displayed through a display.

[0068] Overview of Image Coding Equipment

[0069] Figure 2 is a view schematically showing an image encoding device to which an embodiment of the present disclosure is applicable.

[0070] like Figure 2 As shown, the image encoding device 100 may include an image segmenter 110, a subtractor 115, a transformer 120, a quantizer 130, a dequantizer 140, an inverse transformer 150, an adder 155, a filter 160, a memory 170, an inter-frame predictor 180, an intra-frame predictor 185, and an entropy encoder 190. The inter-frame predictor 180 and the intra-frame predictor 185 may be collectively referred to as a "predictor". The transformer 120, the quantizer 130, the dequantizer 140, and the inverse transformer 150 may be included in a residual processor. The residual processor may also include a subtractor 115.

[0071] In some embodiments, all or at least some of the components configuring the image encoding apparatus 100 may be configured by one hardware component (eg, an encoder or a processor). In addition, the memory 170 may include a decoded picture buffer (DPB) and may be configured by a digital storage medium.

[0072] The image segmenter 110 may segment the input image (or picture or frame) input to the image encoding device 100 into one or more processing units. For example, the processing unit may be referred to as a coding unit (CU). The coding unit may be obtained by recursively segmenting a coding tree unit (CTU) or a maximum coding unit (LCU) according to a quadtree binary tree ternary tree (QT / BT / TT) structure. For example, a coding unit may be segmented into a plurality of coding units of a deeper depth based on a quadtree structure, a binary tree structure, and / or a ternary tree structure. For the segmentation of the coding unit, the quadtree structure may be applied first, and then the binary tree structure and / or the ternary tree structure may be applied. The encoding process according to the present disclosure may be performed based on the final coding unit that is no longer segmented. The maximum coding unit may be used as the final coding unit, and the coding unit of a deeper depth obtained by segmenting the maximum coding unit may also be used as the final coding unit. Here, the encoding process may include the prediction, transformation, and reconstruction processes described later. As another example, the processing unit of the encoding process may be a prediction unit (PU) or a transformation unit (TU). The prediction unit and the transform unit may be divided or partitioned from the final coding unit. The prediction unit may be a sample prediction unit, and the transform unit may be a unit for deriving a transform coefficient and / or a unit for deriving a residual signal from the transform coefficient.

[0073] The predictor (inter predictor 180 or intra predictor 185) may perform prediction on the block to be processed (current block) and generate a prediction block including prediction samples of the current block. The predictor may determine whether to apply intra prediction or inter prediction based on the current block or CU. The predictor may generate various information related to the prediction of the current block and transmit the generated information to the entropy encoder 190. Information about the prediction may be encoded in the entropy encoder 190 and output in the form of a bitstream.

[0074] The intra-frame predictor 185 can predict the current block by referring to the samples in the current picture. Depending on the intra-frame prediction mode and / or the intra-frame prediction technology, the reference samples can be located in the neighbors of the current block or can be placed separately. The intra-frame prediction mode may include multiple non-directional modes and multiple directional modes. The non-directional mode may include, for example, a DC mode and a plane mode. Depending on the level of detail of the prediction direction, the directional mode may include, for example, 33 directional prediction modes or 65 directional prediction modes. However, this is only an example, and more or fewer directional prediction modes may be used according to the settings. The intra-frame predictor 185 may determine the prediction mode applied to the current block by using the prediction mode applied to the neighboring blocks.

[0075] The inter-frame predictor 180 may derive a prediction block of the current block based on a reference block (reference sample array) specified by a motion vector on a reference picture. In this case, in order to reduce the amount of motion information transmitted in the inter-frame prediction mode, the motion information may be predicted in units of blocks, sub-blocks, or samples based on the correlation of motion information between neighboring blocks and the current block. The motion information may include a motion vector and a reference picture index. The motion information may also include inter-frame prediction direction (L0 prediction, L1 prediction, dual prediction, etc.) information. In the case of inter-frame prediction, the neighboring blocks may include spatial neighboring blocks present in the current picture and temporal neighboring blocks present in the reference picture. The reference picture including the reference block and the reference picture including the temporal neighboring block may be the same or different. The temporal neighboring block may be referred to as a collocated reference block, a collocated CU (colCU), etc. The reference picture including the temporal neighboring block may be referred to as a collocated picture (colPic). For example, the inter-frame predictor 180 may configure a motion information candidate list based on the neighboring blocks and generate information indicating which candidate is used to derive the motion vector and / or reference picture index of the current block. Inter-frame prediction may be performed based on various prediction modes. For example, in the case of skip mode and merge mode, the inter-frame predictor 180 may use the motion information of the neighboring blocks as the motion information of the current block. In the case of skip mode, unlike the merge mode, the residual signal may not be transmitted. In the case of motion vector prediction (MVP) mode, the motion vector of the neighboring block may be used as a motion vector predictor, and the motion vector of the current block may be signaled by encoding the motion vector difference and an indicator of the motion vector predictor. The motion vector difference may mean the difference between the motion vector of the current block and the motion vector predictor.

[0076] The predictor can generate a prediction signal based on various prediction methods and prediction techniques described below. For example, the predictor can apply not only intra prediction or inter prediction, but also intra prediction and inter prediction at the same time to predict the current block. The prediction method that applies both intra prediction and inter prediction to predict the current block can be called combined inter and intra prediction (CIIP). In addition, the predictor can perform intra block copy (IBC) to predict the current block. Intra block copy can be used for content image / video encoding of games, etc., for example, screen content coding (SCC). IBC is a method of predicting the current picture using a previously reconstructed reference block in the current picture at a position separated by a predetermined distance from the current block. When IBC is applied, the position of the reference block in the current picture can be encoded as a vector (block vector) corresponding to a predetermined distance. IBC basically performs prediction in the current picture, but can be performed similarly to inter prediction because the reference block is derived within the current picture. That is, IBC can use at least one inter prediction technique described in the present disclosure.

[0077] The prediction signal generated by the predictor can be used to generate a reconstruction signal or to generate a residual signal. The subtractor 115 can generate a residual signal (residual block or residual sample array) by subtracting the prediction signal (prediction block or prediction sample array) output from the predictor from the input image signal (original block or original sample array). The generated residual signal can be transmitted to the transformer 120.

[0078] The transformer 120 may generate transform coefficients by applying a transform technique to the residual signal. For example, the transform technique may include at least one of a discrete cosine transform (DCT), a discrete sine transform (DST), a karhunen-loève transform (KLT), a graph-based transform (GBT), or a conditional nonlinear transform (CNT). Here, GBT refers to a transform obtained from a graph when relationship information between pixels is represented by a graph. CNT refers to a transform obtained based on a prediction signal generated using all previously reconstructed pixels. In addition, the transform process may be applied to square pixel blocks of the same size or may be applied to blocks of variable size instead of square.

[0079] The quantizer 130 may quantize the transform coefficients and transmit them to the entropy encoder 190. The entropy encoder 190 may encode the quantized signal (information about the quantized transform coefficients) and output a bitstream. The information about the quantized transform coefficients may be referred to as residual information. The quantizer 130 may rearrange the quantized transform coefficients of the block type into a one-dimensional vector form based on the coefficient scanning order, and generate information about the quantized transform coefficients based on the quantized transform coefficients in the one-dimensional vector form.

[0080] The entropy encoder 190 may perform various encoding methods, such as exponential Golomb, context adaptive variable length coding (CAVLC), context adaptive binary arithmetic coding (CABAC), etc. The entropy encoder 190 may encode information required for video / image reconstruction other than quantized transform coefficients together or separately (e.g., values ​​of syntax elements, etc.). The encoded information (e.g., encoded video / image information) may be transmitted or stored in units of a network abstraction layer (NAL) in the form of a bitstream. The video / image information may also include information about various parameter sets, such as an adaptive parameter set (APS), a picture parameter set (PPS), a sequence parameter set (SPS), or a video parameter set (VPS). In addition, the video / image information may also include general constraint information. The signaled information, transmitted information, and / or syntax elements described in the present disclosure may be encoded and included in a bitstream through the above-described encoding process.

[0081] The bitstream may be transmitted over a network or may be stored in a digital storage medium. The network may include a broadcast network and / or a communication network, and the digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. A transmitter (not shown) for transmitting a signal output from the entropy encoder 190 and / or a storage unit (not shown) for storing the signal may be included as an internal / external element of the image encoding device 100. Alternatively, a transmitter may be provided as a component of the entropy encoder 190.

[0082] The quantized transform coefficients output from the quantizer 130 may be used to generate a residual signal. For example, the residual signal (residual block or residual sample) may be reconstructed by applying dequantization and inverse transformation to the quantized transform coefficients through the dequantizer 140 and the inverse transformer 150.

[0083] The adder 155 adds the reconstructed residual signal to the prediction signal output from the inter-frame predictor 180 or the intra-frame predictor 185 to generate a reconstructed signal (reconstructed picture, reconstructed block, reconstructed sample array). If the block to be processed has no residual, such as when the skip mode is applied, the prediction block can be used as a reconstructed block. The adder 155 can be called a reconstructor or a reconstructed block generator. The generated reconstructed signal can be used for intra-frame prediction of the next block to be processed in the current picture, and can be used for inter-frame prediction of the next picture by filtering as described below.

[0084] The filter 160 can improve the subjective / objective image quality by applying filtering to the reconstructed signal. For example, the filter 160 can generate a modified reconstructed picture by applying various filtering methods to the reconstructed picture, and store the modified reconstructed picture in the memory 170, specifically, in the DPB of the memory 170. Various filtering methods may include, for example, deblocking filtering, sample adaptive offset, adaptive loop filtering, bilateral filtering, etc. The filter 160 can generate various information related to filtering and transmit the generated information to the entropy encoder 190, as described later in the description of each filtering method. The information related to filtering can be encoded by the entropy encoder 190 and output in the form of a bit stream.

[0085] The modified reconstructed picture transferred to the memory 170 may be used as a reference picture in the inter predictor 180. When inter prediction is applied by the image encoding apparatus 100, prediction mismatch between the image encoding apparatus 100 and the image decoding apparatus may be avoided and encoding efficiency may be improved.

[0086] The DPB of the memory 170 may store the modified reconstructed picture for use as a reference picture in the inter-frame predictor 180. The memory 170 may store the motion information of the block from which the motion information in the current picture is derived (or encoded) and / or the motion information of the reconstructed block in the picture. The stored motion information may be transmitted to the inter-frame predictor 180 and used as the motion information of the spatial neighboring block or the motion information of the temporal neighboring block. The memory 170 may store the reconstructed samples of the reconstructed blocks in the current picture and may transmit the reconstructed samples to the intra-frame predictor 185.

[0087] Overview of Image Decoding Equipment

[0088] Figure 3 is a view schematically showing an image decoding device to which an embodiment of the present disclosure is applicable.

[0089] like Figure 3 As shown, the image decoding device 200 may include an entropy decoder 210, a dequantizer 220, an inverse transformer 230, an adder 235, a filter 240, a memory 250, an inter-frame predictor 260, and an intra-frame predictor 265. The inter-frame predictor 260 and the intra-frame predictor 265 may be collectively referred to as a "predictor". The dequantizer 220 and the inverse transformer 230 may be included in a residual processor.

[0090] According to an embodiment, all or at least some of the plurality of components configuring the image decoding apparatus 200 may be configured by hardware components (eg, a decoder or a processor). In addition, the memory 250 may include a decoded picture buffer (DPB) or may be configured by a digital storage medium.

[0091] The image decoding device 200 having received a bit stream including video / image information may perform the same Figure 2 The image encoding device 100 may reconstruct the image by processing corresponding to the processing performed by the image encoding device 100. For example, the image decoding device 200 may perform decoding using a processing unit applied in the image encoding device. Therefore, the processing unit of decoding may be, for example, a coding unit. The coding unit may be obtained by partitioning the coding tree unit or the maximum coding unit. The reconstructed image signal decoded and output by the image decoding device 200 may be reproduced by a reproduction device (not shown).

[0092] The image decoding device 200 may receive the image in the form of a bit stream from Figure 2The received signal may be decoded by the entropy decoder 210. For example, the entropy decoder 210 may parse the bitstream to derive information (e.g., video / image information) required for image reconstruction (or picture reconstruction). The video / image information may also include information about various parameter sets, such as an adaptive parameter set (APS), a picture parameter set (PPS), a sequence parameter set (SPS), or a video parameter set (VPS). In addition, the video / image information may also include general constraint information. The image decoding device may also decode the picture based on the information about the parameter set and / or the general constraint information. The information and / or syntax elements signaled / received described in the present disclosure may be decoded and obtained from the bitstream through a decoding process. For example, the entropy decoder 210 may decode the information in the bitstream based on a coding method such as exponential Golomb coding, CAVLC, or CABAC, and output the values ​​of the syntax elements required for image reconstruction and the quantized values ​​of the transform coefficients of the residual. More specifically, the CABAC entropy decoding method can receive a bin corresponding to each syntax element in the bitstream, use the decoding target syntax element information, the decoding information of the neighboring block and the decoding target block, or the information of the symbol / bin decoded in the previous stage to determine the context model, perform arithmetic decoding on the bin by predicting the probability of occurrence of the bin according to the determined context model, and generate a symbol corresponding to the value of each syntax element. In this case, the CABAC entropy decoding method can update the context model by using the information of the decoded symbol / bin for the context model of the next symbol / bin after determining the context model. The information related to the prediction in the information decoded by the entropy decoder 210 can be provided to the predictor (inter-frame predictor 260 and intra-frame predictor 265), and the residual value of the entropy decoding performed on it in the entropy decoder 210, that is, the quantized transform coefficient and the related parameter information can be input to the dequantizer 220. In addition, the information about filtering in the information decoded by the entropy decoder 210 can be provided to the filter 240. In addition, a receiver (not shown) for receiving a signal output from the image encoding apparatus may be further configured as an internal / external element of the image decoding apparatus 200 , or the receiver may be a component of the entropy decoder 210 .

[0093] In addition, the image decoding device according to the present disclosure may be referred to as a video / image / picture decoding device. The image decoding device may be divided into an information decoder (video / image / picture information decoder) and a sample decoder (video / image / picture sample decoder). The information decoder may include an entropy decoder 210. The sample decoder may include at least one of a dequantizer 220, an inverse transformer 230, an adder 235, a filter 240, a memory 250, an inter-frame predictor 260, or an intra-frame predictor 265.

[0094] The dequantizer 220 may dequantize the quantized transform coefficients and output the transform coefficients. The dequantizer 220 may rearrange the quantized transform coefficients in the form of two-dimensional blocks. In this case, the rearrangement may be performed based on the coefficient scanning order performed in the image encoding device. The dequantizer 220 may dequantize the quantized transform coefficients by using a quantization parameter (e.g., quantization step size information) and obtain the transform coefficients.

[0095] The inverse transformer 230 may inversely transform the transform coefficients to obtain a residual signal (residual block, residual sample array).

[0096] The predictor may perform prediction on the current block and generate a prediction block including prediction samples of the current block. The predictor may determine whether to apply intra prediction or inter prediction to the current block based on the information about prediction output from the entropy decoder 210, and may determine a specific intra / inter prediction mode (prediction technique).

[0097] The same as described in the predictor of the image encoding device 100 , the predictor can generate a prediction signal based on various prediction methods (techniques) described later.

[0098] The intra predictor 265 may predict the current block by referring to samples in the current picture. The description of the intra predictor 185 is also applicable to the intra predictor 265.

[0099] The inter-frame predictor 260 can derive the prediction block of the current block based on the reference block (reference sample array) specified by the motion vector on the reference picture. In this case, in order to reduce the amount of motion information transmitted in the inter-frame prediction mode, the motion information can be predicted in units of blocks, sub-blocks or samples based on the correlation of the motion information between the neighboring blocks and the current block. The motion information may include a motion vector and a reference picture index. The motion information may also include inter-frame prediction direction (L0 prediction, L1 prediction, dual prediction, etc.) information. In the case of inter-frame prediction, the neighboring blocks may include spatial neighboring blocks existing in the current picture and temporal neighboring blocks existing in the reference picture. For example, the inter-frame predictor 260 may configure a motion information candidate list based on the neighboring blocks, and derive the motion vector and / or reference picture index of the current block based on the received candidate selection information. Inter-frame prediction may be performed based on various prediction modes, and the information about the prediction may include information indicating the inter-frame prediction mode of the current block.

[0100] The adder 235 can generate a reconstruction signal (reconstructed picture, reconstructed block, reconstructed sample array) by adding the obtained residual signal to the prediction signal (prediction block, prediction sample array) output from the predictor (including the inter-frame predictor 260 and / or the intra-frame predictor 265). If the block to be processed has no residual (for example, when the skip mode is applied), the prediction block can be used as the reconstructed block. The description of the adder 155 is also applicable to the adder 235. The adder 235 can be called a reconstructor or a reconstructed block generator. The generated reconstruction signal can be used for intra-frame prediction of the next block to be processed in the current picture, and can be used for inter-frame prediction of the next picture by filtering as described below.

[0101] The filter 240 can improve the subjective / objective image quality by applying filtering to the reconstructed signal. For example, the filter 240 can generate a modified reconstructed picture by applying various filtering methods to the reconstructed picture, and store the modified reconstructed picture in the memory 250, specifically, in the DPB of the memory 250. The various filtering methods may include, for example, deblocking filtering, sample adaptive offset, adaptive loop filtering, bilateral filtering, etc.

[0102] The (modified) reconstructed picture stored in the DPB of the memory 250 can be used as a reference picture in the inter-frame predictor 260. The memory 250 can store the motion information of the block from which the motion information in the current picture is derived (or decoded) and / or the motion information of the reconstructed block in the picture. The stored motion information can be transmitted to the inter-frame predictor 260 to be used as the motion information of the spatial neighboring block or the motion information of the temporal neighboring block. The memory 250 can store the reconstructed samples of the reconstructed block in the current picture and transmit the reconstructed samples to the intra-frame predictor 265.

[0103] In the present disclosure, the embodiments described in the filter 160 , the inter-frame predictor 180 , and the intra-frame predictor 185 of the image encoding device 100 may be equally or correspondingly applied to the filter 240 , the inter-frame predictor 260 , and the intra-frame predictor 265 of the image decoding device 200 .

[0104] General image / video encoding process

[0105] In image / video coding, the pictures configuring the image / video can be encoded / decoded according to the decoding order. The picture order corresponding to the output order of the decoded pictures can be set differently from the decoding order, and based on this, not only forward prediction but also backward prediction can be performed during inter-frame prediction.

[0106] Figure 4 An example of a schematic picture decoding process to which the embodiments of the present disclosure are applicable is shown.

[0107] Figure 4The various processes shown can be Figure 3 For example, step S410 may be performed by the entropy decoder 210, step S420 may be performed by a predictor including predictors 265 and 260, step S430 may be performed by residual processors 220 and 230, step S440 may be performed by the adder 235, and step S450 may be performed by the filter 240. Step S410 may include the information decoding process described in the present disclosure, step S420 may include the inter / intra prediction process described in the present disclosure, step S430 may include the residual processing process described in the present disclosure, step S440 may include the block / picture reconstruction process described in the present disclosure, and step S450 may include the in-loop filtering process described in the present disclosure.

[0108] Reference Figure 4 , the picture decoding process may schematically include a process (S410) for obtaining image / video information (by decoding) from a bitstream, a picture reconstruction process (S420 to S440), and an in-loop filtering process (S450) for the reconstructed picture. The picture reconstruction process may be performed based on the prediction samples and residual samples obtained by the inter / intra prediction (S420) and residual processing (S430) described in the present disclosure (dequantization and inverse transformation of quantized transform coefficients). A modified reconstructed picture may be generated by an in-loop filtering process for the reconstructed picture generated by the picture reconstruction process. In this case, the modified reconstructed picture may be output as a decoded picture, stored in a decoded picture buffer (DPB) of the memory 250, and used as a reference picture in an inter-frame prediction process when the picture is decoded later. The in-loop filtering process (S450) may be omitted. In this case, the reconstructed picture may be output as a decoded picture, stored in a DPB of the memory 250, and used as a reference picture in an inter-frame prediction process when the picture is decoded later. As described above, the in-loop filtering process (S450) may include a deblocking filtering process, a sample adaptive offset (SAO) process, an adaptive loop filter (ALF) process, and / or a bilateral filter process, some or all of which may be omitted. In addition, one or some of the deblocking filtering process, the sample adaptive offset (SAO) process, the adaptive loop filter (ALF) process, and / or the bilateral filter process may be applied sequentially, or all of them may be applied sequentially. For example, after applying the deblocking filtering process to the reconstructed picture, the SAO process may be performed. Alternatively, after applying the deblocking filtering process to the reconstructed picture, the ALF process may be performed. This may even be performed similarly in an encoding device.

[0109] Figure 5 An example of an exemplary picture encoding process to which the embodiments of the present disclosure are applicable is shown.

[0110] Figure 5 The various processes shown can be Figure 2 For example, step S510 may be performed by the predictors 185 and 180, step S520 may be performed by the residual processors 115, 120 and 130, and step S530 may be performed in the entropy encoder 190. Step S510 may include the inter / intra prediction process described in the present disclosure, step S520 may include the residual processing process described in the present disclosure, and step S530 may include the information encoding process described in the present disclosure.

[0111] Reference Figure 5 The picture encoding process may schematically include not only a process of encoding information for picture reconstruction (e.g., prediction information, residual information, segmentation information, etc.) and outputting it in the form of a bit stream, but also a process of generating a reconstructed picture for the current picture and a process of applying in-loop filtering to the reconstructed picture (optional), as shown in FIG. Figure 2 Described. The encoding device can derive (modified) residual samples from the quantized transform coefficients through the dequantizer 140 and the inverse transformer 150, and generate a reconstructed picture based on the predicted samples and the (modified) residual samples as the output of step S510. The reconstructed picture generated in this way can be equal to the reconstructed picture generated in the decoding device. The modified reconstructed picture can be generated by the in-loop filtering process of the reconstructed picture. In this case, the modified reconstructed picture can be stored in the decoded picture buffer or the memory 170, and similar to the decoding device, it can be used as a reference picture in the inter-frame prediction process when the picture is encoded later. As described above, in some cases, some or all of the in-loop filtering processes can be omitted. When performing the in-loop filtering process, the (in-loop) filtering related information (parameters) can be encoded in the entropy encoder 190 and output in the form of a bitstream, and the decoding device can perform the in-loop filtering process based on the filtering related information using the same method as the encoding device.

[0112] By this in-loop filtering process, noise (e.g., block artifacts and ringing artifacts) occurring during image / video encoding can be reduced, and subjective / objective visual quality can be improved. In addition, by performing the in-loop filtering process in both the encoding device and the decoding device, the encoding device and the decoding device can derive the same prediction result, picture encoding reliability can be increased, and the amount of data to be transmitted for picture encoding can be reduced.

[0113] As described above, the picture reconstruction process can be performed not only in the image decoding device, but also in the image encoding device. Reconstructed blocks can be generated in units of blocks based on intra-frame prediction / inter-frame prediction, and a reconstructed picture including the reconstructed blocks can be generated. When the current picture / slice / patchwork group is an I picture / slice / patchwork group, the blocks included in the current picture / slice / patchwork group can be reconstructed based only on intra-frame prediction. On the other hand, when the current picture / slice / patchwork group is a P or B picture / slice / patchwork group, the blocks included in the current picture / slice / patchwork group can be reconstructed based on intra-frame prediction or inter-frame prediction. In this case, inter-frame prediction can be applied to some blocks in the current picture / slice / patchwork group, and intra-frame prediction can be applied to the remaining blocks. The color components of the picture may include luminance components and chrominance components, and unless explicitly limited in the present disclosure, the methods and embodiments of the present disclosure are applicable to both luminance components and chrominance components.

[0114] Example of coding layer structure

[0115] For example, the encoded video / image according to the present disclosure may be processed according to the following encoding layers and structures.

[0116] Figure 6 is a diagram showing the layer structure of a coded image.

[0117] The coded image is classified into a video coding layer (VCL) for image decoding processing and processing itself, a lower system for transmitting and storing coding information, and a network abstraction layer (NAL) that exists between the VCL and the lower system and is responsible for network adaptation functions.

[0118] In the VCL, VCL data including compressed image data (slice data) can be generated, or a supplemental enhancement information (SEI) message additionally required for the decoding process of the image or a parameter set including information such as a picture parameter set (PPS), a sequence parameter set (SPS), or a video parameter set (VPS) can be generated.

[0119] In NAL, header information (NAL unit header) can be added to the raw byte sequence payload (RBSP) generated in the VCL to generate a NAL unit. In this case, RBSP refers to slice data, parameter sets, SEI messages generated in the VCL. The NAL unit header can include NAL unit type information specified according to the RBSP data included in the corresponding NAL unit.

[0120] like Figure 6As shown, NAL units can be classified into VCL NAL units and non-VCL NAL units according to the RBSP generated in the VCL. A VCL NAL unit may refer to a NAL unit including information about an image (slice data), and a non-VCL NAL unit may refer to a NAL unit including information required for decoding an image (parameter set or SEI message).

[0121] The VCL NAL unit and the non-VCL NAL unit may be attached with header information according to the data standard of the lower layer system and sent over the network. For example, the NAL unit may be modified into a data format of a predetermined standard (e.g., H.266 / VVC file format, RTP (Real-time Transport Protocol) or TS (Transport Stream)) and sent over various networks.

[0122] As described above, in the NAL unit, the NAL unit type can be specified according to the RBSP data structure included in the corresponding NAL unit, and information about the NAL unit type can be stored in the NAL unit header and signaled. For example, this can be roughly classified into VCL NAL unit types and non-VCL NAL unit types according to whether the NAL unit includes information about the image (slice data). The VCL NAL unit type can be classified according to the nature and type of the picture included in the VCL NAL unit, and the non-VCL NAL unit type can be classified according to the type of parameter set.

[0123] Examples of NAL unit types specified according to the types of parameter sets / information included in the non-VCL NAL unit type will be listed below.

[0124] -DCI (Decoding Capability Information) NAL Unit Type (NUT): NAL unit type including DCI

[0125] -VPS (Video Parameter Set) NUT: includes the NAL unit type of VPS

[0126] -SPS (Sequence Parameter Set) NUT: NAL unit type including SPS

[0127] -PPS (Picture Parameter Set) NUT: NAL unit type including PPS

[0128] -APS (Adaptation Parameter Set) NUT: NAL unit type including APS

[0129] -PH (picture header) NUT: includes the NAL unit type of PH

[0130] The above-mentioned NAL unit type may have syntax information of the NAL unit type, and the syntax information may be stored in the NAL unit header and signaled. For example, the syntax information may be nal_unit_type, and the NAL unit type may be specified using the nal_unit_type value.

[0131] In addition, as described above, a picture may include multiple slices, and a slice may include a slice header and slice data. In this case, a picture header may be further added to multiple slices (slice headers and slice data sets) in a picture. The picture header (picture header syntax) may include information / parameters that are commonly applicable to the picture. The slice header (slice header syntax) may include information / parameters that are commonly applicable to the slice. APS (APS syntax) or PPS (PPS syntax) may include information / parameters that are commonly applicable to one or more slices or pictures. SPS (SPS syntax) may include information / parameters that are commonly applicable to one or more sequences. VPS (VPS syntax) may be information / parameters that are commonly applicable to multiple layers. DCI (DCI syntax) may include information / parameters related to decoding capabilities.

[0132] In the present disclosure, the high-level syntax (HLS) may include at least one of APS syntax, PPS syntax, SPS syntax, VPS syntax, DCI syntax, picture header syntax, or slice header syntax. In addition, in the present disclosure, the low-level syntax (LLS) may include, for example, slice data syntax, CTU syntax, coding unit syntax, transform unit syntax, etc.

[0133] In addition, in the present disclosure, the image / video information encoded in the encoding device and notified to the decoding device in the form of a bitstream may include not only information related to intra-picture segmentation, intra-frame / inter-frame prediction information, residual information, and in-loop filtering information, but also information about a slice header, information about a picture header, information about an APS, information about a PPS, information about an SPS, information about a VPS, and / or information about a DCI. In addition, the image / video information may also include general constraint information and / or information about a NAL unit header.

[0134] High-level syntax signaling and semantics

[0135] As described above, the image / video information according to the present disclosure may include a high level syntax (HLS). The image encoding method and / or the image decoding method may be performed based on the image / video information.

[0136] Video Parameter Set Signaling

[0137] A video parameter set (VPS) is a parameter set for carrying layer information. For example, the layer information may include information about an output layer set (OLS), information about a profile layer level, information about the relationship between an OLS and a hypothetical reference decoder, and information about the relationship between an OLS and a decoded picture buffer (DPB). For decoding of a bitstream, a VPS may not be necessary.

[0138] The VPS Raw Byte Sequence Payload (RBSP) shall be available for decoding processing and then referenced, included in at least one Access Unit (AU) with TemporalId equal to 0 or provided by external means.

[0139] All VPSNAL units in a coded video sequence (CVS) with a specific value of vps_video_parameter_set_id shall have the same content.

[0140] Figure 7 is a view illustrating a syntax structure of a VPS according to an embodiment of the present disclosure.

[0141] Figure 7 The syntax structure of the VPS shown only includes syntax elements related to the present disclosure. Figure 7 Various other syntax elements not shown in FIG. 1 may be included in the VPS.

[0142] exist Figure 7 In the example shown, vps_video_parameter_set_id provides an identifier of the VPS. Other syntax elements may use vps_video_parameter_set_id to reference the VPS. The value of vps_video_parameter_set_id shall be greater than 0.

[0143] vps_max_layers_minus1 plus 1 can specify the maximum allowed number of layers in each CVS of the reference VPS.

[0144] vps_max_sublayers_minus1 plus 1 may specify the maximum number of temporal sublayers that may exist in a layer in each CVS of a reference VPS. vps_max_sublayers_minus1 may be in the range of 0 to 6, inclusive.

[0145] When vps_max_layers_minus1 is greater than 0 and vps_max_sublayers_minus1 is greater than 0, vps_all_layers_same_num_sublayers_flag may be signaled. vps_all_layers_same_num_sublayers_flag being equal to a first value (e.g., 1) may specify that the number of temporal sublayers is the same for all layers in the respective CVSs of the reference VPS. vps_all_layers_same_num_sublayers_flag being equal to a second value (e.g., 0) may specify that the layers in the respective CVSs of the reference VPS may not have the same number of temporal sublayers. When vps_all_layers_same_num_sublayers_flag is not present, its value may be inferred to be equal to the first value (e.g., 1).

[0146] When vps_max_layers_minus1 is greater than 0, vps_all_independent_layers_flag may be signaled. vps_all_independent_layers_flag equal to a first value (e.g., 1) may specify that all layers in the CVS are independently coded without using inter-layer prediction. vps_all_independent_layers_flag equal to a second value (e.g., 0) may specify that one or more layers in the CVS may use inter-layer prediction. When vps_all_independent_layers_flag is not present, its value may be inferred to be equal to the first value (e.g., 1).

[0147] When vps_max_layers_minus1 is greater than 0, each_layer_is_an_ols_flag may be signaled. Additionally, when vps_all_independent_layers_flag is equal to a first value, each_layer_is_an_ols_flag may be signaled. each_layer_is_an_ols_flag being equal to a first value (e.g., 1) may specify that each OLS contains only one layer. Additionally, each_layer_is_an_ols_flag being equal to a first value (e.g., 1) may specify that each layer in the CVS of the reference VPS is itself an OLS (i.e., one of the layers contained in the OLS is the only output layer). Additionally, each_layer_is_an_ols_flag being equal to a second value (e.g., 0) may specify that at least one OLS may contain more than one layer. If vps_max_layers_minus1 is equal to 0, the value of each_layer_is_an_ols_flag may be inferred to be equal to 1. Otherwise, when vps_all_independent_layers_flag is equal to 0, the value of each_layer_is_an_ols_flag may be inferred to be equal to 0.

[0148] When each_layer_is_an_ols_flag is equal to the second value (eg, 0) and vps_all_independent_layers_flag is equal to the second value (eg, 0), ols_mode_idc may be signaled.

[0149] ols_mode_idc equal to the first value (e.g., 0) may specify that the total number of OLSs specified by the VPS is equal to vps_max_layers_minus1 + 1. In this case, the i-th OLS may contain layers with layer indices from 0 to i (inclusive). In addition, for each OLS, only the layer with the highest layer index (the highest layer) in the OLS may be output.

[0150] ols_mode_idc equal to the second value (eg, 1) may specify that the total number of OLSs specified by the VPS is equal to vps_max_layers_minus1 + 1. In this case, the i-th OLS may contain layers with layer indices from 0 to i (inclusive). Additionally, for each OLS, all layers in the OLS may be output.

[0151] ols_mode_idc equal to a third value (e.g., 2) may specify that the total number of OLSs specified by the VPS is explicitly signaled. In addition, for each OLS, the output layer is explicitly signaled. Other layers that are not output layers are layers that are direct or indirect reference layers to the output layer of the OLS.

[0152] When vps_all_independent_layers_flag is equal to 1 and each_layer_is_an_ols_flag is equal to 0, the value of ols_mode_idc may be inferred to be equal to a third value (eg, 2).

[0153] When ols_mode_idc is 2, num_output_layer_sets_minus1 and ols_output_layer_flag[i][j] may be explicitly signaled.

[0154] num_output_layer_sets_minus1 plus 1 can specify the total number of OLSs specified by the VPS.

[0155] When ols_mode_idc is 2, ols_output_layer_flag[i][j] may specify whether the jth layer of the i-th OLS is an output layer. ols_output_layer_flag[i][j] equal to a first value (e.g., 1) may specify that the layer whose layer identifier nuh_layer_id is equal to vps_layer_id[j] is the output layer of the i-th OLS. ols_output_layer_flag[i][j] equal to a second value (e.g., 0) may specify that the layer whose layer identifier nuh_layer_i is equal to vps_layer_id[j] is not the output layer of the i-th OLS.

[0156] Hereinafter, HRD parameters signaled in the VPS will be described.

[0157] When each_layer_is_an_ols_flag is equal to a second value (e.g., 0), vps_general_hrd_params_present_flag may be signaled. vps_general_hrd_params_present_flag equal to a first value (e.g., 1) may specify that a general_hrd_parameters() syntax structure and other HRD parameters are present in the VPS. vps_general_hrd_params_present_flag equal to a second value (e.g., 0) may specify that a general_hrd_parameters() syntax structure and other HRD parameters are not present in the VPS. When vps_general_hrd_params_present_flag is not present, its value may be inferred to be equal to the second value (e.g., 0).

[0158] When the i th OLS contains one layer (NumLayersInOls[i] is equal to 1), the general_hrd_parameters() syntax structure applicable to the i th OLS may be present in the sequence parameter set (SPS) referenced by the layer in the i th OLS.

[0159] When vps_max_sublayers_minus1 is greater than 0, vps_sublayer_cpb_params_present_flag may be signaled. vps_sublayer_cpb_params_present_flag equal to a first value (e.g., 1) may specify that the i ols_hrd_parameters() syntax structure in the VPS contains HRD parameters for sublayers whose temporal layer identifier TemporalId is in the range of 0 to hrd_max_tid[i], inclusive. vps_sublayer_cpb_params_present_flag equal to a second value (e.g., 0) may specify that the i ols_hrd_parameters() syntax structure in the VPS contains only HRD parameters for sublayers whose temporal layer identifier TemporalId is equal to hrd_max_tid[i]. When vps_max_sublayers_minus1 is equal to 0, vps_sublayer_cpb_params_present_flag may be inferred to be equal to a second value (e.g., 0).

[0160] When vps_sublayer_cpb_params_present_flag is equal to the second value (eg, 0), the HRD parameters of the sublayers with TemporalId in the range of 0 to hrd_max_tid[i]-1 (inclusive) are inferred to be the same as the sublayers with temporal layer identifier TemporalId equal to hrd_max_tid[i].

[0161] num_ols_hrd_params_minus1 plus 1 may specify the number of ols_hrd_parameters() syntax structures in the VPS. num_ols_hrd_params_minus1 may be in the range of 0 to TotalNumOlss-1. TotalNumOlss may specify the total number of OLSs specified by the VPS. In the present disclosure, HRD parameters may refer to ols_hrd_parameters(). Thus, the number of HRD parameter syntax structures may refer to the number of ols_hrd_parameters() syntax structures.

[0162] When vps_max_sublayers_minus1 is greater than 0 and vps_all_layers_same_num_sublayers_flag is equal to a second value (eg, 0), hrd_max_tid[i] may be signaled. hrd_max_tid[i] may specify the temporal layer identifier TemporalId of the highest sublayer containing the associated HRD parameters in the ith ols_hrd_parameters() syntax structure.

[0163] hrd_max_tid[i] may be in the range of 0 to vps_max_sublayers_minus1, inclusive. When vps_max_sublayers_minus1 is equal to 0, the value of hrd_max_tid[i] may be inferred to be equal to 0. When vps_max_sublayers_minus1 is greater than 0 and vps_all_layers_same_num_sublayers_flag is equal to 1, the value of hrd_max_tid[i] may be inferred to be equal to vps_max_sublayers_minus1.

[0164] like Figure 7As shown, the variable firstSubLayer specifying the temporal layer identifier TemporalId of the first sublayer can be derived as 0 or hrd_max_tid[i] based on vps_sublayer_cpb_params_present_flag. Specifically, when vps_sublayer_cpb_params_present_flag is equal to 1, firstSubLayer can be derived as 0, otherwise, firstSubLayer can be derived as hrd_max_tid[i]. Based on the derived firstSubLayer and hrd_max_tid[i], the ols_hrd_parameters() syntax structure can be signaled.

[0165] When num_ols_hrd_params_minus1 plus 1 and TotalNumOlss are not equal and num_ols_hrd_params_minus1 is greater than 0, ols_hrd_idx[i] may be signaled. In this case, when the number of layers (NumLayersInOls[i]) contained in the i-th OLS is greater than 1, ols_hrd_idx[i] may be signaled for the i-th OLS. ols_hrd_idx[i] specifies the index of the ols_hrd_parameters() syntax structure in the list of ols_hrd_parameters() syntax structures in the VPS that applies to the i-th OLS. The value of ols_hrd_idx[[i]] may be in the range of 0 to num_ols_hrd_params_minus1, inclusive. When the number of layers included in the i-th OLS (NumLayersInOls[i]) is equal to 1, the ols_hrd_parameters() syntax structure applied to the i-th OLS may exist in the SPS referenced by the layers in the i-th OLS.

[0166] In the present disclosure, ols_hrd_idx[i] is an index of ols_hrd_parameters() applied to the ith OLS or the ith multi-layer OLS, and may be referred to as mapping information between a (multi-layer) OLS and an HRD parameter syntax structure (ols_hrd_parameters()).

[0167] When num_ols_hrd_param_minus1 plus 1 is equal to TotalNumOlss, the value of ols_hrd_idx[i] may be inferred to be equal to i. Otherwise, when NumLayersInOls[i] is greater than 1 and num_ols_hrd_params_minus1 is equal to 0, the value of ols_hrd_idx[i] may be inferred to be equal to 0.

[0168] HRD signaling in VPS and SPS

[0169] In the following, the signaling of HRD parameters according to the present disclosure will be described in more detail. HRD parameters may be signaled for each output layer set (OLS). A hypothetical reference decoder (HRD) is a hypothetical decoder model that specifies constraints on the variability of a consistent NAL unit stream or a consistent byte stream that may be generated by an encoding process.

[0170] HRD parameters can be as shown in Figure 7 The may be included in the VPS and signaled, or may be included in the SPS and signaled.

[0171] Figure 8 is a view illustrating a syntax structure of an SPS for signaling HRD parameters according to an embodiment of the present disclosure.

[0172] exist Figure 8In the example shown, sps_ptl_dpb_hrd_params_present_flag is equal to a first value (e.g., 1) to specify that the profile_tier_level() syntax structure and the dpb_parameters() syntax structure are present in the SPS. profile_tier_level() can be a syntax structure for sending parameters of the profile layer level, and dpb_parameters() can be a syntax structure for sending decoded picture buffer (DPB) parameters. In addition, sps_ptl_dpb_hrd_params_present_flag is equal to a first value (e.g., 1) to specify that the general_hrd_parameters() syntax structure and the ols_hrd_parameters() syntax structure can be present in the SPS. sps_ptl_dpb_hrd_params_present_flag is equal to a second value (e.g., 0) to specify that the above four syntax structures are not present in the SPS. The value of sps_ptl_dpb_hrd_params_present_flag may be equal to the value of vps_independent_layer_flag[GeneralLayerIdx[nuh_layer_id]]. That is, the value of sps_ptl_dpb_hrd_params_present_flag may be encoded as the value of vps_independent_layer_flag[GeneralLayerIdx[nuh_layer_id]].

[0173] In the above, vps_independent_layer_flag[i] may be a syntax element included in the VPS and transmitted. vps_independent_layer_flag[i] being equal to a first value (e.g., 1) may specify that the layer of index i is an independent layer that does not use inter-layer prediction. vps_independent_layer_flag[i] being equal to a second value (e.g., 0) may specify that the layer of index i may use inter-layer prediction. When vps_independent_layer_flag[i] does not exist, its value is inferred to be equal to the first value (e.g., 1).

[0174] When sps_ptl_dpb_hrd_params_present_flag is equal to 1, sps_general_hrd_params_present_flag may be signaled.

[0175] sps_general_hrd_params_present_flag equal to a first value (eg, 1) may specify that the SPS includes a general_hrd_parameters() syntax structure and an ols_hrd_parameters() syntax structure. sps_general_hrd_params_present_flag equal to a second value (eg, 0) may specify that the SPS does not include a general_hrd_parameters() syntax structure or an ols_hrd_parameters() syntax structure.

[0176] like Figure 8 As shown, when sps_max_sublayers_minus1 is greater than 0, sps_sublayer_cpb_params_present_flag may be signaled. In this case, sps_max_sublayers_minus1 plus 1 may specify the maximum number of temporal sublayers that may be present in each coded layered video sequence (CLVS) that references the SPS. sps_sublayer_cpb_params_present_flag equal to a first value (e.g., 1) may specify that the ols_hrd_parameters() syntax structure in the SPS includes HRD parameters for sublayers whose temporal layer identifier TemporalId is in the range of 0 to sps_max_sublayers_minus1, inclusive. sps_sublayer_cpb_params_present_flag equal to a second value (e.g., 0) may specify that the ols_hrd_parameters() syntax structure in the SPS includes only HRD parameters for sublayers whose temporal layer identifier TemporalId is equal to sps_max_sublayers_minus1. When sps_max_sublayers_minus1 is equal to 0, the value of sps_sublayer_cpb_params_present_flag is inferred to be equal to the second value (eg, 0).

[0177] When sps_sublayer_cpb_params_present_flag is equal to the second value (eg, 0), HRD parameters of sublayers with temporal layer identifier TemporalId in the range of 0 to sps_max_sublayers_minus1-1 (inclusive) are inferred to be the same as sublayers with temporal layer identifier TemporalId equal to sps_max_sublayers_minus1.

[0178] Fig. 9is a view illustrating a general_hrd_parameters() syntax structure according to an embodiment of the present disclosure.

[0179] like Fig. 9 As shown, the general_hrd_parameters() syntax structure may include some sequence level HRD parameters used in HRD operation. A requirement for bitstream conformance is that the contents of general_hrd_parameters() in any VPS or SPS present in the bitstream shall be the same.

[0180] When the general_hrd_parameters() syntax structure is included in a VPS, the general_hrd_parameters() syntax structure can be applied to all OLSs specified by the VPS. When the general_hrd_parameters() syntax structure is included in an SPS, the general_hrd_parameters() syntax structure can be applied to the OLS of the lowest layer among the layers that only contain the reference SPS. In this case, the lowest layer is an independent layer.

[0181] like Fig. 9 As shown, the general_hrd_parameters() syntax structure is an HRD parameter and may include syntax elements such as num_units_in_tick, time_scale, and general_nal_hrd_params_present_flag. Fig. 9 The HRD parameters shown may have the same meaning as conventional HRD parameters. Therefore, detailed description of HRD parameters that are not very relevant to the present disclosure will be omitted.

[0182] Fig.10 is a view illustrating a syntax structure of ols_hrd_parameters() according to an embodiment of the present disclosure.

[0183] When the ols_hrd_parameters() syntax structure is included in the VPS, the OLS to which the ols_hrd_parameters() syntax structure is applied can be specified by the VPS. When the ols_hrd_parameters() syntax structure is included in the SPS, the ols_hrd_parameters() syntax structure can be applied to the OLS of the lowest layer among the layers containing the reference SPS. In this case, the lowest layer is an independent layer.

[0184] like Fig.10As shown, the ols_hrd_parameters() syntax structure is an HRD parameter and may include syntax elements such as fixed_pic_rate_general_flag, fixed_pic_rate_within_cvs_flag, and elemental_duration_in_tc_minus1. Fig.10 The HRD parameters shown may have the same meaning as conventional HRD parameters. Therefore, detailed description of HRD parameters that are not very relevant to the present disclosure will be omitted.

[0185] Fig.11 is a view illustrating a sublayer_hrd_parameters() syntax structure according to an embodiment of the present disclosure.

[0186] The sublayer_hrd_parameters() syntax structure may be included in Fig.10 The ols_hrd_parameters() syntax structure is used and signaled.

[0187] like Fig.11 As shown, the sublayer_hrd_parameters() syntax structure is an HRD parameter and may include syntax elements such as bit_rate_value_minus1, cpb_size_value_minus1, and cpb_size_du_value_minus1. Fig.11 The HRD parameters shown may have the same meaning as conventional HRD parameters. Therefore, detailed description of HRD parameters that are not very relevant to the present disclosure will be omitted.

[0188] For reference, the output time may be the time at which the reconstructed picture is to be output from the DPB. The output time may be designated by the HRD according to the output timing DPB operation.

[0189] Two HRD parameter sets may be used, such as NAL HRD parameters and VCL HRD parameters. The HRD parameters may be signaled via a general_hrd_parameters() syntax structure and an ols_hrd_parameters() syntax structure. The general_hrd_parameters() syntax structure and the ols_hrd_parameters() syntax structure may be included in a VPS and signaled, or may be included in an SPS and signaled.

[0190] For example, DPB management may be performed based on HRD parameters.As an example, removing a picture from the DPB before decoding of a current picture and / or (decoded) picture output may be performed based on HRD parameters.

[0191] In conventional video coding techniques, for hypothetical reference decoder (HRD) operation and conformance testing, a hypothetical stream scheduler (HSS), as an entity that operates the HRD and conformance testing, selects a specific scheduling index (ScIdx) at the start of the operation / test to determine the HRD structure to be used for the operation. For some operations, the scheduling index is used from the beginning to the end of the bitstream. In the prior art, the HSS may change the scheduling index in the following cases: when the contents of the selected and previous general_timing_hrd_parameters() syntax structures of the AU containing access unit (AU) m and the previous AU are different, the HSS selects the value of ScIdx1 of ScIdx from the value of ScIdx provided in the selected general_timing_hrd_parameters() syntax structure of the AU containing AU m, which results in BitRate[Htid][ScIdx1] or CpbSize[Htid][ScIdx1] of the AU containing AU m. The value of BitRate[Htid][ScIdx1] or CpbSize[Htid][ScIdx1] may be different from the value of BitRate[Htid][ScIdx0] or CpbSize[Htid][ScIdx0] used for the ScIdx0 value of the previous AU's ScIdx. Otherwise, the HSS will continue to operate with the previous values ​​of ScIdx, BitRate[Htid][ScIdx], and CpbSize[Htid][ScIdx]. This situation may be suitable for splicing situations. When two bitstreams are spliced ​​and merged together, the HRD and general timing parameters may need to be updated because the bitstream may begin to experience significant changes at the splicing point. When splicing occurs, the conditions related to the change in picture resolution may change. Similarly, it may be desirable to allow the HSS to change the selected scheduling index even when the picture resolution changes due to the use of the RPR feature rather than due to splicing. However, in the prior art, such functionality is not defined / allowed (eg, the ability to change the selected scheduling index when the resolution changes), which may cause problems.

[0192] In addition, HRD operation and timing signaling have multiple challenges. First, timing signaling and operation may be problematic when reference picture resampling (RPR) is applicable. When sending / transmitting coded media / video from an encoder to a decoder, current HRD-related operations and timing signaling are based on the assumption that certain conditions (e.g., network bandwidth) do not change within a coded video sequence (CVS) or a coded layered video sequence (CLVS). In other words, timing signaling exists at the beginning of the CVS, i.e., at the access unit (AU) carrying the IRAP or GDR picture, and consistency testing is initialized only at the beginning of the CVS. This assumption (i.e., that transmission conditions do not change within the CVS) may be reasonable in most cases, but if the RPR feature is allowed, the size of the coded picture in the middle of the CVS may change due to changes in certain conditions (e.g., changes in network bandwidth). For example, a coded picture with a resolution of 1920×1080 at the beginning of the CVS may be reduced to a resolution of 1280×720 in the middle of the CVS due to a reduction in network bandwidth. In this case, when the picture size changes, it may be necessary to allow the timing operation calculations to be reinitialized in the middle of the CVS (e.g., reinitializing the consistency test of the CPB timing including the arrival and removal timings and the DPB timing including the removal timings). Alternatively, it may be necessary to provide updated parameter / timing information rather than reinitialization.

[0193] Furthermore, according to the prior art, problems may arise when calculating the number of consistency tests.

[0194] In order to solve the above-mentioned prior art problems, the following implementation methods are proposed:

[0195] Aspect 1. If DU-based consistency is allowed, DU-based tests can be applied to all test points, such as two types of tests (i.e., type 1 and type 2) starting at each CRA, each IDR, each GDR, and each DRAP. The number of bitstream consistency tests performed can be as follows: n0*n1*(2*n2+n3+n4)*n5. In an example, n0, n1, n2, n3, n4, and n5 may be the same as described in the prior art (including VVC standard documents). In an example, n0 may be 2, n1 may be hrd_cpb_cnt_minus1+1, and n2 may be the number of AUs in BitstreamToDecode associated with the BP SEI message applicable to TargetOlsIdx, for which all of the following conditions are true, respectively.

[0196] -nal_unit_type is CRA_NUT.

[0197] - bp_alt_cpb_params_present_flag in the associated BP SEI message is 1.

[0198] - There is at least one RASL picture with pps_mixed_nalu_types_in_pic_flag equal to 0 associated with the AU.

[0199] n3 may be the number of IRAP or GDR AUs in BitstreamToDecode associated with the BP SEI message applicable to TargetOlsIdx respectively, for which at least one of the following conditions is false:

[0200] -nal_unit_type is CRA_NUT.

[0201] - bp_alt_cpb_params_present_flag in the associated BP SEI message is 1.

[0202] - There is at least one RASL picture with pps_mixed_nalu_types_in_pic_flag equal to 0 associated with the AU.

[0203] n4 may be the number of AUs of BitstreamToDecode, each of which is associated with a DRAP indication SEI message applicable to TargetOlsIdx and whose associated PT SEI message has pt_cpb_alt_timing_info_present_flag equal to 1.

[0204] n5 can be derived as follows:

[0205] - If General_du_hrd_params_present_flag of the selected General_timing_hrd_parameters() syntax structure is 0, then n5 may be 1.

[0206] Otherwise, n5 can be 2.

[0207] Aspect 2. If RPR is applicable, the updated parameters and timing information in the conformance test / timing operations can be used to update the timing derivation of the decoding operations (e.g., picture arrival in CPB, picture removal from CPB, picture removal from DPB, etc.).

[0208] Aspect 3: In the middle of CVS, parameters and timing information can be updated, timing operations can be performed, and consistency tests can be reinitialized for screens that meet the following conditions:

[0209] -a. The screen with temporalId equal to 0

[0210] -b. A picture that references one or more reference pictures with different resolutions / picture sizes

[0211] Aspect 4: For HRD operations and consistency testing, the selected scheduling index can be updated or changed in the middle of CVS or CLVS.

[0212] Aspect 5: For HRD operation and conformance testing, the selected scheduling index may be updated or changed in AUs where significant bit rate changes occur. (i.e., the coded data of the current AU may be much larger or much smaller than the previous AU.)

[0213] Aspect 6: For pictures with temporalId 0, scheduling index changes can be further restricted to pictures with significant bit rate changes in the middle of CVS / CLVS.

[0214] Aspect 7: It may be further restricted to allow the scheduling index change among CVS / CLVS only when the picture resolution (ie, RPR) within CLVS is allowed to change.

[0215] Aspect 8: There may be additional HRD parameters associated with some pictures, which may be used to update or reinitialize the conformance test / timing operations.

[0216] Aspect 9: Additional HRD parameters may be present in the picture parameter set (PPS) referenced by a picture.

[0217] Aspect 10: Additional HRD parameters may be present in the picture header of a picture.

[0218] Aspect 11: There may be a flag specifying whether additional HRD parameters are present in the PPS or picture header.

[0219] Furthermore, the above aspects are described individually for clarity of illustration only and these aspects may be used in combination.

[0220] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings.

[0221] Each conformance test may consist of a combination of options selected at each step. If there is more than one option for a step, only one option may be selected for a particular conformance test. All possible combinations of all steps may constitute a complete set of conformance tests, and the number of bitstream conformance tests to be performed for each operating point to be tested may be equal to n0*n1*(2*n2+n3+n4)*n5 n0*n1*(2*n2+n3+n4+n5), where the values ​​of n0, n1, n2, n3, n4, and n5 may be specified as follows:

[0222] -n0 can be 2.

[0223] -n1 can be hrd_cpb_cnt_minus1+1.

[0224] -n2 may be the number of AUs in BitstreamToDecode, each AU being associated with a BPSEI message applicable to TargetOlsIdx, and for which all of the following conditions are true:

[0225] -nal_unit_type may be equal to CRA_NUT.

[0226] - The bp_alt_cpb_params_present_flag of the associated BP SEI message may be 1.

[0227] - There may be one or more RASL pictures with pps_mixed_nalu_types_in_pic_flag equal to 0 associated with the AU.

[0228] -n3 may be the number of IRAP or GDR AUs in BitstreamToDecode, each AU being associated with a BP SEI message applicable to TargetOlsIdx, for which one or more of the following conditions are false:

[0229] 1.nal_unit_type can be CRA_NUT.

[0230] 2. The bp_alt_cpb_params_present_flag of the associated BP SEI message may be 1.

[0231] 3. There may be one or more RASL pictures with pps_mixed_nalu_types_in_pic_flag equal to 0 associated with the AU.

[0232] -n4 The number of AUs in BitstreamToDecode, each AU is associated with a DRAP indication SEI message applicable to each TargetOlsIdx and its associated PT SEI message has pt_cpb_alt_timing_info_present_flag equal to 1.

[0233] -n5 can be derived as follows:

[0234] 1. If General_du_hrd_params_present_flag of the selected General_timing_hrd_parameters() syntax structure is 0, then n5 may be 1.

[0235] 2. Otherwise, n5 can be 2.

[0236] In addition, n0 may correspond to a consistency test of type I bitstream consistency and type II bitstream consistency. n1 may correspond to a consistency test of each CPB transmission schedule. n2 may correspond to a consistency test of a bitstream starting from each CRA picture with associated RASL pictures and delays and delay offsets removed in place of the initial CPB. These tests may be performed twice: once for bitstream preservation and once for removing the bitstream of the RASL pictures associated with the CRA. n3 may correspond to a consistency test of a bitstream starting from each IRAP or GDR AU that is not a CRA with associated RASL pictures and delays and delay offsets removed in place of the initial CPB. n4 may correspond to a consistency test of a bitstream starting from each IRAP with associated DRAP pictures with alternative timing information and removing the result of all AUs between the DRAP picture with alternative timing information and the previous IRAP. n5 may correspond to AU-based consistency and DU-based consistency tests when General_du_hrd_params_present_flag is 1.

[0237] In addition, as an example, the following steps may be applied to various tests (eg, consistency test, etc.), but some steps may be deleted, other steps may be added, or the order in which the steps are performed may be changed.

[0238] First, the test target operation point indicated by targetOp can be selected by selecting a list of target sub-picture index values ​​opSubpicIdxList[j], i.e., an OLS index opOlsIdx, a highest TemporalId value opTid, and a target sub-picture index value opSubpicIdxList[j], which is in the range of 0 to NumLayersIdOls[OlsIdx]-1. The opOlsIdx value can be in the range of 0 to TotalNumOlss-1, and the opTid value can be in the range of 0 to vps_max_sublayers_minus1. As an example, if opSubpicIdxList[] exists, targetOp consists of pictures, and for each pair of selected opOlsIdx and opTid values, the sub-bitstream BitstreamToDecode output by calling the sub-bitstream extraction process with enitreBitstream, opOlsIdx, and opTid as input can satisfy the following conditions:

[0239] BitstreamToDecode has at least one VCL NAL unit with TemporalId equal to opTid. Otherwise (if opSubpicIdxList[] exists), targetOp consists of sub-pictures, and for each selected set of values ​​of opOlsIdx, opTid and opSubpicIdxList[j] (j is in the range from 0 to numLayersInOls[opOlsIdx]-1), the sub-bitstream BitStreamToDecode output by the sub-picture sub-bitstream extraction process called with entireBitstream, opOlsIdx, opTid and opSubpicIdxList[j] (j is in the range from 0 to numLayersInOls[opOlsIdx]-1) as input can satisfy the following conditions:

[0240] - There is at least one VCL NAL unit in BitstreamToDecode with TemporalId equal to opTid.

[0241] - For each j in the range NumLayerInOls[opOlsIdx]-1, there is at least one VCL unit with nuh_layer_id equal to LayerIdInOls[opOlsIdx][j] and sh_subpic_id equal to SubpicIdVal[opSubpicIdxList[j].

[0242] Here, regardless of whether opSubpicIdxList[] exists, since the bitstream consistency requirements of each IRAP or GDR AU are met, for each j in the range from 0 to NumLayerInOls[OpOlIdx]-1, there may be at least one VCL NAL unit with nuh_layer_id equal to layerIdInOls[j].

[0243] Additionally, if opSubpicIdxList[] does not exist, the following applies:

[0244] - If the layers of targetOp contain all layers of the entire bitstream and opTid is equal to the highest TemporalId value among all NAL units in the entire bitstream, then BitstreamToDecode can be set equal to entireBitstream.

[0245] - Otherwise, the sub-bitstream extraction process may be called with entireBitstream, opOlsIdx and opTid as input, and may output BitstreamToDecode.

[0246] In other cases (when opSubpicIdxList[] exists), the sub-picture sub-bitstream extraction process can be called with opSubpicIdxList[j], entireBitstream, opOlsIdx and opTid as input (j is in the range from 0 to NumLayerSinOls[opOlsIdx]-1), and BitstreamToDecode can be output.

[0247] In addition, the values ​​of TargetOlsIdx and Htid may be set equal to opOlsIdx and opTid of targetOp, respectively.

[0248] The general_timing_hrd_parameters() syntax structure, ols_timing_hrd_parameters() syntax structure, and sublayer_hrd_parameters() syntax structure applicable to BitstreamToDecode may be selected as follows:

[0249] - If NumLayersInOls[TargetOlsIdx] is 1, the general_timing_hrd_parameters() syntax structure and the ols_timing_hrd_parameters() syntax structure of the SPS may be selected. Otherwise, the general_timing_hrd_parameters() syntax structure and the vps_ols_timing_hrd_idx[MultiLayerOlsIdx[TargetOlsIdx]ols_timing_hrd_parameters() syntax structure may be selected.

[0250] -To test a Type I bitstream conformance point within a selected ols_timing_hrd_parameters() syntax structure, the sublayer_hrd_parameters(Htid) syntax structure immediately following the condition "if(general_vcl_hrd_params_present_flag) (determines whether general_vcl_hrd_params_present_flag is 1)" may be selected and the variable NalHrdModeFlag may be set to 0, and the sublayer_hrd_parameters(Htid) syntax structure immediately following the condition "if(general_nal_hrd_params_present_flag) (determines whether general_nal_hrd_params_present_flag is 1)" may be selected and the variable NalHrdModeFlag may be set to 1. When BitstreamToDecode is a type II bitstream and NalHrdModeFlag is 0, all non-VCL NAL units except PH and padding data NAL units and all leading_zero_8bits, zero_byte, start_code_prefix_one_3bytes, and trailing_zero_8bits syntax elements present in BitstreamToDecode are discarded, and the remaining bitstream can be assigned to BitstreamTodecode.

[0251] The AU associated with the BP SEI message applicable to TargetOp (present in BitstreamToDecode or obtained via external means) may be selected as the HRD initialization point and referred to as AU 0.

[0252] When General_du_hrd_params_present_flag is 1 in the selected General_timing_hrd_parameters() syntax structure, the CPB may be scheduled to operate at the AU level (in which case the DecodingUnitHrdFlag variable is set to 0) or the DU level (in which case the DecodingUnitHrdFlag variable is set equal to 1). Otherwise, DecodingUnitHrdFlag is set to 0 and the CPB may be scheduled to operate at the AU level.

[0253] For each AU in BitstreamToDecode starting from AU 0, the BP SEI message associated with the AU and applicable to TargetOlsIdx (present in BitstreamToDecode or obtained via external means) can be selected, the PT SEI message (present in BitstreamToDecode associated with the AU and applicable to TargetOlsIdx or obtained via external means) can be selected, and if DecodingUnitHrdFlag is 1 and bp_du_cpb_params_in_pic_timing_sei_flag is 0, the item associated with the DU of the AU and applicable to TargetOlsIdx of the DUI SEI message (present in BitstreamToDecode or obtained via external means, not specified in this specification) can be selected.

[0254] The ScIdx value can be selected. The selected ScIdx should be in the range of 0 to hrd_cpb_cnt_minus1.

[0255] When the BP SEI message associated with AU 0 has bp_alt_cpb_params_present_flag equal to 0, the variable DefaultInitCpbParamsFlag is set equal to 1. When the BP SEI message associated with AU 0 has bp_alt_cpb_params_present_flag equal to 1, one of the following may apply for the initial CPB removal delay and delay offset selection:

[0256] - If NalHrdModeFlag is 1, the default initial CPB removal delay and delay offset represented by bp_nal_initial_cpb_removal_delay[Htid][ScIdx] and bp_nal_initial_cpb_removal_offset[Htid][ScIdx], respectively, in the selected BP SEI message may be selected. Otherwise, the default initial CPB removal delay and delay offset represented by bp_vcl_initial_cpb_removal_delay[Htid][ScIdx] and bp_vcl_initial_cpb_removal_offset[Htid][ScIdx], respectively, in the selected BP SEI message may be selected. The DefaultInitCpbParamsFlag variable may be set to 1.

[0257] - When NalHrdModeFlag is 1, the alternate initial CPB removal delay and delay offset_delta[Htid][ScIdx] and pt_nal_cpb_alt_initial_removal_offset[Htid][ScIdx] represented by bp_nal_initial_cpb_removal_delay[Htid][ScIdx] and bp_nal_initial_cpb_removal_offset[Htid][ScIdx] and pt_nal_cpb_alt_initial_removal_offset_delta[Htid][ScIdx], respectively, in the selected BP SEI message may be selected from the PT SEI message associated with the AU following AU 0 in decoding order. Otherwise, the alternative initial CPB removal delay and delay offset represented by bp_vcl_initial_cpb_removal_delay[Htid][ScIdx] and bp_vcl_initial_cpb_removal_offset[Htid][ScIdx] and pt_vcl_cpb_alt_initial_removal_delay_delta[Htid][ScIdx], respectively, in the selected BP SEI message may be selected from the PT SEI message associated with the AU following AU 0 in decoding order. The variable DefaultInitCpbParamsFlag is set to 0, and one of the following may apply:

[0258] - RASL AUs containing RASL pictures with pps_mixed_nalu_types_in_pic_flag equal to 0 and associated with CRA pictures contained in AU 0 are discarded from BitstreamToDecode, the remaining bitstream may be assigned to BitstreamToDecode.

[0259] - Discarding all AUs after AU 0 in decoding order up to the AU associated with the DRAP indication SEI message from BitstreamToDecode, the remaining bitstream may be assigned to BitstreamToDecode.

[0260] If a current block (e.g., AU) with temporalId equal to 0 is associated with a BP SEI message, and a picture of the block references at least one reference picture with a different resolution, and the referenced PPS includes HRD parameters (i.e., the value of pps_hrd_parameters_update_present_flag in the referenced PPS is 1), the CPB and DPB operation timings of the current block and the AU following the current CU in decoding order can utilize the HRD parameters signaled in the referenced PPS and the BP SEI associated with the current AU.

[0261] As another example, if a current block with temporalId equal to 0 is associated with a BP SEI message and the picture of the block references at least one reference picture with a different resolution, the selected ScIdx may be updated. When ScIdx is updated, the CPB and DPB operation timing in the current block and the blocks following the current block in decoding order may need to utilize HRD parameters, BP SEI, PT SEI, DUI SEI according to the new / updated ScIdx.

[0262] Buffering period SEI message semantics

[0263] The BP SEI message may provide initial CPB removal delay and initial CPB removal delay offset information for HRD initialization of associated AU positions according to the decoding order.

[0264] When the BP SEI message is present, when an AU with TemporalId equal to 0 is not a RASL or RADL picture and has at least one picture with ph_non_ref_pic_flag equal to 0, the AU may be referred to as notDiscardableAu.

[0265] If the current AU is not the first AU of the bitstream in decoding order, AU prevNonDiscardableAu may be the previous AU in decoding order having at least one picture that is not a RASL or RADL picture and ph_non_ref_pic_flag equal to 0, but TemporalId equal to 0. Furthermore, the presence or absence of a BP SEI message may be specified as follows.

[0266] - If NalHrdBpPresentFlag is equal to 1 or VclHrdBpPresentFlag is equal to 1, the following applies to each AU of the CVS:

[0267] - If the AU is an IRAP or GDR AU, the BP SEI message applicable to the operation point can be associated with the AU.

[0268] - Otherwise, if the AU is a notDiscardableAu that references at least one reference picture with a different resolution, a BP SEI message applicable to the operation point may be associated with the AU.

[0269] - Otherwise, if the AU is notDiscardableAu, the BP SEI message applicable to the operation point may or may not be associated with the AU.

[0270] - Otherwise, the AU may not be associated with the BP SEI message applicable to the operation point.

[0271] - Otherwise (both NalHrdBpPresentFlag and VclHrdBpPresentFlag are 0), the AU of CVS may not be associated with the BP SEI message.

[0272] However, for some applications, it may be desirable to make the BP SEI messages more frequent (eg, for random access or bitstream splicing in IRAP AUs or non-IRAP AUs).

[0273] Table 1 below is a diagram showing an example of a sequence parameter set.

[0274] [Table 1]

[0275] seq_parameter_set_rbsp() { Descriptors ... sps_extension_flag u(1) if (sps_extension_flag ) { sps_hrd_parameters_update_present_flag u(1) while(more_rbsp_data() ) sps_extension_data_flag u(1) } rbsp_trailing_bits() }

[0276] The syntax sps_extension_flag may indicate the possible presence of sps_extension_data_flag within the SPS RBSP syntax structure. A first value (eg, 0) may indicate that sps_extension_data_flag is not present, and a second value (eg, 1) may indicate that sps_extension_data_flag may be present.

[0277] The syntax sps_hrd_parameters_update_present_flag may indicate whether additional general timing and HRD parameters may be present. A first value (e.g., 0) may indicate that additional general timing and HRD parameters are not present in the PPS, and a second value (e.g., 1) may indicate that additional general timing and HRD parameters are present in the PPS. If the current syntax is not present in the bitstream, the value of this flag may be inferred to be 0.

[0278] There is no restriction on the value that the syntax sps_extension_data_flag can have, it can have any value.

[0279] Table 2 below shows an example of a picture parameter set.

[0280] [Table 2]

[0281]

[0282] The syntax pps_extension_flag may indicate whether pps_extension_data_flag may be present in the PPS RBSP syntax structure. A first value (e.g., 0) may indicate that pps_extension_data_flag is not present and a second value (e.g., 1) may indicate that pps_extension_data_flag may be present. The syntax pps_hrd_parameters_update_present_flag may indicate whether general timing and HRD parameters may be present. A first value (e.g., 0) may indicate that general timing and HRD parameters are not present and a second value (e.g., 1) may indicate that general timing and HRD parameters are present. If the current syntax is not present in the bitstream, the value of this flag may be inferred to be 0.

[0283] The syntax structure general_timing_hrd_parameters() may provide some sequence level HRD parameters used in HRD operation.

[0284] If the syntax ols_timing_hrd_parameters() is included in the VPS, the OLS to which the syntax is applied may be specified by the VPS.If the syntax ols_timing_hrd_parameters() is included in the SPS, the syntax may be applied to the OLS including only the lowest layer among layers referring to the SPS, and the lowest layer may be an independent layer.

[0285] There is no restriction on the value that the syntax pps_extension_data_flag can have, it can have any value.

[0286] As an example, where RPR is applicable, updated parameters and timing information may be provided for consistency testing / timing operations, which may be used to update the timing derivation of decoding operations (e.g., arrival of a picture in the CPB, removal of a picture from the CPB, removal of a picture from the DPB, etc.). Updated parameters and timing information / timing operations / reinitialization of consistency testing may be performed for pictures with temporalID equal to 0 in the middle of the CVS and / or pictures that reference at least one reference picture with a different resolution / picture size. In addition, additional HRD parameters may be associated with a particular picture, which may be used to update or reinitialize consistency testing / timing operations. Additional HRD parameters may be present in a picture parameter set (PPS) referenced by the picture. Alternatively, additional HRD parameters may be present in a picture header of the picture. There may be a flag specifying whether the additional HRD parameters are present in the PPS or in the picture header.

[0287] As an example, for HRD operation and consistency testing, the selected scheduling index may be updated or changed in the middle of a CVS or CLVS. For HRD operation and consistency testing, the selected scheduling index may be changed or updated in blocks where significant bit rate changes occur (i.e., the coded data of the current block (e.g., AU) may be much larger or smaller than the previous block). It may be further constrained that scheduling index changes are allowed in pictures where significant bit rate changes occur in the middle of a CVS / CLVS in pictures with a temporal identifier (e.g., temporalId) equal to 0. It may be further constrained that scheduling index changes in the middle of a CVS / CLVS are allowed only when picture resolution (i.e., RPR) changes in the CLVS are allowed.

[0288] As an example, the values ​​of ScIdx, BitRate[Htid][ScIdx], and CpbSize[Htid][ScIdx] may be constrained as follows:

[0289] - The HSS may generate the BitRate[HtId] or CpBSize[HtId][ScIdx1] of the AU including AU m by selecting the ScIdx1 value of ScIdx among the ScIdx values ​​provided in the selected general_timing_hrd_parameters() syntax structure including AU m if at least one of the following conditions is met. The value of BitRate[Htid][ScIdx1] or CpBSize[HtId][ScIdx1] may be different from the value of BitRate[Htid][ScIdx0] or CpBSize[HtId][ScIdx0] used for the previous AU:

[0290] 1. The contents of the general_timing_hrd_parameters() syntax structure selected for the AU including AU m and the previous AU may be different.

[0291] 2. The picture sizes of the coded pictures of the current AU and the previous AU may be different. Here, the temporalId of the current AU may be 0.

[0292] Alternatively, RPR may be applicable (ie, sps_res_change_in_clvs_allowed_flag value is 1) and the picture sizes of the coded pictures of the current AU and the previous AU may be different, wherein the temporalId of the current AU may be 0.

[0293] - Otherwise, the HSS may continue to operate with the previous values ​​of ScIdx, BitRate[Htid][ScIdx] and CpbSize[Htid][ScIdx].

[0294] When the HSS selects a BitRate[Htid][ScIdx] or CpbSize[Htid][ScIdx] value that is different from the previous AU's value, the following may apply:

[0295] -The variable BitRate[Htid][ScIdx] can be applied to the initial CPB arrival time of the current AU.

[0296] -The variable CpbSize[Htid][ScIdx] can be used as follows:

[0297] 1. If the new value of CpbSize[Htid][ScIdx] is larger than the previous CPB size, it may be applied to the initial CPB arrival time of the current AU.

[0298] 2. Otherwise, the new value of CpbSize[Htid][ScIdx] may be applied to the CPB removal time of the current AU.

[0299] Fig.12 is a view for explaining an example to which an image encoding method according to an embodiment of the present disclosure can be applied.

[0300] The image encoding apparatus may derive HRD parameters (S1210) and encode image / video information (S1220). At this time, the image / video information may include information related to the derived HRD parameters.

[0301] although Fig.12 Not shown in FIG. 12 , the image encoding apparatus may perform DPB management based on the HRD parameters derived in step S1210 .

[0302] Fig.13 is a view for explaining an example to which an image decoding method according to an embodiment of the present disclosure can be applied.

[0303] The image decoding apparatus may obtain image / video information from a bitstream (S1310). At this time, the image / video information may include information related to the HRD parameter.

[0304] The image decoding apparatus may perform decoding of the picture based on the obtained HRD parameter ( S1320 ).

[0305] Fig.14 is a view for explaining another example of the image decoding method according to an embodiment of the present disclosure.

[0306] The image decoding apparatus may obtain image / video information from a bitstream (S1410). At this time, the image / video information may include information related to the HRD parameter.

[0307] The image decoding apparatus may perform DPB management based on the obtained HRD parameters (S1420).

[0308] The image decoding apparatus may perform decoding of the picture based on the DPB (S1430). For example, a block / slice in the current picture may be decoded based on inter-frame prediction using a picture that has been reconstructed in the DPB as a reference picture.

[0309] In reference Figure 12 to Figure 14 In the described example, the information related to the HRD parameter may include at least one information / syntax element described with respect to at least one embodiment of the present disclosure. In addition, as described above, DPB management may be performed based on the HRD parameter. For example, before decoding of the current picture, picture deletion from the DPB and / or (decoding) picture output may be performed based on the HRD parameter.

[0310] Fig.15It is a view for illustrating an image processing method according to an embodiment of the present disclosure. The image processing method includes an image encoding method, and may be an image encoding or decoding method. In addition, it may be performed by an image processing device, and the image processing device may include an image encoding device. The image encoding device may include both an image encoding device and an image decoding device.

[0311] As an example, the image decoding method may include the above-mentioned implementation manners and components. Here, some components may be combined, the order may be changed, some components may be deleted, or other components may be added.

[0312] As an example, it may be determined whether the image decoding method allows a reference sample resampling (RPR) operation (S1510). When the resolutions of the reference picture and the current picture are different, the reference sample resampling operation may include an upscaling or downscaling operation, and since this is the same as described above, repeated description is omitted. In addition, based on allowing the RPR operation, a hypothetical reference decoder (HRD) operation (S1520) may be performed. The HRD operation includes at least one of an operation to determine the consistency of a bitstream including encoded video data with a video coding standard, an operation to determine the consistency of a video decoder with a video coding standard, or an HRD picture output timing signaling operation, wherein the HRD operation may be performed by updating a scheduling index of a hypothetical stream scheduler (HSS). For example, the HRD operation is performed by appending specific HRD parameters, and the HRD parameters may be included in a picture header or a picture parameter set. The additional specific HRD parameters may only exist when the additional parameter presence flag is a specific value. In addition, as an example, HRD operations (e.g., operations to determine the consistency of a bitstream with a video coding standard) may be determined based on at least one of the number of access units associated with a BP (buffering period) SEI (supplemental enhancement information) message, the number of IRAP (intra-frame random access point) or GDR (gradual decoding refresh) access units, or the number of access units associated with a DRAP (dependent random access point) indication SEI message. As an example, the scheduling index may be updated in the middle of a CVS (coded video sequence) or a CLVS (coded layered video sequence) and may be updated based on a temporal identifier value. In addition, it may be updated in an AU where a significant bitrate change occurs. As an example, the HRD operation may include an operation to update the output timing derivation based on the updated HRD parameters and output timing information. In addition, the above-mentioned output timing derivation may include deriving at least one of the arrival timing of a picture in a CPB, the removal timing of a picture in a CPB, or the removal timing of a picture in a DPB. As an example, an HRD operation (e.g., an operation to determine the conformance of a bitstream with a video coding standard) may be performed a number of times derived based on the product of a first variable indicating the number of AUs of BitstreamToDecode associated with a BP SEI message applicable to TargetOlsIdx and a second variable associated with the value of general_du_hrd_params_present_flag. Additionally, an HRD operation (e.g., determining the conformance of a bitstream with a video coding standard) may be performed a number of times derived based on the product of a first variable indicating the number of IRAP or GDR AUs of BitstreamToDecode associated with a BP SEI message applicable to TargetOlsIdx and a second variable associated with the value of general_du_hrd_params_present_flag.Additionally, HRD operations (e.g., determining conformance of a bitstream with a video coding standard) may be performed a number of times derived based on the product of a first variable indicating the number of AUs of BitstreamToDecode associated with a DRAP indication SEI message applicable to TargetOlsIdx and having an associated PT SEI message with pt_cpb_alt_timing_info_present_flag equal to 1, and a second variable associated with the value of general_du_hrd_params_present_flag.

[0313] In addition, due to Fig.15 Corresponding to one embodiment of the present disclosure, some components not shown in the drawings may be added, some components may be deleted, or the order of the above-described embodiments may be changed.

[0314] Fig.16 It is a diagram for explaining an image encoding method according to an embodiment of the present disclosure. Fig.16 The image encoding method may be performed by an image encoding device and may include the above-mentioned embodiments and components. Here, some components may be combined, the order may be changed, some components may be deleted, or other components may be added.

[0315] As an example, according to the image coding method, it can be determined whether a reference sample resampling (RPR) operation is allowed, and the reference sample resampling permission information can be encoded (S1610). The repeated description of the reference sample resampling will be omitted. Then, HRD parameters for the assumed reference decoder (HRD) operation can be encoded (S1620). Based on allowing the RPR operation, the HRD parameters can be encoded. The HRD operation may include at least one of an operation of determining the consistency of a bitstream including encoded video data with a video coding standard, an operation of determining the consistency of a video decoder with a video coding standard, or an HRD picture output timing signaling operation, wherein the HRD operation may be performed by updating a scheduling index of a assumed stream scheduler (HSS). The repeated description of this will be omitted.

[0316] In addition, due to Fig.15 Corresponding to one embodiment of the present disclosure, some components not shown in the drawings may be added, some components may be deleted, or the order of the above-described embodiments may be changed.

[0317] In addition, although not shown in the figure, according to an embodiment of the present disclosure, a bitstream transmission method and a bitstream recording medium may be proposed. The bitstream transmission method includes transmitting a bitstream generated by an image encoding method, and the image encoding method may include Fig.16In addition, the bit stream recording medium may be a recording medium storing a bit stream generated by the image encoding method. Similarly, the image encoding method may include Fig.16 image coding method.

[0318] Although for the sake of clarity of description, the exemplary method of the present disclosure described above is represented as a series of operations, it is not intended to limit the order of executing the steps, and these steps can be performed simultaneously or in different orders when necessary. In order to implement the method according to the present invention, the steps described may further include other steps, may include the remaining steps except some steps, or may include other additional steps except some steps.

[0319] In the present disclosure, an image encoding device or an image decoding device that performs a predetermined operation (step) may perform an operation (step) of confirming an execution condition or situation of the corresponding operation (step). For example, if it is described that a predetermined operation is performed when a predetermined condition is met, the image encoding device or the image decoding device may perform the predetermined operation after determining whether the predetermined condition is met.

[0320] The various embodiments of the present disclosure are not a list of all possible combinations and are intended to describe representative aspects of the present disclosure, and matters described in the various embodiments may be applied independently or in combination of two or more.

[0321] Various embodiments of the present disclosure may be implemented in hardware, firmware, software or a combination thereof. In the case of implementing the present disclosure in hardware, the present disclosure may be implemented in an application specific integrated circuit (ASIC), a digital signal processor (DSP), a digital signal processing device (DSPD), a programmable logic device (PLD), a field programmable gate array (FPGA), a general purpose processor, a controller, a microcontroller, a microprocessor, etc.

[0322] In addition, the image decoding device and the image encoding device to which the embodiments of the present disclosure are applied may be included in a multimedia broadcast transmission and reception device, a mobile communication terminal, a home theater video device, a digital theater video device, a surveillance camera, a video chat device, a real-time communication device such as video communication, a mobile streaming device, a storage medium, a camera, a video on demand (VoD) service provider, an OTT video (over the top video) device, an Internet streaming service provider, a three-dimensional (3D) video device, a video phone video device, a medical video device, etc., and may be used to process a video signal or a data signal. For example, an OTT video device may include a game console, a Blu-ray player, an Internet access TV, a home theater system, a smart phone, a tablet PC, a digital video recorder (DVR), etc.

[0323] Fig.17is a diagram showing a content streaming system to which an embodiment of the present disclosure can be applied.

[0324] like Fig.17 As shown in , a content streaming system to which the embodiments of the present disclosure are applied may mainly include an encoding server, a streaming server, a network server, a media storage device, a user device, and a multimedia input device.

[0325] The encoding server compresses the content input from a multimedia input device such as a smart phone, camera, or camcorder into digital data to generate a bitstream and sends the bitstream to the streaming server. As another example, when a multimedia input device such as a smart phone, camera, or camcorder directly generates a bitstream, the encoding server can be omitted.

[0326] The bitstream may be generated by the image encoding method or the image encoding device to which the embodiments of the present disclosure are applied, and the streaming server may temporarily store the bitstream in the process of transmitting or receiving the bitstream.

[0327] The streaming server sends multimedia data to the user device based on the user's request through the network server, and the network server is used as a medium to inform the user of the service. When the user requests the required service from the network server, the network server can deliver it to the streaming server, and the streaming server can send the multimedia data to the user. In this case, the content streaming system may include a separate control server. In this case, the control server is used to control the command / response between the devices in the content streaming system.

[0328] The streaming server may receive content from a media storage device and / or an encoding server. For example, when receiving content from an encoding server, the content may be received in real time. In this case, in order to provide a smooth streaming service, the streaming server may store the bitstream within a predetermined time.

[0329] Examples of user devices may include mobile phones, smart phones, laptop computers, digital broadcast terminals, personal digital assistants (PDAs), portable multimedia players (PMPs), navigation devices, slate PCs, tablet PCs, ultrabooks, wearable devices (e.g., smart watches, smart glasses, head-mounted displays), digital televisions, desktop computers, digital signage, etc.

[0330] Each server in the content streaming system may operate as a distributed server, in which case data received from each server may be distributed.

[0331] The scope of the present disclosure includes software or machine-executable commands (e.g., operating systems, applications, firmware, programs, etc.) for enabling operations of methods according to various embodiments to be performed on a device or computer, and non-transitory computer-readable media having such software or commands stored thereon and executable on a device or computer.

[0332] Industrial Applicability

[0333] The embodiments of the present disclosure may be used to encode or decode an image.

Claims

1. An image decoding method, the image decoding method comprising the following steps: Determine whether to allow reference sample resampling RPR operation; as well as Based on enabling said RPR operation, performing a hypothetical reference decoder HRD operation, wherein the HRD operation comprises at least one of an operation of determining consistency of a bitstream comprising encoded video data with a video coding standard, an operation of determining consistency of a video decoder with the video coding standard, or an HRD picture output timing signaling operation, and The HRD operation is performed by updating a scheduling index selected for a hypothetical flow scheduler HSS.

2. The image decoding method according to claim 1, wherein: The HRD operation is performed by adding specific HRD parameters, and the HRD parameters are included in a picture header or a picture parameter set.

3. The image decoding method according to claim 2, wherein: The additional specific HRD parameter exists only when the additional parameter existence flag is a specific value.

4. The image decoding method according to claim 1, wherein: The HRD operation is determined based on at least one of the number of access units associated with a buffering period BP supplemental enhancement information SEI message, the number of intra random access points IRAP or gradual decoding refresh GDR access units, or the number of access units associated with a dependent random access point DRAP indication SEI message.

5. The image decoding method according to claim 1, wherein: The scheduling index is updated in the middle of a coded video sequence CVS or a coded layered video sequence CLVS.

6. The image decoding method according to claim 5, wherein: The scheduling index is updated based on the time identifier value.

7. The image decoding method according to claim 1, wherein: The scheduling index is updated in the access unit where the bit rate change occurs.

8. The image decoding method according to claim 1, wherein: The HRD operation includes an operation of updating an output timing derivation based on updated HRD parameters and output timing information.

9. The image decoding method according to claim 8, wherein: The output timing derivation includes deriving at least one of an arrival timing of a picture in the CPB, a removal timing of a picture in the CPB, or a removal timing of a picture in the DPB.

10. The image decoding method according to claim 1, wherein: The HRD operation is performed a number of times derived based on the product of a first variable indicating the number of AUs of BitstreamToDecode associated with the BP SEI message applicable to TargetOlsIdx and a second variable associated with the value of general_du_hrd_params_present_flag.

11. The image decoding method according to claim 1, wherein: The HRD operation is performed a number of times derived based on the product of a first variable indicating the number of IRAPs or GDR AUs of BitstreamToDecode associated with the BP SEI message applicable to TargetOlsIdx and a second variable associated with the value of general_du_hrd_params_present_flag.

12. The image decoding method according to claim 1, wherein: The HRD operation is performed a number of times derived based on the product of a first variable indicating the number of AUs of BitstreamToDecode associated with a DRAP indication SEI message applicable to TargetOlsIdx and having an associated PT SEI message with pt_cpb_alt_timing_info_present_flag equal to 1, and a second variable associated with the value of general_du_hrd_params_present_flag.

13. A method for encoding an image, the method comprising the following steps: Determine whether to allow a reference sample resampling RPR operation and encode reference sample resampling permission information; as well as encoding HRD parameters for assuming reference decoder HRD operation based on allowing said RPR operation, wherein the HRD operation comprises at least one of an operation of determining consistency of a bitstream comprising encoded video data with a video coding standard, an operation of determining consistency of a video decoder with the video coding standard, or an HRD picture output timing signaling operation, and The HRD operation is performed by updating a scheduling index for a hypothetical stream scheduler HSS.

14. A method for transmitting a bit stream, the method comprising the following steps: sending the bit stream generated by the image encoding method, The image encoding method comprises the following steps: determining whether a reference sample resampling (RPR) operation is permitted and encoding reference sample resampling permission information; and encoding HRD parameters for assuming reference decoder HRD operation based on allowing said RPR operation, wherein the HRD operation comprises at least one of an operation of determining consistency of a bitstream comprising encoded video data with a video coding standard, an operation of determining consistency of a video decoder with the video coding standard, or an HRD picture output timing signaling operation, and The HRD operation is performed by updating a scheduling index for a hypothetical stream scheduler HSS. 15 . A non-transitory computer-readable recording medium storing a bit stream generated by the image encoding method according to claim 13 .

Images