Method and apparatus for image encoding / decoding based on poc msb information and recording medium storing bitstream

Abstract

A method and apparatus for image encoding / decoding are provided. The image decoding method according to the disclosure includes the steps of determining whether first information indicating a value of a picture order count (POC) most significant bit (MSB) cycle is present in a picture header, determining a POC MSB of a current picture based on whether the first information is present in the picture header, and deriving a POC of the current picture based on the determined POC MSB, wherein, based on the first information not being present in the picture header, the current picture not being a coded layer video sequence start (CLVSS) picture, and a POC difference between the current picture and a picture preceding the current picture in decoding order can be less than half of a maximum value of a POC least significant bit (LSB) of the current picture.

Description

Technical Field

[0001] This disclosure relates to image encoding / decoding methods and apparatus, and recording media for storing bitstreams; more specifically, it relates to image encoding / decoding methods and apparatus based on POC MSB information, and recording media for storing bitstreams generated by the image encoding methods / apparatus of this disclosure. Background Technology

[0002] Recently, there has been an increasing demand across various fields for high-resolution and high-quality images, such as high-definition (HD) and ultra-high-definition (UHD) images. With the increase in image data resolution and quality, the amount of information or bits transmitted increases relative to existing image data. This increase in the amount of information or bits transmitted leads to increased transmission and storage costs.

[0003] Therefore, efficient image compression techniques are needed to effectively transmit, store, and reproduce information about high-resolution and high-quality images. Summary of the Invention

[0004] Technical issues

[0005] The purpose of this disclosure is to provide an image encoding / decoding method and apparatus with improved encoding / decoding efficiency.

[0006] Another object of this disclosure is to provide an image encoding / decoding method and device based on POC MSB information.

[0007] Another object of this disclosure is to provide an image encoding / decoding method and apparatus based on the step of forcibly notifying POC MSB loop information for intra-frame images using signals.

[0008] Another object of this disclosure is to provide an image encoding / decoding method and apparatus based on a POC difference between frames that is forced to be less than half the maximum value of the POC LSB.

[0009] Another object of this disclosure is to provide a non-transitory recording medium for storing bit streams generated by an image encoding method or device according to this disclosure.

[0010] Another object of this disclosure is to provide a non-transitory recording medium for storing a bitstream received, decoded and used to reconstruct an image by an image decoding device according to this disclosure.

[0011] Another object of this disclosure is to provide a method for transmitting a bitstream generated by an image encoding method or device according to this disclosure.

[0012] The technical problems solved by this disclosure are not limited to those described above. Other technical problems not described herein will become clear to those skilled in the art through the following description.

[0013] Technical solution

[0014] An image decoding method according to one aspect of this disclosure includes the following steps: determining whether first information exists in the frame header regarding the value of the most significant bit (MSB) cycle of the frame sequence count (POC) for a specified current frame; determining the POC MSB of the current frame based on the presence or absence of the first information in the frame header; and deriving the POC of the current frame based on the determined POC MSB. If the first information does not exist in the frame header and the current frame is not a CLVSS (Cyclic Layer Video Sequence Start) frame, the POC difference between the current frame and a previous frame preceding the current frame in the decoding order is less than half the maximum value of the least significant bit (LSB) of the current frame's POC.

[0015] According to another aspect of this disclosure, an image decoding apparatus includes a memory and at least one processor. The at least one processor can determine whether first information, specifying the value of the most significant bit (MSB) cycle of the Frame Sequence Count (POC) for the current frame, exists in the frame header; determine the POC MSB of the current frame based on the presence or absence of the first information in the frame header; and deduce the POC of the current frame based on the determined POC MSB. If the first information does not exist in the frame header and the current frame is not a CLVSS (Coding Layer Video Sequence Start) frame, the POC difference between the current frame and a previous frame preceding the current frame in decoding order is less than half the maximum value of the least significant bit (LSB) of the current frame's POC.

[0016] An image coding method according to another aspect of this disclosure includes the following steps: determining the most significant bit (MSB) of the Frame Sequence Count (POC) of the current frame; deriving the POC of the current frame based on the determined POC MSB; and encoding information about the derived POC. First information based on the value of the cyclic POC MSB of the specified current frame is not encoded, and the current frame is not a CLVSS (Cycling Layer Video Sequence Start) frame, the POC difference between the current frame and a previous frame preceding the current frame in the decoding order is less than half the maximum value of the least significant bit (LSB) of the current frame's POC.

[0017] Additionally, according to another aspect of this disclosure, a computer-readable recording medium can store a bitstream generated by the image encoding device or image encoding method of this disclosure.

[0018] In another aspect of the transmission method according to this disclosure, a bit stream generated by the image encoding method or image encoding device of this disclosure may be transmitted.

[0019] The features described above in this brief overview are merely exemplary aspects of the following detailed description of this disclosure and do not limit the scope of this disclosure.

[0020] Beneficial effects

[0021] According to this disclosure, an image encoding / decoding method and apparatus with improved encoding / decoding efficiency can be provided.

[0022] According to this disclosure, an image encoding / decoding method and apparatus based on POC MSB information can be provided.

[0023] According to this disclosure, an image encoding / decoding method and apparatus can be provided based on the step of forcibly notifying POC MSB loop information for intra-frame images using signals.

[0024] According to this disclosure, an image encoding / decoding method and apparatus based on a POC difference between frames that is forced to be less than half the maximum value of the POC LSB can be provided.

[0025] Furthermore, according to this disclosure, a computer-readable recording medium may be provided for storing a bitstream generated by an image encoding method or apparatus according to this disclosure.

[0026] Furthermore, according to this disclosure, a computer-readable recording medium may be provided that stores a bitstream received, decoded, and used by an image decoding device according to this disclosure for reconstructing an image.

[0027] Furthermore, according to this disclosure, a method for transmitting a bit stream generated by an image encoding method or device according to this disclosure can be provided.

[0028] Those skilled in the art will understand that the effects achievable through this disclosure are not limited to those specifically described above, and that other advantages of this disclosure will become clearer from the detailed description. Attached Figure Description

[0029] Figure 1 This is a diagram that schematically illustrates a video coding system to which embodiments of the present disclosure are applicable.

[0030] Figure 2 This is a diagram that schematically illustrates an image encoding device to which embodiments of the present disclosure are applicable.

[0031] Figure 3 This is a diagram that schematically illustrates an image decoding apparatus to which embodiments of the present disclosure are applicable.

[0032] Figure 4 This is a diagram illustrating an example of the layered structure for encoding images / videos.

[0033] Figure 5This is a schematic block diagram of a multilayer coding apparatus to which embodiments of the present disclosure are applicable, and wherein the step of encoding multilayer video / image signals is performed.

[0034] Figure 6 This is a schematic block diagram of a decoding device to which embodiments of the present disclosure are applicable, and wherein the step of decoding multi-layer video / image signals is performed.

[0035] Figure 7 This is a diagram illustrating a method for encoding an image based on a multi-layer structure using an image encoding device according to an embodiment.

[0036] Figure 8 This is a diagram illustrating a method for decoding an image based on a multi-layer structure using an image decoding device according to an embodiment.

[0037] Figure 9 This is a diagram illustrating potential problems during POC derivation in a multi-layered structure.

[0038] Figure 10 This is a diagram illustrating an example of SPS syntax.

[0039] Figure 11 This is an example diagram illustrating the syntax of the screen header.

[0040] Figure 12 This is a flowchart illustrating an image encoding method according to an embodiment of the present disclosure.

[0041] Figure 13 This is a flowchart illustrating an image decoding method according to an embodiment of the present disclosure.

[0042] Figure 14 A diagram illustrating the content streaming system to which the embodiments of this disclosure are applicable. Detailed Implementation

[0043] The embodiments of this disclosure will now be described in detail with reference to the accompanying drawings to facilitate implementation by those skilled in the art. However, this disclosure can be implemented in various different forms and is not limited to the embodiments described herein.

[0044] In describing this disclosure, detailed descriptions of relevant known functions or constructions will be omitted if they unnecessarily obscure the scope of the disclosure. In the accompanying drawings, portions irrelevant to the description of this disclosure are omitted, and similar reference numerals are assigned to similar portions.

[0045] In this disclosure, when a component is "connected," "linked," or "coupled" to another component, it may include not only direct connections but also indirect connections where intermediate components exist. Furthermore, when a component "comprises" or "has" other components, unless otherwise stated, it means that other components may be included, not excluded.

[0046] In this disclosure, the terms first, second, etc., are used only for the purpose of distinguishing one component from other components and do not limit the order or importance of the components, unless otherwise stated. Accordingly, within the scope of this disclosure, a first component in one embodiment may be referred to as a second component in another embodiment, and similarly, a second component in one embodiment may be referred to as a first component in another embodiment.

[0047] In this disclosure, the components are distinguished from each other to clearly describe each feature, but this does not mean that the components must be separate. That is, multiple components may be integrated into a single hardware or software unit, or a single component may be distributed and implemented across multiple hardware or software units. Therefore, unless otherwise specified, implementations of these integrated or distributed components are included within the scope of this disclosure.

[0048] In this disclosure, the components described in the various embodiments are not necessarily essential components, and some components may be optional. Therefore, embodiments consisting of a subset of the components described in the embodiments are also included within the scope of this disclosure. Furthermore, embodiments that include other components besides those described in the various embodiments are also included within the scope of this disclosure.

[0049] This disclosure relates to the encoding and decoding of images. Unless redefined in this disclosure, the terms used herein may have the general meaning commonly used in the art to which this disclosure pertains.

[0050] In this disclosure, a "picture" generally refers to a unit representing an image within a specific time period, while a slice / tile is a coding unit that constitutes part of a picture. A picture can be composed of one or more slices / tiles. Furthermore, a slice / tile may include one or more coding tree units (CTUs).

[0051] In this disclosure, "pixel" or "pixel" can refer to the smallest unit that constitutes a frame (or image). Furthermore, "sample" can be used as a term corresponding to a pixel. A sample can generally represent a pixel or a pixel value, or it can represent only the pixel / pixel value of the luminance component or only the pixel / pixel value of the chrominance component.

[0052] In this disclosure, "unit" can refer to a basic unit of image processing. A unit may include at least one of a specific region of a picture and information associated with that region. In some cases, the term "unit" may be used interchangeably with terms such as "sample array," "block," or "region." Generally, an M×N block may include a set (or array) of samples (or transform coefficients) with M columns and N rows.

[0053] In this disclosure, "current block" can mean one of "current coding block," "current coding unit," "coding target block," "decoding target block," or "processing target block." When performing prediction, "current block" can mean "current prediction block" or "prediction target block." When performing transform (inverse transform) / quantization (dequantization), "current block" can mean "current transform block" or "transform target block." When performing filtering, "current block" can mean "filter target block."

[0054] Furthermore, in this disclosure, unless explicitly stated as a chroma block, "current block" may mean a block that includes both luma component blocks and chroma component blocks, or "the luma block of the current block." The luma component block of the current block can be represented by an explicit description including terms such as "luma block" or "current luma block." Similarly, "the chroma component block of the current block" can be represented by an explicit description including terms such as "chroma block" or "current chroma block."

[0055] In this disclosure, the terms “ / ” or “,” can be interpreted as indicating “and / or”. For example, “A / B” and “A, B” can mean “A and / or B”. Furthermore, “A / B / C” and “A / B / C” can mean “at least one of A, B and / or C”.

[0056] In this disclosure, the term "or" should be interpreted to indicate "and / or". For example, the expression "A or B" can include 1) only "A", 2) only "B", or 3) both "A and B". In other words, in this disclosure, "or" should be interpreted to indicate "additionally or alternatively".

[0057] Overview of Video Encoding Systems

[0058] Figure 1 This is a diagram illustrating a video coding system to which embodiments of the present disclosure are applicable.

[0059] The video encoding system according to the embodiment may include an encoding device 10 and a decoding device 20. The encoding device 10 may deliver encoded video and / or image information or data to the decoding device 20 in the form of a file or stream via a digital storage medium or network.

[0060] The encoding device 10 according to an embodiment may include a video source generator 11, an encoding unit 12, and a transmitter 13. The decoding device 20 according to an embodiment may include a receiver 21, a decoding unit 22, and a renderer 23. The encoding unit 12 may be referred to as a video / image encoding unit, and the decoding unit 22 may be referred to as a video / image decoding unit. The transmitter 13 may be included in the encoding unit 12. The receiver 21 may be included in the decoding unit 22. The renderer 23 may include a display, and the display may be configured as a separate device or an external component.

[0061] The video source generator 11 can acquire video / images through a process of capturing, compositing, or generating video / images. The video source generator 11 may include a video / image capture device and / or a video / image generation device. The video / image capture device may include, for example, one or more cameras, a video / image archive including previously captured video / images, etc. The video / image generation device may include, for example, a computer, tablet computer, and smartphone, and can generate video / images (electronically). For example, virtual video / images can be generated by a computer, etc. In this case, the video / image capture process can be replaced by a process of generating related data.

[0062] The encoding unit 12 can encode the input video / image. For compression and encoding efficiency, the encoding unit 12 can perform a series of processes, such as prediction, transformation, and quantization. The encoding unit 12 can output encoded data (encoded video / image information) in the form of a bitstream.

[0063] Transmitter 13 can transmit encoded video / image information or data, output in bitstream form, to receiver 21 of decoding device 20 in the form of a file or stream via digital storage medium or network. Digital storage medium can include various storage media, such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. Transmitter 13 can include elements for generating media files according to a predetermined file format and may include elements for transmission via broadcast / communication networks. Receiver 21 can extract / receive bitstreams from storage medium or network and transmit the bitstreams to decoding unit 22.

[0064] The decoding unit 22 can decode video / images by performing a series of processes corresponding to the operations of the encoding unit 12, such as dequantization, inverse transform, and prediction.

[0065] Renderer 23 can render decoded video / images. The rendered video / images can be displayed on a monitor.

[0066] Overview of Image Encoding Devices

[0067] Figure 2This is a diagram schematically illustrating an image encoding apparatus to which embodiments of the present disclosure may be applied.

[0068] like Figure 2 As shown, the image encoding device 100 may include an image segmenter 110, a subtractor 115, a transformer 120, a quantizer 130, a dequantizer 140, an inverse transformer 150, an adder 155, a filter 160, a memory 170, an inter-frame prediction unit 180, an intra-frame prediction unit 185, and an entropy encoder 190. The inter-frame prediction unit 180 and the intra-frame prediction unit 185 may be collectively referred to as "predictors". The transformer 120, quantizer 130, dequantizer 140, and inverse transformer 150 may be included in a residual processor. The residual processor may also include a subtractor 115.

[0069] In some implementations, all or at least some of the components configuring the image encoding device 100 may be configured by a single hardware component (e.g., an encoder or a processor). Furthermore, the memory 170 may include a decoded screen buffer (DPB) and may be configured by a digital storage medium.

[0070] Image segmenter 110 can segment an input image (or picture or frame) input to image encoding device 100 into one or more processing units. For example, a processing unit may be called an encoding unit (CU). Encoding units can be obtained by recursively segmenting encoding tree units (CTUs) or maximum encoding units (LCUs) according to a quadtree / binary tree / tritree (QT / BT / TT) structure. For example, an encoding unit can be segmented into multiple encoding units of greater depth based on a quadtree structure, a binary tree structure, and / or a ternary tree structure. For the segmentation of encoding units, a quadtree structure can be applied first, followed by a binary tree structure and / or a ternary tree structure. The encoding process according to this disclosure can be performed based on the final encoding unit that is no longer segmented. The maximum encoding unit can be used as the final encoding unit, or a deeper encoding unit obtained by segmenting the maximum encoding unit can be used as the final encoding unit. Here, the encoding process may include prediction, transformation, and reconstruction processes, which will be described later. As another example, the processing unit of the encoding process may be a prediction unit (PU) or a transformation unit (TU). Prediction units and transform units can be partitioned or segmented from the final coding unit. Prediction units can be sample prediction units, and transform units can be units used to derive transform coefficients and / or units used to derive residual signals from transform coefficients.

[0071] The prediction unit (inter-frame prediction unit 180 or intra-frame prediction unit 185) can perform prediction on the block to be processed (the current block) and generate a prediction block that includes prediction samples of the current block. The prediction unit can determine whether to apply intra-frame prediction or inter-frame prediction based on the current block or CU. The prediction unit can generate various information related to the prediction of the current block and transmit the generated information to the entropy encoder 190. The information about the prediction can be encoded in the entropy encoder 190 and output as a bitstream.

[0072] Intra-prediction unit 185 can predict the current block by referencing samples in the current frame. Depending on the intra-prediction mode and / or intra-prediction technique, the reference samples may be located among the neighbors of the current block or may be placed separately. Intra-prediction modes may include multiple non-directional modes and multiple directional modes. Non-directional modes may include, for example, DC mode and planar mode. Depending on the level of detail in the prediction direction, directional modes may include, for example, 33 or 65 directional prediction modes. However, this is merely an example, and more or fewer directional prediction modes may be used depending on the settings. Intra-prediction unit 185 can determine the prediction mode to be applied to the current block by using prediction modes applied to neighboring blocks.

[0073] The inter-frame prediction unit 180 can deduce the prediction block of the current block based on a reference block (reference sample array) specified by motion vectors on a reference frame. In this case, to reduce the amount of motion information transmitted in the inter-frame prediction mode, motion information can be predicted on a block, sub-block, or sample basis based on the correlation between motion information between neighboring blocks and the current block. Motion information may include motion vectors and reference frame indices. Motion information may also include inter-frame prediction direction (L0 prediction, L1 prediction, dual prediction, etc.) information. In the case of inter-frame prediction, neighboring blocks may include spatially neighboring blocks existing in the current frame and temporally neighboring blocks existing in the reference frame. The reference frame including the reference block and the reference frame including the temporally neighboring block may be the same or different. The temporally neighboring block may be referred to as a juxtaposed reference block, a juxtaposed CU (colCU), etc. The reference frame including the temporally neighboring block may be referred to as a juxtaposed frame (colPic). For example, the inter-frame prediction unit 180 can configure a motion information candidate list based on neighboring blocks and generate information indicating which candidate to use to deduce the motion vector and / or reference frame index of the current block. Inter-frame prediction can be performed based on various prediction modes. For example, in skip mode and merge mode, the inter-frame prediction unit 180 can use motion information from neighboring blocks as motion information for the current block. In skip mode, unlike merge mode, residual signals may not be transmitted. In motion vector prediction (MVP) mode, motion vectors from neighboring blocks can be used as motion vector predictors, and the motion vector of the current block can be signaled by encoding motion vector differences and indicators of the motion vector predictors. The motion vector difference can refer to the difference between the motion vector of the current block and the motion vector predictor.

[0074] The prediction unit can generate a prediction signal based on various prediction methods and techniques described below. For example, the prediction unit can apply not only intra-frame prediction or inter-frame prediction, but also both intra-frame prediction and inter-frame prediction simultaneously to predict the current block. A prediction method that simultaneously applies both intra-frame prediction and inter-frame prediction to predict the current block can be called Combined Inter-Frame and Intra-Frame Prediction (CIIP). Furthermore, the prediction unit can perform Intra-Frame Block Copy (IBC) to predict the current block. Intra-Frame Block Copy can be used for content image / video coding in games, such as Screen Content Coding (SCC). IBC is a method of predicting the current frame using a previously reconstructed reference block in the current frame at a predetermined distance from the current block. When IBC is applied, the position of the reference block in the current frame can be encoded as a vector (block vector) corresponding to the predetermined distance. IBC essentially performs prediction in the current frame, but can be performed similarly to inter-frame prediction because the reference block is derived within the current frame. That is, IBC can use at least one inter-frame prediction technique described in this disclosure.

[0075] The prediction signal generated by the prediction unit can be used to generate a reconstructed signal or a residual signal. Subtractor 115 can generate a residual signal (residual block or residual sample array) by subtracting the prediction signal (prediction block or prediction sample array) output from the prediction unit from the input image signal (original block or original sample array). The generated residual signal can be transmitted to converter 120.

[0076] Transformer 120 can generate transform coefficients by applying transform techniques to the residual signal. For example, the transform techniques may include at least one of Discrete Cosine Transform (DCT), Discrete Sine Transform (DST), Karhunen-Loève Transform (KLT), Graph-Based Transform (GBT), or Conditional Nonlinear Transform (CNT). Here, GBT refers to a transform obtained from a graph when the relationship information between pixels is represented graphically. CNT refers to a transform obtained based on a prediction signal generated using all previously reconstructed pixels. Furthermore, the transform processing can be applied to square pixel blocks of the same size or to blocks of variable size instead of square.

[0077] Quantizer 130 can quantize the transform coefficients and transmit them to entropy encoder 190. Entropy encoder 190 can encode the quantized signal (information about the quantized transform coefficients) and output a bitstream. The information about the quantized transform coefficients can be referred to as residual information. Quantizer 130 can rearrange the block-type quantized transform coefficients into a one-dimensional vector form based on the coefficient scan order, and generate information about the quantized transform coefficients based on the one-dimensional vector form of the quantized transform coefficients.

[0078] The entropy encoder 190 can perform various encoding methods, such as exponential Columbus coding, context-adaptive variable-length coding (CAVLC), and context-adaptive binary arithmetic coding (CABAC). The entropy encoder 190 can encode, either together or separately, the information required for video / image reconstruction other than the quantization transform coefficients (e.g., values ​​of syntax elements). The encoded information (e.g., encoded video / image information) can be transmitted or stored in bitstream form at the Network Abstraction Layer (NAL) level. The video / image information may also include information about various parameter sets, such as Adaptive Parameter Set (APS), Picture Parameter Set (PPS), Sequence Parameter Set (SPS), or Video Parameter Set (VPS). Furthermore, the video / image information may also include general constraint information. The signaled information, transmitted information, and / or syntax elements described in this disclosure can be encoded and included in the bitstream through the above encoding process.

[0079] The bitstream can be transmitted over a network or stored in a digital storage medium. The network may include broadcast networks and / or communication networks, and the digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, and SSD. A transmitter (not shown) for transmitting the signal output from the entropy encoder 190 and / or a storage unit (not shown) for storing the signal may be included as internal / external components of the image encoding device 100. Alternatively, a transmitter may be provided as a component of the entropy encoder 190.

[0080] The quantization transform coefficients output from quantizer 130 can be used to generate residual signals. For example, the residual signals (residual blocks or residual samples) can be reconstructed by applying dequantization and inverse transform to the quantization transform coefficients through dequantizer 140 and inverse transformer 150.

[0081] Adder 155 adds the reconstructed residual signal to the prediction signal output from inter-frame prediction unit 180 or intra-frame prediction unit 185 to generate a reconstructed signal (reconstructed frame, reconstructed block, reconstructed sample array). If the block to be processed has no residual, such as in the case of applying skip mode, the prediction block can be used as a reconstructed block. Adder 155 can be referred to as a reconstructor or reconstructed block generator. The generated reconstructed signal can be used for intra-frame prediction of the next block to be processed in the current frame, and can be used for inter-frame prediction of the next frame by filtering as described below.

[0082] Filter 160 can improve the subjective / objective image quality by applying filtering to the reconstructed signal. For example, filter 160 can generate a modified reconstructed image by applying various filtering methods to the reconstructed image and store the modified reconstructed image in memory 170, specifically in the DPB of memory 170. Various filtering methods can include, for example, deblocking filtering, sample adaptive offsetting, adaptive loop filtering, bilateral filtering, etc. Filter 160 can generate various filtering-related information and transmit the generated information to entropy encoder 190, as described later in the description of each filtering method. The filtering-related information can be encoded by entropy encoder 190 and output as a bitstream.

[0083] The modified reconstructed frame transmitted to memory 170 can be used as a reference frame in inter-frame prediction unit 180. When inter-frame prediction is applied by image encoding device 100, prediction mismatch between image encoding device 100 and image decoding device can be avoided and coding efficiency can be improved.

[0084] The DPB of memory 170 can store modified reconstructed frames for use as reference frames in inter-frame prediction unit 180. Memory 170 can store motion information of blocks from which motion information in the current frame is derived (or encoded) and / or motion information of already reconstructed blocks in the frame. The stored motion information can be transmitted to inter-frame prediction unit 180 and used as motion information for spatially or temporally neighboring blocks. Memory 170 can store reconstructed samples of reconstructed blocks in the current frame and can transmit the reconstructed samples to intra-frame prediction unit 185.

[0085] Overview of image decoding devices

[0086] Figure 3 This is a diagram schematically illustrating an image decoding device to which embodiments of the present disclosure may be applied.

[0087] like Figure 3 As shown, the image decoding device 200 may include an entropy decoder 210, a dequantizer 220, an inverse transformer 230, an adder 235, a filter 240, a memory 250, an inter-frame prediction unit 260, and an intra-frame prediction unit 265. The inter-frame prediction unit 260 and the intra-frame prediction unit 265 may be collectively referred to as "predictors". The dequantizer 220 and the inverse transformer 230 may be included in a residual processor.

[0088] According to an implementation, all or at least some of the components of the image decoding device 200 can be configured by hardware components (e.g., a decoder or a processor). Furthermore, the memory 250 may include a decoded screen buffer (DPB) or may be configured by a digital storage medium.

[0089] The image decoding device 200, having received a bitstream including video / image information, can perform operations related to... Figure 2 The image is reconstructed by processing corresponding to the processing performed by the image encoding device 100. For example, the image decoding device 200 can perform decoding using a processing unit applied in the image encoding device. Therefore, the decoding processing unit can be, for example, an encoding unit. The encoding unit can be obtained by segmenting a coding tree unit or a maximum coding unit. The reconstructed image signal decoded and output by the image decoding device 200 can be reproduced by a reproduction device (not shown).

[0090] Image decoding device 200 can receive data in bitstream form from... Figure 2The signal output by the image encoding device. The received signal can be decoded by the entropy decoder 210. For example, the entropy decoder 210 can parse the bitstream to derive the information (e.g., video / image information) required for image reconstruction (or picture reconstruction). The video / image information may also include information about various parameter sets, such as adaptive parameter sets (APS), picture parameter sets (PPS), sequence parameter sets (SPS), or video parameter sets (VPS). In addition, the video / image information may also include general constraint information. The image decoding device can also decode the picture based on the information about the parameter sets and / or general constraint information. The information and / or syntax elements notified / received by signals described in this disclosure can be decoded and obtained from the bitstream through the decoding process. For example, the entropy decoder 210 decodes the information in the bitstream based on encoding methods such as exponential Golomb coding, CAVLC, or CABAC, and outputs the values ​​of the syntax elements required for image reconstruction and the quantized values ​​of the transform coefficients of the residuals. More specifically, the CABAC entropy decoding method can receive bins corresponding to each syntax element in the bitstream, determine the context model using information about the target syntax element, decoding information of neighboring blocks and the target block, or information about symbols / bins decoded in the previous stage, perform arithmetic decoding on the bins based on the determined context model by predicting the occurrence probability of the bins, and generate symbols corresponding to the value of each syntax element. In this case, the CABAC entropy decoding method can update the context model after determining the context model by using the information of the decoded symbols / bins for the context model of the next symbol / bin. The prediction-related information in the information decoded by the entropy decoder 210 can be provided to the prediction units (inter-frame prediction unit 260 and intra-frame prediction unit 265), and the residual value of entropy decoding performed in the entropy decoder 210, i.e., the quantization transform coefficients and related parameter information, can be input to the dequantizer 220. In addition, the filtering information in the information decoded by the entropy decoder 210 can be provided to the filter 240. Furthermore, the receiver (not shown) for receiving signals output from the image encoding device may be further configured as an internal / external element of the image decoding device 200, or the receiver may be a component of the entropy decoder 210.

[0091] Furthermore, the image decoding apparatus according to this disclosure can be referred to as a video / image / screen decoding apparatus. The image decoding apparatus can be divided into an information decoder (video / image / screen information decoder) and a sample decoder (video / image / screen sample decoder). The information decoder may include an entropy decoder 210. The sample decoder may include at least one of a dequantizer 220, an inverse transformer 230, an adder 235, a filter 240, a memory 250, an inter-frame prediction unit 260, or an intra-frame prediction unit 265.

[0092] Dequantizer 220 can dequantize the quantized transform coefficients and output transform coefficients. Dequantizer 220 can rearrange the quantized transform coefficients in the form of two-dimensional blocks. In this case, the rearrangement can be performed based on the coefficient scan order performed in the image encoding device. Dequantizer 220 can obtain transform coefficients by performing dequantization on the quantized transform coefficients using quantization parameters (e.g., quantization step size information).

[0093] The inverse transformer 230 can perform inverse transformation on the transformation coefficients to obtain the residual signal (residual block, residual sample array).

[0094] The prediction unit can perform prediction on the current block and generate a prediction block that includes prediction samples of the current block. The prediction unit can determine whether to apply intra-frame prediction or inter-frame prediction to the current block based on the prediction information output from the entropy decoder 210, and can determine a specific intra-frame / inter-frame prediction mode (prediction technique).

[0095] Similar to that described in the prediction unit of the image coding device 100, the prediction unit can generate a prediction signal based on various prediction methods (techniques) described later.

[0096] Intra-prediction unit 265 can predict the current block by referring to samples in the current frame. The description of intra-prediction unit 185 also applies to intra-prediction unit 265.

[0097] The inter-frame prediction unit 260 can deduce the prediction block of the current block based on a reference block (reference sample array) specified by a motion vector on a reference frame. In this case, to reduce the amount of motion information transmitted in the inter-frame prediction mode, motion information can be predicted on a block, sub-block, or sample basis based on the correlation of motion information between neighboring blocks and the current block. Motion information may include motion vectors and reference frame indices. Motion information may also include inter-frame prediction direction (L0 prediction, L1 prediction, dual prediction, etc.) information. In the case of inter-frame prediction, neighboring blocks may include spatially neighboring blocks existing in the current frame and temporally neighboring blocks existing in the reference frame. For example, the inter-frame prediction unit 260 can configure a motion information candidate list based on neighboring blocks and deduce the motion vector and / or reference frame index of the current block based on the received candidate selection information. Inter-frame prediction can be performed based on various prediction modes, and the information about the prediction may include information indicating the inter-frame prediction mode of the current block.

[0098] Adder 235 generates a reconstruction signal (reconstructed frame, reconstruction block, reconstruction sample array) by adding the obtained residual signal to the prediction signal (prediction block, prediction sample array) output from the prediction unit (including inter-frame prediction unit 260 and / or intra-frame prediction unit 265). If the block to be processed has no residual (e.g., in the case of applying skip mode), the prediction block can be used as a reconstruction block. The description of adder 155 also applies to adder 235. Adder 235 may be referred to as a reconstructor or reconstruction block generator. The generated reconstruction signal can be used for intra-frame prediction of the next block to be processed in the current frame, and can be used for inter-frame prediction of the next frame by filtering as described below.

[0099] Filter 240 can improve the subjective / objective image quality by applying filtering to the reconstructed signal. For example, filter 240 can generate a modified reconstructed image by applying various filtering methods to the reconstructed image and store the modified reconstructed image in memory 250, specifically in the DPB of memory 250. Various filtering methods may include, for example, deblocking filtering, adaptive sample shifting, adaptive loop filtering, bilateral filtering, etc.

[0100] The (modified) reconstructed frame stored in the DPB of memory 250 can be used as a reference frame in inter-frame prediction unit 260. Memory 250 can store motion information of blocks from which motion information in the current frame is derived (or decoded) and / or motion information of already reconstructed blocks in the frame. The stored motion information can be transmitted to inter-frame prediction unit 260 to be used as motion information for spatially or temporally neighboring blocks. Memory 250 can store reconstructed samples of reconstructed blocks in the current frame and transmit the reconstructed samples to intra-frame prediction unit 265.

[0101] In this disclosure, the embodiments described in the filter 160, inter-frame prediction unit 180 and intra-frame prediction unit 185 of the image encoding device 100 can be equally or correspondingly applied to the filter 240, inter-frame prediction unit 260 and intra-frame prediction unit 265 of the image decoding device 200.

[0102] Example of encoding layer structure

[0103] The encoded video / images according to this disclosure can be processed, for example, according to the coding layers and structures described below.

[0104] Figure 4 This is a diagram illustrating an example of the layered structure for encoding images / videos.

[0105] Encoded images / videos are classified into a Video Coding Layer (VCL) for image / video decoding and processing, a lower-layer system for sending and storing encoded information, and a Network Abstraction Layer (NAL) that exists between the VCL and the lower-layer system and is responsible for network adaptation functions.

[0106] In VCL, VCL data that includes compressed image data (slice data) can be generated, or additional enhancement information (SEI) messages required for image decoding processing or parameter sets that include information such as picture parameter set (PPS), sequence parameter set (SPS), or video parameter set (VPS) can be generated.

[0107] In NAL, header information (NAL unit header) can be added to the raw byte sequence payload (RBSP) generated in VCL to generate NAL units. In this case, RBSP refers to the slice data, parameter set, and SEI message generated in VCL. The NAL unit header may include NAL unit type information specified according to the RBSP data included in the corresponding NAL unit.

[0108] like Figure 4 As shown, NAL units can be classified into VCL NAL units and non-VCL NAL units based on the type of RBSP generated in the VCL. A VCL NAL unit can refer to a NAL unit that includes information about the image (slice data), while a non-VCL NAL unit can refer to a NAL unit that includes information required for decoding the image (parameter set or SEI message).

[0109] VCL NAL units and non-VCL NAL units can be appended with header information and sent over the network according to the data standard of the underlying system. For example, NAL units can be modified to have a data format with a predetermined standard (e.g., H.266 / VVC file format, RTP (Real-Time Transport Protocol), or TS (Transport Stream)) and sent over various networks.

[0110] As described above, within a NAL unit, the NAL unit type can be specified based on the RBSP data structure included in the corresponding NAL unit, and information about the NAL unit type can be stored in the NAL unit header and signaled. For example, this can be broadly categorized into VCL NAL unit types and non-VCL NAL unit types based on whether the NAL unit includes information about the image (slice data). VCL NAL unit types can be categorized based on the nature and type of the image included in the VCL NAL unit, while non-VCL NAL unit types can be categorized based on the type of parameter set.

[0111] Below are examples of NAL cell types specified based on the type of parameter set / information included in non-VCLNAL cell types.

[0112] - DCI (Decoding Capability Information) NAL Unit: Includes NAL unit types for DCI.

[0113] -VPS (Video Parameter Set) NAL Unit: Includes the NAL unit type of VPS.

[0114] -SPS (Sequence Parameter Set) NAL Unit: Includes NAL unit types related to SPS.

[0115] -PPS (Picture Parameter Set) NAL Unit: Includes the NAL unit type of PPS.

[0116] -APS (Adaptive Parameter Set) NAL Units: Includes NAL unit types related to APS.

[0117] -PH (Picture Header) NAL Unit: Includes NAL unit types for the picture header.

[0118] The aforementioned NAL unit type can have syntax information for the NAL unit type, and this syntax information can be stored in the NAL unit header and signaled. For example, this syntax information can be nal_unit_type, and the NAL unit type can be specified using the nal_unit_type value.

[0119] Furthermore, a frame can include multiple slices, and a slice can include a slice header and slice data. In this case, a frame header can be further added to the multiple slices (slice headers and slice data sets) within a frame. The frame header (frame header syntax) can include information / parameters common to the frame. The slice header (slice header syntax) can include information / parameters common to the slice. APS (APS syntax) or PPS (PPS syntax) can include information / parameters common to one or more slices or frames. SPS (SPS syntax) can include information / parameters common to one or more sequences. VPS (VPS syntax) can be information / parameters common to multiple layers. DCI (DCI syntax) can include information / parameters related to decoding capabilities.

[0120] In this disclosure, the high-level syntax (HLS) may include at least one of APS syntax, PPS syntax, SPS syntax, VPS syntax, DCI syntax, picture header syntax, or slice header syntax. Additionally, in this disclosure, the low-level syntax (LLS) may include, for example, slice data syntax, CTU syntax, coding unit syntax, transform unit syntax, etc.

[0121] In this disclosure, the image / video information encoded in the encoding device and signaled to the decoding device in the form of a bitstream may include not only intra-frame segmentation related information, intra / inter-frame prediction information, residual information, and intra-loop filtering information, but also information about slice headers, frame headers, APS, PPS, SPS, VPS, and / or DCI. Additionally, the image / video information may also include general constraint information and / or information about NAL unit headers.

[0122] Overview of Hybrid NAL Unit Types

[0123] Typically, a single NAL unit type can be set for a single frame. As mentioned above, the syntax information specifying the NAL unit type can be signaled in the NAL unit header, which includes the slice (data). Here, the syntax information can be `nal_unit_type`, and the NAL unit type can be specified using the `nal_unit_type` value.

[0124] Table 1 below shows examples of NAL unit types to which embodiments of this disclosure are applicable.

[0125] [Table 1]

[0126]

[0127]

[0128]

[0129] Referring to Table 1, a CRA frame can have associated RASL frames or RADL frames that exist in the bitstream.

[0130] An IDR frame with a nal_unit_type such as IDR_N_LP may not have an associated leading frame present in the bitstream. In contrast, an IDR frame with a nal_unit_type such as IDR_W_RADL may not have an associated RASL frame present in the bitstream, but may have an associated RADL frame present in the bitstream.

[0131] Furthermore, the VVC standard allows slices within a frame to have different NAL unit types. In this case, the syntax element pps_mixed_nalu_types_in_pic_flag in the Picture Parameter Set (PPS) can have a first value (e.g., 1) and the following constraints can be applied.

[0132] - The frame is constrained to include at least two sub-frames.

[0133] - The VCL NAL unit of the screen is constrained to have two or more different nal_unit_type values.

[0134] - VCL NAL units with nal_unit_type such as GDR_NUT in the VCL NAL units of the screen are constrained to be non-existent.

[0135] - When at least one sub-frame of a frame has a VCLNAL unit with a nal_unit_type such as IDR_W_RADL, IDR_N_LP, or CRA_NUT, all VCL NAL units of other sub-frames in the frame are constrained to have a nal_unit_type such as TRAIL_NUT.

[0136] Additionally, when a picture contains a mix of RASL_NUT and RADL_NUT slices, the picture can be considered a RASL picture, and the variable PictureOutputFlag, which specifies whether to output the picture, can be derived as follows.

[0137] (1) When sps_video_parameter_set_id is greater than 0 and the current layer containing the current frame is not an output layer (i.e., for i in the range of 0 to NumOutputLayersInOls[TargetOlsIdx]-1, nuh_layer_id is not OutputLayerIdInOls[TargetOlsIdx][i]), the PictureOutputFlag of the current frame can be set to the second value (e.g., 0). Alternatively, the PictureOutputFlag of the current frame can be set to the second value (e.g., 0) when either the first or second condition below is true.

[0138] - (First Condition) The current screen is a RASL screen and the NoOutputBeforeRecoveryFlag of the associated IRAP screen has a first value (e.g., 1). Here, the first value (e.g., 1) of NoOutputBeforeRecoveryFlag can specify that screens before the recovery point screen in the decoding order may not be output before the recovery point screen.

[0139] - (Second condition) The current screen is either a GDR screen with a NoOutputBeforeRecoveryFlag of the first value (e.g., 1) or a recovery screen of a GDR screen with a NoOutputBeforeRecoveryFlag of the first value (e.g., 1).

[0140] (2) In another case, PictureOutputFlag can be set to the same value as ph_pic_output_flag.

[0141] In this implementation, the decoder can output frames that do not belong to the output layer. For example, even if there is only one output layer in the access unit (AU), when the output layer's frames are unavailable (e.g., due to loss or layer downswitching), the decoder can set the PictureOutputFlag to the first value (e.g., 1) for the frame with the highest nuh_layer_id among all frames available to the decoder's access unit (AU) and a ph_pic_output_flag with a first value (e.g., 1). Alternatively, in this case, the decoder can set the PictureOutputFlag to a second value (e.g., 0) for all other frames available to the decoder's access unit (AU).

[0142] Multi-layer coding

[0143] Image / video coding according to this disclosure may include multi-layer-based image / video coding. Multi-layer-based image / video coding may include scalable coding. In multi-layer-based coding or scalable coding, input signals can be processed for individual layers. Depending on the layer, the input signal (input image / video) may have different values ​​in at least one aspect: resolution, frame rate, bit depth, color format aspect ratio, or view. In this case, performing inter-layer prediction by using the differences between layers (e.g., based on scalability) can reduce redundant information transmission / processing and increase compression efficiency.

[0144] Figure 5 This is a schematic block diagram of a multilayer encoding device 500 to which embodiments of the present disclosure are applicable for encoding multilayer video / image signals.

[0145] Figure 5 The multi-layer coding device 500 may include Figure 2 Encoding devices. With Figure 2 In comparison, Figure 5 The image segmenter 110 and adder 155 are not shown in the multi-layer coding device 500, but the multi-layer coding device 500 may include the image segmenter 110 and adder 155. In this case, the image segmenter 110 and adder 155 may be included on a layer-by-layer basis. Hereinafter, Figure 5 The description will focus on multi-layer-based prediction. For example, in addition to the following description, the multi-layer coding device 500 may include the above-mentioned references. Figure 2 The technical concept of the described encoding device.

[0146] For ease of description, Figure 5 The diagram illustrates a multi-layer structure consisting of two layers. However, embodiments of this disclosure are not limited to two layers, and multi-layer structures applying embodiments of this disclosure may include two or more layers.

[0147] Reference Figure 5 The encoding device 500 includes an encoder 500-1 for layer 1 and an encoder 500-0 for layer 0. Layer 0 can be a base layer, a reference layer, or a lower layer, and layer 1 can be an enhancement layer, a current layer, or a higher layer.

[0148] The encoder 500-1 of layer 1 may include a predictor 520-1, a residual processor 530-1, a filter 560-1, a memory 570-1, an entropy encoder 540-1, and a multiplexer (MUX) 570. In an embodiment, the MUX may be included as an external component.

[0149] The encoder 500-0 of layer 0 may include a predictor 520-0, a residual processor 530-0, a filter 560-0, a memory 570-0, and an entropy encoder 540-0.

[0150] Predictors 520-0 and 520-1 can perform predictions on the input image based on various prediction schemes as described above. For example, predictors 520-0 and 520-1 can perform inter-frame prediction and intra-frame prediction. Predictors 520-0 and 520-1 can perform predictions in a predetermined processing unit. The prediction unit can be a coding unit (CU) or a transform unit (TU). Prediction blocks (including prediction samples) can be generated based on the prediction results, and based on this, the residual processor can derive residual blocks (including residual samples).

[0151] Inter-frame prediction can generate prediction blocks based on information about at least one of the previous and / or next frames in the current frame. Intra-frame prediction can generate prediction blocks based on neighboring samples in the current frame.

[0152] As inter-frame prediction modes or methods, the various prediction modes or methods described above can be used. In inter-frame prediction, a reference frame can be selected for the current block to be predicted, and a reference block corresponding to the current block can be selected from the reference frame. Predictors 520-0 and 520-1 can generate prediction blocks based on the reference blocks.

[0153] Additionally, predictor 520-1 can use information about layer 0 to perform predictions for layer 1. In this disclosure, for ease of description, the method of using information about another layer to predict information about the current layer is referred to as inter-layer prediction.

[0154] Information about the current layer that is predicted using information about another layer (e.g., predicted via inter-layer prediction) can be at least one of texture, motion information, unit information, or predetermined parameters (e.g., filtering parameters, etc.).

[0155] Additionally, the information about another layer used for prediction of the current layer (e.g., for inter-layer prediction) can be at least one of texture, motion information, unit information, or predetermined parameters (e.g., filtering parameters, etc.).

[0156] Inter-layer prediction: The current block can be a block in the current frame of the current layer (e.g., layer 1) or the block to be encoded. The reference block is a block in the frame (reference frame) of the same access unit (AU) as the frame (current frame) to which the prediction of the current block belongs, on the layer (reference layer, e.g., layer 0) to which the prediction of the current block belongs, and can be a block corresponding to the current block.

[0157] As an example of inter-layer prediction, there exists inter-layer motion prediction that uses motion information from a reference layer to predict the motion information of the current layer. Based on inter-layer motion prediction, the motion information of the current block can be predicted using the motion information of the reference block. That is, when deriving motion information according to the inter-frame prediction mode described below, motion information candidates can be derived based on the motion information of the inter-layer reference block rather than temporally neighboring blocks.

[0158] When applying interlayer motion prediction, predictor 520-1 can scale and use motion information from the reference block of the reference layer (i.e., the interlayer reference block).

[0159] As another example of inter-layer prediction, inter-layer texture prediction can use the texture of a reconstructed reference block as the predicted value for the current block. In this case, predictor 520-1 can scale the texture of the reference block by up-scaling. Inter-layer texture prediction can be referred to as inter-layer (reconstructed) sample prediction or simply inter-layer prediction.

[0160] In inter-layer parameter prediction, which is another example of inter-layer prediction, the derivation parameters of the reference layer can be reused in the current layer, or the parameters of the current layer can be derived based on the parameters used in the reference layer.

[0161] In interlayer residual prediction, which is another example of interlayer prediction, residual information from another layer can be used to predict the residual information of the current layer, and based on this, prediction of the current block can be performed.

[0162] In inter-layer difference prediction, which is another example of inter-layer prediction, the prediction of the current block can be performed using the difference between the images obtained by upsampling or downsampling the reconstructed image of the current layer and the reconstructed image of the reference layer.

[0163] In inter-layer syntax prediction, another example of inter-layer prediction, the syntax information of a reference layer can be used to predict or generate the texture of the current block. In this case, the syntax information of the reference layer can include information about the intra-frame prediction mode and motion information.

[0164] The various prediction methods described above can be used when predicting specific blocks.

[0165] Here, as examples of interlayer prediction, although interlayer texture prediction, interlayer motion prediction, interlayer unit information prediction, interlayer parameter prediction, interlayer residual prediction, interlayer difference prediction, interlayer syntax prediction, etc. are described, the interlayer prediction applicable in this disclosure is not limited to these.

[0166] For example, inter-layer prediction can be applied as an extension of inter-frame prediction for the current layer. That is, inter-frame prediction can be performed for the current block by including the reference frame derived from the reference layer in the reference frames that the inter-frame prediction of the current block can refer to.

[0167] In this case, inter-layer reference frames can be included in the reference frame list for the current block. Predictor 520-1 can use inter-layer reference frames to perform inter-frame prediction for the current block.

[0168] Here, the inter-layer reference frame can be a reference frame constructed by sampling the reconstructed frame of a reference layer to correspond with the current layer. Therefore, when the reconstructed frame of the reference layer corresponds to the frame of the current layer, the reconstructed frame of the reference layer can be used as the inter-layer reference frame without sampling. For example, when the width and height of the samples in the reconstructed frames of the reference layer and the current layer are the same, and the offset between the top left, top right, bottom left, and bottom right edges of the reference layer and the top left, top right, bottom left, and bottom right edges of the current layer is 0, the reconstructed frame of the reference layer can be used as the inter-layer reference frame of the current layer without resampling.

[0169] Furthermore, the reconstructed frame of the reference layer of the interlayer reference frame can be a frame belonging to the same AU as the current frame to be encoded.

[0170] When performing inter-frame prediction for the current block by including inter-layer reference frames in the reference frame list, the positions of the inter-layer reference frames in the reference frame list can differ between reference frame lists L0 and L1. For example, in reference frame list L0, the inter-layer reference frame can be located after a short reference frame preceding the current frame, while in reference frame list L1, the inter-layer reference frame can be located at the end of the reference frame list.

[0171] Here, reference frame list L0 is a list of reference frames used for inter-frame prediction of P-slices or a list of reference frames used as the first reference frame list in inter-frame prediction of B-slices. Reference frame list L1 can be a second reference frame list used for inter-frame prediction of B-slices.

[0172] Therefore, the reference frame list L0 can be composed, in sequence, short-term reference frames before the current frame, inter-layer reference frames, short-term reference frames after the current frame, and long-term reference frames. The reference frame list L1 can be composed, in sequence, short-term reference frames after the current frame, short-term reference frames before the current frame, long-term reference frames, and inter-layer reference frames.

[0173] In this context, a prediction (P) slice is a slice in which intra-frame prediction is performed or inter-frame prediction is performed using at most one motion vector and a reference frame index per prediction block. A double prediction (B) slice is a slice in which intra-frame prediction is performed or prediction is performed using at most two motion vectors and a reference frame index per prediction block. In this respect, an intra (I) slice is a slice in which intra-frame prediction is applied only.

[0174] Additionally, when performing inter-frame prediction for the current block based on a list of reference frames that includes inter-layer reference frames, the list of reference frames may include multiple inter-layer reference frames derived from multiple layers.

[0175] When multiple inter-layer reference frames are included, they can be arranged alternately in reference frame lists L0 and L1. For example, suppose two inter-layer reference frames, such as inter-layer reference frames ILRPi and ILRPiJ, are included in the reference frame list for inter-frame prediction of the current block. In this case, in reference frame list L0, ILRPi can be located after a short reference frame preceding the current frame, and ILRPi can be located at the end of the list. Similarly, in reference frame list L1, ILRPi can be located at the end of the list, and ILRPi can be located after a short reference frame following the current frame.

[0176] In this case, the reference frame list L0 can be composed, in sequence, of short-term reference frames preceding the current frame, interlayer reference frames (ILRPi), short-term reference frames following the current frame, long-term reference frames, and interlayer reference frames (ILRPj). The reference frame list L1 can be composed, in sequence, of short-term reference frames following the current frame, interlayer reference frames (ILRPj), short-term reference frames preceding the current frame, long-term reference frames, and interlayer reference frames (ILRPi).

[0177] Additionally, one of the two inter-layer reference frames can be an inter-layer reference frame derived from a scalable layer used for resolution, and the other can be an inter-layer reference frame derived from a layer used to provide another view. In this case, for example, if ILRPi is an inter-layer reference frame derived from a layer used to provide a different resolution and ILRPj is an inter-layer reference frame derived from a layer used to provide a different view, then in the case of scalability-only scalable video coding without view scalability support, the reference frame list L0 can be composed in sequence of short-term reference frames before the current frame, inter-layer reference frame ILRPi, short-term reference frames after the current frame, and long-term reference frames, and the reference frame list L1 can be composed in sequence of short-term reference frames after the current frame, short-term reference frames before the current frame, long-term reference frames, and inter-layer reference frame ILRPi.

[0178] Furthermore, in inter-layer prediction, information about the inter-layer reference frame can be obtained using only sample values, only motion information (motion vectors), or both. When the reference frame index indicates an inter-layer reference frame, based on information received from the encoding device, the predictor 520-1 can use only sample values ​​of the inter-layer reference frame, only motion information (motion vectors) of the inter-layer reference frame, or both.

[0179] When using only sample values ​​from the inter-frame reference frame, predictor 520-1 can derive samples of the block specified by the motion vector from the inter-frame reference frame as the prediction samples for the current block. Without considering scalable video coding of the view, the motion vectors in inter-frame prediction (inter-frame prediction) using the inter-frame reference frame can be set to a fixed value (e.g., 0).

[0180] When using only the motion information from the inter-layer reference frame, predictor 520-1 can use the motion vector specified by the inter-layer reference frame as a motion vector predictor for deriving the motion vector of the current block. Alternatively, predictor 520-1 can use the motion vector specified by the inter-layer reference frame as the motion vector of the current block.

[0181] When using both sample values ​​and motion information from the inter-layer reference frame, the predictor 520-1 can use samples from the region in the inter-layer reference frame corresponding to the current block and the motion information (motion vector) specified in the inter-layer reference frame to predict the current block.

[0182] When applying interlayer prediction, the encoding device can send a reference index of the interlayer reference frame in the reference frame list to the decoding device, and can also send information to the decoding device to specify which information (sample information, motion information, or both) to be used from the interlayer reference frame, that is, information to specify the dependency type of the interlayer prediction dependency between the two layers.

[0183] Figure 6 This is a schematic block diagram of a decoding apparatus to which embodiments of the present disclosure are applicable for performing decoding of multi-layer video / image signals. Figure 6 The decoding device may include Figure 3 Decoding devices. Figure 6 The realigner shown can be omitted or included in the dequantizer. This diagram focuses on multi-level prediction. Alternatively, it may include... Figure 3 Description of the decoding device.

[0184] exist Figure 6 In the examples provided, for ease of description, a multi-layer structure consisting of two layers will be described. However, it should be noted that the embodiments of this disclosure are not limited thereto, and the multi-layer structure to which the embodiments of this disclosure are applied may include two or more layers.

[0185] Reference Figure 6 The decoding device 600 may include a layer 1 decoder 600-1 and a layer 1 decoder 600-0. The layer 1 decoder 600-1 may include an entropy decoder 610-1, a residual processor 620-1, a predictor 630-1, an adder 640-1, a filter 650-1, and a memory 660-1. The layer 0 decoder 600-2 may include an entropy decoder 610-0, a residual processor 620-0, a predictor 630-0, an adder 640-0, a filter 650-0, and a memory 660-0.

[0186] When a bitstream containing image information is received from an encoding device, the demultiplexer (DEMUX) 605 can demultiplex the information from each layer and send the information to the decoding devices of each layer.

[0187] Entropy decoders 610-1 and 610-0 can perform decoding corresponding to the encoding method used in the encoding device. For example, when the encoding device uses CABAC, entropy decoders 610-1 and 610-0 can perform entropy decoding using CABAC.

[0188] When the prediction mode of the current block is intra-prediction mode, predictors 630-1 and 630-0 can perform intra-prediction of the current block based on neighboring reconstructed samples in the current frame.

[0189] When the prediction mode for the current block is inter-frame prediction mode, predictors 630-1 and 630-0 can perform inter-frame prediction for the current block based on information included in at least one frame before or after the current frame. Some or all of the motion information required for inter-frame prediction can be derived by examining information received from the encoding device.

[0190] When the skip mode is applied as the inter-frame prediction mode, no residual is sent from the encoding device, and the prediction block can be a reconstruction block.

[0191] Furthermore, the predictor 630-1 of layer 1 can perform inter-frame prediction or intra-frame prediction using only information about layer 1 and perform inter-layer prediction using information about another layer (layer 0).

[0192] As information about the current layer that is predicted using information about another layer (e.g., predicted via inter-layer prediction), it may include at least one of texture, motion information, unit information, and predetermined parameters (e.g., filtering parameters, etc.).

[0193] As information about another layer used for prediction of the current layer (e.g., for inter-layer prediction), at least one of texture, motion information, unit information, and predetermined parameters (e.g., filtering parameters, etc.) may exist.

[0194] In inter-layer prediction, the current block can be a block in the current frame of the current layer (e.g., layer 1), and it can also be the block to be decoded. The reference block can be a block in the frame (reference frame) of the same access unit (AU) as the frame (current frame) to which the prediction of the current block belongs, on the layer (reference layer, e.g., layer 0) to which the prediction of the current block belongs, and it can also be the block corresponding to the current block.

[0195] The multilayer decoding device 600 can perform interlayer prediction as described in the multilayer coding device 500. For example, the multilayer decoding device 600 can perform interlayer texture prediction, interlayer motion prediction, interlayer unit information prediction, interlayer parameter prediction, interlayer residual prediction, interlayer difference prediction, interlayer syntax prediction, etc., as described in the multilayer coding device 500, and the interlayer prediction applicable in this disclosure is not limited thereto.

[0196] When a reference frame index is received from the encoding device or when a reference frame index derived from a neighboring block indicates an interlayer reference frame in the reference frame list, the predictor 630-1 can perform interlayer prediction using the interlayer reference frame. For example, when the reference frame index indicates an interlayer reference frame, the predictor 630-1 can deduce the sample values ​​of the region specified by the motion vector in the interlayer reference frame as the prediction block for the current block.

[0197] In this case, inter-layer reference frames can be included in the reference frame list for the current block. Predictor 630-1 can use inter-layer reference frames to perform inter-frame prediction for the current block.

[0198] As described above in the multi-layer encoding device 500, in the operation of the multi-layer decoding device 600, the inter-layer reference frame can be a reference frame constructed by sampling the reconstructed frame of the reference layer to correspond with the current layer. The processing for the case where the reconstructed frame of the reference layer corresponds to the frame of the current layer can be performed in the same manner as the encoding processing.

[0199] Furthermore, as described above in the multilayer encoding device 500, in the operation of the multilayer decoding device 600, the reconstructed screen of the reference layer from which the interlayer reference screen is derived can be a screen belonging to the same AU as the current screen to be encoded.

[0200] Furthermore, as described above in the multilayer coding device 500, in the operation of the multilayer decoding device 600, when inter-frame prediction of the current block is performed by including interlayer reference frames in the reference frame list, the positions of the interlayer reference frames in the reference frame list L0 and L1 may be different.

[0201] Furthermore, as described above in the multilayer coding device 500, in the operation of the multilayer decoding device 600, when inter-frame prediction of the current block is performed based on a reference frame list including inter-layer reference frames, the reference frame list may include multiple inter-layer reference frames derived from multiple layers, and the arrangement of the inter-layer reference frames may be performed to correspond to the encoding process described.

[0202] Furthermore, as described above in the multilayer encoding device 500, in the operation of the multilayer decoding device 600, information about the interlayer reference frame can be obtained using only sample values, only motion information (motion vectors), or both sample values ​​and motion information.

[0203] The multilayer decoding device 600 can receive reference indices from the multilayer encoding device 500 indicating interlayer reference frames in the reference frame list and perform interlayer prediction based on these indices. Additionally, the multilayer decoding device 600 can receive information from the multilayer encoding device 500 specifying which information (sample information, motion information, or both) is used from the interlayer reference frames; that is, information specifying the dependency type of the interlayer prediction dependency between two layers.

[0204] Reference Figure 7 and Figure 8This document describes an image encoding method and a decoding method performed by a multilayer image encoding device and a multilayer image decoding device according to embodiments. Hereinafter, for ease of description, the multilayer image encoding device may be referred to as an image encoding device. Additionally, the multilayer image decoding device may be referred to as an image decoding device.

[0205] Figure 7 This diagram illustrates a method for encoding an image based on a multi-layer structure using an image encoding device according to an embodiment. The image encoding device according to the embodiment can encode a first layer of the image (S710). Next, the image encoding device can encode a second layer of the image based on the first layer (S720). Next, the image encoding device can output a bitstream (for multiple layers) (S730).

[0206] Figure 8 This diagram illustrates a method for decoding an image based on a multi-layer structure using an image decoding device according to an embodiment. The image decoding device according to the embodiment can obtain video / image information from a bitstream (S810). Next, the image decoding device can decode the first layer of the image based on the video / image information (S820). Next, the image decoding device can decode the second layer of the image based on the video / image information and the first layer (S830).

[0207] In one embodiment, the video / image information may include the High-Level Syntax (HLS) described below. In one embodiment, the HLS may include the SPS and / or PPS disclosed in this disclosure. For example, the video / image information may include the information and / or syntax elements described in this disclosure. As described in this disclosure, the second layer of the image may be encoded based on the motion information / reconstruction samples / parameters of the first layer's image. In one embodiment, the layer may be a layer lower than the second layer. In one embodiment, when the second layer is the current layer, the first layer may be referred to as the reference layer.

[0208] POC (Frame Order Count) Decoding Process

[0209] The process for deriving the POC value of the current image is as follows.

[0210] The output of this process can be PicOrderCntVal, which specifies the POC value of the current screen.

[0211] Each encoded screen can be associated with a POC variable called PicOrderCntVal.

[0212] When vps_independent_layer_flag[GeneralLayerIdx[nuh_layer_id]] has a second value (e.g., 0) and picture picA exists in the current access unit (AU) of the reference layer of the current layer, the PicOrderCntVal of the current picture can be deduced to be equal to the PicOrderCntVal of picture picA, and the value of ph_order_cnt_lsb can be forced to be the same in all VCL NAL units of the current access unit (AU). Alternatively, the PicOrderCntVal of the current picture can be deduced as follows.

[0213] - Assume that `prevTid0Pic` has a `nuh_layer_id` equal to the current frame's `nuh_layer_id`, a `TemporalId` equal to 0, and a `ph_non_ref_pic_flag`, and is a previous frame in decoding order, not a RASL or RADL frame. Here, `ph_non_ref_pic_flag` can specify whether the current frame can be used as a reference frame. For example, a first value (e.g., 1) of `ph_non_ref_pic_flag` can specify that the current frame is never used as a reference frame. In contrast, a second value (e.g., 0) of `ph_non_ref_pic_flag` can specify that the current frame can be used as a reference frame.

[0214] - The variable prevPicOrderCntLsb is set to equal ph_pic_order_cnt_lsb of prevTid0Pic.

[0215] - The variable prevPicOrderCntMsb is set to be equal to the PicOrderCntMsb of prevTid0Pic.

[0216] In this case, the variable PicOrderCntMsb of the current screen can be derived as follows.

[0217] - When ph_poc_msb_cycle_present_flag has a first value (e.g., 1), PicOrderCntMsb can be set to be equal to ph_poc_msb_cycle_val * MaxPicOrderCntLsb.

[0218] - In contrast, PicOrderCntMsb can be set to 0 when ph_poc_msb_cycle_present_flag has a second value (e.g., 0) and the current screen is a CLVSS screen.

[0219] - In other cases, PicOrderCntMsb can be derived as shown in Table 2 below.

[0220] [Table 2]

[0221]

[0222] Referring to Table 2, when prevPicOrderCntLsb is greater than or equal to the value obtained by adding MaxPicOrderCntLsb / 2 to ph_pic_order_cnt_lsb, PicOrderCntMsb can be derived as the value obtained by adding MaxPicOrderCntLsb to prevPicOrderCntMsb. Conversely, when prevPicOrderCntLsb is less than the value obtained by adding MaxPicOrderCntLsb / 2 to ph_pic_order_cnt_lsb, PicOrderCntMsb can be derived as the value obtained by subtracting MaxPicOrderCntLsb from prevPicOrderCntMsb. In another case, PicOrderCntMsb can be derived as the same value as prevPicOrderCntMsb.

[0223] By adding ph_pic_order_cnt_lsb to PicOrderCntMsb, which is derived in this way, PicOrderCntVal can be derived (i.e., PicOrderCntVal = PicOrderCntMsb + ph_pic_order_cnt_lsb). In this case, because PicOrderCntMsb is set to 0 for the frame, all CLVSS frames where ph_poc_msb_cycle_val does not exist can have the same PicOrderCntVal as ph_pic_order_cnt_lsb.

[0224] In one example, the value of PicOrderCntVal can be constrained to a range from -2. 31 Up to 2 31 Within the range of -1.

[0225] In one example, within a CVS, for any two encoded frames with the same nuh_layer_id value, the value of PicOrderCntVal can be constrained to be unequal to each other.

[0226] In one example, all screens in any given access unit (AU) can be constrained to have the same value, PicOrderCntVal.

[0227] In one example, the function PicOrderCnt(picX) can specify the PicOrderCntVal of the image picX (i.e., PicOrderCnt(picX) = PicOrderCntVal of the image picX).

[0228] In one example, the function DiffPicOrderCnt(picA, picB) can specify the POC difference between picture picA and picture picB (i.e., DiffPicOrderCnt(picA, picB) = PicOrderCnt(picA) - PicOrderCnt(picB)). In this case, the bitstream can be constrained such that the value of DiffPicOrderCnt(picA, picB) used during decoding does not include -2. 15 Up to 2 15 Data outside the range of -1. Furthermore, assuming the current frame is X and two different frames in the same CVS are Y and Z, when both DiffPicOrderCnt(X,Y) and DiffPicOrderCnt(X,Z) are positive or negative, Y and Z can be considered to be in the same output order direction from X.

[0229] The following section will describe the problems that may arise during the POC derivation under a multi-layered structure.

[0230] Problems with existing technology

[0231] Figure 9 This is a diagram illustrating potential problems during POC derivation in a multi-layered structure.

[0232] Reference Figure 9 A multi-layer structure can consist of two layers: layer 0 and layer 1. Layer 0 is the lower layer and can be called the base layer or reference layer. In contrast, layer 1 is the higher layer and can be called the enhancement layer or current layer. In the following text, it is assumed that layer 0 is a non-output layer and layer 1 is an output layer.

[0233] Layer 0 may include multiple Intra-Random Access Point (IRAP) frames with a single frame type. For example, Layer 0 may include first through nth Pure Random Access (CRA) frames. In this case, within Layer 0, the first through nth CRA frames may be spaced apart from each other in Frame Order Count (POC) order. For example, the POC value of the first CRA frame may be 0, the POC value of the second CRA frame may be 32, the POC value of the third CRA frame may be 64, and the POC value of the fourth CRA frame may be 96.

[0234] In contrast, Layer 1 may include multiple frames with two or more frame types. For example, Layer 1 may include at least one CRA frame and multiple rear frames. In this case, within Layer 1, at least one CRA frame and multiple rear frames may be arranged sequentially in POC order. For example, the POC value of at least one CRA frame may be 0, and the POC values ​​of multiple rear frames may be greater than or equal to 1.

[0235] Suppose that the value of the syntax element `sps_log2_max_pic_order_cnt_lsb_minus4`, signaled within the High-Level Syntax (HLS), is 0 (e.g., the Sequence Parameter Set (SPS) is 0) and `sps_poc_msb_cycle_flag` has a second value (e.g., 0). Here, `sps_log2_max_pic_order_cnt_lsb_minus4` can specify the maximum value of the least significant bit (LSB) of the POC used for POC derivation (e.g., `MaxPicOrderCntLsb`). When the value of `sps_log2_max_pic_order_cnt_lsb_minus4` is 0, only 4 bits are used for the POC. The signaling uses the LSB, therefore the value of MaxPicOrderCntLsb can be 16. Additionally, sps_poc_msb_cycle_flag can specify whether flag information (e.g., ph_poc_msb_cycle_present_flag) exists in the screen header. This flag information specifies whether the most significant bit (MSB) cycle information for POC derivation (e.g., ph_poc_msb_cycle_val) exists in the screen header. When sps_poc_msb_cycle_flag has a second value (e.g., 0), ph_poc_cycle_present_flag may not exist in the screen header.

[0236] Under the above conditions, when Figure 9When the second, third, or fourth CRA frame in layer 0 is decoded, the decoder may not be able to correctly derive the current POC because it cannot find the previous frame in the decoding order (i.e., prevTid0Pic). Therefore, in the example above, to derive the current POC, the POC MSB information for the CRA frame in the layer needs to be signaled (e.g., ph_poc_msb_cycle_val). However, according to the existing POC derivation process, there is a problem that the signaling of the POC MSB information is not mandatory.

[0237] To address this issue, according to embodiments of this disclosure, POC MSB information can be forcibly notified by signal under predetermined conditions. Furthermore, according to embodiments of this disclosure, when POC MSB information is not notified by signal for the current screen, the extraction process can be modified so that a pre-specified previous screen (i.e., prevTid0Pic) is forcibly present or screens with TemporalId equal to 0 are not removed.

[0238] The embodiments of this disclosure can be implemented by modifying the semantics of the aforementioned sps_poc_msb_cycle_flag and / or ph_poc_cycle_present_flag. The SPS syntax containing sps_poc_msb_cycle_flag and the screen header syntax containing ph_poc_cycle_present_flag will be briefly described, followed by a description of the embodiments of this disclosure.

[0239] Figure 10 This is a diagram illustrating an example of SPS syntax.

[0240] Reference Figure 10 The SPS can include the syntax element sps_seq_parameter_set_id. sps_seq_parameter_set_id can provide an identifier for the SPS to be referenced by other syntax elements.

[0241] Additionally, the SPS can include the syntax element `sps_video_parameter_set_id`. `sps_video_parameter_set_id` can specify the value of `vps_video_parameter_set_id`, which is an identifier for the VPS referenced by the SPS. The presence of a VPS is optional for single-layer bitstreams. When the VPS does not exist, `sps_video_parameter_set_id` can be 0, and the following can be applied.

[0242] - The SPS can be independent of the VPS, and when each coding layer video sequence (CLVS) with reference to the SPS is decoded, no VPS needs to be referenced.

[0243] The value of -vps_max_layers_minus1 can be inferred to be 0. Here, the value obtained by adding 1 to vps_max_layers_minus1 can specify the number of layers specified by the VPS, that is, the maximum allowed number of layers in each encoded video sequence (CVS) of the reference VPS.

[0244] The value of `-vps_max_sublayer_minus1` can be inferred to be 6. Here, the value obtained by adding 1 to `vps_max_sublayer_minus1` specifies the maximum number of time-based sublayers that can exist within the layer specified by the VPS. In one example, the value of `vps_max_sublayer_minus1` can be constrained to a range of 0 to 6.

[0245] - CVS can be constrained to include only one layer. That is, all VCL NAL units in CVS can be constrained to have the same nuh_layer_id value.

[0246] The value of -GeneralLayerIdx[nuh_layer_id] can be inferred to be 0.

[0247] The value of `-vps_independent_layer_flag[GeneralLayerIdx[nuh_layer_id]]` can be inferred to be 1. Here, `vps_independent_layer_flag[i]` can specify whether the layer with index i can use inter-layer prediction. For example, a first value (e.g., 1) of `vps_independent_layer_flag[i]` can specify that the layer with index i does not use inter-layer prediction. In contrast, a second value (e.g., 0) of `vps_independent_layer_flag[i]` can specify that the layer with index i can use inter-layer prediction.

[0248] Additionally, the SPS can include the syntax element `sps_poc_msb_cycle_flag`. `sps_poc_msb_cycle_flag` can specify whether `ph_poc_msb_cycle_present_flag` exists in the reference SPS's screen header. For example, a first value (e.g., 1) of `sps_poc_msb_cycle_flag` can specify that `ph_poc_msb_cycle_present_flag` exists in the reference SPS's screen header. In contrast, a second value (e.g., 0) of `sps_poc_msb_cycle_flag` can specify that `ph_poc_msb_cycle_present_flag` does not exist in the reference SPS's screen header.

[0249] Figure 11 This is an example diagram illustrating the syntax of the screen header.

[0250] Reference Figure 11 The screen header can include the syntax element ph_recovery_poc_cnt. ph_recovery_poc_cnt can specify the recovery point of the decoded screens in the output order. In one example, if the current screen is a GDR screen and the value of ph_recovery_poc_cnt for the current screen is 0, the current screen itself can be a recovery point screen.

[0251] Additionally, the screen header may include the syntax element ph_poc_msb_cycle_present_flag. (See above for reference only.) Figure 10 The description of `sps_poc_msb_cycle_flag` encodes / decodes `ph_poc_msb_cycle_present_flag` when it has a first value (e.g., 1). `ph_poc_msb_cycle_present_flag` can specify whether the syntax element `ph_poc_msb_cycle_val` exists in the screen header. For example, a first value (e.g., 1) of `ph_poc_msb_cycle_present_flag` can specify that the syntax element `ph_poc_msb_cycle_val` exists in the screen header. In contrast, a second value (e.g., 0) of `ph_poc_msb_cycle_present_flag` can specify that the syntax element `ph_poc_msb_cycle_val` does not exist in the screen header. When referring to... Figure 10When the described vps_independent_layer_flag[GeneralLayerIdx[nuh_layer_id]] has a second value (e.g., 0) and the image exists in the current access unit (AU) of the reference layer of the current layer, the value of ph_poc_msb_cycle_present_flag can be constrained to the second value (e.g., 0).

[0252] Additionally, the screen header can have the syntax element ph_poc_msb_cycle_val. ph_poc_msb_cycle_val specifies the POC MSB cycle value for the current screen. In one example, the bit length of ph_poc_msb_cycle_val can be equal to the value obtained by adding 1 to the syntax element sps_poc_msb_cycle_minus1, which is signaled in SPS.

[0253] The embodiments of this disclosure will be described in detail below.

[0254] Implementation methods 1 to 5

[0255] According to embodiments 1 to 5 of this disclosure, for IRAP screens and / or GDR screens with an associated ph_recovery_poc_cnt value equal to 0, POC MSB information can be forcibly notified by signal under predetermined conditions.

[0256] Specifically, according to Embodiment 1 of this disclosure, for an IRAP screen in a layer that is an independent layer and only includes an IRAP screen but not an output layer, it can be constrained to include a step of signaling POC MSB information.

[0257] According to Embodiment 2 of this disclosure, as an IRAP screen included in a layer that is not an independent layer output layer, when other layers only need the IRAP screen for inter-layer prediction, it can be constrained to include a step of signaling the POC MSB information for the IRAP screen.

[0258] According to embodiment 3 of this disclosure, as an independent layer, as an IRAP frame in a layer that includes an output layer in at least one output layer set (OLS) not specified in a video parameter set (VPS), it is constrained to include a step of signaling POC MSB information for the IRAP frame when other layers only require the IRAP frame for inter-layer prediction.

[0259] According to Embodiment 4 of this disclosure, the constraints of Embodiments 1 to 3 described above apply not only to IRAP screens but also to GDR (Progressive Decoding Refresh) screens with an associated ph_recovery_poc_cnt value equal to 0. Here, when the value of ph_recovery_poc_cnt associated with the GDR screen is 0, the GDR screen itself can be a recovery point screen.

[0260] According to Embodiment 5 of this disclosure, the constraints of Embodiments 1 to 4 described above are applicable regardless of whether the layer is an output layer. For example, when the layer in Embodiment 1 described above is an independent layer that only includes the IRAP screen, it can be constrained to include a step of signaling the POC MSB information for the IRAP screen, regardless of whether the layer is an output layer.

[0261] The above-described implementation methods 1 to 5 can be implemented by modifying the semantics of each of the `sps_poc_msb_cycle_flag` and `ph_poc_cycle_present_flag`. Detailed implementation methods are as follows.

[0262] (1) Implementation method 1

[0263] According to implementation method 1 of this disclosure, constraint 1 or constraint 2 for bitstream consistency can be added to the reference as follows. Figure 10 The semantics of the described sps_poc_msb_cycle_flag.

[0264] (Constraint 1) When all of the following conditions are true, sps_poc_msb_cycle_flag can be forced to have a first value (e.g., 1).

[0265] The case where the value of -sps_video_parameter_set_id is not 0.

[0266] - In the following cases: the SPS is referenced by at least one VCL NAL unit of a layer with a layer ID equal to nuh_layer_id, the value of vps_max_tid_il_ref_pics_plus1[i][GeneralLayerIdx[nuh_layer_id]] is 0 for all i values ​​in the range from 0 to vps_max_layers_minus1, and the value of vps_independent_layer_flag[GeneralLayerIdx[nuh_layer_id]] is 0. Here, vps_max_tid_il_ref_pics_plus1[i][j] with a value of 0 can specify that the j-th layer picture, which is neither an IRAP picture nor a GDR picture with ph_recovery_poc_cnt of 0, is not used as the interlayer reference picture (ILRP) when decoding the i-th layer picture. In contrast, vps_max_tid_il_ref_pics_plus1[i][j] with a value greater than 0 can specify that the j-th layer picture with a TemporalId greater than vps_max_tid_il_ref_pics_plus1[i][j]-1 is not used as an inter-layer reference picture when decoding the i-th layer picture, and does not refer to the adaptive parameter set (APS) with a TemporalId greater than vps_max_tid_il_ref_pics_plus1[i][j]-1 and nuh_layer_id equal to vps_layer_id[j].

[0267] (Constraint 2) Alternatively, sps_poc_msb_cycle_flag may be forced to have a first value (e.g., 1) when all of the following conditions are true.

[0268] The case where the value of -sps_video_parameter_set_id is not 0.

[0269] - In the following cases: SPS is referenced by at least one VCL NAL unit of a layer with a layer id equal to nuh_layer_id, and the value of vps_max_tid_il_ref_pics_plus1[i][GeneralLayerIdx[nuh_layer_id]] is 0 for all i values ​​in the range from 0 to vps_max_layers_minus1.

[0270] Additionally, according to implementation 1 of this disclosure, constraint 3 or constraint 4 regarding bitstream consistency can be added to the reference above as follows. Figure 11The semantics of ph_poc_msb_cycle_present_flag.

[0271] (Constraint 3) ph_poc_msb_cycle_present_flag can be forced to have a first value (e.g., 1) when all of the following conditions are true.

[0272] - The case where the associated screen is an IRAP screen or a GDR screen with an associated ph_recovery_poc_cnt value equal to 0, and the screen header includes ph_poc_msb_cycle_present_flag.

[0273] The case where the value of -sps_video_parameter_set_id is not 0.

[0274] - For all i values ​​in the range of 0 to vps_max_layers_minus1, the value of vps_max_tid_il_ref_pics_plus1[i][GeneralLayerIdx[nuh_layer_id]] is 0.

[0275] (Constraint 4) Alternatively, ph_poc_msb_cycle_present_flag may be forced to have a first value (e.g., 1) when all of the following conditions are true.

[0276] - The case where the associated screen is an IRAP screen or a GDR screen with an associated ph_recovery_poc_cnt value equal to 0, and the screen header includes ph_poc_msb_cycle_present_flag.

[0277] The case where the value of -sps_video_parameter_set_id is not 0.

[0278] - For all i values ​​in the range of 0 to vps_max_layers_minus1, the value of vps_max_tid_il_ref_pics_plus1[i][GeneralLayerIdx[nuh_layer_id]] is 0 and the value of vps_independent_layer_flag[GeneralLayerIdx[nuh_layer_id]] is 0.

[0279] (2) Implementation Method 2

[0280] According to implementation 2 of this disclosure, constraint 5 or constraint 6 for bitstream consistency can be added to the reference as follows. Figure 10 The semantics of the described sps_poc_msb_cycle_flag.

[0281] (Constraint 5) When all of the following conditions are true, sps_poc_msb_cycle_flag can be forced to have a first value (e.g., 1).

[0282] The case where the value of -sps_video_parameter_set_id is not 0.

[0283] - In the following cases: the SPS is referenced by at least one VCL NAL unit of a layer with a layer ID equal to nuh_layer_id, and for all i values ​​in the range of 0 to TotalNumOlss-1, the value of NumSubLayersInLayerInOLS[i][GeneralLayerIdx[nuh_layer_id]] is 0. Here, NumSubLayersInLayerInOLS[i][j] can specify the number of sublayers in the j-th output layer of the i-th output layer set (OLS).

[0284] (Constraint 6) Alternatively, sps_poc_msb_cycle_flag may be forced to have a first value (e.g., 1) when all of the following conditions are true.

[0285] The case where the value of -sps_video_parameter_set_id is not 0.

[0286] - In the following cases: SPS is referenced by at least one VCL NAL unit of a layer with a layer ID equal to nuh_layer_id, for all i values ​​in the range of 0 to TotalNumOlss-1, the value of NumSubLayersInLayerInOLS[i][GeneralLayerIdx[nuh_layer_id]] is 0, and the value of vps_independent_layer_flag[GeneralLayerIdx[nuh_layer_id]] is 0.

[0287] Additionally, according to implementation 2 of this disclosure, constraint 7 or constraint 8 for bitstream consistency can be added to the reference above as follows. Figure 11 The semantics of ph_poc_msb_cycle_present_flag are described.

[0288] (Constraint 7) ph_poc_msb_cycle_present_flag may be forced to have a first value (e.g., 1) when all of the following conditions are true.

[0289] - The case where the associated screen is an IRAP screen or a GDR screen with an associated ph_recovery_poc_cnt value equal to 0, and the screen header includes ph_poc_msb_cycle_present_flag.

[0290] The case where the value of -sps_video_parameter_set_id is not 0.

[0291] - For all i values ​​in the range of 0 to TotalNumOlss-1, the value of NumSubLayersInLayerInOLS[i][GeneralLayerIdx[nuh_layer_id]] is 0.

[0292] (Constraint 8) Alternatively, ph_poc_msb_cycle_present_flag may be forced to have a first value (e.g., 1) when all of the following conditions are true.

[0293] - The case where the associated screen header, which includes ph_poc_msb_cycle_present_flag, is an IRAP screen or a GDR screen with an associated ph_recovery_poc_cnt value equal to 0.

[0294] The case where the value of -sps_video_parameter_set_id is not 0.

[0295] - For all i values ​​in the range of 0 to TotalNumOlss-1, the value of NumSubLayersInLayerInOLS[i][GeneralLayerIdx[nuh_layer_id]] is 0 and the value of vps_independent_layer_flag[GeneralLayerIdx[nuh_layer_id]] is 0.

[0296] (3) Implementation method 3

[0297] According to implementation 3 of this disclosure, constraint 9 or constraint 10 for bitstream consistency can be added to the reference as follows. Figure 10 The semantics of the described sps_poc_msb_cycle_flag.

[0298] (Constraint 9) When all of the following conditions are true, sps_poc_msb_cycle_flag can be forced to have a first value (e.g., 1).

[0299] The case where the value of -sps_video_parameter_set_id is not 0.

[0300] -SPS is the case where at least one VCL NAL cell of a layer has a layer ID equal to nuh_layer_id and the value of NumSubLayersInLayerInOLS[i][GeneralLayerIdx[nuh_layer_id]] is 0.

[0301] (Constraint 10) Alternatively, sps_poc_msb_cycle_flag may be forced to have a first value (e.g., 1) when all of the following conditions are true.

[0302] The case where the value of -sps_video_parameter_set_id is not 0.

[0303] - In the following cases: SPS is referenced by at least one VCL NAL unit of a layer with a layer ID equal to nuh_layer_id, NumSubLayersInLayerInOLS[i][GeneralLayerIdx[nuh_layer_id]] is 0, and vps_independent_layer_flag[GeneralLayerIdx[nuh_layer_id]] is 0.

[0304] Additionally, according to implementation 3 of this disclosure, constraint 11 or constraint 12 for bitstream consistency can be added to the reference above as follows. Figure 11 The semantics of ph_poc_msb_cycle_present_flag are described.

[0305] (Constraint 11) ph_poc_msb_cycle_present_flag may be forced to have a first value (e.g., 1) when all of the following conditions are true.

[0306] - The case where the associated screen is an IRAP screen or a GDR screen with an associated ph_recovery_poc_cnt value equal to 0, and the screen header containing ph_poc_msb_cycle_present_flag is included.

[0307] The case where the value of -sps_video_parameter_set_id is not 0.

[0308] The case where -NumSubLayersInLayerInOLS[i][GeneralLayerIdx[nuh_layer_id]] has a value of 0.

[0309] (Constraint 12) Alternatively, ph_poc_msb_cycle_present_flag may be forced to have a first value (e.g., 1) when all of the following conditions are true.

[0310] - The case where the associated screen is an IRAP screen or a GDR screen with an associated ph_recovery_poc_cnt value equal to 0, and the screen header includes ph_poc_msb_cycle_present_flag.

[0311] The case where the value of -sps_video_parameter_set_id is not 0.

[0312] The case where -NumSubLayersInLayerInOLS[i][GeneralLayerIdx[nuh_layer_id]] has a value of 0 and vps_independent_layer_flag[GeneralLayerIdx[nuh_layer_id]] has a value of 0.

[0313] As described above, according to Embodiments 1 to 5 of this disclosure, for IRAP screens and / or GDR screens with an associated ph_recovery_poc_cnt value equal to 0, the step of signaling POCMSB information can be forced under predetermined conditions. Therefore, when the non-output layer only includes IRAP screens and / or GDR screens (see...), Figure 9 When the decoder cannot find the previous frame in the decoding order of the IRAP frame and / or GDR frame (i.e., prevTid0Pic), the problem of incorrect derivation of the POC of the IRAP frame and / or GDR frame can be resolved.

[0314] Implementation Method 6

[0315] According to embodiment 6 of this disclosure, when there is no POC MSB information for the IRAP screen (and the GDR screen with an associated ph_recovery_poc_cnt value equal to 0), it is possible to restrict the extraction of the non-output layer, such that after extraction, only the IRAP screen (and the GDR screen with an associated ph_recovery_poc_cnt value equal to 0) exists in the non-output layer. Here, when the value of ph_recovery_poc_cnt associated with the GDR screen is 0, the GDR screen itself can be a recovery point screen.

[0316] In one implementation, the current layer can be constrained from being extracted only when the current layer is a non-output layer and sps_poc_msb_cycle_flag has a second value (e.g., 0), such that only IRAP frames (and GDR frames with an associated ph_recovery_poc_cnt value equal to 0) exist in the current layer.

[0317] The implementation method 6 described above can be achieved by modifying the semantics of `sps_poc_msb_cycle_flag`. The specific implementation method is as follows.

[0318] (1) Implementation method 4

[0319] According to implementation 4 of this disclosure, constraint 13 or constraint 14 for bitstream consistency can be added to the reference as follows. Figure 10 The semantics of the described sps_poc_msb_cycle_flag.

[0320] (Constraint 13) When all of the following conditions are true, for any layer with a layer ID equal to nuh_layer_id, for i values ​​in the range 0 to TotalNumOlss-1, the value of NumSubLayersInLayerInOLS[i][GeneralLayerIdx[nuh_layer_id]] can be forced to be greater than 0.

[0321] - In the case where the value of sps_video_parameter_set_id is not 0 within the reference SPS.

[0322] - In the reference SPS, sps_poc_msb_cycle_flag has a second value (e.g., 0).

[0323] -vps_independent_layer_flag[GeneralLayerIdx[nuh_layer_id]] has a second value (e.g., 0).

[0324] -SPS is referenced by at least one VCL NAL unit of a layer with a layer ID equal to nuh_layer_id, in the case that the value of NumSubLayersInLayerInOLS[i][GeneralLayerIdx[nuh_layer_id]] is 0.

[0325] (Constraint 14) Alternatively, for any layer with a layer ID equal to nuh_layer_id, the value of NumSubLayersInLayerInOLS[i][GeneralLayerIdx[nuh_layer_id]] may be forced to be greater than 0 for any i value in the range 0 to vps_max_layers_minus1, provided that all of the following conditions are true.

[0326] - In the case where the value of sps_video_parameter_set_id is not 0 within the reference SPS.

[0327] - In the reference SPS, sps_poc_msb_cycle_flag has a second value (e.g., 0).

[0328] -vps_independent_layer_flag[GeneralLayerIdx[nuh_layer_id]] has a second value (e.g., 0).

[0329] -SPS is the case where at least one VCL NAL cell of a layer has a layer ID equal to nuh_layer_id and the value of NumSubLayersInLayerInOLS[i][GeneralLayerIdx[nuh_layer_id]] is 0.

[0330] As described above, according to embodiment 6 of this disclosure, when there is no POC MSB information for the IRAP screen (and the GDR screen with an associated ph_recovery_poc_cnt value equal to 0), it is possible to restrict the extraction of the non-output layer, such that after extraction, only the IRAP screen (and the GDR screen with an associated ph_recovery_poc_cnt value equal to 0) exists in the non-output layer. Therefore, when the non-output layer only includes the IRAP screen (and the GDR screen with an associated ph_recovery_poc_cnt value equal to 0) (see...), Figure 9 When the decoder cannot find the previous image in the decoding order of the IRAP image (and GDR image) (i.e., prevTid0Pic), it can resolve the problem of incorrectly derived POC of the IRAP image (and GDR image).

[0331] Implementation Method 7

[0332] According to Embodiment 7 of this disclosure, when the current frame is not a CLVSS (Cyclic Layer Video Sequence Start) frame (that is, the current frame is not the first frame of the CLVS (Cyclic Layer Video Sequence)) and there is no POC MSB information signaled for the current frame (i.e., ph_poc_msb_cycle_present_flag == 0), it is possible to constrain the existence of a previous frame (i.e., prevTid0Pic) designated as the current frame in the decoding order. Furthermore, in this case, the designated frame can be constrained to be the same as the frame designated by the encoder for the current frame.

[0333] The implementation method 7 described above can be achieved by modifying the POC decoding process described with reference to Table 2 above. The specific implementation method is as follows.

[0334] (1) Implementation method 5

[0335] According to implementation method 5 of this disclosure, the POC decoding process described above with reference to Table 2 can be modified as follows.

[0336] When ph_poc_msb_cycle_present_flag has a second value (e.g., 0) and the current frame is not a CLVSS (Cycated Layer Video Sequence Start) frame, the variables prevPicOrderCntLsb and prevPicOrderCntMsb can be derived as follows.

[0337] - Assume that prevTid0Pic has a nuh_layer_id equal to the nuh_layer_id of the current frame, a TemporalId equal to 0 and ph_non_ref_pic_flag, and is the previous frame in the decoding order, rather than a RASL frame or a RADL frame.

[0338] -prevTid0Pic is constrained to exist, and the POC difference between prevTid0Pic and the current frame is constrained to be less than MaxPicOrderCntLsb / 2.

[0339] - The variable prevPicOrderCntLsb is set to equal ph_pic_order_cnt_lsb of prevTid0Pic.

[0340] - The variable prevPicOrderCntMsb is set to be equal to the PicOrderCntMsb of prevTid0Pic.

[0341] As described above, according to embodiment 7 of this disclosure, when the current frame is not a CLVSS frame and there is no POC MSB information signaled for the current frame, it is possible to constrain the existence of a previous frame (i.e., prevTid0Pic) designated as the current frame in decoding order, and the designated frame can be constrained to be equal to the frame designated by the encoder. Therefore, when the non-output layer only includes IRAP frames and / or GDR frames with an associated ph_recovery_poc_cnt value equal to 0 (see... Figure 9 When the decoder cannot find the previous frame in the decoding order of the IRAP frame and / or GDR frame (i.e., prevTid0Pic), it can resolve the problem of incorrectly derived POC of the IRAP frame and / or GDR frame.

[0342] Implementation Method 8

[0343] According to Embodiment 8 of this disclosure, under certain conditions, the extraction process can be modified so that images with a TemporalId equal to 0 in the layer are not removed. The predetermined conditions are as follows.

[0344] - (Condition 1) is the case where the layer is an independent layer.

[0345] - (Condition 2) is not an output layer.

[0346] - (Condition 3) In the following cases: there is no POC MSB information for the frame in the layer, or there is no POC MSB information for the IRAP frame in the layer or the GDR frame with an associated ph_recovery_poc_cnt value equal to 0. Here, when the value of ph_recovery_poc_cnt associated with the GDR frame is 0, the GDR frame itself can be a recovery point frame.

[0347] Therefore, as in Figure 9 In the example, since the frame with a TemporalId equal to 0 that was not removed during the extraction process can be used as the previous frame of the current frame (i.e., prevTid0Pic), the problem of the POC of the current frame not being correctly derived can be solved.

[0348] Implementation Method 9

[0349] According to Embodiment 9 of this disclosure, when an IRAP screen or a GDR screen with an associated ph_recovery_poc_cnt value equal to 0 is included in a layer that is not an independent output layer, and other layers only require inter-layer prediction of the IRAP screen and the GDR screen, and the distance between the IRAP screen or the GDR screen and the previous IRAP screen or the previous GDR screen is greater than or equal to MaxPicOrderCntLsb / 2, a step can be constrained to include a signal notifying the IRAP screen or the GDR screen of the POC MSB information. Here, when the value of ph_recovery_poc_cnt associated with the GDR screen is 0, the GDR screen itself can be a recovery point screen.

[0350] The above implementation method 9 can be implemented by modifying the semantics of each of sps_poc_msb_cycle_flag and ph_poc_cycle_present_flag. Therefore, implementation methods 1 to 3 described above can be modified and applied.

[0351] (1) Modified implementation method 1

[0352] Specifically, in order to implement implementation method 9, constraints 3 and 4 of implementation method 1 can be modified as follows.

[0353] (Revised Constraint 3) When all of the following conditions are true, ph_poc_msb_cycle_present_flag can be forced to have a first value (e.g., 1).

[0354] - The case where the associated screen is an IRAP screen or a GDR screen with an associated ph_recovery_poc_cnt value equal to 0, and the screen header includes ph_poc_msb_cycle_present_flag.

[0355] - The case where the image associated with the image header is not a CLVSS image.

[0356] The case where the value of -sps_video_parameter_set_id is not 0.

[0357] - For all i values ​​in the range of 0 to vps_max_layers_minus1, the value of vps_max_tid_il_ref_pics_plus1[i][GeneralLayerIdx[nuh_layer_id]] is 0.

[0358] (Modified Constraint 4) Alternatively, ph_poc_msb_cycle_present_flag may be forced to have a first value (e.g., 1) when all of the following conditions are true.

[0359] - The case where the associated screen is an IRAP screen or a GDR screen with an associated ph_recovery_poc_cnt value equal to 0, and the screen header includes ph_poc_msb_cycle_present_flag.

[0360] - The case where the image associated with the image header is not a CLVSS image.

[0361] The case where the value of -sps_video_parameter_set_id is not 0.

[0362] - For all i values ​​in the range from 0 to vps_max_layers_minus1, the value of vps_max_tid_il_ref_pics_plus1[i][GeneralLayerIdx[nuh_layer_id]] is 0 and the value of vps_independent_layer_flag[GeneralLayerIdx[nuh_layer_id]] is 0.

[0363] (2) Modified implementation method 2

[0364] In addition, in order to implement implementation method 9, constraints 7 and 8 of implementation method 2 can be modified as follows.

[0365] (Revised Constraint 7) When all of the following conditions are true, ph_poc_msb_cycle_present_flag can be forced to have a first value (e.g., 1).

[0366] - The case where the associated screen is an IRAP screen or a GDR screen with an associated ph_recovery_poc_cnt value equal to 0, and the screen header includes ph_poc_msb_cycle_present_flag.

[0367] - The case where the image associated with the image header is not a CLVSS image.

[0368] The case where the value of -sps_video_parameter_set_id is not 0.

[0369] - For all i values ​​in the range 0 to TotalNumOlss-1, the value of NumSubLayersInLayerInOLS[i][GeneralLayerIdx[nuh_layer_id]] is 0.

[0370] (Revised Constraint 8) Alternatively, ph_poc_msb_cycle_present_flag may be forced to have a first value (e.g., 1) when all of the following conditions are true.

[0371] - The case where the associated screen is an IRAP screen or a GDR screen with an associated ph_recovery_poc_cnt value equal to 0, and the screen header includes ph_poc_msb_cycle_present_flag.

[0372] - The case where the image associated with the image header is not a CLVSS image.

[0373] The case where the value of -sps_video_parameter_set_id is not 0.

[0374] - For all i values ​​in the range of 0 to TotalNumOlss-1, the value of NumSubLayersInLayerInOLS[i][GeneralLayerIdx[nuh_layer_id]] is 0 and the value of vps_independent_layer_flag[GeneralLayerIdx[nuh_layer_id]] is 0.

[0375] (3) Modified implementation method 3

[0376] In addition, in order to implement implementation method 9, constraints 11 and 12 of implementation method 3 can be modified as follows.

[0377] (Revised Constraint 11) When all of the following conditions are true, ph_poc_msb_cycle_present_flag can be forced to have a first value (e.g., 1).

[0378] - The case where the associated screen is an IRAP screen or a GDR screen with an associated ph_recovery_poc_cnt value equal to 0, and the screen header includes ph_poc_msb_cycle_present_flag.

[0379] - The case where the image associated with the image header is not a CLVSS image.

[0380] The case where the value of -sps_video_parameter_set_id is not 0.

[0381] The case where -NumSubLayersInLayerInOLS[i][GeneralLayerIdx[nuh_layer_id]] has a value of 0.

[0382] (Revised Constraint 12) Alternatively, ph_poc_msb_cycle_present_flag may be forced to have a first value (e.g., 1) when all of the following conditions are true.

[0383] - The case where the associated screen is an IRAP screen or a GDR screen with an associated ph_recovery_poc_cnt value equal to 0, and the screen header includes ph_poc_msb_cycle_present_flag.

[0384] - The case where the image associated with the image header is not a CLVSS image.

[0385] The case where the value of -sps_video_parameter_set_id is not 0.

[0386] The case where -NumSubLayersInLayerInOLS[i][GeneralLayerIdx[nuh_layer_id]] has a value of 0 and vps_independent_layer_flag[GeneralLayerIdx[nuh_layer_id]] has a value of 0.

[0387] The above implementation method 9 can be achieved by modifying the semantics of ph_poc_cycle_present_flag. The specific implementation method is as follows.

[0388] (4) Implementation method 6

[0389] According to implementation 6 of this disclosure, constraints 15 to 17 below can be added to the reference above. Figure 11 The semantics of ph_poc_msb_cycle_present_flag are described.

[0390] (Constraint 15) ph_poc_msb_cycle_present_flag may be forced to have a first value (e.g., 1) when all of the following conditions are true.

[0391] - The case where the associated screen is an IRAP screen or a GDR screen with an associated ph_recovery_poc_cnt value equal to 0, and the screen header includes ph_poc_msb_cycle_present_flag.

[0392] - The screen is not a CLVSS screen.

[0393] - The POC difference between the frame and the IRAP frame that immediately precedes it in the decoding order, or the GDR frame with an associated ph_recovery_poc_cnt value equal to 0, is less than or equal to MaxPicOrderCntLsb / 2.

[0394] The case where the value of -sps_video_parameter_set_id is not 0 and the value of NumSubLayersInLayerInOLS[TargetOlsIdx][GeneralLayerIdx[nuh_layer_id]] is 0.

[0395] (Constraint 16) Alternatively, ph_poc_msb_cycle_present_flag may be forced to have a first value (e.g., 1) when all of the following conditions are true.

[0396] - The case where the associated screen is an IRAP screen or a GDR screen with an associated ph_recovery_poc_cnt value equal to 0, and the screen header includes ph_poc_msb_cycle_present_flag.

[0397] - The screen is not a CLVSS screen.

[0398] - The POC difference between the frame and the IRAP frame that immediately precedes it in the decoding order, or the GDR frame with an associated ph_recovery_poc_cnt value equal to 0, is less than or equal to MaxPicOrderCntLsb / 2.

[0399] The value of -sps_video_parameter_set_id is not 0, the value of NumSubLayersInLayerInOLS[TargetOlsIdx][GeneralLayerIdx[nuh_layer_id]] is 0, and vps_independent_layer_flag[GeneralLayerIdx[nuh_layer_id]] has a first value (e.g., 1).

[0400] (Constraint 17) Alternatively, ph_poc_msb_cycle_present_flag may be forced to have a first value (e.g., 1) when all of the following conditions are true.

[0401] - The case where the associated screen is an IRAP screen or a GDR screen with an associated ph_recovery_poc_cnt value equal to 0, and the screen header includes ph_poc_msb_cycle_present_flag.

[0402] - The screen is not a CLVSS screen.

[0403] - The POC difference between the frame and the IRAP frame that immediately precedes it in the decoding order, or the GDR frame with an associated ph_recovery_poc_cnt value equal to 0, is less than or equal to MaxPicOrderCntLsb / 2.

[0404] - In the following cases: the value of sps_video_parameter_set_id is not 0, for i values ​​in the range of 0 to TotalNulOlss-1, the value of NumSubLayersInLayerInOLS[TargetOlsIdx][GeneralLayerIdx[nuh_layer_id]] is 0, and vps_independent_layer_flag[GeneralLayerIdx[nuh_layer_id]] has a first value (e.g., 1).

[0405] The implementation method 9 described above can be achieved by modifying the POC decoding process described in Table 2 above. The specific implementation method is as follows.

[0406] (5) Implementation method 7

[0407] According to implementation method 7 of this disclosure, the POC decoding process described with reference to Table 2 can be modified as follows.

[0408] When ph_poc_msb_cycle_present_flag has a second value (e.g., 0) and the current frame is not a CLVSS (Cycated Layer Video Sequence Start) frame, the variables prevPicOrderCntLsb and prevPicOrderCntMsb can be derived as follows.

[0409] - Assume that prevTid0Pic has a nuh_layer_id equal to the nuh_layer_id of the current frame, a TemporalId equal to 0 and ph_non_ref_pic_flag, and is the previous frame in the decoding order, rather than a RASL frame or a RADL frame.

[0410] -prevTid0Pic is constrained to exist, and the POC difference between prevTid0Pic and the current frame is constrained to be less than MaxPicOrderCntLsb / 2.

[0411] - The variable prevPicOrderCntLsb can be set to equal ph_pic_order_cnt_lsb of prevTid0Pic.

[0412] - The variable prevPicOrderCntMsb is set to be equal to the PicOrderCntMsb of prevTid0Pic.

[0413] As described above, according to Embodiment 9 of this disclosure, for IRAP screens and / or GDR screens with an associated ph_recovery_poc_cnt value equal to 0, the step of signaling POC MSB information can be forced under predetermined conditions. In this case, the distance between the IRAP screen and / or GDR screen and the previous IRAP screen or the previous GDR screen can be less than or equal to MaxPicOrderCntLsb / 2. Therefore, when the non-output layer only includes IRAP screens and / or GDR screens (see...), Figure 9 When the decoder cannot find the previous frame in the decoding order of the IRAP frame and / or GDR frame (i.e., prevTid0Pic), it can resolve the problem of incorrectly derived POC of the IRAP frame and / or GDR frame.

[0414] In the following, the image encoding / decoding method according to embodiments of the present disclosure will be described in detail with reference to the accompanying drawings.

[0415] Figure 12 This is a flowchart illustrating an image encoding method according to an embodiment of the present disclosure.

[0416] Figure 12 Image encoding methods can be derived from Figure 2 or Figure 5 The image encoding device performs the steps. For example, steps S1210 to S1230 can be performed by prediction unit 180, 520-0 or 520-1 and entropy encoder 190, 540-0 or 540-1.

[0417] Reference Figure 12 The image encoding device can determine the POC (Picture Sequence Count) and MSB (Most Significant Bit) of the current frame (S1210).

[0418] In one example, the POC MSB can be determined based on the presence of first information specifying the value of the POC MSB cycle for the current frame. For instance, when first information exists (e.g., ph_poc_msb_cycle_val), the POC MSB can be derived by multiplying the value of the POC MSB cycle by the maximum value of the POC LSB (least significant bit) of the current frame (e.g., MaxPicOrderCntLsb). In contrast, when first information is absent and the current frame is a CLVSS (Catalyst Video Sequence Start) frame, the POC MSB can be derived as 0. In another case, as described above with reference to Table 2, the POC MSB can be derived based on the POC MSB of the previous frame preceding the current frame in decoding order (e.g., prevPicOrderCntMsb).

[0419] The image encoding device can deduce the POC of the current frame based on the POC MSB determined in step S1210 (S1220). In one example, the POC (e.g., PicOrderCntMsb) can be deduced by adding the POC LSB (e.g., ph_pic_order_cnt_lsb) to the POC MSB (e.g., PicOrderCntMsb).

[0420] The image encoding device can encode information about the POC (S1230).

[0421] Information about the POC may include flags indicating the existence of a first piece of information specifying the value of the specified POC MSB cycle (e.g., sps_poc_msb_cycle_flag and ph_poc_msb_cycle_present_flag). Additionally, based on the existence of this first piece of information (e.g., sps_poc_msb_cycle_flag == 1 && ph_poc_msb_cycle_present_flag == 1), information about the POC may also include first pieces of information specifying the value of the POC MSB cycle (e.g., ph_poc_msb_cycle_val).

[0422] In one example, based on the fact that the information about the POC does not include the first information and the current frame is not a CLVSS (Cyclic Layer Video Sequence Start) frame, the POC difference between the current frame and a previous frame (e.g., Tid0Pic) preceding the current frame in decoding order can be less than half the maximum value of the current frame's POC LSB (Least Significant Bit) (e.g., MaxPicOrderCntLsb). In this case, the time identifier (e.g., TemporalId) of the previous frame can be 0. Additionally, the layer identifier (e.g., nuh_layer_id) for the NAL unit of the previous frame can be equal to the layer identifier value for the NAL unit of the current frame, and the previous frame can be used as a reference frame (e.g., ph_non_ref_pic_flag = 0) and can be neither a RASL (Random Access Skip Preamble) frame nor a RADL (Random Access Decodeable Preamble) frame. The above implementation can be defined as in Implementation 7.

[0423] In one implementation, based on whether the current frame is an IRAP (Intra-Frame Random Access Point) frame or a GDR frame serving as a recovery point frame (i.e., a GDR frame with an associated ph_recovery_poc_cnt_ of equal to 0), the first information can be encoded in the frame header. In this case, the IRAP frame or GDR frame may not be a CLVSS (Coding Layer Video Sequence Start) frame in an independent non-output layer. Additionally, IRAP-only frames and GDR frames in the current layer, including the current frame, can be used for inter-layer prediction by a layer different from the current layer. Furthermore, the distance between the current frame and a previous IRAP frame or GDR frame serving as a recovery point frame that precedes the current frame in decoding order can be greater than or equal to half the maximum value of the POC LSB (e.g., MaxPicOrderCntLsb).

[0424] The above implementation can be defined as a constraint on bitstream consistency in Implementation 9 of this disclosure. That is, as an IRAP frame included in a layer that is not an independent output layer, or a GDR frame with an associated ph_recovery_poc_cnt value equal to 0, other layers only need the IRAP frame and GDR frame for inter-layer prediction, and when the distance between the IRAP frame or GDR frame and the previous IRAP frame or GDR frame is greater than or equal to MaxPicOrderCntLsb / 2, it can be constrained to include a step of signaling POCMSB information to the IRAP frame or GDR frame. Here, when the value of ph_recovery_poc_cnt associated with the GDR frame is 0, the GDR frame itself can be a recovery point frame.

[0425] The information about the POC encoded in step S1230 can be sent to the image decoding device via a bitstream. In this case, the bitstream can be stored in a non-transitory computer-readable recording medium.

[0426] Figure 13 This is a flowchart illustrating an image decoding method according to an embodiment of the present disclosure.

[0427] Figure 13 Image decoding methods can be derived from Figure 3 or Figure 6 The image decoding device performs the steps. For example, steps S1310 to S1330 can be performed by entropy decoder 210, 610-0 or 610-1 and prediction unit 260, 630-0 or 630-1.

[0428] Reference Figure 13 The image decoding device can determine whether first information exists in the frame header to specify the value of the POC (Frame Sequence Count) MSB (Most Significant Bit) cycle for the current frame (S1310). In this disclosure, the first information may be referred to as POC MSB information.

[0429] In one example, the image decoding device can determine whether first information (e.g., ph_poc_msb_cycle_flag) exists in the frame header based on sps_poc_msb_cycle_flag obtained from the SPS (Sequence Parameter Set). For example, when sps_poc_msb_cycle_flag has a first value (e.g., 1), the image decoding device can determine whether first information exists in the frame header based on ph_poc_msb_cycle_present_flag obtained from the frame header (e.g., exists when ph_poc_msb_cycle_present_flag is 1 / does not exist when ph_poc_msb_cycle_present_flag is 0). In contrast, when sps_poc_msb_cycle_flag has a second value (e.g., 0), the image decoding device can determine that first information does not exist in the frame header.

[0430] In one implementation, the first information may be present in the frame header, based on whether the current frame is an IRAP (Intra-Frame Random Access Point) frame or a GDR frame that serves as a recovery point frame (i.e., a GDR frame with an associated ph_recovery_poc_cnt_ of equal to 0). In this case, the IRAP frame or GDR frame may not be a CLVSS (Coding Layer Video Sequence Start) frame in an independent non-output layer. Furthermore, IRAP-only frames and GDR frames in the current layer, including the current frame, may be used for inter-layer prediction by a layer different from the current layer. Additionally, the distance between the current frame and a previous IRAP frame or GDR frame that serves as a recovery point frame preceding the current frame in decoding order may be greater than or equal to half the maximum value of the POC LSB (e.g., MaxPicOrderCntLsb).

[0431] The above implementation can be defined as a constraint on bitstream consistency in Implementation 9 of this disclosure. That is, as an IRAP frame included in a layer that is not an independent output layer, or a GDR frame with an associated ph_recovery_poc_cnt value equal to 0, other layers only need the IRAP frame and GDR frame for inter-layer prediction, and when the distance between the IRAP frame or GDR frame and the previous IRAP frame or GDR frame is greater than or equal to MaxPicOrderCntLsb / 2, it can be constrained to include a step of signaling POCMSB information to the IRAP frame or GDR frame. Here, when the value of ph_recovery_poc_cnt associated with the GDR frame is 0, the GDR frame itself can be a recovery point frame.

[0432] The image decoding device can determine the POC MSB of the current frame based on the presence or absence of first information in the frame header (S1320). In one example, when the first information (e.g., ph_poc_msb_cycle_val) is present in the frame header, the POC MSB can be derived by multiplying the value of the POC MSB cycle by the maximum value of the POC LSB (least significant bit) of the current frame (e.g., MaxPicOrderCntLsb). In contrast, when the first information is absent and the current frame is a CLVSS (Cycated Layer Video Sequence Start) frame, the POC MSB can be derived as 0. In another case, as described above with reference to Table 2, the POC MSB can be derived based on the POC MSB of the previous frame preceding the current frame in the decoding order (e.g., prevPicOrderCntMsb).

[0433] The image decoding device can deduce the POC of the current frame based on the POC MSB determined in step S1320 (S1330). In one example, the POC (e.g., PicOrderCntMsb) can be deduced by adding the POC LSB (e.g., ph_pic_order_cnt_lsb) of the current frame to the POC MSB (e.g., PicOrderCntMsb).

[0434] In one example, based on the fact that the first information is not present in the frame header and the current frame is not a CLVSS frame, the POC difference between the current frame and the previous frame (e.g., Tid0Pic) that precedes the current frame in decoding order can be less than half the maximum value of the current frame's POC LSB (e.g., MaxPicOrderCntLsb). In this case, the time identifier (e.g., TemporalId) of the previous frame can be 0. Additionally, the layer identifier (e.g., nuh_layer_id) for the NAL unit of the previous frame can be equal to the layer identifier for the NAL unit of the current frame, and the previous frame can be used as a reference frame (e.g., ph_non_ref_pic_flag = 0) and can be neither a RASL (Random Access Skip Preamble) frame nor a RADL (Random Access Decodeable Preamble) frame. The above implementation can be defined as in Implementation 7.

[0435] As described above, based on the reference above... Figure 12 and Figure 13 The image encoding / decoding method according to embodiments of this disclosure, for IRAP frames and / or GDR frames with an associated ph_recovery_poc_cnt value equal to 0, can force the step of signaling POC MSB information (e.g., ph_poc_msb_cycle_val) under predetermined conditions. In this case, the distance between the IRAP frame and / or GDR frame and the previous IRAP frame or GDR frame can be less than or equal to MaxPicOrderCntLsb / 2. Therefore, when the non-output layer only includes IRAP frames and / or GDR frames (see...), the process can be more efficient. Figure 9 When the decoder cannot find the previous frame in the decoding order of the IRAP frame and / or GDR frame (i.e., prevTid0Pic), it can resolve the problem of incorrectly derived POC of the IRAP frame and / or GDR frame.

[0436] Although the exemplary methods of this disclosure described above are represented as a series of operations for clarity of description, they are not intended to limit the order in which the steps are performed, and these steps may be performed simultaneously or in different orders if necessary. To implement the methods according to this disclosure, the described steps may also include other steps, including steps in addition to some steps, or additional steps in addition to some steps.

[0437] In this disclosure, an image encoding device or an image decoding device that performs a predetermined operation (step) can perform an operation (step) that confirms the execution conditions or circumstances of the corresponding operation (step). For example, if it is described that a predetermined operation is performed when predetermined conditions are met, the image encoding device or the image decoding device can perform the predetermined operation after determining whether the predetermined conditions are met.

[0438] The various embodiments of this disclosure are not a list of all possible combinations and are intended to describe representative aspects of this disclosure; the matters described in the various embodiments may be applied independently or in combination of two or more.

[0439] Various embodiments of this disclosure can be implemented in hardware, firmware, software, or a combination thereof. When this disclosure is implemented in hardware, it can be implemented using application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers, microprocessors, etc.

[0440] Furthermore, the image decoding and image encoding devices using embodiments of this disclosure can be included in multimedia broadcasting transmission and reception devices, mobile communication terminals, home theater video devices, digital cinema video devices, surveillance cameras, video chat devices, real-time communication devices such as video communication, mobile streaming devices, storage media, cameras, video-on-demand (VoD) service providers, OTT (over-the-top) video devices, internet streaming service providers, three-dimensional (3D) video devices, video telephony devices, medical video devices, etc., and can be used to process video signals or data signals. For example, OTT video devices can include game consoles, Blu-ray players, internet access televisions, home theater systems, smartphones, tablet PCs, digital video recorders (DVRs), etc.

[0441] Figure 14 This is a diagram illustrating a content flow system to which embodiments of the present disclosure can be applied.

[0442] like Figure 14As shown, the content streaming system applying the embodiments of this disclosure may mainly include an encoding server, a streaming server, a network server, a media storage device, a user device, and a multimedia input device.

[0443] The encoding server compresses content input from multimedia input devices such as smartphones, cameras, and camcorders into digital data to generate a bitstream and then sends the bitstream to the streaming server. As another example, when multimedia input devices such as smartphones, cameras, and camcorders directly generate bitstreams, the encoding server can be omitted.

[0444] The bitstream can be generated by an image encoding method or image encoding device applying the embodiments of this disclosure, and the stream server can temporarily store the bitstream during the sending or receiving of the bitstream.

[0445] A streaming server sends multimedia data to a user's device based on a request from a web server, and the web server acts as a medium for informing the user of the service. When a user requests a service from the web server, the web server can deliver it to the streaming server, and the streaming server can send the multimedia data to the user. In this scenario, the content streaming system may include a separate control server. In this case, the control server is used to control the commands / responses between devices in the content streaming system.

[0446] A streaming server can receive content from media storage devices and / or encoding servers. For example, when receiving content from an encoding server, the content can be received in real time. In this case, to provide a smooth streaming service, the streaming server can store the bitstream for a predetermined period of time.

[0447] Examples of user devices may include mobile phones, smartphones, laptop computers, digital broadcasting terminals, personal digital assistants (PDAs), portable multimedia players (PMPs), navigation devices, board PCs, tablet PCs, ultrabooks, wearable devices (e.g., smartwatches, smart glasses, head-mounted displays), digital televisions, desktop computers, digital signage, etc.

[0448] In a content streaming system, each server can operate as a distributed server, in which case the data received from each server can be distributed.

[0449] The scope of this disclosure includes software or machine-executable commands (e.g., operating systems, applications, firmware, programs, etc.) for enabling the operation of methods according to various embodiments to be executed on a device or computer, and non-transitory computer-readable media having such software or commands stored thereon and executable on a device or computer.

[0450] Industrial applicability

[0451] The embodiments disclosed herein can be used to encode or decode images.

Claims

1. An image decoding method performed by an image decoding device, the image decoding method comprising the following steps: Determine if the first piece of information regarding the value of the most significant bit (MSB) of the current frame sequence count (POC) loop exists in the frame header; The POC MSB of the current screen is determined based on whether the first information exists in the screen header; as well as The POC of the current frame is derived based on the determined POC MSB. Specifically, based on the fact that the first information does not exist in the frame header and the current frame is not the starting CLVSS frame of the coding layer video sequence, the POC difference between the current frame and the previous frame preceding the current frame in the decoding order is less than half the maximum value of the least significant bit (LSB) of the current frame's POC, and Wherein, based on whether the current frame is an Intra-Frame Random Access Point (IRAP) frame or a progressively decoded refreshed GDR frame as a recovery point frame, the first information is present in the frame header.

2. The image decoding method according to claim 1, wherein, The time identifier for the previous frame is 0.

3. The image decoding method according to claim 1, in, The layer identifier of the NAL unit in the previous frame is equal to the layer identifier of the NAL unit in the current frame, and The preceding frame can be used as a reference frame and is neither a randomly accessed skip RASL frame nor a randomly accessed decodeable RADL frame.

4. The image decoding method according to claim 1, wherein, The IRAP screen or the GDR screen is not a CLVSS screen in an independent non-output layer.

5. The image decoding method according to claim 1, wherein, The IRAP-only images and GDR images in the current layer, including the current image, are used for inter-layer prediction by a layer different from the current layer.

6. The image decoding method according to claim 1, wherein, The distance between the current frame and the previous IRAP frame or GDR frame that serves as the recovery point frame, which precedes the current frame in the decoding order, is greater than or equal to half the maximum value of the POCLSB.

7. An image encoding method performed by an image encoding device, the image encoding method comprising the following steps: Determine the most significant bit (MSB) of the current screen sequence count (POC); The POC of the current screen is derived based on the determined POC MSB; as well as Information about the derived Proof of Concept (POC) is encoded. Wherein, the first information based on the value of the POC MSB cycle of the specified current frame is not encoded and the current frame is not the starting CLVSS frame of the coding layer video sequence, the POC difference between the current frame and the previous frame preceding the current frame in the decoding order is less than half the maximum value of the least significant bit (LSB) of the current frame's POC, and Specifically, based on whether the current frame is an Intra-Frame Random Access Point (IRAP) frame or a progressively decoded refreshed GDR frame that serves as a recovery point frame, the first information is encoded in the frame header.

8. The image encoding method according to claim 7, wherein, The time identifier for the previous frame is 0.

9. The image encoding method according to claim 7, in, The layer identifier of the NAL unit in the previous frame is equal to the layer identifier of the NAL unit in the current frame, and The preceding frame can be used as a reference frame and is neither a randomly accessed skip RASL frame nor a randomly accessed decodeable RADL frame.

10. The image encoding method according to claim 7, wherein, The IRAP screen or the GDR screen is not a CLVSS screen in an independent non-output layer.

11. The image encoding method according to claim 7, wherein, The IRAP-only images and GDR images in the current layer, including the current image, are used for inter-layer prediction by a layer different from the current layer.

12. A method for transmitting a bitstream generated by an image encoding method, the image encoding method comprising the following steps: Determine the most significant bit (MSB) of the current screen sequence count (POC); The POC of the current screen is derived based on the determined POC MSB; as well as Information about the derived Proof of Concept (POC) is encoded. Wherein, the first information based on the value of the POC MSB cycle of the specified current frame is not encoded and the current frame is not the starting CLVSS frame of the coding layer video sequence, the POC difference between the current frame and the previous frame preceding the current frame in the decoding order is less than half the maximum value of the least significant bit (LSB) of the current frame's POC, and Specifically, based on whether the current frame is an Intra-Frame Random Access Point (IRAP) frame or a progressively decoded refreshed GDR frame that serves as a recovery point frame, the first information is encoded in the frame header.

Images