Image encoding / decoding methods and devices for signaling APS identifiers, and computer-readable recording media for storing bitstreams.

Abstract

An image encoding / decoding method and apparatus for signaling an adaptive parameter set (APS) identifier is provided, as well as a method for transmitting a bit stream. The image decoding method according to this disclosure may include the following steps: obtaining APS parameter type information indicating the type of APS parameters signaled via the APS; after obtaining the APS parameter type information, obtaining APS identifier information indicating the APS; and reconstructing the image based on the APS identifier information.

Description

Technical Field

[0001] This disclosure relates to an image encoding / decoding method and apparatus, and more specifically, to an image encoding and decoding method and apparatus for signaling an identifier of an adaptive parameter set (APS), and a recording medium for storing a bitstream generated by the image encoding method / apparatus of this disclosure. Background Technology

[0002] Recently, there has been an increasing demand across various fields for high-resolution and high-quality images, such as high-definition (HD) and ultra-high-definition (UHD) images. With the increase in image data resolution and quality, the amount of information or bits transmitted increases relative to existing image data. This increase in the amount of information or bits transmitted leads to increased transmission and storage costs.

[0003] Therefore, efficient image compression techniques are needed to effectively transmit, store, and reproduce information about high-resolution and high-quality images. Summary of the Invention

[0004] Technical issues

[0005] The purpose of this disclosure is to provide an image encoding / decoding method and apparatus with improved encoding / decoding efficiency.

[0006] Another object of this disclosure is to provide an image encoding / decoding method and apparatus for improving encoding / decoding efficiency by effectively signaling the identifier of the APS.

[0007] Another object of this disclosure is to provide a method for transmitting a bitstream generated by an image encoding method or device according to this disclosure.

[0008] Another object of this disclosure is to provide a recording medium for storing bitstreams generated by an image encoding method or apparatus according to this disclosure.

[0009] Another object of this disclosure is to provide a recording medium that stores a bitstream received, decoded and used to reconstruct an image by an image decoding device according to this disclosure.

[0010] The technical problems solved by this disclosure are not limited to those described above. Other technical problems not described herein will become clear to those skilled in the art through the following description.

[0011] Technical solution

[0012] An image decoding method performed by an image decoding device according to one aspect of this disclosure may include the following steps: obtaining APS parameter type information specifying an APS parameter type signaled by an adaptive parameter set (APS); obtaining APS identifier information specifying the APS after obtaining the APS parameter type information; and reconstructing an image based on the APS identifier information.

[0013] In the image decoding method according to this disclosure, the APS parameter type can be determined as the full-loop filter (ALF) parameter type when the value of the APS parameter type information is 0.

[0014] In the image decoding method according to this disclosure, the APS parameter type can be determined as a Luminance Mapping with Chromaticity Scaling (LMCS) parameter type based on the value of the APS parameter type information being 1.

[0015] In the image decoding method according to this disclosure, the APS parameter type can be determined as a scaling list parameter type based on the value of 2 of the APS parameter type information.

[0016] In the image decoding method according to this disclosure, the APS identifier can be determined as a value in the range of 0 to 7, based on the APS parameter type being either a full-loop filter (ALF) parameter or a scaling list parameter.

[0017] In the image decoding method according to this disclosure, based on the APS parameter type being a Luminance Mapping with Chroma Scaling (LMCS) parameter, the APS identifier can be determined as a value in the range of 0 to 3.

[0018] An image decoding apparatus according to another aspect of this disclosure may include a memory and at least one processor. The at least one processor may: obtain APS parameter type information specifying an APS parameter type signaled via an adaptive parameter set (APS); obtain APS identifier information specifying the APS after obtaining the APS parameter type information; and reconstruct an image based on the APS identifier information.

[0019] An image encoding method performed by an image encoding device according to another aspect of this disclosure may include the following steps: determining an adaptive parameter set (APS) parameter type; determining an APS identifier for a specified APS based on the APS parameter type; encoding APS identifier information for a specified APS identifier after encoding APS parameter type information for the specified APS parameter type; and encoding the image based on the APS identifier information.

[0020] In the image coding method according to this disclosure, the value of the APS parameter type information specifying the APS parameter type can be determined to be 0, based on the APS parameter type being the full loop filter (ALF) parameter.

[0021] In the image encoding method according to this disclosure, based on the APS parameter type being a Luminance Mapping with Chromaticity Scaling (LMCS) parameter, the value of the APS parameter type information specifying the APS parameter type can be determined to be 1.

[0022] In the image encoding method according to this disclosure, based on the APS parameter type being a scaling list parameter, the value of the APS parameter type information specifying the APS parameter type can be determined to be 2.

[0023] In the image coding method according to this disclosure, based on the APS parameter type being either a full-loop filter (ALF) parameter or a scaling list parameter, the APS identifier can be determined as a value in the range from 0 to 3.

[0024] In the image encoding method according to this disclosure, based on the APS parameter type being a Luminance Mapping with Chroma Scaling (LMCS) parameter, the APS identifier can be determined as a value in the range from 0 to 7.

[0025] Additionally, according to another aspect of this disclosure, a computer-readable recording medium can store a bitstream generated by the image encoding device or image encoding method of this disclosure.

[0026] In another aspect of the transmission method according to this disclosure, a bit stream generated by the image encoding method or image encoding device of this disclosure can be transmitted.

[0027] The features briefly outlined above are merely exemplary aspects of the detailed description of this disclosure below and do not limit the scope of this disclosure.

[0028] Beneficial effects

[0029] According to this disclosure, an image encoding / decoding method and apparatus with improved encoding / decoding efficiency can be provided.

[0030] Furthermore, according to this disclosure, an image encoding / decoding method and apparatus can be provided for improving encoding / decoding efficiency by effectively signaling the identifier of the APS.

[0031] Furthermore, according to this disclosure, a method for transmitting a bitstream generated by an image encoding method or device according to this disclosure can be provided.

[0032] Furthermore, according to this disclosure, it is possible to provide a recording medium for storing a bitstream generated by an image encoding method or apparatus according to this disclosure.

[0033] Furthermore, according to this disclosure, a recording medium can be provided that stores a bitstream received, decoded, and used to reconstruct an image by an image decoding device according to this disclosure.

[0034] Those skilled in the art will understand that the effects achievable through this disclosure are not limited to those specifically described above, and that other advantages of this disclosure will become clearer from the detailed description. Attached Figure Description

[0035] Figure 1 This is a view that schematically illustrates a video coding system to which embodiments of the present disclosure are applicable.

[0036] Figure 2 This is a view schematically illustrating an image encoding device to which embodiments of the present disclosure are applicable.

[0037] Figure 3 This is a view schematically illustrating an image decoding device to which embodiments of the present disclosure are applicable.

[0038] Figure 4 An example of an illustrative screen decoding process to which embodiments of this disclosure apply is shown.

[0039] Figure 5 An example of an illustrative screen encoding process to which embodiments of this disclosure are applicable is shown.

[0040] Figure 6 This is a view that illustrates an example of a layered structure used for encoding images / videos.

[0041] Figure 7 This is a view that illustrates an example of a syntax structure used to signal information about the APS and screen header.

[0042] Figures 8 to 16 This is a view illustrating the VPS to which the embodiments of this disclosure are applicable.

[0043] Figures 17 to 18 This is a view illustrating the VPS to which the embodiments of this disclosure are applicable.

[0044] Figure 19 This is a view illustrating a method for decoding an image using an image decoding device according to an embodiment.

[0045] Figure 20 This is a view illustrating a method for encoding an image using an image encoding device according to an embodiment.

[0046] Figure 21 This is a view that displays the APS parameter names based on the APS parameter type.

[0047] Figure 22 This is a view that illustrates an example of a syntax structure used to signal APS identifier information based on the APS parameter type.

[0048] Figure 23 This is an example based on a reference. Figure 22 A view of the operation of the image encoding device described in the embodiments.

[0049] Figure 24 This is an example based on a reference. Figure 22 A view of the operation of the image decoding device described in the embodiments.

[0050] Figure 25 This is a view illustrating a content streaming system to which embodiments of this disclosure are applicable. Detailed Implementation

[0051] In the following, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings to facilitate implementation by those skilled in the art. However, the present disclosure may be implemented in various different forms and is not limited to the embodiments described herein.

[0052] In describing this disclosure, detailed descriptions of relevant known functions or constructions will be omitted if they unnecessarily obscure the scope of this disclosure. In the accompanying drawings, portions irrelevant to the description of this disclosure are omitted, and similar reference numerals are assigned to similar portions.

[0053] In this disclosure, when a component is "connected," "linked," or "coupled" to another component, it may include not only direct connections but also indirect connections where intermediate components exist. Furthermore, when a component "comprises" or "has" other components, unless otherwise stated, it means that other components may be included, not excluded.

[0054] In this disclosure, the terms first, second, etc., are used only for the purpose of distinguishing one component from other components and do not limit the order or importance of the components, unless otherwise stated. Accordingly, within the scope of this disclosure, a first component in one embodiment may be referred to as a second component in another embodiment, and similarly, a second component in one embodiment may be referred to as a first component in another embodiment.

[0055] In this disclosure, the components are distinguished from each other to clearly describe each feature, but this does not mean that the components must be separate. That is, multiple components may be integrated into a single hardware or software unit, or a single component may be distributed and implemented across multiple hardware or software units. Therefore, unless otherwise specified, implementations of these integrated or distributed components are included within the scope of this disclosure.

[0056] In this disclosure, the components described in the various embodiments are not necessarily essential components, and some components may be optional. Therefore, embodiments consisting of a subset of the components described in the embodiments are also included within the scope of this disclosure. Furthermore, embodiments that include other components besides those described in the various embodiments are also included within the scope of this disclosure.

[0057] This disclosure relates to the encoding and decoding of images. Unless redefined in this disclosure, the terms used herein may have the general meaning commonly used in the art to which this disclosure pertains.

[0058] In this disclosure, a "picture" generally refers to a unit representing an image within a specific time period, while a slice / tile is a coding unit that constitutes part of a picture. A picture can be composed of one or more slices / tiles. Furthermore, a slice / tile may include one or more coding tree units (CTUs).

[0059] In this disclosure, "pixel" or "pixel" can refer to the smallest unit that constitutes a frame (or image). Furthermore, "sample" can be used as a term corresponding to a pixel. A sample can generally represent a pixel or a pixel value, or it can represent only the pixel / pixel value of the luminance component or only the pixel / pixel value of the chrominance component.

[0060] In this disclosure, "unit" can refer to a basic unit of image processing. A unit may include at least one of a specific region of a picture and information associated with that region. In some cases, the term "unit" may be used interchangeably with terms such as "sample array," "block," or "region." Generally, an M×N block may include a set (or array) of samples (or transform coefficients) with M columns and N rows.

[0061] In this disclosure, "current block" can mean one of "current coding block," "current coding unit," "coding target block," "decoding target block," or "processing target block." When performing prediction, "current block" can mean "current prediction block" or "prediction target block." When performing transform (inverse transform) / quantization (dequantization), "current block" can mean "current transform block" or "transform target block." When performing filtering, "current block" can mean "filter target block."

[0062] Furthermore, in this disclosure, unless explicitly stated as a chroma block, "current block" may mean a block that includes both luma component blocks and chroma component blocks, or "the luma block of the current block." The luma component block of the current block can be represented by an explicit description including terms such as "luma block" or "current luma block." Similarly, "the chroma component block of the current block" can be represented by an explicit description including terms such as "chroma block" or "current chroma block."

[0063] In this disclosure, "A or B" can mean "A only", "B only", or "both A and B". In other words, in this disclosure, "A or B" can be interpreted as "A and / or B". For example, in this disclosure, "A, B or C" can mean "A only", "B only", "C only", or "any combination of A, B and C".

[0064] As used in this disclosure, a forward slash ( / ) or a comma can mean "and / or". For example, "A / B" can mean "A and / or B". Therefore, "A / B" can mean "A only", "B only", or "both A and B". For example, "A, B, C" can mean "A, B, or C".

[0065] Furthermore, in this disclosure, "at least one of A and B" may mean "only A", "only B" or "any combination of A and B". Additionally, in this disclosure, "at least one of A or B" or "at least one of A and / or B" may be interpreted as the same as "at least one of A and B".

[0066] Additionally, in this disclosure, "at least one of A, B, and C" means "only A," "only B," "only C," or "A, any combination of A, A, B, and C." Furthermore, "at least one of A, B, or C" or "at least one of A, B, and / or C" means that it can mean "at least one of A, B, and C."

[0067] Furthermore, the parentheses used in this disclosure may mean "for example". Specifically, when describing "prediction (intra-frame prediction)", "intra-frame prediction" may be cited as an example of "prediction". In other words, the "prediction" in this disclosure is not limited to "intra-frame prediction", and "intra-frame prediction" may be cited as an example of "prediction". Additionally, even when describing "prediction (i.e., intra-frame prediction)", "intra-frame prediction" may be cited as an example of "prediction".

[0068] In this disclosure, a technical feature described individually in a single figure may be implemented individually or simultaneously.

[0069] Overview of Video Encoding Systems

[0070] Figure 1 This is a view illustrating a video coding system according to this disclosure.

[0071] The video encoding system according to the embodiment may include an encoding device 10 and a decoding device 20. The encoding device 10 may deliver encoded video and / or image information or data to the decoding device 20 in the form of a file or stream via a digital storage medium or network.

[0072] The encoding device 10 according to an embodiment may include a video source generator 11, an encoding unit 12, and a transmitter 13. The decoding device 20 according to an embodiment may include a receiver 21, a decoding unit 22, and a renderer 23. The encoding unit 12 may be referred to as a video / image encoding unit, and the decoding unit 22 may be referred to as a video / image decoding unit. The transmitter 13 may be included in the encoding unit 12. The receiver 21 may be included in the decoding unit 22. The renderer 23 may include a display, and the display may be configured as a separate device or an external component.

[0073] The video source generator 11 can acquire video / images through a process of capturing, compositing, or generating video / images. The video source generator 11 may include a video / image capture device and / or a video / image generation device. The video / image capture device may include, for example, one or more cameras, a video / image archive including previously captured video / images, etc. The video / image generation device may include, for example, a computer, tablet computer, and smartphone, and can generate video / images (electronically). For example, virtual video / images can be generated by a computer, etc. In this case, the video / image capture process can be replaced by a process of generating related data.

[0074] The encoding unit 12 can encode the input video / image. For compression and encoding efficiency, the encoding unit 12 can perform a series of processes, such as prediction, transformation, and quantization. The encoding unit 12 can output encoded data (encoded video / image information) in the form of a bitstream.

[0075] Transmitter 13 can transmit encoded video / image information or data, output in bitstream form, to receiver 21 of decoding device 20 in the form of a file or stream via digital storage medium or network. Digital storage medium can include various storage media, such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. Transmitter 13 can include elements for generating media files according to a predetermined file format and may include elements for transmission via broadcast / communication networks. Receiver 21 can extract / receive bitstreams from storage medium or network and transmit the bitstreams to decoding unit 22.

[0076] The decoding unit 22 can decode video / images by performing a series of processes corresponding to the operations of the encoding unit 12, such as dequantization, inverse transform, and prediction.

[0077] Renderer 23 can render decoded video / images. The rendered video / images can be displayed on a monitor.

[0078] Overview of Image Encoding Devices

[0079] Figure 2 This is a schematic view illustrating an image encoding device to which embodiments of this disclosure may be applied.

[0080] like Figure 2 As shown, the image encoding device 100 may include an image segmenter 110, a subtractor 115, a transformer 120, a quantizer 130, a dequantizer 140, an inverse transformer 150, an adder 155, a filter 160, a memory 170, an inter-frame predictor 180, an intra-frame predictor 185, and an entropy encoder 190. The inter-frame predictor 180 and the intra-frame predictor 185 may be collectively referred to as "predictors". The transformer 120, quantizer 130, dequantizer 140, and inverse transformer 150 may be included in a residual processor. The residual processor may also include a subtractor 115.

[0081] In some implementations, all or at least some of the components configuring the image encoding device 100 may be configured by a single hardware component (e.g., an encoder or a processor). Furthermore, the memory 170 may include a decoded screen buffer (DPB) and may be configured by a digital storage medium.

[0082] Image segmenter 110 can segment an input image (or picture or frame) input to image encoding device 100 into one or more processing units. For example, a processing unit may be called an encoding unit (CU). Encoding units can be obtained by recursively segmenting encoding tree units (CTUs) or maximum encoding units (LCUs) according to a quadtree / binary tree / tritree (QT / BT / TT) structure. For example, an encoding unit can be segmented into multiple encoding units of greater depth based on a quadtree structure, a binary tree structure, and / or a ternary tree structure. For the segmentation of encoding units, a quadtree structure can be applied first, followed by a binary tree structure and / or a ternary tree structure. The encoding process according to this disclosure can be performed based on the final encoding unit that is no longer segmented. The maximum encoding unit can be used as the final encoding unit, or a deeper encoding unit obtained by segmenting the maximum encoding unit can be used as the final encoding unit. Here, the encoding process may include prediction, transformation, and reconstruction processes, which will be described later. As another example, the processing unit of the encoding process may be a prediction unit (PU) or a transformation unit (TU). Prediction units and transform units can be partitioned or segmented from the final coding unit. Prediction units can be sample prediction units, and transform units can be units used to derive transform coefficients and / or units used to derive residual signals from transform coefficients.

[0083] The predictor (inter-frame predictor 180 or intra-frame predictor 185) can perform prediction on the block to be processed (the current block) and generate a prediction block that includes prediction samples of the current block. The predictor can determine whether to apply intra-frame prediction or inter-frame prediction based on the current block or CU. The predictor can generate various information related to the prediction of the current block and transmit the generated information to the entropy encoder 190. The information about the prediction can be encoded in the entropy encoder 190 and output as a bitstream.

[0084] Intra-predictor 185 can predict the current block by referencing samples in the current frame. Depending on the intra-prediction mode and / or intra-prediction technique, the reference samples may be located among the neighbors of the current block or may be placed separately. Intra-prediction modes may include multiple non-directional modes and multiple directional modes. Non-directional modes may include, for example, DC mode and planar mode. Depending on the level of detail in the prediction direction, directional modes may include, for example, 33 or 65 directional prediction modes. However, this is merely an example, and more or fewer directional prediction modes may be used depending on the settings. Intra-predictor 185 can determine the prediction mode to be applied to the current block by using prediction modes applied to neighboring blocks.

[0085] Inter-frame predictor 180 can deduce the predicted block of the current block based on a reference block (reference sample array) specified by motion vectors on a reference frame. In this case, to reduce the amount of motion information transmitted in inter-frame prediction mode, motion information can be predicted on a block, sub-block, or sample basis based on the correlation of motion information between neighboring blocks and the current block. Motion information may include motion vectors and reference frame indices. Motion information may also include inter-frame prediction direction (L0 prediction, L1 prediction, dual prediction, etc.) information. In the case of inter-frame prediction, neighboring blocks may include spatially neighboring blocks existing in the current frame and temporally neighboring blocks existing in the reference frame. The reference frame including the reference block and the reference frame including the temporally neighboring block may be the same or different. The temporally neighboring block may be referred to as a juxtaposed reference block, a juxtaposed CU (colCU), etc. The reference frame including the temporally neighboring block may be referred to as a juxtaposed frame (colPic). For example, inter-frame predictor 180 can configure a motion information candidate list based on neighboring blocks and generate information indicating which candidate to use to deduce the motion vector and / or reference frame index of the current block. Inter-frame prediction can be performed based on various prediction modes. For example, in skip mode and merge mode, the inter-frame predictor 180 can use motion information from neighboring blocks as motion information for the current block. In skip mode, unlike merge mode, residual signals may not be transmitted. In motion vector prediction (MVP) mode, motion vectors from neighboring blocks can be used as motion vector predictors, and the motion vector of the current block can be signaled by encoding motion vector differences and indicators of the motion vector predictors. The motion vector difference can refer to the difference between the motion vector of the current block and the motion vector predictor.

[0086] The predictor can generate a prediction signal based on various prediction methods and techniques described below. For example, the predictor can apply not only intra-frame prediction or inter-frame prediction, but also both intra-frame prediction and inter-frame prediction simultaneously to predict the current block. A prediction method that simultaneously applies both intra-frame prediction and inter-frame prediction to predict the current block can be called Combined Intra-Frame and Inter-Frame Prediction (CIIP). Furthermore, the predictor can perform Intra-Frame Block Copy (IBC) to predict the current block. Intra-Frame Block Copy can be used for content image / video coding in games, such as Screen Content Coding (SCC). IBC is a method of predicting the current frame using a previously reconstructed reference block in the current frame at a predetermined distance from the current block. When IBC is applied, the position of the reference block in the current frame can be encoded as a vector (block vector) corresponding to the predetermined distance. IBC essentially performs prediction in the current frame, but can be performed similarly to inter-frame prediction because the reference block is derived within the current frame. That is, IBC can use at least one inter-frame prediction technique described in this disclosure.

[0087] The predicted signal generated by the predictor can be used to generate a reconstructed signal or a residual signal. Subtractor 115 generates a residual signal (residual block or residual sample array) by subtracting the predicted signal (predicted block or predicted sample array) output from the predictor from the input image signal (original block or original sample array). The generated residual signal can be transmitted to converter 120.

[0088] Transformer 120 can generate transform coefficients by applying transform techniques to the residual signal. For example, the transform techniques may include at least one of Discrete Cosine Transform (DCT), Discrete Sine Transform (DST), Karhunen-Loève Transform (KLT), Graph-Based Transform (GBT), or Conditional Nonlinear Transform (CNT). Here, GBT refers to a transform obtained from a graph when the relationship information between pixels is represented graphically. CNT refers to a transform obtained based on a prediction signal generated using all previously reconstructed pixels. Furthermore, the transform processing can be applied to square pixel blocks of the same size or to blocks of variable size instead of square.

[0089] Quantizer 130 quantizes the transform coefficients and transmits them to entropy encoder 190. Entropy encoder 190 encodes the quantized signal (information about the quantized transform coefficients) and outputs a bitstream. This information about the quantized transform coefficients can be referred to as residual information. Quantizer 130 can rearrange the block-type quantized transform coefficients into a one-dimensional vector based on the coefficient scan order and generate information about the quantized transform coefficients based on this one-dimensional vector form.

[0090] The entropy encoder 190 can perform various encoding methods, such as exponential Columbus coding, context-adaptive variable-length coding (CAVLC), and context-adaptive binary arithmetic coding (CABAC). The entropy encoder 190 can encode, either together or separately, the information required for video / image reconstruction other than the quantization transform coefficients (e.g., values ​​of syntax elements). The encoded information (e.g., encoded video / image information) can be transmitted or stored in bitstream form at the Network Abstraction Layer (NAL) level. The video / image information may also include information about various parameter sets, such as Adaptive Parameter Set (APS), Picture Parameter Set (PPS), Sequence Parameter Set (SPS), or Video Parameter Set (VPS). Furthermore, the video / image information may also include general constraint information. The signaled information, transmitted information, and / or syntax elements described in this disclosure can be encoded and included in the bitstream through the above encoding process.

[0091] The bitstream can be transmitted over a network or stored in a digital storage medium. The network may include broadcast networks and / or communication networks, and the digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, and SSD. A transmitter (not shown) for transmitting the signal output from the entropy encoder 190 and / or a storage unit (not shown) for storing the signal may be included as internal / external components of the image encoding device 100. Alternatively, a transmitter may be provided as a component of the entropy encoder 190.

[0092] The quantization transform coefficients output from quantizer 130 can be used to generate residual signals. For example, the residual signals (residual blocks or residual samples) can be reconstructed by applying dequantization and inverse transform to the quantization transform coefficients through dequantizer 140 and inverse transformer 150.

[0093] Adder 155 adds the reconstructed residual signal to the prediction signal output from inter-frame predictor 180 or intra-frame predictor 185 to generate a reconstructed signal (reconstructed frame, reconstructed block, reconstructed sample array). If the block to be processed has no residual, such as in the case of applying skip mode, the prediction block can be used as a reconstructed block. Adder 155 can be referred to as a reconstructor or reconstructed block generator. The generated reconstructed signal can be used for intra-frame prediction of the next block to be processed in the current frame, and can be used for inter-frame prediction of the next frame by filtering as described below.

[0094] Furthermore, luminance mapping with chroma scaling (LMCS) is applicable during image encoding and / or reconstruction.

[0095] Filter 160 can improve the subjective / objective image quality by applying filtering to the reconstructed signal. For example, filter 160 can generate a modified reconstructed image by applying various filtering methods to the reconstructed image and store the modified reconstructed image in memory 170, specifically in the DPB of memory 170. Various filtering methods can include, for example, deblocking filtering, sample adaptive offsetting, adaptive loop filtering, bilateral filtering, etc. Filter 160 can generate various filtering-related information and transmit the generated information to entropy encoder 190, as described later in the description of each filtering method. The filtering-related information can be encoded by entropy encoder 190 and output as a bitstream.

[0096] The modified reconstructed frame transmitted to memory 170 can be used as a reference frame in inter-frame predictor 180. When inter-frame prediction is applied by image encoding device 100, prediction mismatch between image encoding device 100 and image decoding device can be avoided and coding efficiency can be improved.

[0097] The DPB of memory 170 can store modified reconstructed frames for use as reference frames in inter-frame predictor 180. Memory 170 can store motion information of blocks from which motion information in the current frame is derived (or encoded) and / or motion information of already reconstructed blocks in the frame. The stored motion information can be transmitted to inter-frame predictor 180 and used as motion information for spatially or temporally neighboring blocks. Memory 170 can store reconstructed samples of reconstructed blocks in the current frame and can transmit the reconstructed samples to intra-frame predictor 185.

[0098] Overview of image decoding devices

[0099] Figure 3 This is a schematic view illustrating an image decoding device to which embodiments of the present disclosure may be applied.

[0100] like Figure 3 As shown, the image decoding device 200 may include an entropy decoder 210, a dequantizer 220, an inverse transformer 230, an adder 235, a filter 240, a memory 250, an inter-frame predictor 260, and an intra-frame predictor 265. The inter-frame predictor 260 and the intra-frame predictor 265 may be collectively referred to as "predictors". The dequantizer 220 and the inverse transformer 230 may be included in a residual processor.

[0101] According to an implementation, all or at least some of the components of the image decoding device 200 can be configured by hardware components (e.g., a decoder or a processor). Furthermore, the memory 250 may include a decoded screen buffer (DPB) or may be configured by a digital storage medium.

[0102] The image decoding device 200, having received a bitstream including video / image information, can perform operations related to... Figure 2 The image is reconstructed by processing corresponding to the processing performed by the image encoding device 100. For example, the image decoding device 200 can perform decoding using a processing unit applied in the image encoding device. Therefore, the decoding processing unit can be, for example, an encoding unit. The encoding unit can be obtained by segmenting a coding tree unit or a maximum coding unit. The reconstructed image signal decoded and output by the image decoding device 200 can be reproduced by a reproduction device (not shown).

[0103] Image decoding device 200 can receive data in bitstream form from... Figure 2The signal output by the image encoding device. The received signal can be decoded by the entropy decoder 210. For example, the entropy decoder 210 can parse the bitstream to derive the information (e.g., video / image information) required for image reconstruction (or picture reconstruction). The video / image information may also include information about various parameter sets, such as adaptive parameter sets (APS), picture parameter sets (PPS), sequence parameter sets (SPS), or video parameter sets (VPS). In addition, the video / image information may also include general constraint information. The image decoding device can also decode the picture based on the information about the parameter sets and / or general constraint information. The information and / or syntax elements notified / received by signals described in this disclosure can be decoded and obtained from the bitstream through the decoding process. For example, the entropy decoder 210 decodes the information in the bitstream based on encoding methods such as exponential Golomb coding, CAVLC, or CABAC, and outputs the values ​​of the syntax elements required for image reconstruction and the quantized values ​​of the transform coefficients of the residuals. More specifically, the CABAC entropy decoding method can receive bins corresponding to each syntax element in the bitstream, determine the context model using information about the target syntax element, decoding information of neighboring blocks and the target block, or information about symbols / bins decoded in the previous stage, perform arithmetic decoding on the bins based on the determined context model by predicting the occurrence probability of the bins, and generate symbols corresponding to the value of each syntax element. In this case, the CABAC entropy decoding method can update the context model after determining the context model by using the information of the decoded symbols / bins for the context model of the next symbol / bin. The prediction-related information in the information decoded by the entropy decoder 210 can be provided to the predictors (inter-frame predictor 260 and intra-frame predictor 265), and the residual value of entropy decoding performed in the entropy decoder 210, i.e., the quantization transform coefficients and related parameter information, can be input to the dequantizer 220. In addition, the filtering information in the information decoded by the entropy decoder 210 can be provided to the filter 240. Furthermore, the receiver (not shown) for receiving signals output from the image encoding device may be further configured as an internal / external element of the image decoding device 200, or the receiver may be a component of the entropy decoder 210.

[0104] Furthermore, the image decoding apparatus according to this disclosure can be referred to as a video / image / screen decoding apparatus. The image decoding apparatus can be divided into an information decoder (video / image / screen information decoder) and a sample decoder (video / image / screen sample decoder). The information decoder may include an entropy decoder 210. The sample decoder may include at least one of a dequantizer 220, an inverse transformer 230, an adder 235, a filter 240, a memory 250, an inter-frame predictor 260, or an intra-frame predictor 265.

[0105] Dequantizer 220 can dequantize the quantized transform coefficients and output transform coefficients. Dequantizer 220 can rearrange the quantized transform coefficients in the form of two-dimensional blocks. In this case, the rearrangement can be performed based on the coefficient scan order performed in the image encoding device. Dequantizer 220 can obtain the transform coefficients by performing dequantization on the quantized transform coefficients using quantization parameters (e.g., quantization step size information).

[0106] The inverse transformer 230 can perform inverse transformation on the transformation coefficients to obtain the residual signal (residual block, residual sample array).

[0107] The predictor can perform prediction on the current block and generate a prediction block that includes prediction samples of the current block. The predictor can determine whether to apply intra-frame prediction or inter-frame prediction to the current block based on the prediction information output from the entropy decoder 210, and can determine a specific intra-frame / inter-frame prediction mode (prediction technique).

[0108] Similar to that described in the predictor of the image coding device 100, the predictor can generate a predictive signal based on various prediction methods (techniques) described later.

[0109] Intra-predictor 265 can predict the current block by referring to samples in the current frame. The description of intra-predictor 185 also applies to intra-predictor 265.

[0110] Inter-frame predictor 260 can deduce the predicted block of the current block based on a reference block (reference sample array) specified by a motion vector on a reference frame. In this case, to reduce the amount of motion information transmitted in the inter-frame prediction mode, motion information can be predicted on a block, sub-block, or sample basis based on the correlation of motion information between neighboring blocks and the current block. Motion information may include motion vectors and reference frame indices. Motion information may also include inter-frame prediction direction (L0 prediction, L1 prediction, dual prediction, etc.) information. In the case of inter-frame prediction, neighboring blocks may include spatially neighboring blocks existing in the current frame and temporally neighboring blocks existing in the reference frame. For example, inter-frame predictor 260 can configure a motion information candidate list based on neighboring blocks and deduce the motion vector and / or reference frame index of the current block based on the received candidate selection information. Inter-frame prediction can be performed based on various prediction modes, and the information about the prediction may include information indicating the inter-frame prediction mode of the current block.

[0111] Adder 235 generates a reconstruction signal (reconstructed frame, reconstruction block, reconstruction sample array) by adding the obtained residual signal to the prediction signal (prediction block, prediction sample array) output from the predictor (including inter-frame predictor 260 and / or intra-frame predictor 265). If the block to be processed has no residual (e.g., in the case of applying skip mode), the prediction block can be used as a reconstruction block. The description of adder 155 also applies to adder 235. Adder 235 may be referred to as a reconstructor or reconstruction block generator. The generated reconstruction signal can be used for intra-frame prediction of the next block to be processed in the current frame, and can be used for inter-frame prediction of the next frame by filtering as described below.

[0112] Furthermore, Luminance Mapping with Chroma Scaling (LMCS) is applicable during the image decoding process.

[0113] Filter 240 can improve the subjective / objective image quality by applying filtering to the reconstructed signal. For example, filter 240 can generate a modified reconstructed image by applying various filtering methods to the reconstructed image and store the modified reconstructed image in memory 250, specifically in the DPB of memory 250. Various filtering methods may include, for example, deblocking filtering, adaptive sample shifting, adaptive loop filtering, bilateral filtering, etc.

[0114] The (modified) reconstructed frame stored in the DPB of memory 250 can be used as a reference frame in inter-frame predictor 260. Memory 250 can store motion information of blocks from which motion information in the current frame is derived (or decoded) and / or motion information of already reconstructed blocks in the frame. The stored motion information can be transmitted to inter-frame predictor 260 to be used as motion information for spatially or temporally neighboring blocks. Memory 250 can store reconstructed samples of reconstructed blocks in the current frame and transmit the reconstructed samples to intra-frame predictor 265.

[0115] In this disclosure, the embodiments described in the filter 160, inter-frame predictor 180 and intra-frame predictor 185 of the image encoding device 100 can be applied equally or correspondingly to the filter 240, inter-frame predictor 260 and intra-frame predictor 265 of the image decoding device 200.

[0116] The quantizer of an encoding device derives the quantized transform coefficients by applying quantization to the transform coefficients, and the dequantizer of either the encoding or decoding device derives the transform coefficients by applying dequantization to the quantized transform coefficients. In video coding, the quantization rate can be changed, and the compression ratio can be adjusted using the changed quantization rate. From an implementation perspective, considering complexity, a quantization parameter (QP) can be used instead of the quantization rate directly. For example, a quantization parameter with integer values ​​from 0 to 63 can be used, and each quantization parameter value can correspond to the actual quantization rate. The quantization parameter QP can be set differently for the luma component (luma sample). Y And the quantization parameter QP for chromaticity components (chromaticity samples) C .

[0117] During quantization, the transform coefficient C can be input and divided by the quantization rate Q. step Furthermore, the quantization transform coefficients C' can be derived based on this. In this case, the computational complexity can be considered by multiplying the quantization rate by a scale to form an integer, and a shift operation can be performed using the value corresponding to the scale. The quantization scale can be derived based on the product of the quantization rate and the scale value. That is, the quantization scale can be derived from QP. The quantization scale can be applied to the transform coefficients, and the quantization transform coefficients C' can be derived based on this.

[0118] Dequantization is the inverse of quantization, and the quantized transform coefficients C' can be multiplied by the quantization rate Qstep, from which the reconstructed transform coefficients C' can be derived. In this case, level scaling can be derived from the quantization parameters, applied to the quantized transform coefficients C', and the reconstructed transform coefficients C' can be derived from this. Due to losses during the transform and / or quantization processes, the reconstructed transform coefficients C' may differ slightly from the original transform coefficients C. Therefore, even in the encoding device, dequantization can be performed in the same manner as in the decoding device.

[0119] Furthermore, adaptive frequency-weighted quantization (IFQ) can be applied, where the quantization intensity is adjusted according to the frequency. IFQ corresponds to methods that apply different quantization intensities based on frequency. In IFQ, a predefined quantization scaling matrix can be used to apply different quantization intensities according to the frequency. That is, the quantization / dequantization process described above can also be performed based on a quantization scaling matrix.

[0120] For example, different quantization scaling matrices can be used depending on the size of the current block and / or whether the prediction mode applied to generate the residual signal for the current block is inter-frame prediction or intra-frame prediction. The quantization scaling matrix can be called a quantization matrix or a scaling matrix. The quantization scaling matrix can be predefined. Additionally, for frequency-adaptive scaling, frequency quantization scaling information of the quantization scaling matrix can be constructed / encoded in the encoding device and signaled to the decoding device. Frequency quantization scaling information can be called quantization scaling information. Frequency quantization scaling information may include scaling list data (scaling_list_data).

[0121] The quantization scaling matrix can be derived based on the scaling list data. Additionally, the frequency quantization scaling information can include a current flag indicating the presence of scaling list data. Furthermore, when the scaling list data is signaled at a higher level (e.g., SPS), it can also include information indicating whether the scaling list data has been modified at a lower level (e.g., PPS, APS, or slice header).

[0122] General image / video encoding process

[0123] In image / video coding, the frames of an image / video can be encoded / decoded according to the decoding order. The output order of the decoded frames can be set differently from the decoding order, and based on this, not only forward prediction but also backward prediction can be performed during inter-frame prediction.

[0124] Figure 4 An example of an illustrative screen decoding process to which embodiments of this disclosure apply is shown.

[0125] Figure 4 The processes shown can be derived from Figure 3 The image decoding device performs this. For example, in Figure 4 In this process, step S410 can be executed by entropy decoder 210, step S420 can be executed by a predictor including intra-frame predictor 265 and inter-frame predictor 260, step S430 can be executed by residual processor including dequantizer 220 and inverse transformer 230, step S440 can be executed by adder 235, and step S450 can be executed by filter 240. Step S410 may include the information decoding process described in this disclosure, step S420 may include the inter-frame / intra-frame prediction process described in this disclosure, step S430 may include the residual processing process described in this disclosure, step S440 may include the block / frame reconstruction process described in this disclosure, and step S450 may include the in-loop filtering process described in this disclosure.

[0126] Reference Figure 4The image decoding process can schematically include a process for obtaining image / video information from the bitstream (through decoding) (S410), an image reconstruction process (S420 to S440), and an in-loop filtering process for the reconstructed image (S450). The image reconstruction process can be performed based on prediction samples and residual samples obtained through inter-frame / intra-frame prediction (S420) and residual processing (S430) (dequantization and inverse transform of quantization transform coefficients) described in this disclosure. A modified reconstructed image can be generated by an in-loop filtering process for the reconstructed image generated by the image reconstruction process. The modified reconstructed image can be stored as a decoded image output in the decoded image buffer or memory 250 of the decoding device and used as a reference image in the inter-frame prediction process when the image is later decoded. In some cases, the in-loop filtering process can be omitted. In this case, the reconstructed image can be stored as a decoded image output in the decoded image buffer or memory 250 of the decoding device and used as a reference image in the inter-frame prediction process when the image is later decoded. As described above, the in-loop filtering process (S450) may include a deblocking filtering process, a sample adaptive offset (SAO) process, an adaptive loop filter (ALF) process, and / or a bilateral filter process, some or all of which may be omitted. Furthermore, one or more of the deblocking filtering process, the sample adaptive offset (SAO) process, the adaptive loop filter (ALF) process, and / or the bilateral filter process may be applied sequentially, or all of them may be applied sequentially. For example, the SAO process may be performed after the deblocking filtering process is applied to the reconstructed image. Alternatively, for example, the ALF process may be performed after the deblocking filtering process is applied to the reconstructed image. This can even be performed similarly in an encoding device.

[0127] Figure 5 An example of an illustrative screen encoding process to which embodiments of this disclosure are applicable is shown.

[0128] Figure 5 The processes shown can be derived from Figure 2 The image encoding device performs the operation. For example, step S510 can be performed by a predictor including an intra-frame predictor 185 and an inter-frame predictor 180, step S520 can be performed by residual processors 115, 120, and 130, and step S530 can be performed by an entropy encoder 190. Step S510 may include the inter-frame / intra-frame prediction process described in this disclosure, step S520 may include the residual processing process described in this disclosure, and step S530 may include the information encoding process described in this disclosure.

[0129] Reference Figure 5The image encoding process can schematically include not only the process of encoding information used for image reconstruction (e.g., prediction information, residual information, segmentation information, etc.) and outputting it as a bitstream, but also the process of generating a reconstructed image for the current image and the process of applying in-loop filtering to the reconstructed image (optional). The encoding device can derive (modified) residual samples from the quantization transform coefficients using dequantizer 140 and inverse transformer 150, and generate a reconstructed image based on the prediction samples and (modified) residual samples as the output of step S510. The reconstructed image generated in this way can be equal to the reconstructed image generated in the decoding device. A modified reconstructed image can be generated through an in-loop filtering process on the reconstructed image. In this case, the modified reconstructed image can be stored in the decoded image buffer of memory 170 and, similar to the decoding device, can be used as a reference image during inter-frame prediction when encoding images later. As mentioned above, in some cases, some or all of the in-loop filtering processes can be omitted. When performing the in-loop filtering process, the filtering-related information (parameters) can be encoded in the entropy encoder 190 and output as a bit stream, and the decoding device can perform the in-loop filtering process using the same method as the encoding device based on the filtering-related information.

[0130] This in-loop filtering process reduces noise (e.g., block artifacts and ringing artifacts) that occurs during image / video encoding, and improves subjective / objective visual quality. Furthermore, by performing the in-loop filtering process in both the encoding and decoding devices, the encoding and decoding devices can derive the same prediction results, increasing the reliability of image encoding and reducing the amount of data that needs to be sent for image encoding.

[0131] As described above, the image reconstruction process can be performed not only in an image decoding device but also in an image encoding device. Reconstruction blocks can be generated on a block-by-block basis based on intra-frame prediction / inter-frame prediction, and a reconstructed image including these blocks can be generated. When the current image / slice / patch group is an I-frame / slice / patch group, the blocks included in the current image / slice / patch group can be reconstructed based solely on intra-frame prediction. On the other hand, when the current image / slice / patch group is a P-frame / slice / patch group or a B-frame / slice / patch group, the blocks included in the current image / slice / patch group can be reconstructed based on either intra-frame prediction or inter-frame prediction. In this case, inter-frame prediction can be applied to some blocks in the current image / slice / patch group, and intra-frame prediction can be applied to the remaining blocks. The color components of the image can include luminance components and chrominance components, and unless explicitly limited in this disclosure, the methods and implementations of this disclosure are applicable to both luminance and chrominance components.

[0132] Example of encoding layer structure

[0133] The encoded video / images according to this disclosure can be processed, for example, according to the coding layers and structures described below.

[0134] Figure 6 This is a view that illustrates an example of a layered structure for encoding images / videos.

[0135] Encoded images / videos are classified into a Video Coding Layer (VCL) for image / video decoding and processing, a lower-layer system for sending and storing encoded information, and a Network Abstraction Layer (NAL) that exists between the VCL and the lower-layer system and is responsible for network adaptation functions.

[0136] In VCL, VCL data that includes compressed image data (slice data) can be generated, or additional enhancement information (SEI) messages required for image decoding processing or parameter sets that include information such as picture parameter set (PPS), sequence parameter set (SPS), or video parameter set (VPS) can be generated.

[0137] In NAL, header information (NAL unit header) can be added to the raw byte sequence payload (RBSP) generated in VCL to generate NAL units. In this case, RBSP refers to the slice data, parameter set, and SEI message generated in VCL. The NAL unit header may include NAL unit type information specified according to the RBSP data included in the corresponding NAL unit.

[0138] like Figure 6 As shown, NAL units can be classified into VCL NAL units and non-VCL NAL units based on the type of RBSP generated in the VCL. A VCL NAL unit can refer to a NAL unit that includes information about the image (slice data), while a non-VCL NAL unit can refer to a NAL unit that includes information required for decoding the image (parameter set or SEI message).

[0139] VCL NAL units and non-VCL NAL units can be appended with header information and transmitted over the network according to the data standard of the underlying system. For example, NAL units can be modified to have a data format with a predetermined standard (e.g., H.266 / VVC file format, RTP (Real-Time Transport Protocol), or TS (Transport Stream)) and transmitted over various networks.

[0140] As described above, within a NAL unit, the NAL unit type can be specified based on the RBSP data structure included in the corresponding NAL unit, and information about the NAL unit type can be stored in the NAL unit header and signaled. For example, this can be broadly categorized into VCL NAL unit types and non-VCL NAL unit types based on whether the NAL unit includes information about the image (slice data). VCL NAL unit types can be categorized based on the nature and type of the image included in the VCL NAL unit, while non-VCL NAL unit types can be categorized based on the type of parameter set.

[0141] Below are examples of NAL cell types specified based on the type of parameter set / information included in non-VCL NAL cell types.

[0142] - DCI (Decoding Capability Information) NAL Unit Type (NUT): Includes the NAL unit type of DCI.

[0143] -VPS (Video Parameter Set) NUT: Includes the NAL unit type of the VPS.

[0144] -SPS (Sequence Parameter Set) NUT: NAL unit type including SPS

[0145] -PPS (Picture Parameter Set) NUT: Includes the NAL unit type of PPS.

[0146] -APS (Adaptive Parameter Set) NUT: Includes NAL unit types of APS.

[0147] -PH (Picture Header) NUT: NAL unit type including the picture header.

[0148] The aforementioned NAL unit type can have syntax information specific to the NAL unit type, and this syntax information can be stored in the NAL unit header and signaled. For example, this syntax information can be nal_unit_type, and the NAL unit type can be specified using the nal_unit_type value.

[0149] Furthermore, a frame can include multiple slices, and a slice can include a slice header and slice data. In this case, a frame header can be further added to the multiple slices (slice headers and slice data sets) within a frame. The frame header (frame header syntax) can include information / parameters common to the frame. The slice header (slice header syntax) can include information / parameters common to the slice. APS (APS syntax) or PPS (PPS syntax) can include information / parameters common to one or more slices or frames. SPS (SPS syntax) can include information / parameters common to one or more sequences. VPS (VPS syntax) can be information / parameters common to multiple layers. DCI (DCI syntax) can include information / parameters related to decoding capabilities.

[0150] In this disclosure, the high-level syntax (HLS) may include at least one of APS syntax, PPS syntax, SPS syntax, VPS syntax, DCI syntax, picture header syntax, or slice header syntax. Additionally, in this disclosure, the low-level syntax (LLS) may include, for example, slice data syntax, CTU syntax, coding unit syntax, transform unit syntax, etc.

[0151] In this disclosure, the image / video information encoded in the encoding device and signaled to the decoding device in the form of a bitstream may include not only intra-frame segmentation related information, intra / inter-frame prediction information, residual information, and intra-loop filtering information, but also information about slice headers, frame headers, APS, PPS, SPS, VPS, and / or DCI. Additionally, the image / video information may also include general constraint information and / or information about NAL unit headers.

[0152] Figure 7This is a view illustrating an example of a syntax structure used to signal information about the APS and the picture header. Image information may include a High-Level Syntax (HLS). Image encoding methods can be performed based on the image information. An encoded picture may consist of one or more slices. Parameters describing the encoded picture can be signaled in the picture header. Parameters describing the slices can be signaled in the slice header. The picture header can be carried in a NAL cell type. The slice header may exist at the beginning of the NAL cell containing the slice payload. Luminance Mapping with Chroma Scaling (LMCS) can be a term encompassing both a luminance mapping process and a chroma scaling process. For example, a luminance mapping process and / or a chroma scaling process can be performed before in-loop filtering. For example, a luminance mapping process can be performed relative to a prediction block of the current luminance block to generate a prediction block with a changed dynamic range. In this case, the current luminance block can be reconstructed based on the prediction block with the changed dynamic range. Alternatively, for example, the chroma scaling process can be a process of scaling the chroma component residual signal based on the relationship between the luminance component signal and the chroma component signal. In this case, the current chroma block can be reconstructed based on the scaled chroma component residual signal.

[0153] Reference Figure 7 The APS identifier of the ALF APS referenced by the chroma element of the slice in the current frame can be specified using a signal (e.g., ph_alf_aps_id_chroma). The APS identifier of the LMCS APS referenced by the chroma element of the slice in the current frame can be specified using a signal (e.g., ph_lmcs_aps_id). Additionally, the APS identifier of the APS scaling list can be specified using a signal (e.g., ph_scaling_list_aps_id).

[0154] Video Parameter Set Signaling

[0155] For multi-layer bitstreams with inter-layer dependencies, the layers can be decoded using a set of parameters. The Video Parameter Set (VPS) is a set of parameters used to transmit layer information. Layer information may include, for example, information about the Output Layer Set (OLS), information about the profile layer level, information about the relationship between the OLS and the hypothetical reference decoder, and information about the relationship between the OLS and the DPB, etc.

[0156] The VPS RBSP (Raw Byte Sequence Payload) can be included in at least one Access Unit (AU) with a TemporalID of 0 or provided by external means before reference, thus enabling its use in the decoding process. All VPS NAL units with a specific value of vps_video_parameter_set_id in CVS will have the same content.

[0157] Figure 8 and Figure 9 The grammar examples exemplify the grammatical structure according to embodiments of this disclosure. Hereinafter, a description will be provided. Figures 8 to 9 Syntax elements.

[0158] `vps_video_parameter_set_id` provides an identifier for the VPS that will be referenced by another syntax element. Other syntax elements can use `vps_video_parameter_set_id` to reference a VPS. The value of `vps_video_parameter_set_id` should be greater than 0.

[0159] `vps_max_layers_minus1` can specify the maximum number of layers that can exist in a single CVS of the reference VPS. For example, the value obtained by incrementing `vps_max_layers_minus1` by 1 can specify the maximum number of layers that can exist in a single CVS of the reference VPS.

[0160] The value obtained by incrementing vps_max_sublayer_minus1 by 1 can specify the maximum number of time sublayers that can exist in a single CVS of the reference VPS.

[0161] A value of 1 for `vps_all_layers_same_num_sublayer_flag` specifies that the number of time sublayers is the same across all layers in a single CVS of the reference VPS. A value of 0 for `vps_all_layers_same_num_sublayer_flag` specifies that the number of time sublayers can be the same or different across layers in a single CVS of the reference VPS. When a value for `vps_all_layers_same_num_sublayer_flag` is not provided in the bitstream, its value can be inferred as 1.

[0162] A value of 1 for `vps_all_independent_layers_flag` specifies that all layers belonging to CVS are encoded independently without using inter-layer prediction. A value of 0 for `vps_all_independent_layers_flag` specifies that at least one layer belonging to CVS can be encoded using inter-layer prediction.

[0163] `vps_layer_id[i]` can specify the `nuh_layer_id` value of the `i`-th layer. For any two non-negative integer values ​​`m` and `n`, `vps_layer_id[m]` can be constrained to have a value less than `vps_layer_id[n]` when `m` is less than `n`. Here, `nuh_layer_id` is a syntax element signaled in the NAL cell header and can specify the identifier of the NAL cell.

[0164] A value of 1 for `vps_independent_layer_flag[i]` specifies that inter-layer prediction is not applied to the layer corresponding to index `i`. A value of 0 for `vps_independent_layer_flag[i]` specifies that inter-layer prediction is applied to the layer corresponding to index `i`, and the syntax element `vps_direct_ref_layer_flag[i][j]` is obtained from the VPS. Here, `j` can have a value from 0 to `i-1`. Furthermore, when a value for `vps_independent_layer_flag[i]` does not exist in the bitstream, its value can be deduced to be 1.

[0165] A value of 1 for `vps_max_tid_ref_present_flag[i]` specifies that the syntax element `vps_max_tid_il_ref_pics_plus1[i][j]` is provided from the bitstream. A value of 0 for `vps_max_tid_ref_present_flag[i]` specifies that the syntax element `vps_max_tid_il_ref_pics_plus1[i][j]` is not provided from the bitstream.

[0166] A value of 0 for `vps_direct_ref_layer_flag[i][j]` specifies that the layer with index `j` is not a direct reference layer to the layer with index `i`. A value of 1 for `vps_direct_ref_layer_flag[i][j]` specifies that the layer with index `j` is a direct reference layer to the layer with index `i`. For `i` and `j` in the range from 0 to `vps_max_layers_minus1`, the value of `vps_direct_ref_layer_flag[i][j]` can be deduced to be 0 when the value is not obtained from the bitstream. When the value of `vps_independent_layer_flag[i]` is 0, there can exist at least one `j` that makes the value of `vps_direct_ref_layer_flag[i][j]` equal to 1. In this case, the value of `j` can range from 0 to `i-1`.

[0167] It is possible Figure 10The derived variables shown are NumDirectRefLayers[i], DirectRefLayerIdx[i][d], NumRefLayers[i], RefLayerIdx[i][r], and LayerUsedAsRefLayerFlag[j].

[0168] The variable GeneralLayerIdx[i] specifies the layer index of the layer where the value of nuh_layer_id is the same as that of vps_layer_id[i], and can be as follows: Figure 11 The derivation is shown in the figure.

[0169] A value of 0 for `vps_max_tid_il_ref_pics_plus1[i][j]` specifies that a frame in layer j, whose `ph_recovery_poc_cnt` is neither a GDR nor an IRAP frame, is not used as an interlayer reference frame for decoding a frame in layer i. A value greater than 0 for `vps_max_tid_il_ref_pics_plus1[i][j]` specifies that a frame in layer j with a `TemporalId` greater than `vps_max_tid_il_ref_pics_plus1[i][j]-1` is not used as an interlayer reference frame for decoding a frame in layer i. Furthermore, when the value of `vps_max_tid_il_ref_pics_plus1[i][j]` is not obtained from the bitstream, its value can be deduced as `vps_max_sublayer_minus1+1`.

[0170] A value of 1 for `vps_each_layer_is_a_ols_flag` specifies that a single OLS has only one layer, and that a single layer belonging to the CVS of the reference VPS is an OLS with a single containing layer that is the only output layer. A value of 0 for `vps_each_layer_is_a_ols_flag` specifies that an OLS may contain more than one layer. In one implementation, the value of `vps_each_layer_is_a_ols_flag` can be deduced to be 1 when the value of `vps_max_layers_minus1` is 0. Otherwise, the value of `vps_each_layer_is_a_ols_flag` can be deduced to be 0 when the value of `vps_all_independent_layers_flag` is 0.

[0171] A value of 0 for `vps_ols_mode_idc` allows you to specify that the total number of OLSs specified by the VPS is equal to `vps_max_layers_minus1+1`. The i-th OLS can include layers with layer indices from 0 to i. Additionally, for a single OLS, the highest layer of the OLS can be output.

[0172] A value of 1 for `vps_ols_mode_idc` specifies that the total number of OLSs specified by the VPS is equal to `vps_max_layers_minus1+1`. The i-th OLS can include layers with layer indices from 0 to i. Additionally, for a single OLS, all layers of the OLS can be output.

[0173] A value of 2 for vps_ols_mode_idc can specify the total number of OLSs specified by the VPS, explicitly signal the output layer for a single OLS, and other layers are direct or reference layers to the output layer of the OLS.

[0174] vps_ols_mode_idc can have values ​​from 0 to 2. The value 3 of vps_ols_mode_idc can be reserved for future use. When the value of vps_all_independent_layers_flag is 1 and the value of vps_each_layer_is_a_ols_flag is 0, the value of vps_ols_mode_idc can be deduced to be 2.

[0175] The value obtained by incrementing `vps_num_output_layer_sets_minus1` by 1 specifies the total number of OLS specified by the VPS when `vps_ols_mode_idc` is a predetermined value (e.g., when the value is 2). This can be achieved as follows: Figure 12 The derivation shown specifies the variable TotalNumOlss, which represents the total number of OLS specified by the VPS.

[0176] A value of 1 for `vps_ols_output_layer_flag[i][j]` specifies that when `vps_ols_mode_idc` is 2, the layer whose `nuh_layer_id` is equal to `vps_layer_id[j]` is the output layer of the i-th OLS. A value of 0 for `vps_ols_output_layer_flag[i][j]` specifies that when `vps_ols_mode_idc` is 2, the layer whose `nuh_layer_id` is equal to `vps_layer_id[j]` is not the output layer of the i-th OLS.

[0177] The variables NumOutputLayersInOls[i] specifying the number of output layers in the i-th OLS, NumSubLayersInLayerInOLS[i][j] specifying the number of sublayers existing in the j-th layer of the i-th OLS, OutputLayerIdInOls[i][j] specifying the nuh_layer_id value of the j-th output layer in the i-th OLS, and LayerUsedAsOutputLayerFlag[k] specifying whether the k-th layer is used as an output layer in at least one OLS can be used as... Figures 13 to 14 The pseudocode shown in the middle is derived.

[0178] For each value from 0 to vps_max_layers_minus1, the values ​​of LayerUsedAsRefLayerFlag[i] and LayerUsedAsOutputLayerFlag[i] can be forced to be non-zero. For example, a layer that is neither the output layer of at least one OLS nor a direct reference layer of another layer can be forced to not exist.

[0179] For each OLS, it is permissible to have at least one layer as the output layer. For example, for each i value from 0 to TotalNumOlss-1, the value of NumOutputLayersInOls[i] can be permissible to have a value greater than or equal to 1.

[0180] The variable NumLayersInOls[i] specifies the number of layers in the i-th OLS and the variable LayerIdInOls[i][j] specifies the nuh_layer_id value of the j-th layer in the i-th OLS.

[0181] When the value of NumLayersInOls[i] is greater than 0, the variable NumMultiLayerOlss, which specifies the number of multi-layer OLS (e.g., OLS containing one or more layers), and the variable MultiLayerOlsIdx[i], which specifies the index of the list of multi-layer OLS for the i-th OLS, can be used as... Figure 15 The pseudocode was derived.

[0182] The variable OlsLayerIdx[i][j], which specifies the OLS layer index of the layer with the same nuh_layer_id as LayerIdInOls[i][j], can be used as... Figure 16 The pseudocode was derived.

[0183] The lowest layer present in each OLS can be constrained to be an independent layer. For example, for each i with 0 to TotalNumOlss-1, the value of vps_independent_layer_flag[GeneralLayerIdx[LayerIdInOls[i]]] can be forced to 1. Each layer can be forced to be included in at least one OLS specified by the VPS.

[0184] Improvements to APS signaling

[0185] When decoding of the output layer is not required, VCL NAL units with a TemporalId equal to or less than the target maximum TemporalId can be removed from non-output layers during extraction. However, APS NAL units present in such layers and temporal sublayers are not properly removed. Typically, in this case, the highest output layer does not use the VCL from the temporal sublayer, nor does it use the parameter set from it, such as APS and / or PPS. Here, APS can include adaptive loop filter (ALF) parameters according to the APS type, luminance mapping (LMCS) parameters with chroma scaling, or scaling list parameters.

[0186] To address the aforementioned issues, during the extraction process, PPS and / or APS can be selected as parameter sets that are not used as references for decoding the images in the output layer. Additionally, besides the parameter sets, non-VCL NAL units that are not used as references during the extraction process for decoding the images in the output layer can also be selected as non-references for decoding the images in the output layer. Furthermore, APS extraction from the output bitstream can be prevented.

[0187] For this type of processing, the syntax element `vps_max_tid_il_ref_pics_plus1[i][j]` can be redefined. For example, a first value (e.g., 0) of `vps_max_tid_il_ref_pics_plus1[i][j]` can specify that a frame that is neither a GDR nor an IRAP frame and has a `ph_recovery_poc_cnt` value of 0 in layer j is not used as an inter-layer reference frame for decoding frames in layer i. A value greater than 0 for `vps_max_tid_il_ref_pics_plus1[i][j]` can specify that frames and parameter sets with a `TemporalId` greater than `vps_max_tid_il_ref_pics_plus1[i][j] - 1` in layer j are not used as references for decoding frames in layer i. Furthermore, when the value of `vps_max_tid_il_ref_pics_plus1[i][j]` is not obtained from the bitstream, its value can be derived as `vps_max_sublayer_minus1 + 1`.

[0188] Furthermore, the sub-bitstream can be derived according to the following sub-bitstream extraction procedure. The sub-bitstream extraction procedure can take the variable `inBitstream` specifying the input bitstream, the target OLS index `targetOlsIdx`, and `tIdTarget` as the target `TemporalId` as input. The variable `OutBitstream` specifying the output sub-bitstream can be derived as follows.

[0189] First, the sub-bitstream can be determined as the value of the input bitstream (S1710). For example, the variable outBitstream, which specifies the output sub-bitstream, can be set to be equal to the variable inBitstream, which specifies the input bitstream.

[0190] All PPS, APS, and VCL NAL units and their associated non-VCL NAL units that satisfy all of the following conditions 1 to 3 can be removed from outBitstream (S1720). Here, the associated non-VCL NAL units can be non-VCL NAL units with a PayloadType other than 0, 1, or 130 and a nal_unit_type of any of PH_NUT, FD_NUT, SUFFIX_SEI_NUT, and PREFIX_SEI_NUT. Figure 18 The NAL unit type is shown.

[0191] (Condition 1) The nal_unit_type of the VCL NAL unit is equal to TRAIL_NUT, STSA_NUT, RADL_NUT or RASL_NUT, the nal_unit_type is equal to GDR_NUT and the associated ph_recovery_poc_cnt is not 0.

[0192] (Condition 2) nuh_layer_id equals LayerIdInOls[targetOlsIdx][j]. Here, j has a value from 0 to NumLayersInOls[targetOlsIdx]-1.

[0193] (Condition 3) TemporalId has a value greater than or equal to NumSubLayersInLayerInOLS[targetOlsIdx][GeneralLayerIdx[nuh_layer_id]].

[0194] Encoding methods and decoding methods

[0195] The image encoding and image decoding methods performed by the image encoding and image decoding devices according to embodiments will be described below. First, the operation of the decoding device will be described.

[0196] As described above, when vps_max_tid_il_ref_pics_plus1[i][j] has a value greater than 0, for inter-layer prediction at layer i (current layer), the frames and parameter sets at layer j (reference layer) with TemporalId greater than vps_max_tid_il_ref_pics_plus1[i][j]-1 can be disregarded as references. Additionally, as referenced... Figure 14 and Figure 15 As mentioned above, NumSubLayersInLayerInOLS[targetOlsIdx][GeneralLayerIdx[nuh_layer_id]] can be derived based on vps_max_tid_il_ref_pics_plus1[i][j]. Therefore, frames and parameter sets with TemporalIds equal to or greater than NumSubLayersInLayerInOLS[targetOlsIdx][GeneralLayerIdx[nuh_layer_id]] can be disregarded as references for inter-layer predictions for the current layer.

[0197] Figure 19This is a view illustrating a method for decoding an image using an image decoding device according to one embodiment. The image decoding device according to the embodiment may include a memory and a processor, and the decoding device can perform decoding through operation of a processor according to the embodiment described below.

[0198] According to the implementation method, the decoding device can obtain maximum time identifier information for inter-layer prediction (S1910). Specifically, the decoding device can obtain maximum time identifier information from the bitstream that specifies the maximum time identifier of the reference layer for inter-layer prediction reference of the current layer. In this disclosure, "inter-layer prediction reference for the current layer" can mean "it can be referenced for inter-layer prediction of the current layer". Here, the maximum time identifier information can be the above syntax element vps_max_tid_il_ref_pics_plus1[i][j].

[0199] Next, the decoding device can perform a sub-bitstream extraction process (S1920) from the bitstream based on the maximum time identifier information. During the sub-bitstream extraction process in step S1920, the decoding device can remove unreferenced inter-layer prediction parameter sets from the reference layer's parameter set. Here, the parameter set can be an adaptive parameter set (APS).

[0200] Specifically, the decoding device can derive `NumSubLayersInLayerInOLS[targetOlsIdx][GeneralLayerIdx[nuh_layer_id]]` based on the maximum time identifier information obtained from the bitstream. In this case, for example, it can use... Figure 13 and Figure 14 The method. The decoding device can perform a reference operation based on the derived NumSubLayersInLayerInOLS. Figure 17 The described sub-bitstream extraction process extracts a sub-bitstream from which the set of parameters not used for inter-layer prediction reference in the current layer has been removed.

[0201] The maximum time identifier information specifies the maximum time identifier of the reference layer used for decoding the current layer's frame. Parameter sets in the reference layer's parameter set that have time identifiers greater than the maximum time identifier can be used without referencing the current layer's frame.

[0202] Additionally, the maximum time identifier information can specify the maximum time identifier of the reference layer used for decoding the current layer's frame. Frames in the reference layer with time identifiers greater than the maximum time identifier can be used without referencing the current layer's frame for decoding.

[0203] Furthermore, the decoding device may also include a step of obtaining (extracting) a sub-bitstream from the bitstream based on the number of sub-layers for the current layer. The sub-bitstream can be obtained by removing a predetermined parameter set from the bitstream, and the predetermined parameter set can be determined based on a comparison between a time identifier of the predetermined parameter set and the number of sub-layers determined for the layer corresponding to the predetermined parameter set.

[0204] The predetermined parameter set can be determined based on whether the value of the time identifier of the predetermined parameter set is equal to or greater than the value corresponding to the number of sub-layers determined for the layer corresponding to the predetermined parameter set in the layers of the predetermined output layer set (OLS).

[0205] For example, for the predefined parameter set nuh_layer_id and TemporalId, whether to remove the predefined parameter set can be determined based on whether TemporalId has a value equal to or greater than NumSubLayersInLayerInOLS[targetOlsIdx][GeneralLayerIdx[nuh_layer_id]]. For example, NumSubLayersInLayerInOLS[][] can be derived based on vps_max_tid_il_ref_pics_plus1.

[0206] Furthermore, the pre-defined OLS can be any of the OLS identified by information signaled by the Video Parameter Set (VPS).

[0207] In reference Figure 19 In the example described, the current layer can be an output layer, and the reference layer can be a non-output layer.

[0208] Figure 20 This is a view illustrating a method of encoding an image using an image encoding device according to one embodiment. The image encoding device according to the embodiment may include a memory and a processor, and the encoding device can perform encoding by operating the processor in a manner corresponding to the decoding steps of a decoding device according to the embodiment described below.

[0209] According to one embodiment, the encoding device can determine maximum time identifier information based on the maximum time identifier used for inter-layer prediction (S2010). Specifically, the encoding device can determine the maximum time identifier of the reference layer used for inter-layer prediction reference of the current layer, and determine maximum time identifier information specifying the determined maximum time identifier. In this disclosure, "inter-layer prediction reference for the current layer" can mean "it can be referenced for inter-layer prediction of the current layer".

[0210] Next, the encoding device can generate a bitstream by encoding the maximum time identifier information of the specified maximum time identifier (S2020). Here, the maximum time identifier information can be the above syntax element vps_max_tid_il_ref_pics_plus1[i][j].

[0211] The maximum time identifier information can be used during sub-bitstream extraction. Specifically, the maximum time identifier information can be used to remove parameter sets from the reference layer's parameter set that are not used for inter-layer prediction reference in the current layer. Here, the parameter set can be the Adaptive Parameter Set (APS).

[0212] Specifically, based on the maximum time identifier information, NumSubLayersInLayerInOLS[targetOlsIdx][GeneralLayerIdx[nuh_layer_id]] can be derived. In this case, for example, one could use... Figure 13 and Figure 14 The method. Additionally, a reference is performed based on the derived NumSubLayersInLayerInOLS. Figure 17 The described sub-bitstream extraction process can extract sub-bitstreams from which the parameter set of the reference layer that is not used for inter-layer prediction reference in the current layer has been removed.

[0213] The maximum time identifier can specify the maximum time identifier value of the reference layer used to encode the current layer's frame. Parameter sets in the reference layer's parameter set that have time identifiers greater than the maximum time identifier can be used without referencing the current layer's frame.

[0214] Additionally, the maximum time identifier can specify the maximum time identifier of the reference layer used to encode the current layer's frame. Frames in the reference layer with time identifiers greater than the maximum time identifier can be used without being referenced for encoding the current layer's frame.

[0215] Furthermore, a sub-bit stream can be obtained (extracted) from the bit stream generated according to the encoding method based on the number of sub-layers for the current layer. For example, a sub-bit stream can be obtained by removing a predetermined parameter set from the bit stream. Additionally, the predetermined parameter set can be determined based on a comparison between the time identifier of the predetermined parameter set and the number of sub-layers determined for the layer corresponding to the predetermined parameter set.

[0216] For example, a predetermined parameter set can be determined based on whether the value of the time identifier of the predetermined parameter set is equal to or greater than a value corresponding to the number of sub-layers determined for the layer corresponding to the predetermined parameter set in the layers of the predetermined output layer set (OLS).

[0217] For example, for nuh_layer_id and TemporalId of a predefined parameter set, it can be determined whether to remove the predefined parameter set based on the fact that TemporalId has a value equal to or greater than NumSubLayersInLayerInOLS[targetOlsIdx][GeneralLayerIdx[nuh_layer_id]].

[0218] Furthermore, the pre-defined OLS can be any of the OLS identified by information signaled by the Video Parameter Set (VPS).

[0219] In reference Figure 20 In the example described, the current layer can be an output layer, and the reference layer can be a non-output layer.

[0220] As described above, based on the maximum time identifier information of the reference layer specifying the inter-layer prediction reference for the current layer, the adaptive parameter set (APS) used as the inter-layer prediction reference for the current layer can be removed from the parameter set of the reference layer. An image encoding or decoding device can encode / decode the sub-bitstream extracted by removing the unreferenced parameter set. In this case, the image encoding / decoding device can perform encoding / decoding by referencing the APS included in the extracted sub-bitstream. Hereinafter, a method for signaling / resolving the APS by an image encoding / decoding device according to this disclosure will be described.

[0221] According to another embodiment of this disclosure, the method of signaling APS identifier information can be improved based on the APS parameter type, which relates to the encoding and decoding method described above. Figure 21 This is a view that displays the APS parameter names based on the APS parameter type.

[0222] Reference Figure 21Information about the type of APS parameters sent in the APS (e.g., aps_params_type) can be signaled. The value of aps_params_type can correspond to a range from 0 to 7 in the bitstream. When the value of aps_params_type is 0, the name of aps_params_type can correspond to ALF_APS and the type of the APS parameter can correspond to the ALF parameter. When the value of aps_params_type is 1, the name of aps_params_type can correspond to LMCS_APS and the type of the APS parameter can correspond to the LMCS parameter. When aps_params_type is 2, the name of aps_params_type can correspond to SCALING_APS and the type of the APS parameter can correspond to the scaling list parameter. When the value of aps_params_type is from 3 to 7, aps_params_type can be reserved for future use.

[0223] Figure 22 This is a view illustrating an example of a syntax structure for signaling APS identifier information based on the APS parameter type. In an implementation, within the VVC, the range of the APS identifier can vary depending on the APS type. For example, in the cases of ALFAPS and scaling list APS, the range of the APS identifier can correspond to a range from 0 to 7. In the case of LMCS APS, the range of the APS identifier can correspond to a range from 0 to 3. Therefore, instead of signaling the APS identifier with a fixed length, the required number of bits can be used to signal the APS identifier based on the APS type.

[0224] Reference Figure 22After signaling information specifying the type of APS parameters sent in the APS (e.g., aps_params_type), signaling information specifying the APS identifier (e.g., aps_adaptation_parameter_set_id) can be used. aps_adaptation_parameter_set_id can provide the identifier of the APS to be referenced in another syntax element. When aps_params_type is equal to ALF_APS or SCALING_APS, the length of aps_adaptation_parameter_set_id can correspond to 3 bits. Alternatively, when aps_params_type is LMCS_APS, the length of aps_adaptation_parameter_set_id can correspond to 2 bits. apsLayerId can be set to the nuh_layer_id value of a specific APS NAL unit, and vclLayerId can be set to the nuh_layer_id value of a specific VCL NAL unit. A specific VCL NAL unit may not reference a specific APS NAL unit until apsLayerId is less than or equal to vclLayerId. Additionally, all OLS specified in the VPS that include layers where nuh_layer_id is equal to vclLayerId can include layers where nuh_layer_id is equal to apslayerId.

[0225] According to another embodiment of this disclosure, all APS NAL units with a specific value for `aps_params_type` and those that are prefixed or suffixed APS NAL units can share the same value space for `aps_adaptation_parameter_set_id`, independent of `nuh_layer_id`. APS NAL units with different values ​​for `aps_params_type` can use separate value spaces for `aps_adaptation_parameter_set_id`. When `aps_params_type` is ALF_APS or SCALING_APS, the value of `aps_adaptation_parameter_set_id` can correspond to a range from 0 to 7. When `aps_params_type` is LMCS_APS, the value of `aps_adaptation_parameter_set_id` can correspond to a range from 0 to 3.

[0226] Figure 23 This is an example based on a reference. Figure 22This describes a view of the operation of an image encoding device according to a specific implementation. Information associated with the APS identifier may include at least one of the value of the APS identifier or a range of APS identifiers. The range of APS identifiers can be expressed in terms of the number of bits.

[0227] Reference Figure 23 It can be determined whether `aps_params_type` corresponds to 0 or 2 (S2310). When the above condition is met (S2310 - Yes), the bit length of the APS identifier information (e.g., `aps_adaptation_parameter_set_id`) can be set to a length of 3 bits (S2320). Furthermore, the 3-bit APS identifier information (e.g., `aps_adaptation_parameter_set_id`) can be encoded (S2330). Additionally, the range of values ​​for the APS identifier information (e.g., `aps_adaptation_parameter_set_id`) can be set to a range from 0 to 7. Furthermore, the APS identifier information (e.g., `aps_adaptation_parameter_set_id`) can be encoded as any value within the range from 0 to 7.

[0228] When the condition of step S2310 is not met (S2310-No), that is, when aps_params_type does not correspond to 0 or 2, the bit length of the APS identifier information (e.g., aps_adaptation_parameter_set_id) can be set to a length of 2 bits (S2340). Alternatively, the 2-bit identifier information (e.g., aps_adaptation_parameter_set_id) can be encoded (S2350). Therefore, the range of values ​​for the APS identifier information (e.g., aps_adaptation_parameter_set_id) can be set to a range from 0 to 3. Furthermore, the APS identifier information (e.g., aps_adaptation_parameter_set_id) can be encoded as any value within the range from 0 to 3.

[0229] Figure 24 This is an example based on a reference. Figure 22 A view of the operation of the image decoding device described in the embodiments.

[0230] Reference Figure 24It can be determined whether `aps_params_type` corresponds to 0 or 2 (S2410). When the condition is met (S2410 - Yes), 3 bits of APS identifier information (e.g., `aps_adaptation_parameter_set_id`) can be obtained (S2420). Therefore, the range of values ​​for the APS identifier information (e.g., `aps_adaptation_parameter_set_id`) is from 0 to 7. In this case, the image (not shown) can be reconstructed based on the values ​​of the APS identifier information (e.g., `aps_adaptation_parameter_set_id`) from 0 to 7.

[0231] When the condition of step S2410 is not met (S2410-No), that is, when aps_params_type does not correspond to 0 or 2, 2 bits of APS identifier information (e.g., aps_adaptation_parameter_set_id) can be obtained (S2430). Therefore, the range of values ​​for the APS identifier information (e.g., aps_adaptation_parameter_set_id) that can be obtained is from 0 to 3. In this case, the image (not shown) can be reconstructed based on the values ​​of the APS identifier information (e.g., aps_adaptation_parameter_set_id) from 0 to 3.

[0232] Furthermore, according to another embodiment of this disclosure, when nal_unit_type is equal to DCI_NUT, VPS_NUT, PREFIX_APS_NUT, SUFFIX_APS_NUT, PPS_NUT or SPS_NUT, it can force TemporalId to be equal to 0 and the TemporalId of the AU including the NAL unit to be equal to 0.

[0233] Additionally, according to another embodiment of this disclosure, the output bitstream can be set to be equal to the input bitstream. All VCL NAL units with a TemporalId greater than tIdTarget can be removed from the output bitstream. Associated non-VCL NAL units with the same nal_unit_type as PH_NUT, FD_NUT, and SUFFIX_SEI_NUT can be removed from the output bitstream. PREFIX_SEI_NUTs with PayloadType not equal to 0, 1, or 130 can be removed from the output bitstream. All NAL units with nal_unit_type not equal to VPS_NUT or DCI_NUT can be removed from the output bitstream. nuh_layer_id not included in EOB_NUT and LayerIdInOls[targetOlsIdx] can be removed from the output bitstream.

[0234] All VCL NAL units can be removed from the output bitstream when the following conditions are met: Associated non-VCL NAL units with the same nal_unit_type as PH_NUT, FD_NUT, and SUFFIX_SEI_NUT can be removed from the output bitstream when the following conditions are met: PREFIX_SEI_NUTs with PayloadType not equal to 0, 1, or 130 can be removed from the output bitstream when the following conditions are met:

[0235] Condition 1: nal_unit_type is equal to TRAIL_NUT, STSA_NUT, RADL_NUT or RASL_NUT, or nal_unit_type is equal to GDR_NUT and the associated ph_recovery_poc_cnt is not 0.

[0236] Condition 2: For j in the range from 0 to NumLayersInOls[targetOlsIdx]-1, nuh_layer_id is equal to LayerIdInOls[targetOlsIdx][j].

[0237] Condition 3: TemporalId is greater than or equal to NumSubLayersInLayerInOLS[targetOlsIdx][GeneralLayerIdx[nuh_layer_id]].

[0238] Application and Implementation Methods

[0239] Although the exemplary methods of this disclosure described above are represented as a series of operations for clarity of description, they are not intended to limit the order in which the steps are performed, and these steps may be performed simultaneously or in different orders if necessary. To implement the methods according to this disclosure, the described steps may further include other steps, including steps in addition to some steps, or may include additional steps in addition to some steps.

[0240] In this disclosure, the image encoding device or image decoding device that performs a predetermined operation (step) can perform an operation (step) that confirms the execution conditions or circumstances of the corresponding operation (step). For example, if it is described that a predetermined operation is performed when predetermined conditions are met, the image encoding device or image decoding device can perform the predetermined operation after determining whether the predetermined conditions are met.

[0241] The various embodiments of this disclosure are not a list of all possible combinations and are intended to describe representative aspects of this disclosure; the matters described in the various embodiments may be applied independently or in combination of two or more.

[0242] Various embodiments of this disclosure can be implemented in hardware, firmware, software, or a combination thereof. When this disclosure is implemented in hardware, it can be implemented using application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers, microprocessors, etc.

[0243] Furthermore, the image decoding and image encoding devices applying the embodiments of this disclosure can be included in multimedia broadcasting transmission and receiving devices, mobile communication terminals, home theater video devices, digital cinema video devices, surveillance cameras, video chat devices, real-time communication devices such as video communication, mobile streaming devices, storage media, cameras, video-on-demand (VoD) service providers, OTT (over-the-top) video devices, internet streaming service providers, three-dimensional (3D) video devices, video telephony devices, medical video devices, etc., and can be used to process video signals or data signals. For example, OTT video devices can include game consoles, Blu-ray players, internet access televisions, home theater systems, smartphones, tablet PCs, digital video recorders (DVRs), etc.

[0244] Figure 25 This is a view illustrating a content streaming system to which embodiments of the present disclosure can be applied.

[0245] like Figure 25As shown, the content streaming system applying the embodiments of this disclosure may mainly include an encoding server, a streaming server, a network server, a media storage device, a user device, and a multimedia input device.

[0246] The encoding server compresses content input from multimedia input devices such as smartphones, cameras, and camcorders into digital data to generate a bitstream and then sends the bitstream to the streaming server. As another example, when multimedia input devices such as smartphones, cameras, and camcorders directly generate bitstreams, the encoding server can be omitted.

[0247] The bitstream can be generated by an image encoding method or image encoding device applying the embodiments of this disclosure, and the stream server can temporarily store the bitstream during the sending or receiving of the bitstream.

[0248] A streaming server sends multimedia data to a user's device based on a request from a web server, and the web server acts as a medium for informing the user about the service. When a user requests a service from the web server, the web server can deliver it to the streaming server, and the streaming server can send the multimedia data to the user. In this scenario, the content streaming system may include a separate control server. In this case, the control server is used to control the commands / responses between devices in the content streaming system.

[0249] A streaming server can receive content from media storage devices and / or encoding servers. For example, when receiving content from an encoding server, the content can be received in real time. In this case, to provide a smooth streaming service, the streaming server can store the bitstream for a predetermined period of time.

[0250] Examples of user devices may include mobile phones, smartphones, laptop computers, digital broadcasting terminals, personal digital assistants (PDAs), portable multimedia players (PMPs), navigation devices, board PCs, tablet PCs, ultrabooks, wearable devices (e.g., smartwatches, smart glasses, head-mounted displays), digital televisions, desktop computers, digital signage, etc.

[0251] In a content streaming system, each server can operate as a distributed server, in which case the data received from each server can be distributed.

[0252] The scope of this disclosure includes software or machine-executable commands (e.g., operating systems, applications, firmware, programs, etc.) for enabling the operation of methods according to various embodiments to be executed on a device or computer, and non-transitory computer-readable media having such software or commands stored thereon and executable on a device or computer.

[0253] Industrial applicability

[0254] The embodiments disclosed herein can be used to encode or decode images.

Claims

1. An image decoding method performed by an image decoding device, the image decoding method comprising the following steps: Obtain APS parameter type information from the bit stream, specifying the APS parameter type signaled by the adaptive parameter set APS. After obtaining the APS parameter type information from the bit stream, obtain the APS identifier information specifying the APS from the bit stream; as well as The image is reconstructed based on the APS identifier information.

2. The image decoding method according to claim 1, wherein, Based on the value of 0 for the APS parameter type information, the APS parameter type is determined to be the full-loop filter ALF parameter type.

3. The image decoding method according to claim 1, wherein, Based on the value of 1 in the APS parameter type information, the APS parameter type is determined to be a luminance mapping LMCS parameter type with chroma scaling.

4. The image decoding method according to claim 1, wherein, Based on the value of 2 in the APS parameter type information, the APS parameter type is determined to be a scaling list parameter type.

5. The image decoding method according to claim 1, wherein, Based on the APS parameter type being either a full-loop filter ALF parameter or a scaling list parameter, the APS identifier is determined to be a value in the range of 0 to 7.

6. The image decoding method according to claim 1, wherein, Based on the fact that the APS parameter type is a luminance mapping LMCS parameter with chroma scaling, the APS identifier is determined to be a value in the range of 0 to 3.

7. An image encoding method performed by an image encoding device, the image encoding method comprising the following steps: Determine the type of the adaptive parameter set (APS) parameters; The APS identifier for the specified APS is determined based on the APS parameter type; After encoding the APS parameter type information specifying the APS parameter type into a bit stream, the APS identifier information specifying the APS identifier is encoded into the bit stream; as well as The image is encoded based on the APS identifier.

8. The image encoding method according to claim 7, wherein, Based on the fact that the APS parameter type is a full-loop filter ALF parameter, the value of the APS parameter type information specifying the APS parameter type is determined to be 0.

9. The image encoding method according to claim 7, wherein, Based on the fact that the APS parameter type is a luminance mapping LMCS parameter with chroma scaling, the value of the APS parameter type information specifying the APS parameter type is determined to be 1.

10. The image encoding method according to claim 7, wherein, Based on the fact that the APS parameter type is a scaling list parameter, the value of the APS parameter type information specifying the APS parameter type is determined to be 2.

11. The image encoding method according to claim 7, wherein, Based on the APS parameter type being either a full-loop filter ALF parameter or a scaling list parameter, the APS identifier is determined to be a value in the range of 0 to 3.

12. The image encoding method according to claim 7, wherein, Since the APS parameter type is a luminance mapping LMCS parameter with chroma scaling, the APS identifier is determined to be a value in the range of 0 to 7.

13. A non-transitory computer-readable recording medium storing a bitstream generated by the image encoding method according to claim 7.

14. A method for transmitting a bitstream generated by an image encoding method, the image encoding method comprising the following steps: Determine the type of the adaptive parameter set (APS) parameters; The APS identifier for the specified APS is determined based on the APS parameter type; After encoding the APS parameter type information specifying the APS parameter type into the bit stream, the APS identifier information specifying the APS identifier is encoded into the bit stream; as well as The image is encoded based on the APS identifier.

Images