Video decoding method and apparatus for coding video information including a picture header.

Abstract

Providing a video decoding method. [Solution] The video decoding method performed by the decoding device according to this document is characterized by including the steps of: obtaining a flag for the presence or absence of a PH (Picture Header, PH)NAL (Network Abstraction Layer) unit; obtaining a PH based on the flag; and decoding the current picture associated with the PH based on the PH.

Description

TECHNICAL FIELD

[0001] This document relates to video coding technology, and more specifically, to a video decoding method and apparatus for adaptively coding a PH NAL unit in a video coding system.

BACKGROUND ART

[0002] Recently, demand for high-resolution, high-quality images such as HD (High Definition) images and UHD (Ultra High Definition) images has been increasing in various fields. As image data becomes higher in resolution and quality, the amount of information or bits to be transmitted relatively increases compared to existing image data. Therefore, when image data is transmitted using a medium such as existing wired and wireless broadband lines, or image data is stored using an existing recording medium, transmission costs and storage costs increase.

[0003] Accordingly, highly efficient image compression technology is required to effectively transmit, store, and reproduce information of high-resolution and high-quality images.

SUMMARY OF THE INVENTION

PROBLEMS TO BE SOLVED BY THE INVENTION

[0004] A technical problem of this document is to provide a method and apparatus for improving image coding efficiency.

[0005] Another technical problem of this document is to provide a method and apparatus for coding a flag indicating the presence or absence of a PH NAL unit.

MEANS FOR SOLVING THE PROBLEMS

[0006] According to one embodiment of this document, a video decoding method performed by a decoding device is provided. The method is characterized by including the steps of: obtaining a flag indicating the presence or absence of a PH (Picture Header, PH)NAL (Network Abstraction Layer) unit; obtaining a PH based on the flag; and decoding the current picture associated with the PH based on the PH.

[0007] Another embodiment of this document provides a decoding device for performing video decoding. The decoding device is characterized by including an entropy decoding unit that acquires a flag indicating the presence or absence of a PH (Picture Header, PH)NAL (Network Abstraction Layer) unit and acquires a PH based on the flag, and a prediction unit that decodes the current picture associated with the PH based on the PH.

[0008] Another embodiment of this document provides a video encoding method performed by an encoding device. The method is characterized by including the steps of: determining whether a PH NAL (Network Abstraction Layer) unit containing a PH (Picture Header, PH) currently associated with a picture exists; generating a flag indicating the presence or absence of the PH NAL unit based on the result of the determination; and encoding video information including the flag.

[0009] Another embodiment of this document provides a video encoding device. The encoding device is characterized by including an entropy encoding unit that determines whether a PH NAL (Network Abstraction Layer) unit containing a PH (Picture Header, PH) associated with a picture currently exists, generates a flag indicating the presence or absence of the PH NAL unit based on the result of the determination, and encodes video information including the flag.

[0010] Another embodiment of this document provides a computer-readable digital storage medium that stores a bitstream containing video information for performing a video decoding method. In the computer-readable digital storage medium, the video decoding method is characterized by including the steps of: obtaining a flag for the presence or absence of a PH (Picture Header, PH)NAL (Network Abstraction Layer) unit; obtaining a PH based on the flag; and decoding the current picture associated with the PH based on the PH. [Effects of the Invention]

[0011] According to this document, a flag can be signaled to indicate whether a PH NAL unit is present, and based on this flag, the NAL unit can be adaptively adjusted to the bitrate of the bitstream, thereby improving overall coding efficiency.

[0012] According to this document, based on a flag indicating whether or not PH NAL units exist, constraints can be set on the number of slices in the current picture and on the existence of PH NAL units in related pictures. This allows for adaptive adjustment of NAL units to the bitstream bitrate, thereby improving overall coding efficiency. [Brief explanation of the drawing]

[0013] [Figure 1] An example of a video / image coding system to which the embodiments described herein may be applied is schematically shown. [Figure 2] This figure schematically illustrates the configuration of a video / image encoding device to which the embodiments described herein may be applied. [Figure 3] This figure schematically illustrates the configuration of a video / image decoding device to which the embodiments described herein may be applied. [Figure 4] This illustrates the hierarchical structure of coded video information. [Figure 5] The encoding procedure described in the examples in this document is illustrated as an example. [Figure 6] The decoding procedure described in the examples in this document is illustrated. [Figure 7] This example illustrates the picture header configuration within a NAL unit depending on the presence or absence of a PH NAL unit. [Figure 8] This document outlines the image encoding method using the encoding device described herein. [Figure 9] A schematic diagram of the encoding device used for the image encoding method described in this document is shown below. [Figure 10] The image decoding method using the decoding device described in this document is outlined below. [Figure 11] A schematic diagram of a decoding device that performs the image decoding method described in this document is shown. [Figure 12] An illustrative diagram of a content streaming system structure to which the embodiments described herein apply is shown. [Modes for carrying out the invention]

[0014] This document may be subject to various modifications and may have multiple embodiments, and specific embodiments are illustrated in the drawings and will be described in detail. However, this is not intended to limit this document to the specific embodiments. Terms used in this specification are merely used to describe specific embodiments, and are not intended to limit the technical spirit of this document. Singular expressions include plural expressions unless the context clearly dictates otherwise. As used herein, terms such as "comprise" or "have" are intended to specify the presence of the features, numbers, steps, operations, components, parts, or combinations thereof described in the specification, and it should be understood that they do not preclude the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof in advance.

[0015] Meanwhile, each configuration in the drawings described in this document is illustrated independently for the convenience of explanation relating to different characteristic functions, and does not mean that each configuration is implemented by separate hardware or separate software. For example, among the respective configurations, two or more configurations may be combined to form one configuration, and one configuration may be divided into a plurality of configurations. Embodiments in which respective configurations are integrated and / or separated are also included in the scope of rights of this document without departing from the essence of this document.

[0016] Hereinafter, preferred embodiments of this document will be described in more detail with reference to the accompanying drawings. Hereinafter, the same reference numerals are used for the same components in the drawings, and overlapping descriptions of the same components may be omitted.

[0017] Figure 1 schematically shows an example of a video / image coding system to which the embodiments of this document can be applied.

[0018] As shown in Figure 1, a video / image coding system may include a first apparatus (source device) and a second apparatus (receiving device). The source device may transmit encoded video / image information or data to the receiving device via a digital recording medium or a network in the form of a file or streaming.

[0019] The source device may comprise a video source, an encoding apparatus, and a transmission unit. The receiving device may comprise a receiving unit, a decoding apparatus, and a renderer. The encoding apparatus may be referred to as a video / image encoding apparatus, and the decoding apparatus may be referred to as a video / image decoding apparatus. A transmitter may be included in the encoding apparatus. A receiver may be included in the decoding apparatus. The renderer may comprise a display unit, and the display unit may be configured as a separate device or an external component.

[0020] The video source may obtain a video / image through a process such as capturing, synthesizing, or generating the video / image. The video source may include a video / image capture device and / or a video / image generation device. The video / image capture device may comprise, for example, one or more cameras, a video / image archive containing previously captured video / images, and the like. The video / image generation device may comprise, for example, a computer, a tablet, a smartphone, and the like, and may (electronically) generate a video / image. For example, a virtual video / image may be generated via a computer or the like, and in this case, the video / image capture process may be replaced in the process of generating related data.

[0021] An encoding device can encode input video / images. For compression and coding efficiency, the encoding device can perform a series of steps, including prediction, transformation, and quantization. The encoded data (encoded video / image information) can be output in bitstream format.

[0022] The transmitting unit can transmit encoded video / image information or data output in bitstream format to the receiving unit of a receiving device via a digital recording medium or network in file or streaming format. The digital recording medium can include various recording media such as USB, SD, CD, DVD, Blu-ray, HDD, and SSD. The transmitting unit may include elements for generating media files via a predetermined file format and may include elements for transmission via a broadcast / communication network. The receiving unit can receive / extract the bitstream and transmit it to a decoding device.

[0023] A decoding device can decode video / images by performing a series of steps, such as inverse quantization, inverse transformation, and prediction, corresponding to the operation of the encoding device.

[0024] The renderer can render the decoded video / image. The rendered video / image can be displayed via the display unit.

[0025] This document relates to video / image coding. For example, the methods / embodiments disclosed in this document can be applied to methods disclosed in the VVC (versatile video coding) standard, EVC (essential video coding) standard, AV1 (AOMedia Video 1) standard, AVS2 (2nd generation of audio video coding standard), or next-generation video / image coding standards (e.g., H.267 or H.268).

[0026] This document presents various embodiments relating to video / image coding, and unless otherwise noted, these embodiments may be implemented in combination with each other.

[0027] In this document, "video" can mean a collection of images over time. "Picture" generally refers to a unit representing a single image at a specific time point in time, while "subpicture," "slice," and "tile" are units that constitute a part of a picture in coding. A subpicture, slice, or tile may contain one or more CTUs (coding tree units). A single picture may consist of one or more subpictures, slices, or tiles. A single picture may consist of one or more groups of tiles. A group of tiles may contain one or more tiles. A brick may represent a rectangular region of CTU rows within a tile in a picture. A tile may be partitioned into multiple bricks, each of which consists of one or more CTU rows within the tile. A tile that is not partitioned into multiple bricks may also be referred to as a brick.A brick scan is a specific sequential ordering of CTUs partitioning a picture in which the CTUs are ordered consecutively in CTU raster scan in a brick, bricks within a tile are ordered consecutively in a raster scan of the bricks of the tile, and tiles in a picture are ordered consecutively in a raster scan of the tiles of the picture. A subpicture may represent a rectangular region of one or more slices within a picture. In other words, a subpicture contains one or more slices that collectively cover a rectangular region of a picture. A tile is a rectangular region of CTUs within a particular tile column and a particular tile row in a picture.The tile column is a rectangular region of CTUs having a height equal to the height of the picture and a width specified by syntax elements in the picture parameter set. The tile row is a rectangular region of CTUs having a width specified by syntax elements in the picture parameter set and a height equal to the width of the picture. A tile scan is a specific sequential ordering of CTUs partitioning a picture in which the CTUs are ordered consecutively in CTU raster scan in a tile whereas tiles in a picture are ordered consecutively in a raster scan of the tiles of the picture.A slice includes an integer number of bricks of a picture that may be exclusively contained in a single NAL unit. A slice may consist of either a number of complete tiles or only a consecutive sequence of complete bricks of one tile. In this document, tile groups and slices may be used interchangeably. For example, in this document, a tile group / tile group header may be called a slice / slice header.

[0028] A pixel or pel can refer to the smallest unit that makes up a picture (or image). Alternatively, the term "sample" can be used as a counterpart to pixel. A sample can generally represent a pixel or a pixel value, and can represent only the luma component pixel / pixel value, or only the chroma component pixel / pixel value.

[0029] A unit can represent a basic unit of image processing. A unit can contain at least one of a specific region of a picture and information associated with that region. A unit can contain one luma block and two chroma (e.g., cb, cr) blocks. The term unit may sometimes be used interchangeably with terms such as block or area. In general, an M×N block can contain a sample (or sample array) consisting of M columns and N rows, or a set (or array) of transform coefficients.

[0030] In this specification, "A or B" may mean "A only," "B only," or "both A and B." In other words, in this specification, "A or B" may be interpreted as "A and / or B." For example, in this specification, "A, B or C" may mean "A only," "B only," "C only," or "any combination of A, B and C."

[0031] In this specification, slashes ( / ) and commas may mean "and / or". For example, "A / B" may mean "A and / or B". Thus, "A / B" may mean "A only", "B only", or "both A and B". For example, "A, B, C" may mean "A, B or C".

[0032] In this specification, "at least one of A and B" may mean "A only," "B only," or "both A and B." Furthermore, in this specification, the expressions "at least one of A or B" and "at least one of A and / or B" may be interpreted similarly to "at least one of A and B."

[0033] Furthermore, in this specification, "at least one of A, B and C" may mean "A only," "B only," "C only," or "any combination of A, B and C." Also, "at least one of A, B or C" or "at least one of A, B and / or C" may mean "at least one of A, B and C."

[0034] Furthermore, parentheses used in this specification may mean "for example." Specifically, when "prediction (intra-prediction)" is indicated, "intra-prediction" may be proposed as an example of "prediction." In other words, "prediction" in this specification is not limited to "intra-prediction," and "intra-prediction" may be proposed as an example of "prediction." Also, when "prediction (i.e., intra-prediction)" is indicated, "intra-prediction" may be proposed as an example of "prediction."

[0035] Technical features described individually in each drawing in this specification may be implemented individually or simultaneously.

[0036] The following drawings have been prepared to illustrate a specific example of this specification. The names of specific devices and signals / messages / fields shown in the drawings are illustrative and not limited to the specific names used in the following drawings.

[0037] Figure 2 is a schematic diagram illustrating the configuration of a video / image encoding device to which the embodiments described in this document may be applied. Hereinafter, the term "video encoding device" may include an image encoding device.

[0038] As shown in Figure 2, the encoding device 200 can be configured to include an image partitioner 210, a predictor 220, a residual processor 230, an entropy encoder 240, an adder 250, a filter 260, and a memory 270. The predictor 220 may include an inter-predictor 221 and an intra-predictor 222. The residual processor 230 may include a transformer 232, a quantizer 233, a dequantizer 234, and an inverse transformer 235. The residual processor 230 may further include a subtractor 231. The adder 250 may be called a reconstructor or a reconstructed block generator. The aforementioned image segmentation unit 210, prediction unit 220, residual processing unit 230, entropy encoding unit 240, addition unit 250, and filtering unit 260 can be configured by one or more hardware components (e.g., an encoder chipset or processor) depending on the embodiment. Furthermore, the memory 270 may include a DPB (decoded picture buffer) and may be configured by a digital recording medium. The hardware components may also further include the memory 270 as an internal / external component.

[0039] The image splitting unit 210 can split an input image (or picture, frame) input to the encoding device 200 into one or more processing units. For example, one of these processing units may be called a coding unit (CU). In this case, a coding unit can be recursively split from a coding tree unit (CTU) or the largest coding unit (LCU) using a QTBTTT (Quad-tree binary-tree ternary-tree) structure. For example, one coding unit can be split into multiple coding units of deeper depth based on a quad-tree structure, a binary-tree structure, and / or a ternary structure. In this case, for example, the quad-tree structure may be applied first, followed by the binary-tree structure and / or the ternary structure. Alternatively, the binary-tree structure may be applied first. The coding procedure described in this document can be executed based on the final coding unit that cannot be further split. In this case, based on coding efficiency according to image characteristics, the largest coding unit can be immediately used as the final coding unit, or, if necessary, the coding unit can be recursively divided into lower-depth coding units so that the optimally sized coding unit is used as the final coding unit. Here, the coding procedure may include procedures such as prediction, transformation, and restoration, which will be described later. As another example, the processing unit may further comprise a prediction unit (PU) or a transformation unit (TU). In this case, the prediction unit and the transformation unit can each be separated or partitioned from the final coding unit described above.The prediction unit is a unit of sample prediction, and the conversion unit is a unit that derives a conversion coefficient and / or a unit that derives a residual signal from the conversion coefficient.

[0040] The term "unit" can sometimes be used interchangeably with terms such as "block" or "area." Generally, an M×N block can represent a set of samples or transform coefficients consisting of M columns and N rows. A sample can generally represent a pixel or a pixel value, and may represent only the luminance (luma) component pixel / pixel value, or only the chroma component pixel / pixel value. A sample can be used as the term corresponding to a single picture (or image) pixel or pel.

[0041] The encoding device 200 can generate a residual signal (residual block, residual sample array) by subtracting the prediction signal (predicted block, predicted sample array) output from the inter-prediction unit 221 or intra-prediction unit 222 from the input image signal (original block, original sample array), and the generated residual signal is transmitted to the conversion unit 232. In this case, as shown in the figure, the unit that subtracts the prediction signal (predicted block, predicted sample array) from the input image signal (original block, original sample array) within the encoder 200 can be called the subtraction unit 231. The prediction unit can perform a prediction for the block to be processed (hereinafter referred to as the current block) and generate a predicted block that includes the predicted sample for the current block. The prediction unit can determine whether intra-prediction or inter-prediction is applied on a current block or CU basis. The prediction unit can generate various prediction-related information, such as prediction mode information, and transmit it to the entropy encoding unit 240, as will be described later in the explanation of each prediction mode. The prediction information can be encoded by the entropy encoding unit 240 and output in bitstream format.

[0042] The intra-prediction unit 222 can predict the current block by referring to a sample in the current picture. The referenced sample can be located in the vicinity (neighbor) of the current block or at a distance, depending on the prediction mode. In intra-prediction, the prediction mode can include multiple non-directional modes and multiple directional modes. Non-directional modes can include, for example, DC mode and Planar mode. Directional modes can include, for example, 33 directional prediction modes or 65 directional prediction modes, depending on the degree of fineness of the prediction direction. However, this is merely an example, and more or fewer directional prediction modes can be used depending on the settings. The intra-prediction unit 222 can also determine the prediction mode to be applied to the current block using the prediction modes applied to adjacent blocks.

[0043] The interprediction unit 221 can derive a predicted block relative to the current block based on a reference block (reference sample array) identified by motion vectors on the reference picture. In this case, in order to reduce the amount of motion information transmitted in interprediction mode, motion information can be predicted in units of blocks, subblocks, or samples based on the correlation of motion information between adjacent blocks and the current block. The motion information may include motion vectors and reference picture indices. The motion information may further include interprediction direction information (L0 prediction, L1 prediction, Bi prediction, etc.). In the case of interprediction, adjacent blocks may include spatially adjacent blocks existing in the current picture and temporally adjacent blocks existing in the reference picture. The reference picture containing the reference block and the reference picture containing the temporally adjacent block may be the same or different. The temporally adjacent block may be called a collocated reference block, colCU, etc., and the reference picture containing the temporally adjacent block may be called a collocated picture (colPic). For example, the interpretation unit 221 can construct a motion information candidate list based on adjacent blocks and generate information indicating which candidates are used to derive the motion vector and / or reference picture index of the current block. Interpretation can be performed based on various prediction modes; for example, in skip mode and merge mode, the interpretation unit 221 can use the motion information of adjacent blocks as the motion information of the current block. In skip mode, unlike merge mode, a residual signal may not be transmitted.In motion vector prediction (MVP) mode, the motion vector of an adjacent block is used as a motion vector predictor, and the motion vector difference is signaled to indicate the motion vector of the current block.

[0044] The prediction unit 220 can generate prediction signals based on various prediction methods described later. For example, the prediction unit can apply intra-prediction or inter-prediction for predictions on a single block, and can also apply intra-prediction and inter-prediction simultaneously. This can be called combined inter and intra prediction (CIIP). The prediction unit can also be based on intra-block copy (IBC) prediction mode or palette mode for predictions on blocks. The IBC prediction mode or palette mode can be used for content image / video coding such as in games, for example, as in SCC (screen content coding). IBC basically performs predictions within the current picture, but can be performed similarly to inter-prediction in that it derives reference blocks within the current picture. That is, IBC can utilize at least one of the inter-prediction techniques described in this document. Palette mode can be seen as an example of intra-coding or intra-prediction. When palette mode is applied, sample values ​​within the picture can be signaled based on information about the palette table and palette index.

[0045] The prediction signal generated via the prediction unit (including the inter-prediction unit 221 and / or the intra-prediction unit 222) can be used to generate a reconstructed signal or a residual signal. The transformation unit 232 can generate transformation coefficients by applying a transformation technique to the residual signal. For example, the transformation technique may include at least one of the following: DCT (Discrete Cosine Transform), DST (Discrete Sine Transform), KLT (Karhunen-Loeve Transform), GBT (Graph-Based Transform), or CNT (Conditionally Non-linear Transform). Here, GBT means a transformation obtained from a graph when the relationship information between pixels is represented by this graph. CNT means a transformation obtained by generating a prediction signal using all previously reconstructed pixels and obtaining a transformation based on it. The transformation process can also be applied to pixel blocks of the same size and square, or to non-square, variable-sized blocks.

[0046] The quantization unit 233 quantizes the conversion coefficients and transmits them to the entropy encoding unit 240, which can encode the quantized signal (information about the quantized conversion coefficients) and output it as a bitstream. The information about the quantized conversion coefficients can be called residual information. The quantization unit 233 can rearrange the block-form quantized conversion coefficients into a one-dimensional vector form based on the coefficient scan order, and can also generate information about the quantized conversion coefficients based on the one-dimensional vector form of the quantized conversion coefficients. The entropy encoding unit 240 can perform various encoding methods, such as exponential Golomb, CAVLC (context-adaptive variable length coding), and CABAC (context-adaptive binary arithmetic coding). In addition to the quantized conversion coefficients, the entropy encoding unit 240 can also encode information necessary for video / image restoration (e.g., the values ​​of syntax elements) together with or separately from the quantized conversion coefficients. Encoded information (e.g., encoded video / image information) can be transmitted or stored in bitstream form in units of network abstraction layer (NAL) units. The video / image information may further include information about various parameter sets, such as adaptation parameter sets (APS), picture parameter sets (PPS), sequence parameter sets (SPS), or video parameter sets (VPS). The video / image information may also further include general constraint information. Information and / or syntax elements transmitted / signaled from the encoding device to the decoding device in this document may be included in the video / image information. The video / image information may be encoded via the encoding procedure described above and included in the bitstream.The bitstream can be transmitted over a network or stored on a digital recording medium. Here, the network may include broadcast networks and / or communication networks, and the digital recording medium may include various recording media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. The signal output from the entropy encoding unit 240 can be transmitted by a transmitting unit (not shown) and / or stored by a storage unit (not shown) which are configured as internal / external elements of the encoding device 200, or the transmitting unit may be included in the entropy encoding unit 240.

[0047] The quantized conversion coefficients output from the quantization unit 233 can be used to generate a prediction signal. For example, a residual signal (residual block or residual sample) can be reconstructed by applying inverse quantization and inverse transformation to the quantized conversion coefficients via the inverse quantization unit 234 and the inverse transformation unit 235. The adder 250 can generate a reconstructed signal (reconstructed picture, reconstructed block, reconstructed sample array) by adding the reconstructed residual signal to the prediction signal output from the inter-prediction unit 221 or the intra-prediction unit 222. If there is no residual for the block to be processed, such as when skip mode is applied, the predicted block can be used as the reconstructed block. The adder 250 can be called the reconstructor unit or reconstructed block generator. The generated reconstructed signal can be used for intra-prediction of the next block to be processed in the current picture, and can also be used for inter-prediction of the next picture after filtering, as described later.

[0048] On the other hand, LMCS (luma mapping with chroma scaling) can also be applied during the picture encoding and / or restoration process.

[0049] The filtering unit 260 can improve subjective / objective image quality by applying filtering to the restored signal. For example, the filtering unit 260 can apply various filtering methods to the restored picture to generate a modified restored picture, and the modified restored picture can be stored in the memory 270, specifically in the DPB of the memory 270. The various filtering methods can include, for example, deblocking filtering, sample adaptive offset, adaptive loop filter, and bilateral filter. The filtering unit 260 can generate various filtering-related information and transmit it to the entropy encoding unit 240, as will be described later in the explanation of each filtering method. The filtering-related information can be encoded by the entropy encoding unit 240 and output in bitstream format.

[0050] The corrected restored picture sent to memory 270 can be used as a reference picture in the interpretation unit 221. When interpretation is applied via this, the encoding device can avoid prediction mismatches between the encoding device 200 and the decoding device 300, and can also improve encoding efficiency.

[0051] Memory 270DPB can store the corrected restored picture for use as a reference picture in the inter-prediction unit 221. Memory 270 can store motion information of blocks from which motion information has been derived (or encoded) in the current picture and / or motion information of blocks in the picture that have already been restored. The stored motion information can be transmitted to the inter-prediction unit 221 for use as motion information of spatially adjacent blocks or motion information of temporally adjacent blocks. Memory 270 can store restored samples of restored blocks in the current picture and transmit them to the intra-prediction unit 222.

[0052] Figure 3 is a schematic diagram illustrating the configuration of a video / image decoding device to which the embodiments described in this document may be applied.

[0053] As shown in Figure 3, the decoding device 300 can be configured to include an entropy decoder 310, a residual processor 320, a predictor 330, an adder 340, a filter 350, and a memory 360. The predictor 330 can include an inter-prediction unit 331 and an intra-prediction unit 332. The residual processor 320 can include a dequantizer 321 and an inverse transformer 322. The aforementioned entropy decoder 310, residual processor 320, predictor 330, adder 340, and filtering unit 350 can be configured by a single hardware component (e.g., a decoder chipset or processor) depending on the embodiment. The memory 360 can also include a decoded picture buffer (DPB) and can be configured by a digital recording medium. The aforementioned hardware component may also further include memory 360 as an internal / external component.

[0054] When a bitstream containing video / image information is input, the decoding device 300 can reconstruct the image in accordance with the process by which the video / image information was processed in the encoding device shown in Figure 2. For example, the decoding device 300 can derive units / blocks based on block division-related information obtained from the bitstream. The decoding device 300 can perform decoding using the processing units applied in the encoding device. Thus, the decoding processing units are, for example, coding units, which can be divided from a coding tree unit or a maximum coding unit according to a quad-tree structure, a binary tree structure, and / or a terminally tree structure. One or more conversion units can be derived from the coding unit. The reconstructed image signal decoded and output via the decoding device 300 can then be reproduced via a playback device.

[0055] The decoding device 300 can receive the signal output from the encoding device shown in Figure 2 in bitstream form, and the received signal can be decoded via the entropy decoding unit 310. For example, the entropy decoding unit 310 can parse the bitstream to derive information necessary for image restoration (or picture restoration) (e.g., video / image information). The video / image information may further include information about various parameter sets, such as the adaptation parameter set (APS), picture parameter set (PPS), sequence parameter set (SPS), or video parameter set (VPS). The video / image information may also further include general constraint information. The decoding device can further decode the picture based on the parameter set information and / or the general constraint information. The signaling / received information and / or syntax elements described later in this document can be decoded via the decoding procedure and obtained from the bitstream. For example, the entropy decoding unit 310 can decode information in the bitstream based on a coding method such as exponential Golomb coding, CAVLC, or CABAC, and output the values ​​of syntax elements necessary for image reconstruction, quantized values ​​of conversion coefficients related to residuals, etc. More specifically, the CABAC entropy decoding method receives bins corresponding to each syntax element in the bitstream, determines a context model using the information of the syntax element to be decoded, the decoded information of the surrounding and decoded blocks, or the symbol / bin information decoded in a previous step, predicts the probability of bin occurrence based on the determined context model, and performs arithmetic decoding of the bins to generate symbols corresponding to the values ​​of each syntax element. At this time, after determining the context model, the CABAC entropy decoding method can update the context model using the decoded symbol / bin information for the context model of the next symbol / bin.Of the information decoded by the entropy decoding unit 310, information related to prediction is provided to the prediction unit (inter-prediction unit 332 and intra-prediction unit 331), and the residual values ​​from which entropy decoding has been performed in the entropy decoding unit 310, i.e., quantized conversion coefficients and related parameter information, can be input to the residual processing unit 320. The residual processing unit 320 can derive residual signals (residual blocks, residual samples, residual sample arrays). In addition, of the information decoded by the entropy decoding unit 310, information related to filtering can be provided to the filtering unit 350. On the other hand, a receiving unit (not shown) that receives signals output from the encoding device can be further configured as an internal / external element of the decoding device 300, or the receiving unit is a component of the entropy decoding unit 310. On the other hand, the decoding device relating to this document may be called a video / image / picture decoding device, and the decoding device may also be divided into an information decoder (video / image / picture information decoder) and a sample decoder (video / image / picture sample decoder). The information decoder may include the entropy decoding unit 310, and the sample decoder may include at least one of the inverse quantization unit 321, inverse transformation unit 322, addition unit 340, filtering unit 350, memory 360, inter-prediction unit 332, and intra-prediction unit 331.

[0056] The inverse quantization unit 321 can inverse quantize the quantized transformation coefficients and output the transformation coefficients. The inverse quantization unit 321 can rearrange the quantized transformation coefficients in a two-dimensional block form. In this case, the rearrangement can be performed based on the coefficient scan order performed by the encoding device. The inverse quantization unit 321 can perform inverse quantization on the quantized transformation coefficients using quantization parameters (e.g., quantization step size information) to obtain the transformation coefficients.

[0057] In the inverse conversion unit 322, the conversion coefficients are inversely converted to obtain a residual signal (residual block, residual sample array).

[0058] The prediction unit can perform a prediction on the current block and generate a predicted block containing prediction samples for the current block. Based on the prediction information output from the entropy decoding unit 310, the prediction unit can determine whether intra-prediction or inter-prediction is applied to the current block and can determine a specific intra / inter-prediction mode.

[0059] The prediction unit 320 can generate prediction signals based on various prediction methods described later. For example, the prediction unit can apply intra-prediction or inter-prediction for prediction of a single block, and can also apply intra-prediction and inter-prediction simultaneously. This can be called combined inter and intra prediction (CIIP). The prediction unit can also be based on intra-block copy (IBC) prediction mode or palette mode for prediction of a block. The IBC prediction mode or palette mode can be used for content image / video coding such as in games, for example, as in SCC (screen content coding). IBC basically performs prediction within the current picture, but can be performed similarly to inter-prediction in that it derives reference blocks within the current picture. That is, IBC can utilize at least one of the inter-prediction techniques described in this document. Palette mode can be seen as an example of intra-coding or intra-prediction. When palette mode is applied, information about the palette table and palette index can be included in the video / image information and signaled.

[0060] The intra-prediction unit 331 can predict the current block by referring to a sample in the current picture. The referenced sample can be located in the vicinity (neighbor) of the current block or at a distance from it, depending on the prediction mode. In intra-prediction, the prediction mode can include a plurality of non-directional modes and a plurality of directional modes. The intra-prediction unit 331 can also determine the prediction mode to be applied to the current block using the prediction modes applied to adjacent blocks.

[0061] The interprediction unit 332 can derive a predicted block for the current block based on a reference block (reference sample array) identified by motion vectors on a reference picture. In this case, in order to reduce the amount of motion information transmitted from the interprediction mode, motion information can be predicted in blocks, subblocks, or samples based on the correlation of motion information between adjacent blocks and the current block. The motion information may include motion vectors and reference picture indices. The motion information may further include interprediction direction information (L0 prediction, L1 prediction, Bi prediction, etc.). In the case of interprediction, adjacent blocks may include spatially adjacent blocks that exist in the current picture and temporally adjacent blocks that exist in the reference picture. For example, the interprediction unit 332 can construct a motion information candidate list based on adjacent blocks and derive the motion vector and / or reference picture index of the current block based on the received candidate selection information. Interprediction can be performed based on various prediction modes, and the prediction information may include information indicating the mode of interprediction for the current block.

[0062] The summing unit 340 can generate a restored signal (restored picture, restored block, restored sample array) by adding the acquired residual signal to the predicted signal (predicted block, predicted sample array) output from the prediction unit (including the inter-prediction unit 332 and / or intra-prediction unit 331). If there is no residual for the block to be processed, such as when skip mode is applied, the predicted block can be used as the restored block.

[0063] The summing unit 340 may be called the restoration unit or restoration block generation unit. The generated restoration signal can be used for intra-prediction of the next block to be processed in the current picture, and can be output after filtering as described later, or it can be used for intra-prediction of the next picture.

[0064] On the other hand, LMCS (luma mapping with chroma scaling) can also be applied during the picture decoding process.

[0065] The filtering unit 350 can apply filtering to the restored signal to improve subjective / objective image quality. For example, the filtering unit 350 can apply various filtering methods to the restored picture to generate a modified restored picture, and can transmit the modified restored picture to the memory 360, specifically to the DPB of the memory 360. The various filtering methods may include, for example, deblocking filtering, sample adaptive offset, adaptive loop filter, and bilateral filter.

[0066] The (modified) restored picture stored in the DPB of memory 360 can be used as a reference picture by the inter-prediction unit 332. Memory 360 can store motion information of blocks from which motion information in the current picture has been derived (or decoded) and / or motion information of blocks in the picture that have already been restored. The stored motion information can be transmitted to the inter-prediction unit 260 for use as motion information of spatially adjacent blocks or motion information of temporally adjacent blocks. Memory 360 can store restored samples of restored blocks in the current picture and transmit them to the intra-prediction unit 331.

[0067] In this specification, the embodiments described for the filtering unit 260, inter-prediction unit 221, and intra-prediction unit 222 of the encoding device 200 can be applied identically or in a corresponding manner to the filtering unit 350, inter-prediction unit 332, and intra-prediction unit 331 of the decoding device 300, respectively.

[0068] In this document, at least one of quantization / inverse quantization and / or transformation / inverse transformation may be omitted. If quantization / inverse quantization is omitted, the quantized transformation coefficient may be called a transformation coefficient. If transformation / inverse transformation is omitted, the transformation coefficient may be called a coefficient or residual coefficient, or for consistency of expression, may still be called a transformation coefficient.

[0069] In this document, quantized transformation coefficients and transformation coefficients may be referred to as transformation coefficients and scaled transformation coefficients, respectively. In this case, residual information may include information about the transformation coefficients (etc.), and such information may be signaled via residual coding syntax. Transformation coefficients may be derived based on the residual information (or information about the transformation coefficients (etc.)), and scaled transformation coefficients may be derived via inverse transformation (scaling) of the transformation coefficients. Residual samples may be derived based on inverse transformation (transformation) of the scaled transformation coefficients. This may be applied / expressed similarly in other parts of this document.

[0070] Figure 4 illustrates the hierarchical structure of coded video information.

[0071] Figure 4 illustrates a video / image coded according to the coding layers and structure described in this document. Referring to Figure 4, the coded video / image can be divided into a Video Coding Layer (VCL) that handles the video / image and the video / image decoding process, a subsystem that transmits and stores the coded information, and a Network Abstraction Layer (NAL) that exists between the VCL and the subsystem and is responsible for its functions.

[0072] For example, VCL can generate VCL data containing compressed image data (slice data), or it can generate a Picture Parameter Set (PPS), Sequence Parameter Set (SPS), Video Parameter Set (VPS), or a parameter set containing SEI (Supplemental Enhancement Information, SEI) messages additionally required during the video decoding process.

[0073] Furthermore, in NAL, for example, a NAL unit can be generated by adding header information (NAL unit header) to an RBSP (Raw Byte Sequence Payload) generated by VCL. In this case, the RBSP can refer to slice data, parameter sets, SEI messages, etc., generated by VCL. The NAL unit header can include NAL unit type information specified by the RBSP data contained in the NAL unit.

[0074] Furthermore, as shown in Figure 4, for example, NAL units can be classified into VCL NAL units and Non-VCL NAL units by the RBSP generated in VCL. VCL NAL units can represent NAL units containing information (slice data) for the image, while Non-VCL NAL units can represent NAL units containing information (parameter sets or SEI messages) necessary for image decoding.

[0075] The aforementioned VCL NAL units and Non-VCL NAL units can be transmitted over a network with header information attached according to the subsystem's data standard. For example, NAL units can be converted to predetermined standard data formats such as the H.266 / VVC file format, the Real-Time Transport Protocol (RTP), and the Transport Stream (TS), and transmitted over a variety of networks.

[0076] Furthermore, as mentioned above, the NAL unit type can be specified by the RBSP data structure contained within the NAL unit, and information regarding the NAL unit type can be stored in the NAL unit header and signaled.

[0077] For example, NAL units can be classified into VCL NAL unit types and Non-VCL NAL unit types depending on whether or not they contain information (slice data) related to the image. Furthermore, VCL NAL unit types can be classified by the characteristics and type of the picture contained within the VCL NAL unit, and Non-VCL NAL unit types can be classified by the type of parameter set.

[0078] The following is an example of a NAL unit type specified by the type of parameter set included in the Non-VCL NAL unit type.

[0079] -APS (Adaptation Parameter Set) NAL Unit: Type for NAL units that include APS

[0080] -DPS (Decoding Parameter Set) NAL Unit: Type for NAL units that include DPS

[0081] -VPS (Video Parameter Set) NAL Unit: Type for NAL units including VPS

[0082] -SPS (Sequence Parameter Set) NAL Unit: Type for NAL units that include SPS

[0083] -PPS (Picture Parameter Set) NAL Unit: Type for NAL units that include PPS

[0084] -PH (Picture header) NAL unit: Type for NAL units that include PH

[0085] The aforementioned NAL unit type may have syntax information for the NAL unit type, and this syntax information may be stored and signaled in the NAL unit header. For example, the syntax information may be nal_unit_type, and the NAL unit type may be specified in the nal_unit_type value.

[0086] On the other hand, as mentioned above, a single picture can contain multiple slices, and a slice can contain a slice header and slice data. In this case, a single picture header can be added (embedded) to multiple slices (sets of slice headers and slice data). A picture header (picture header syntax) can contain information / parameters that can be applied commonly to pictures. A slice header (slice header syntax) can contain information / parameters that can be applied commonly to slices. APS (APS syntax) or PPS (PPS syntax) can contain information / parameters that can be applied commonly to one or more slices or pictures. SPS (SPS syntax) can contain information / parameters that can be applied commonly to one or more sequences. VPS (VPS syntax) can contain information / parameters that can be applied commonly to multiple layers. DPS (DPS syntax) can contain information / parameters that can be applied commonly to the entire image. DPS may include information / parameters related to the concatenation of coded video sequences (CVS). In this document, HLS (High Level Syntax) may include at least one of the following: APS syntax, PPS syntax, SPS syntax, VPS syntax, DPS syntax, picture header syntax, and slice header syntax.

[0087] On the other hand, as mentioned above, generally one NAL unit type can be assigned to a single picture, and the NAL unit type can be signaled via nal_unit_type in the NAL unit header of the NAL unit containing the slice, as described above. The following table shows examples of NAL unit type codes and NAL unit type classes.

[0088] [Table 1-1]

[0089] [Table 1-2]

[0090] On the other hand, as mentioned above, a picture can consist of one or more slices. Parameters describing the picture can be signaled in a Picture Header (PH), and parameters describing a slice can be signaled in a Slice Header (SH). The PH can be transmitted to the NAL unit type itself. The SH can be present at the beginning of the NAL unit containing the slice payload (i.e., slice data).

[0091] For example, the syntax elements of the signaled PH are as follows:

[0092] [Table 2-1]

[0093] [Table 2-2]

[0094] [Table 2-3]

[0095] [Table 2-4]

[0096] [Table 2-5]

[0097] [Table 2-6]

[0098] On the other hand, the adoption of PH can mean that there should be at least two NAL units for every coded picture. For example, one of the two units would be the NAL unit for the PH, and the other would be the NAL unit for the coded slice containing the slice header (SH) and slice data. This can be problematic for low-bitrate bitstreams because additional NAL units per picture can have a significant impact on the bitrate. Therefore, it is preferable that the PH has a mode in which it does not consume new NAL units.

[0099] Therefore, this document proposes the following embodiments to solve the aforementioned problems. The proposed embodiments can be applied individually or in combination.

[0100] As an example, we propose signaling a flag in a high-level parameter set that indicates whether a PH NAL unit exists in a CLVS (Coded Layer Video Sequence). That is, the flag can indicate whether a picture header exists in a NAL unit (i.e., a PH NAL unit) or in a slice header. Here, for example, the CLVS can mean a sequence of Picture Units (PUs) having the same nuh_layer_id value. The picture unit is a set of NAL units (etc.) for a coded picture. Also, for example, the high-level parameter set can be an SPS (Sequence Parameter Set), a PPS (Picture Parameter Set), or a slice header. The flag can be called ph_nal_present_flag, or it can be called a PH NAL presence flag.

[0101] Furthermore, this document proposes an embodiment that, in relation to the PH NAL presence flag, constrains that the value of ph_nal_present_flag must be the same for all SPS referenced by CVS pictures with the same value. This constraint can mean that the value of ph_nal_present_flag must be the same for a single coded video sequence in a multi-layer bitstream.

[0102] Furthermore, as an example, we propose that if the value of ph_nal_present_flag is the same as 1, then one PH NAL unit exists, and that this PH NAL unit is associated with the picture's VCL (Video Coding Layer, VCL) NAL unit.

[0103] As an example, we propose a method in which the following constraints are applied when the value of ph_nal_present_flag is the same as 0 (i.e., when no PH NAL unit exists for each picture).

[0104] For example, the aforementioned constraints are as follows:

[0105] Firstly, every picture in CLVS can contain only one slice.

[0106] Secondly, the PH NAL unit may not exist. The PH syntax table can exist in the slice layer RBSP along with the SH (Slice Header) and slice data (slicedata). In other words, the PH syntax table can exist in the slice header.

[0107] Thirdly, the PH syntax table and the SH syntax table can begin at a byte-aligned position. To realize this, a byte alignment bit can be added between PH and SH.

[0108] Fourthly, the value of picture_header_extension_present_flag in all PPS that reference SPS is 0.

[0109] Fifth, all syntax elements that can exist in PH or SH can also exist in PH that is not SH.

[0110] As an example, a method for updating access unit detection can be proposed. That is, instead of checking for PH, all new VCL NAL units can represent a new AU (Access Unit). In other words, if the value of ph_nal_present_flag indicates that the PH NAL unit does not exist, then the VCL NAL unit containing ph_nal_present_flag can mean that the VCL NAL unit for the new AU (i.e., the picture for the new AU) is being parsed, rather than the VCL NAL unit for the previously ordered AU (i.e., the picture for the previously ordered AU). Therefore, if the value of ph_nal_present_flag indicates that the PH NAL unit does not exist, then the VCL NAL unit containing ph_nal_present_flag is the first VCL NAL unit for the picture of the new AU (e.g., the current picture on which decoding is performed). Here, AU can mean a set of Picture Units (PUs) that belong to different layers and contain coded pictures associated with the same time for the output of a Decoded Picture Buffer (DPB). Alternatively, the PUs can mean a set of NAL units that contain a single coded picture that is associated with each other and has a sequential decoding order. That is, the PUs can mean a set of NAL units for a single coded picture that is associated with each other and has a sequential decoding order. On the other hand, if the bitstream is a single-layer bitstream and not a multi-layer bitstream, then AU is the same as PU.

[0111] The embodiments proposed in this document can be implemented as described later.

[0112] For example, the SPS syntax in which the ph_nal_present_flag proposed in the examples of this document is signaled is as follows:

[0113] [Table 3]

[0114] Referring to Table 3, SPS can include ph_nal_present_flag.

[0115] For example, the semantics of the syntax element ph_nal_present_flag are as shown in the table below.

[0116] [Table 4]

[0117] For example, referring to Table 4, the syntax element ph_nal_present_flag can indicate whether a NAL unit with the same nal_unit_type as PH_NUT exists for each coded picture in a CLVS etc. (CLVSs) that references an SPS. For example, a ph_nal_present_flag equal to 1 can indicate that a NAL unit with the same nal_unit_type as PH_NUT exists for each coded picture in a CLVS etc. (CLVSs) that references an SPS. Also, for example, a ph_nal_present_flag equal to 0 can indicate that no NAL unit with the same nal_unit_type as PH_NUH exists for each coded picture in a CLVS etc. (CLVSs) that references an SPS.

[0118] Furthermore, for example, if ph_nal_present_flag is 1, the following can be applied.

[0119] NAL units with -nal_unit_type PH_NUT (i.e., PH NAL units) do not exist in CLVS referencing SPS.

[0120] -Each picture in a CLVS that references an SPS can contain one slice.

[0121] -PH can exist in the slice layer RBSP.

[0122] On the other hand, while Tables 3 and 4 propose a method in which ph_nal_present_flag is signaled to the SPS, the methods shown in Tables 3 and 4 are merely one example proposed in this document, and examples in which ph_nal_present_flag is signaled to PPS or slice headers other than the SPS can also be proposed.

[0123] On the other hand, according to one embodiment proposed in this document, for example, the picture header syntax table and picture header RBSP can be separated and signaled as shown in the table below.

[0124] [Table 5]

[0125] Furthermore, the signaled picture header syntax table is as follows:

[0126] [Table 6-1]

[0127] [Table 6-2]

[0128] [Table 6-3]

[0129] [Table 6-4]

[0130] [Table 6-5]

[0131] [Table 6-6]

[0132] Furthermore, for example, according to one embodiment proposed in this document, the slice layer RBSP can be signaled as follows.

[0133] [Table 7]

[0134] Furthermore, according to one embodiment proposed in this document, for example, one or more of the constraints shown in the table below can be applied.

[0135] [Table 8]

[0136] For example, referring to Table 8, if ph_nal_present_flag is 0, the value of picture_header_extension_present_flag can be 0.

[0137] Furthermore, for example, if both of the following conditions are true, the value of pic_rpl_present_flag must be the same as 1, which is a requirement for bitstream conformance.

[0138] -ph_nal_present_flag is 0, and the picture associated with PH is not an IDR picture.

[0139] -ph_nal_present_flag is 0, the picture associated with PH is an IDR picture, and sps_id_rpl_present_flag is the same as 1.

[0140] Here, rpl can mean reference picture list.

[0141] Furthermore, a bitstream conformance requirement is that, for example, if the value of ph_nal_present_flag is 0, the value of pic_sao_enabled_present_flag must be the same as 1.

[0142] Furthermore, a bitstream conformance requirement is that, for example, if the value of ph_nal_present_flag is 0, the value of pic_alf_enabled_present_flag must be the same as 1.

[0143] Furthermore, a bitstream conformance requirement is that, for example, if the value of ph_nal_present_flag is 0, the value of pic_deblocking_filter_override_present_flag must be the same as 1.

[0144] On the other hand, for example, the embodiment(s) can be applied by the following procedure.

[0145] Figure 5 illustrates the encoding procedure according to the examples in this document.

[0146] Referring to Figure 5, the encoding device can generate a NAL unit (etc.) containing information for the picture (S500). The encoding device can determine whether a NAL unit exists for the picture header (S510), and can determine whether a NAL unit exists for the picture header (S520).

[0147] For example, if a NAL unit exists for the picture header, the encoding device can generate a bitstream that includes a VCL NAL unit including a slice header and a PH NAL unit including a picture header (S530).

[0148] On the other hand, for example, if there is no NAL unit for the picture header, the encoding device can generate a bitstream that includes a slice header and a VCL NAL unit including the picture header (S540). That is, for example, the picture header syntax structure can be present in the slice header.

[0149] Figure 6 illustrates the decoding procedure according to the examples in this document.

[0150] Referring to Figure 6, the decoding device can receive a bitstream containing NAL units (etc.) (S600). Subsequently, the decoding device can determine whether or not a NAL unit exists for the picture header (S610).

[0151] For example, if a NAL unit exists for the picture header, the decoding device can decode / restore the picture / slice / block / sample based on the slice header in the VCL NAL unit and the picture header in the PH NAL unit (S620).

[0152] On the other hand, for example, if there is no NAL unit for the picture header, the decoding device can decode / restore the picture / slice / block / sample based on the slice header and picture header in the VCL NAL unit (S630).

[0153] Here, the (coded) bitstream may contain one or more NAL units for decoding the picture. The NAL units may be VCL NAL units or non-VCL NAL units. For example, a VCL NAL unit may contain information for a coded slice, and a VCL NAL unit may have a NAL unit type with the NAL unit type class "VCL" as described in Table 1 above.

[0154] On the other hand, according to the embodiment proposed in this document, the bitstream may contain PH NAL units (NAL units for picture headers), or the bitstream may not currently contain PH NAL units for pictures. Information indicating the presence or absence of PH NAL units (e.g., ph_nal_present_flag) can be signaled to the HLS (e.g., VPS, DPS, SPS, or slice header).

[0155] Figure 7 illustrates the picture header configuration within a NAL unit depending on the presence or absence of a PH NAL unit. For example, Figure 7(a) can show the case where a PH NAL unit currently exists for the picture, and Figure 7(b) can show the case where a PH NAL unit currently does not exist for the picture, but the picture header is included in the VCL NAL unit.

[0156] For example, if the PH NAL unit exists, the picture header can be contained within the PH NAL unit. Conversely, if the PH NAL unit does not exist, the picture header can still be configured but can be contained within another NAL unit. For example, the picture header can be contained within a VCL NAL unit. The VCL NAL unit can contain information about coded slices. The VCL unit can contain slice headers for coded slices. For example, if a particular slice header contains information that the coded / associated slice is the first slice in a picture or subpicture, the picture header can be contained within a particular VCL NAL unit that contains the particular slice header. Alternatively, for example, if the PH NAL unit does not exist, the picture header can be contained within a Non-VCL NAL unit such as a PPS NAL unit or an APS NAL unit.

[0157] Figure 8 schematically illustrates the video encoding method using the encoding device described in this document. The method disclosed in Figure 8 can be performed using the encoding device disclosed in Figure 2. Specifically, for example, steps S800 to S820 in Figure 8 can be performed by the entropy encoding unit of the encoding device. Although not shown, the process of decoding the current picture can be performed by the prediction unit and residual processing unit of the encoding device.

[0158] The encoding device determines whether a PH NAL (Network Abstraction Layer) unit containing a PH (Picture Header, PH) associated with the current picture exists (S800). The encoding device can generate a NAL unit for the current picture. For example, the NAL unit for the current picture may include a PH NAL unit containing a PH associated with the current picture and / or a VCL (Video Coded Layer) NAL unit containing information for a slice of the current picture (e.g., slice header and slice data). The encoding device can determine whether the PH NAL unit exists. For example, if the PH NAL unit exists, the encoding device can generate a VCL (Video Coded Layer) NAL unit containing a PH NAL unit containing a PH associated with the current picture and / or information for a slice of the current picture (e.g., slice header and slice data). Alternatively, for example, if the PH NAL unit does not exist, the encoding device can generate a VCL (Video Coded Layer) NAL unit containing a PH associated with the current picture and information for one slice of the current picture (e.g., slice header and slice data). Furthermore, for example, if the flag indicates that the PH NAL unit does not exist, the current picture may include a single slice. Here, for example, the PH may include syntax elements indicating parameters for the current picture.

[0159] The encoding device generates a flag indicating the presence or absence of the PH NAL unit based on the result of the determination (S810). The encoding device can generate a flag indicating the presence or absence of the PH NAL unit based on the result of the determination. For example, the flag can indicate whether the PH NAL unit exists or not. For example, if the value of the flag is 1, the flag can indicate that the PH NAL unit exists, and if the value of the flag is 0, the flag can indicate that the PH NAL unit does not exist. Alternatively, for example, if the value of the flag is 0, the flag can indicate that the PH NAL unit exists, and if the value of the flag is 1, the flag can indicate that the PH NAL unit does not exist. The syntax element of the flag is the ph_nal_present_flag described above.

[0160] The encoding device encodes the video information including the flag (S820). The encoding device can encode the video information including the flag. The video information may include the flag. Also, for example, the video information may include a high-level syntax, and the flag may be included in the high-level syntax. For example, the high-level syntax is an SPS (Sequence Parameter Set). Or, for example, the high-level syntax is a Slice Header (SH). That is, for example, the flag may be included in the Slice Header.

[0161] On the other hand, for example, if the flag indicates that the PH NAL unit exists, the PH can be included in the PH NAL unit, and if the flag indicates that the PH NAL unit does not exist, the PH can be included in the slice header associated with the current picture. For example, if the flag indicates that the PH NAL unit exists, the PH can be included in the PH NAL unit, and if the flag indicates that the PH NAL unit does not exist, the PH can be included in a VCL NAL unit including a slice header. That is, for example, if the flag indicates that the PH NAL unit exists, the PH can be included in the PH NAL unit, and if the flag indicates that the PH NAL unit does not exist, the PH can be included in a slice header. For example, if the flag indicates that the PH NAL unit exists, the video information can include at least one VCL NAL unit including the PH NAL unit and a slice header associated with the current picture, and if the flag indicates that the PH NAL unit does not exist, the video information can include a VCL NAL unit including the PH and a slice header. Furthermore, for example, if the flag indicates that the PH NAL unit does not exist, the video information may not include the PH NAL unit.

[0162] Furthermore, for example, if the flag indicates that the PH NAL unit does not exist, then the PH NAL unit may not exist for any picture in the CLVS (Coded Layer Video Sequence) that includes the current picture. That is, for example, the flag value for the presence or absence of the PH NAL unit for all pictures in the CLVS (Coded Layer Video Sequence) is the same. Also, for example, if the flag indicates that the PH NAL unit does not exist, then the picture headers for all the pictures in the CLVS may be included in the slice headers of all the pictures.

[0163] On the other hand, for example, AU detection can be modified to differ from existing schemes. For example, a new VCL NAL unit can signify a new AU. That is, for example, if the flag indicates that the PH NAL unit does not exist, the VCL NAL unit including the slice header is the first VCL NAL unit of the current picture (for the new AU (i.e., the AU for the current picture)). For example, the flag can be included in the slice header of the VCL NAL unit. Or, for example, if the flag indicates that the PH NAL unit exists, it follows the PH NAL unit, i.e., the first VCL NAL unit of the current picture that signals after the PH NAL unit.

[0164] On the other hand, the encoding device can decode the current picture. For example, the encoding device can decode the current picture based on the syntax elements of the PH. For example, the syntax elements of the PH are the syntax elements shown in Table 6 above. The PH may include syntax elements indicating parameters for the current picture, and the encoding device can decode the current picture based on the syntax elements. Also, for example, the VCL NAL unit including the slice header may include slice data for slices in the current picture, and the encoding device can decode the slices of the current picture based on the slice data. For example, the decoding device can derive predicted samples and residual samples of the current picture, and can generate restored samples / restored pictures for the current picture based on the predicted samples and residual samples.

[0165] On the other hand, for example, an encoding device can generate and encode prediction information for the current picture block. In this case, various prediction methods disclosed in this document, such as interpretation or intrapretation, can be applied. For example, the encoding device can decide whether to perform interpretation or intrapretation on the block, and can determine a specific interpretation mode or a specific intraprediction mode based on RD cost. Depending on the determined mode, the encoding device can derive prediction samples for the block. The prediction information may include prediction mode information for the block. The video information may include the prediction information.

[0166] Furthermore, for example, the encoding device can encode residual information for the current picture block.

[0167] For example, the encoding device can derive the residual sample by subtracting the original sample from the predicted sample for the block.

[0168] Thereafter, for example, the encoding device can quantize the residual sample to derive a quantized residual sample, derive conversion coefficients based on the quantized residual sample, and generate and encode the residual information based on the conversion coefficients. Alternatively, for example, the encoding device can quantize the residual sample to derive a quantized residual sample, convert the quantized residual sample to derive conversion coefficients, and generate and encode the residual information based on the conversion coefficients. The video information may include the residual information. Furthermore, for example, the encoding device can encode the video information and output it in bitstream format.

[0169] The encoding device can generate a restored sample and / or restored picture by adding the predicted sample and the residual sample. Thereafter, as described above, in-loop filtering procedures such as deblocking filtering, SAO and / or ALF procedures can be applied to the restored sample to improve subjective / objective image quality as needed.

[0170] On the other hand, the bitstream containing the video information can be transmitted to a decoding device via a network or a (digital) storage medium. Here, the network may include broadcast networks and / or communication networks, and the digital storage medium may include a variety of storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc.

[0171] Figure 9 schematically shows an encoding device that performs the video encoding method according to this document. The method disclosed in Figure 8 can be performed by the encoding device disclosed in Figure 9. Specifically, for example, the entropy encoding unit of the encoding device in Figure 9 can perform steps S800 to S820. Although not shown, the process of decoding the current picture can be performed by the prediction unit and residual processing unit of the encoding device.

[0172] Figure 10 schematically illustrates the video decoding method using the decoding device described in this document. The method disclosed in Figure 10 can be performed using the decoding device disclosed in Figure 3. Specifically, for example, steps S1000 to S1010 in Figure 10 can be performed by the entropy decoding unit of the decoding device, and step S1020 in Figure 10 can be performed by the prediction unit and residual processing unit of the decoding device.

[0173] The decoding device obtains a flag indicating the presence or absence of a PH (Picture Header, PH)NAL (Network Abstraction Layer) unit (S1000). The decoding device can obtain the flag indicating the presence or absence of the PH NAL unit via the bitstream. For example, the decoding device can obtain video information via the bitstream, and the video information may include the flag. Alternatively, for example, the video information may include a high-level syntax, and the flag may be included in the high-level syntax. For example, the high-level syntax may be an SPS (Sequence Parameter Set, SPS). Or, for example, the high-level syntax may be a Slice Header (SH). That is, for example, the flag may be included in the Slice Header.

[0174] For example, the flag can indicate whether the PH NAL unit exists. For example, if the value of the flag is 1, the flag can indicate that the PH NAL unit exists, and if the value of the flag is 0, the flag can indicate that the PH NAL unit does not exist. Alternatively, for example, if the value of the flag is 0, the flag can indicate that the PH NAL unit exists, and if the value of the flag is 1, the flag can indicate that the PH NAL unit does not exist. The syntax element of the flag is the ph_nal_present_flag described above.

[0175] The decoding device acquires the PH based on the flag (S1010). The decoding device may acquire the PH in the PH NAL unit or in a VCL NAL unit including a slice header based on the flag. That is, for example, the decoding device may acquire the PH in the PH NAL unit or in a slice header based on the flag.

[0176] For example, if the flag indicates that the PH NAL unit exists, the PH can be obtained in the PH NAL unit, and if the flag indicates that the PH NAL unit does not exist, the PH can be obtained in the slice header. For example, if the flag indicates that the PH NAL unit exists, the PH can be included in the PH NAL unit, and if the flag indicates that the PH NAL unit does not exist, the PH can be included in a VCL NAL unit including a slice header. That is, for example, if the flag indicates that the PH NAL unit exists, the PH can be included in the PH NAL unit, and if the flag indicates that the PH NAL unit does not exist, the PH can be included in a slice header. For example, if the flag indicates that the PH NAL unit exists, the video information can include the PH NAL unit including the PH and a VCL NAL unit including a slice header, and if the flag indicates that the PH NAL unit does not exist, the video information can include the PH and a VCL NAL unit including a slice header. Furthermore, for example, if the flag indicates that the PH NAL unit does not exist, the video information may not include the PH NAL unit.

[0177] Furthermore, for example, if the flag indicates that the PH NAL unit does not exist, the current picture associated with the PH may include one slice. That is, for example, if the flag indicates that the PH NAL unit does not exist, the video information may include a VCL NAL unit that includes a slice header for one slice in the current picture.

[0178] Furthermore, for example, if the flag indicates the presence of the PH NAL unit, then at least one VCL NAL unit, including the PH NAL unit and slice header for the current picture, can be obtained via the bitstream. That is, for example, if the flag indicates the presence of the PH NAL unit, the video information may include a VCL NAL unit, including the PH NAL unit for the current picture and a slice header for at least one slice within the current picture.

[0179] Furthermore, for example, if the flag indicates that the PH NAL unit does not exist, then the PH NAL unit may not exist for any picture in the CLVS (Coded Layer Video Sequence) that includes the current picture. That is, for example, the flag value for the presence or absence of the PH NAL unit for all pictures in the CLVS (Coded Layer Video Sequence) is the same. Also, for example, if the flag indicates that the PH NAL unit does not exist, then the picture headers for all the pictures in the CLVS may be included in the slice headers of all the pictures.

[0180] On the other hand, for example, AU detection can be modified to differ from existing schemes. For example, a new VCL NAL unit can signify a new AU. That is, for example, if the flag indicates that the PH NAL unit does not exist, the VCL NAL unit including the slice header is the first VCL NAL unit of the current picture (for the new AU (i.e., the AU for the current picture)). For example, the flag can be included in the slice header of the VCL NAL unit. Or, for example, if the flag indicates that the PH NAL unit exists, it follows the PH NAL unit, i.e., the first VCL NAL unit of the current picture that signals after the PH NAL unit.

[0181] The decoding device decodes the current picture associated with the PH based on the PH (S1020). The decoding device can decode the current picture based on the syntax elements of the PH. For example, the syntax elements of the PH are the syntax elements shown in Table 6 above. The PH may include syntax elements indicating parameters for the current picture, and the decoding device can decode the current picture based on the syntax elements. Also, for example, the VCL NAL unit including the slice header may include slice data for slices in the current picture, and the decoding device can decode the slices of the current picture based on the slice data. For example, the decoding device can derive predicted samples and residual samples of the current picture, and can generate restored samples / restored pictures for the current picture based on the predicted samples and residual samples.

[0182] As previously mentioned, in-loop filtering procedures such as deblocking filtering, SAO, and / or ALF procedures may be applied to the restored samples thereafter, if necessary, to improve subjective / objective image quality.

[0183] Figure 11 schematically shows a decoding apparatus that performs the video decoding method according to this document. The method disclosed in Figure 10 can be performed by the decoding apparatus disclosed in Figure 11. Specifically, for example, the entropy decoding unit of the decoding apparatus in Figure 11 can perform S1000 to S1010 in Figure 10, and the prediction unit and residual processing unit of the decoding apparatus in Figure 11 can perform S1020 in Figure 10.

[0184] As detailed in this document, a flag can be signaled to indicate whether a PH NAL unit is present, and based on this flag, the NAL unit can be adaptively adjusted to the bitstream bitrate, thereby improving overall coding efficiency.

[0185] Furthermore, according to this document, constraints can be set on the number of slices in the current picture and on the presence of PH NAL units in related pictures based on a flag indicating whether or not PH NAL units exist. This allows for adaptive adjustment of NAL units to the bitstream bitrate, thereby improving overall coding efficiency.

[0186] In the embodiments described above, the method is explained based on a flowchart in a series of steps or blocks, but this document is not limited to the order of the steps, and some steps may occur in a different order or simultaneously with other steps than those described above. Furthermore, those skilled in the art will understand that the steps shown in the flowchart are not exclusive, other steps may be included, or one or more steps in the flowchart may be deleted without affecting the scope of this document.

[0187] The embodiments described herein can be implemented and executed on a processor, microprocessor, controller, or chip. For example, the functional units illustrated in each drawing can be implemented and executed on a computer, processor, microprocessor, controller, or chip. In this case, information on implementation (e.g., information on instructions) or algorithms can be stored on a digital recording medium.

[0188] Furthermore, the decoding and encoding devices to which the embodiments of this document apply can include multimedia broadcasting transceivers, mobile communication terminals, home cinema video equipment, digital cinema video equipment, surveillance cameras, video interaction devices, real-time communication devices such as video communications, mobile streaming devices, recording media, camcorders, video-on-demand (VoD) service providers, over-the-top (OTT) video equipment, internet streaming service providers, 3D video equipment, image-phone video equipment, transportation terminals (e.g., vehicle terminals, airplane terminals, ship terminals, etc.), and medical video equipment, and can be used to process video signals or data signals. For example, over-the-top (OTT) video equipment can include game consoles, Blu-ray players, internet-connected TVs, home theater systems, smartphones, tablet PCs, and DVRs (Digital Video Recorders).

[0189] Furthermore, the processing methods to which the embodiments of this document apply can be produced in the form of programs executed on a computer and stored on a computer-readable recording medium. Multimedia data having the data structure relating to this document can also be stored on a computer-readable recording medium. The computer-readable recording medium includes all types of storage devices and distributed storage devices that store data that can be read by a computer. The computer-readable recording medium can include, for example, Blu-ray discs (BDs), general-purpose serial buses (USBs), ROMs, PROMs, EPROMs, EEPROMs, RAMs, CD-ROMs, magnetic tapes, floppy disks, and optical data storage devices. The computer-readable recording medium also includes media implemented in the form of carrier waves (e.g., transmission over the Internet). Furthermore, bitstreams generated by encoding methods can be stored on a computer-readable recording medium or transmitted over a wireless network.

[0190] Furthermore, the embodiments described in this document can be implemented as a computer program product using program code, and the program code can be executed on a computer according to the embodiments described in this document. The program code can be stored on a computer-readable carrier.

[0191] Figure 12 illustrates a content streaming system structure diagram to which the embodiments described in this document apply.

[0192] The content streaming system to which the embodiments described herein apply may broadly include an encoding server, a streaming server, a web server, a media storage facility, user equipment, and multimedia input devices.

[0193] The encoding server is responsible for compressing content input from multimedia input devices such as smartphones, cameras, and camcorders into digital data to generate a bitstream, and then transmitting this bitstream to the streaming server. In other cases, if a multimedia input device such as a smartphone, camera, or camcorder directly generates the bitstream, the encoding server can be omitted.

[0194] The bitstream can be generated by an encoding method or bitstream generation method to which an embodiment of this document applies, and the streaming server can temporarily store the bitstream in the process of transmitting or receiving the bitstream.

[0195] The streaming server transmits multimedia data to user devices based on user requests via a web server, and the web server acts as an intermediary to inform users about available services. When a user requests a desired service from the web server, the web server transmits this to the streaming server, and the streaming server transmits multimedia data to the user. In this case, the content streaming system may include a separate control server, in which case the control server controls the commands / responses between each device within the content streaming system.

[0196] The streaming server can receive content from a media storage and / or encoding server. For example, if it starts receiving content from the encoding server, it can receive the content in real time. In this case, in order to provide a smooth streaming service, the streaming server can store the bitstream for a certain period of time.

[0197] Examples of user devices include mobile phones, smartphones, laptop computers, digital broadcasting terminals, PDAs (personal digital assistants), PMPs (portable multimedia players), navigation systems, slate PCs, tablet PCs, ultrabooks, wearable devices (such as smartwatches, smart glasses, and HMDs), digital TVs, desktop computers, and digital signage. Each server in the content streaming system can be operated as a distributed server, in which case the data received by each server can be processed in a distributed manner.

[0198] The claims described herein can be combined in various ways. For example, the technical features of the method claims herein can be combined to realize an apparatus, and the technical features of the apparatus claims herein can be combined to realize a method. Furthermore, the technical features of the method claims and the technical features of the apparatus claims herein can be combined to realize an apparatus, and the technical features of the method claims and the technical features of the apparatus claims herein can be combined to realize a method.

Claims

1. In a video decoding method performed by a decoding device, Steps include obtaining a flag indicating whether a PH (Picture Header) NAL (Network Abstraction Layer) unit exists, Based on the fact that the aforementioned flag indicates the presence of the PH NAL unit, the steps include obtaining the picture header for the current picture from the PH NAL unit, A step of obtaining the picture header for the current picture from the slice layer RBSP (raw byte sequence payload) of the current picture, based on the fact that the flag indicates that the PH NAL unit does not exist, wherein the slice layer RBSP includes a slice header, The steps include decoding the current picture based on the picture header, Based on the fact that the aforementioned flag indicates that the PH NAL unit does not exist, the current picture contains only one slice. A method in which, based on the fact that the flag indicates the absence of the PH NAL unit, the slice layer RBSP includes the picture header, the slice header, and the slice data.

2. In a video encoding method performed by an encoding device, The current steps include determining whether a PH (Picture Header) NAL (Network Abstraction Layer) unit containing a picture header associated with the picture exists, Based on the determination that the PH NAL unit exists, the steps include generating the PH NAL unit including the picture header, Based on the determination that the PH NAL unit does not exist, a step of generating a slice layer RBSP (raw byte sequence payload) of the current picture including the picture header, wherein the slice layer RBSP includes the slice header, A step of generating a flag indicating whether the PH NAL unit exists based on the result of the above determination, The step includes encoding video information including the aforementioned flag, Based on the fact that the aforementioned flag indicates that the PH NAL unit does not exist, the current picture contains only one slice. A method in which, based on the fact that the flag indicates the absence of the PH NAL unit, the slice layer RBSP includes the picture header, the slice header, and the slice data.

3. A method for transmitting video data, The bitstream of video information, The current steps include determining whether a PH (Picture Header) NAL (Network Abstraction Layer) unit containing a picture header associated with the picture exists, Based on the determination that the PH NAL unit exists, the steps include generating the PH NAL unit including the picture header, Based on the determination that the PH NAL unit does not exist, a step of generating a slice layer RBSP (raw byte sequence payload) of the current picture including the picture header, wherein the slice layer RBSP includes the slice header, A step of generating a flag indicating whether the PH NAL unit exists based on the result of the above determination, A step of encoding the video information including the flag, and a step of generating based on, The step of transmitting the data, which includes the bitstream, Based on the fact that the aforementioned flag indicates that the PH NAL unit does not exist, the current picture contains only one slice. A method in which, based on the fact that the flag indicates the absence of the PH NAL unit, the slice layer RBSP includes the picture header, the slice header, and the slice data.

Images