System and method for signaling hypothetical reference decoder parameters in video coding

Abstract

The present disclosure relates to video encoding, and more particularly to techniques for signaling hypothetical reference decoder (HRD) parameters of an encoded video. In one example, a method of decoding video data includes decoding a first syntax element in a sequence parameter set, wherein the first syntax element specifies whether the HRD parameter syntax structure includes a parameter of a temporal sub-layer having an identifier in a range from 0 to a maximum number of temporal sub-layers minus one or whether the HRD parameter syntax structure includes a parameter of a temporal sub-layer having an identifier only equal to a maximum number of temporal sub-layers minus one; defining a starting sub-layer of the HRD parameter syntax structure according to a value of the first syntax element; and decoding each of the parameters in the HRD parameter syntax structure by using the starting sub-layer and a maximum number of sub-layers.

Description

Technical Field

[0001] The present disclosure relates to video coding, and more particularly to techniques for signaling hypothetical reference decoder parameters for encoded video. Background Art

[0002] Digital video capabilities can be incorporated into a variety of devices, including digital televisions, laptop or desktop computers, tablet computers, digital recording devices, digital media players, video game devices, cellular phones (including so-called smart phones), medical imaging devices, etc. Digital video can be encoded according to a video coding standard. Video coding standards define the format of a compliant bitstream that encapsulates coded video data. A compliant bitstream is a data structure that can be received and decoded by a video decoding device to generate reconstructed video data. Video coding standards can be combined with video compression techniques. Examples of video coding standards include ISO / IEC MPEG-4 Visual and ITU-T H.264 (also known as ISO / IEC MPEG-4 AVC) and High Efficiency Video Coding (HEVC). HEVC is described in High Efficiency Video Coding (HEVC) of the ITU-T H.265 Recommendation of December 2016, which is incorporated herein by reference and is referred to herein as ITU-T H.265. ITU-T H.265 is currently being considered for expansion and improvement to develop the next generation of video coding standards. For example, the ITU-T Video Coding Experts Group (VCEG) and the ISO / IEC Moving Picture Experts Group (MPEG), collectively referred to as the Joint Video Study Group (JVET), are working to standardize video coding techniques with compression capabilities significantly exceeding the current HEVC standard. The Joint Exploration Model 7 (JEM 7), the algorithm description of the Joint Exploration Test Model 7 (JEM 7), ISO / IEC JTC1 / SC29 / WG11 document: JVET-G1001 (July 2017, Torino, Italy), incorporated herein by reference, describes the coding features studied by JVET under the Joint Test Model, which is a potential enhanced video coding technology that exceeds the capabilities of ITU-T H.265. It should be noted that the coding features of JEM 7 are implemented in the JEM reference software. As used herein, the term JEM may refer collectively to the algorithms included in JEM 7 and the specific implementation of the JEM reference software. In addition, in response to the "Joint Call for Proposals on Video Compression with Capabilities beyond HEVC" jointly issued by VCEG and MPEG, at the 10th meeting of ISO / IEC JTC1 / SC29 / WG11 held in San Diego, California (San Diego, CA) from April 16 to 20, 2018, various groups proposed multiple descriptions of video coding tools.Based on the various descriptions of the video coding tools, the final initial draft text of the video coding specification is described in "Versatile Video Coding (Draft 1)" document JVET-J1001-v2 at the 10th meeting of ISO / IEC JTC1 / SC29 / WG11 held in San Diego, California from April 16 to 20, 2018, which is incorporated herein by reference and is referred to as JVET-J1001. The current development of the next generation video coding standard by VCEG and MPEG is known as the Versatile Video Coding (VVC) project. "Versatile Video Coding (Draft 5)" at the 14th meeting of ISO / IEC JTC1 / SC29 / WG11 held in Geneva, Switzerland from March 19 to 27, 2019 (document JVET-N1001-v7, which is incorporated herein by reference and is referred to as JVET-N1001) represents the current iteration of the draft text of the video coding specification corresponding to the VVC project.

[0003] Video compression technology can reduce the data requirements for storing and transmitting video data. Video compression technology can reduce data requirements by exploiting the redundancy inherent in video sequences. Video compression technology can divide the video sequence into smaller and continuous parts (i.e., a group of pictures within a video sequence, pictures within a group of pictures, regions within pictures, sub-regions within regions, etc.). Intra-frame prediction coding techniques (e.g., spatial prediction techniques within pictures) and inter-frame prediction techniques (i.e., techniques (time) between pictures) can be used to generate the difference between the unit video data to be encoded and the reference unit of the video data. The difference can be called residual data. The residual data can be encoded as quantized transform coefficients. Syntax elements can involve residual data and reference coding units (e.g., intra-frame prediction mode indexes and motion information). The residual data and syntax elements can be entropy encoded. Entropy-encoded residual data and syntax elements can be included in the data structure that forms a compatible bitstream. Summary of the invention

[0004] In one example, a method for decoding video data, the method comprising: decoding a first syntax element in a sequence parameter set, wherein the first syntax element specifies whether a hypothetical reference decoder parameter syntax structure includes parameters of temporal sublayers whose identifiers are in the range of 0 to a maximum number of temporal sublayers minus one or whether a hypothetical reference decoder parameter syntax structure includes parameters of temporal sublayers whose identifiers are only equal to a maximum number of temporal sublayers minus one; defining a starting sublayer of the hypothetical reference decoder parameter syntax structure according to a value of the first syntax element; and decoding each of the parameters in the hypothetical reference decoder parameter syntax structure by using the starting sublayer and the maximum number of sublayers.

[0005] In one example, a method for encoding image data, the method comprising: encoding a first syntax element in a sequence parameter set, wherein the first syntax element specifies whether a hypothetical reference decoder parameter syntax structure includes parameters of temporal sublayers whose identifiers are in the range of 0 to a maximum number of temporal sublayers minus one or whether a hypothetical reference decoder parameter syntax structure includes parameters of temporal sublayers whose identifiers are only equal to a maximum number of temporal sublayers minus one; defining a starting sublayer of the hypothetical reference decoder parameter syntax structure according to a value of the first syntax element; and encoding each of these parameters in the hypothetical reference decoder parameter syntax structure by using the starting sublayer and the maximum number of sublayers.

[0006] In one example, a decoder for decoding video data comprises: a processor, and a memory associated with the processor; wherein the processor is configured to perform the following steps: decode a first syntax element in a sequence parameter set, wherein the first syntax element specifies whether a hypothetical reference decoder parameter syntax structure includes parameters of temporal sublayers whose identifiers are in the range of 0 to a maximum number of temporal sublayers minus one or whether a hypothetical reference decoder parameter syntax structure includes parameters of temporal sublayers whose identifiers are only equal to the maximum number of temporal sublayers minus one; define a starting sublayer of the hypothetical reference decoder parameter syntax structure according to the value of the first syntax element; and decode each of these parameters in the hypothetical reference decoder parameter syntax structure by using the starting sublayer and the maximum number of sublayers.

[0007] In one example, an encoder for encoding video data comprises: a processor, and a memory associated with the processor; wherein the processor is configured to perform the following steps: encode a first syntax element in a sequence parameter set, wherein the first syntax element specifies whether a hypothetical reference decoder parameter syntax structure includes parameters of temporal sublayers whose identifiers are in the range of 0 to a maximum number of temporal sublayers minus one or whether a hypothetical reference decoder parameter syntax structure includes parameters of temporal sublayers whose identifiers are only equal to a maximum number of temporal sublayers minus one; define a starting sublayer of the hypothetical reference decoder parameter syntax structure according to the value of the first syntax element; and encode each of these parameters in the hypothetical reference decoder parameter syntax structure by using the starting sublayer and the maximum number of sublayers. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] [ Figure 1 ] Figure 1 is a block diagram illustrating an example of a system that may be configured to encode and decode video data in accordance with one or more techniques of this disclosure. [ Figure 2 ] Figure 2is a conceptual diagram illustrating encoded video data and corresponding data structures in accordance with one or more techniques of this disclosure. [ Figure 3 ] Figure 3 is a conceptual diagram illustrating a data structure that encapsulates encoded video data and corresponding metadata in accordance with one or more techniques of this disclosure. [ Figure 4 ] Figure 4 is a conceptual diagram illustrating an example of components that may be included in an implementation of a system that may be configured to encode and decode video data in accordance with one or more techniques of this disclosure. [ Figure 5 ] Figure 5 is a block diagram illustrating an example of a video encoder that may be configured to encode video data in accordance with one or more techniques of this disclosure. [ Figure 6 ] Figure 6 is a block diagram illustrating an example of a video decoder that may be configured to decode video data in accordance with one or more techniques of this disclosure. DETAILED DESCRIPTION

[0009] In general, the present disclosure describes various techniques for encoding video data. Specifically, the present disclosure describes techniques for signaling hypothetical reference decoder parameters for encoded video. It should be noted that although the techniques of the present disclosure are described with respect to ITU-T H.264, ITU-T H.265, JEM, and JVET-N1001, the techniques of the present disclosure may be generally applied to video encoding. For example, in addition to those techniques included in ITU-T H.265, JEM, and JVET-N1001, the encoding techniques described herein may be incorporated into video encoding systems (including video encoding systems based on future video encoding standards), including video block structures, intra-frame prediction techniques, inter-frame prediction techniques, transform techniques, filtering techniques, and / or other entropy encoding techniques. Therefore, references to ITU-T H.264, ITU-T H.265, JEM, and / or JVET-N1001 are for descriptive purposes and should not be construed as limiting the scope of the techniques described herein. In addition, it should be noted that the incorporation of references herein is for descriptive purposes and should not be construed as limiting or creating ambiguity regarding the terms used herein. For example, where a definition of a term provided in an incorporated reference differs from that provided in another incorporated reference and / or as used herein, the term should be interpreted in a manner that broadly includes each corresponding definition and / or in a manner that includes each specific definition in alternatives.

[0010] In one example, a method for encoding video data includes: sending a signal notification mark, wherein the value of the mark indicates whether to signal a corresponding instance of an HRD parameter for each temporal sublayer of the video data or to signal a single instance of the HRD parameter for all temporal sublayers of the video data; and sending a plurality of instances of the HRD parameter according to the number of temporal sublayers of the video data and the value of the mark.

[0011] In one example, a device includes one or more processors configured to: send a signal notification mark, wherein the value of the mark indicates whether to signal a corresponding instance of the HRD parameter for each temporal sublayer of the video data or to signal a single instance of the HRD parameter for all temporal sublayers of the video data; and send a plurality of signal notification instances of the HRD parameter based on the number of temporal sublayers of the video data and the value of the mark.

[0012] In one example, a non-transitory computer-readable storage medium includes instructions stored thereon that, when executed, cause one or more processors of a device to: send a signal notification mark, wherein the value of the mark indicates whether to signal a corresponding instance of an HRD parameter for each temporal sublayer of video data or to signal a single instance of the HRD parameter for all temporal sublayers of the video data; and signal multiple instances of the HRD parameter based on the number of temporal sublayers of the video data and the value of the mark.

[0013] In one example, the device includes: a device for sending a signal notification mark, wherein the value of the mark indicates whether to send a corresponding instance of the HRD parameter to be signaled for each temporal sublayer of the video data or a single instance of the HRD parameter to be signaled for all temporal sublayers of the video data; and a device for sending a plurality of instances of the HRD parameter to be signaled based on the number of temporal sublayers of the video data and the value of the mark.

[0014] In one example, a method for decoding video data includes: parsing a tag that indicates whether a corresponding instance of an HRD parameter is signaled for each temporal sublayer of the video data or a single instance of the HRD parameter is signaled for all temporal sublayers of the video data; and parsing multiple instances of the HRD parameter based on the number of temporal sublayers of the video data and the value of the tag.

[0015] In one example, a device includes one or more processors configured to: parse a tag that indicates whether a corresponding instance of an HRD parameter is signaled for each temporal sublayer of video data or a single instance of the HRD parameter is signaled for all temporal sublayers of the video data; and parse multiple instances of the HRD parameter based on the number of temporal sublayers of the video data and the value of the tag.

[0016] In one example, a non-transitory computer-readable storage medium includes instructions stored thereon that, when executed, cause one or more processors of a device to: parse a tag that indicates whether a corresponding instance of an HRD parameter is signaled for each temporal sublayer of video data or a single instance of the HRD parameter is signaled for all temporal sublayers of the video data; and parse multiple instances of the HRD parameter based on the number of temporal sublayers of the video data and the value of the tag.

[0017] In one example, the device includes: a device for parsing a tag that indicates whether a corresponding instance of an HRD parameter is signaled for each temporal sublayer of video data or a single instance of the HRD parameter is signaled for all temporal sublayers of the video data; and a device for parsing multiple instances of the HRD parameter based on the number of temporal sublayers of the video data and the value of the tag.

[0018] The details of one or more examples are set forth in the accompanying drawings and the description below. Other features, objects, and advantages will be apparent from the description and drawings, and from the claims.

[0019] The video content includes a video sequence consisting of a series of frames (or pictures). A series of frames may also be referred to as a group of pictures (GOP). Each video frame or picture may be divided into one or more regions. The region may be defined according to a set of rules for defining a basic unit (e.g., a video block) and a defined region. For example, the rule defining the region may be that the region must be an integer number of video blocks arranged in a rectangle. In addition, the video blocks in the region may be sorted according to a scanning mode (e.g., a raster scan). As used herein, the term "video block" may generally refer to a region of a picture, or may more specifically refer to a maximum array of sample values ​​that may be predictively encoded, its sub-partitions, and / or corresponding structures. In addition, the term "current video block" may refer to a region of a picture that is being encoded or decoded. A video block may be defined as an array of sample values. It should be noted that in some cases, a pixel value may be described as a sample value comprising a corresponding component of video data, which may also be referred to as a color component (e.g., brightness (Y) and chrominance (Cb and Cr) components or red, green, and blue components). It should be noted that in some cases, the terms "pixel value" and "sample value" may be used interchangeably. Additionally, in some cases, a pixel or sample may be referred to as a pel. A video sampling format (which may also be referred to as a chroma format) may define the number of chroma samples included in a video block relative to the number of luma samples included in the video block. For example, for a 4:2:0 sampling format, the sampling rate of the luma component is twice the sampling rate of the chroma components in both the horizontal and vertical directions.

[0020] The video encoder may perform predictive coding on a video block and its sub-partitions. A video block and its sub-partitions may be referred to as a node. ITU-T H.264 specifies a macroblock including 16×16 luminance samples. That is, in ITU-T H.264, a picture is segmented into macroblocks. ITU-T H.265 specifies a similar coding tree unit (CTU) structure (which may be referred to as a maximum coding unit (LCU)). In ITU-TH.265, a picture is segmented into CTUs. In ITU-T H.265, for a picture, the CTU size may be set to include 16×16, 32×32, or 64×64 luminance samples. In ITU-T H.265, a CTU is composed of a corresponding coding tree block (CTB) for each component of the video data (e.g., luminance (Y) and chrominance (Cb and Cr)). It should be noted that a video having a luminance component and two corresponding chrominance components may be described as having two channels, i.e., a luminance channel and a chrominance channel. In addition, in ITU-T H.265, CTUs can be divided according to a quadtree (QT) partitioning structure, which allows the CTBs of a CTU to be divided into coding blocks (CBs). That is, in ITU-T H.265, CTUs can be divided into quadtree leaf nodes. According to ITU-T H.265, a luma CB together with two corresponding chroma CBs and associated syntax elements is referred to as a coding unit (CU). In ITU-TH.265, a signal can be sent to notify the minimum allowed size of a CB. In ITU-T H.265, the minimum allowed minimum size of a luma CB is 8×8 luma samples. In ITU-T H.265, the decision to encode a picture area using intra prediction or inter prediction is made at the CU level.

[0021] In ITU-T H.265, a CU is associated with a prediction unit (PU) structure having its root at the CU. In ITU-TH.265, the PU structure allows the partitioning of luma CB and chroma CB to generate corresponding reference samples. That is, in ITU-T H.265, the luma CB and chroma CB can be partitioned into corresponding luma prediction blocks and chroma prediction blocks (PBs), wherein the PBs include blocks of sample values ​​to which the same prediction is applied. In ITU-T H.265, the CB can be divided into 1, 2, or 4 PBs. ITU-T H.265 supports PB sizes from 64×64 samples down to 4×4 samples. In ITU-T H.265, square PBs are supported for intra prediction, where the CB can form a PB or the CB can be partitioned into four square PBs. In ITU-T H.265, in addition to square PBs, rectangular PBs are also supported for inter prediction, where the CB can be halved vertically or horizontally to form a PB. Furthermore, it should be noted that in ITU-T H.265, for inter prediction, four asymmetric PB partitions are supported, where a CB is divided into two PBs at one quarter of the height (top or bottom) or width (left or right) of the CB. Intra prediction data (e.g., intra prediction mode syntax elements) or inter prediction data (e.g., motion data syntax elements) corresponding to the PB are used to generate reference and / or prediction sample values ​​for the PB.

[0022] JEM specifies a CTU with a maximum size of 256×256 luma samples. JEM specifies a quadtree plus binary tree (QTBT) block structure. In JEM, the QTBT structure allows the quadtree leaf nodes to be further divided by the binary tree (BT) structure. That is, in JEM, the binary tree structure allows the quadtree leaf nodes to be recursively divided vertically or horizontally. In JVET-N1001, CTUs are divided according to a quadtree plus multi-type tree (QTMT or QT+MTT) structure. The QTMT in JVET-N1001 is similar to the QTBT in JEM. However, in JVET-N1001, in addition to indicating binary partitioning, the multi-type tree can also indicate a so-called ternary (or ternary tree (TT)) partitioning. Ternary partitioning divides a block vertically or horizontally into three blocks. In case of vertical TT splitting, the block is split at one quarter of its width from the left edge and at one quarter of its width from the right edge, and in case of horizontal TT splitting, the block is split at one quarter of its height from the top edge and at one quarter of its height from the bottom edge.

[0023] As described above, each video frame or picture may be divided into one or more regions. For example, according to ITU-T H.265, each video frame or picture may be divided into one or more slices, and further divided into one or more tiles, wherein each slice includes a CTU sequence (e.g., arranged in a raster scan order), and wherein the tile is a CTU sequence corresponding to a rectangular area of ​​the picture. It should be noted that in ITU-T H.265, a slice is a sequence of one or more slice segments starting from an independent slice segment and containing all subsequent subordinate slice segments (if any) before the next independent slice segment (if any). Slice segments (such as slices) are CTU sequences. Therefore, in some cases, the terms "slice" and "slice segment" may be used interchangeably to indicate a sequence of CTUs arranged in a raster scan order. In addition, it should be noted that in ITU-T H.265, a tile may be composed of CTUs contained in more than one slice, and a slice may be composed of CTUs contained in more than one tile. However, ITU-T H.265 specifies that one or both of the following conditions should be met: (1) all CTUs in a segment belong to the same tile; and (2) all CTUs in a tile belong to the same slice.

[0024] Regarding JVET-N1001, a slice needs to consist of an integer number of bricks, rather than just an integer number of CTUs. In JVET-N1001, a brick is a rectangular CTU row area within a specific tile in a picture. In addition, in JVET-N1001, a tile can be divided into multiple bricks, each brick consisting of one or more CTU rows within the tile. Tiles that are not divided into multiple bricks are also referred to as bricks. However, bricks that are true subsets of tiles are not referred to as tiles. Therefore, in some video coding techniques, slices that include a group of CTUs that do not form a rectangular area of ​​a picture may or may not be supported. In addition, it should be noted that in some cases, a slice may need to consist of an integer number of complete tiles, and in this case, the slice is referred to as a tile group. The techniques described herein may be applicable to bricks, slices, tiles, and / or tile groups. Figure 2 is a conceptual diagram showing an example of a picture group including slices. Figure 2 In the example shown, Pic3 is shown to include two slices (ie, slice 0 and slice 1). Figure 2 In the example shown, slice 0 includes one brick, brick 0, and slice 1 includes two bricks, brick 1 and brick 2. It should be noted that in some cases, slice 0 and slice 1 may meet the requirements of a tile and / or tile group and be classified as a tile and / or tile group.

[0025] For intra-frame prediction coding, the intra-frame prediction mode may specify the location of the reference sample within the picture. In ITU-TH.265, the defined possible intra-frame prediction modes include a plane (ie, surface fitting) prediction mode, a DC (ie, flat overall average) prediction mode, and 33 angle prediction modes (predMode:2-34). In JEM, the defined possible intra-frame prediction modes include a plane prediction mode, a DC prediction mode, and 65 angle prediction modes. It should be noted that the plane prediction mode and the DC prediction mode may be referred to as a non-directional prediction mode, and the angle prediction mode may be referred to as a directional prediction mode. It should be noted that regardless of the number of possible prediction modes defined, the techniques described herein may be generally applicable.

[0026] For inter-frame prediction coding, a reference picture is determined, and a motion vector (MV) identifies samples in the reference picture that are used to generate a prediction for the current video block. For example, the current video block may be predicted using reference sample values ​​located in one or more previously coded pictures, and a motion vector is used to indicate the position of the reference block relative to the current video block. The motion vector may describe, for example, the horizontal displacement component of the motion vector (i.e., the MV x ), the vertical displacement component of the motion vector (i.e., MV y) and the resolution of the motion vector (e.g., quarter pixel accuracy, half pixel accuracy, one pixel accuracy, two pixel accuracy, four pixel accuracy). Previously decoded pictures (which may include pictures output before or after the current picture) may be organized into one or more reference picture lists and identified using reference picture index values. In addition, in inter-frame prediction coding, uni-prediction refers to generating predictions using sample values ​​from a single reference picture, and bi-prediction refers to generating predictions using corresponding sample values ​​from two reference pictures. That is, in uni-prediction, a single reference picture and a corresponding motion vector are used to generate a prediction for the current video block, while in bi-prediction, a first reference picture and a corresponding first motion vector and a second reference picture and a corresponding second motion vector are used to generate a prediction for the current video block. In bi-prediction, corresponding sample values ​​are combined (e.g., added, rounded and cropped, or averaged according to weights) to generate a prediction. Pictures and their regions may be classified based on what types of prediction modes may be used to encode their video blocks. That is, for a region with a B type (e.g., a B slice), bi-prediction, uni-prediction, and intra-prediction modes can be utilized, for a region with a P type (e.g., a P slice), uni-prediction and intra-prediction modes can be utilized, and for a region with an I type (e.g., an I slice), only an intra-prediction mode can be utilized. As described above, a reference picture is identified by a reference index. For example, for a P slice, there may be a single reference picture list RefPicList0, and for a B slice, there may be a second independent reference picture list RefPicList1 in addition to RefPicList0. It should be noted that for uni-prediction in a B slice, one of RefPicList0 or RefPicList1 may be used to generate a prediction. Furthermore, it should be noted that during the decoding process, when decoding of a picture begins, one or more reference picture lists are generated from previously decoded pictures stored in a decoded picture buffer (DPB).

[0027] In addition, the coding standard may support various motion vector prediction modes. Motion vector prediction enables the value of the motion vector for the current video block to be derived based on another motion vector. For example, a set of candidate blocks with associated motion information can be derived from the spatial neighboring blocks and the temporal neighboring blocks of the current video block. In addition, the generated (or default) motion information can be used for motion vector prediction. Examples of motion vector prediction include advanced motion vector prediction (AMVP), temporal motion vector prediction (TMVP), so-called "merge" mode, and "skip" and "direct" motion reasoning. In addition, other examples of motion vector prediction include advanced temporal motion vector prediction (ATMVP) and space-time motion vector prediction (STMVP). For motion vector prediction, both the video encoder and the video decoder perform the same process to derive a set of candidates. Therefore, for the current video block, the same set of candidates is generated during encoding and decoding.

[0028] As described above, for inter-frame prediction coding, reference samples in previously encoded pictures are used to encode video blocks in the current picture. Previously encoded pictures that can be used as references when encoding the current picture are called reference pictures. It should be noted that the decoding order does not necessarily correspond to the picture output order, that is, the temporal order of pictures in a video sequence. In ITU-T H.265, when a picture is decoded, it is stored in a decoded picture buffer (DPB) (which may be called a frame buffer, a reference buffer, a reference picture buffer, etc.). In ITU-T H.265, pictures stored in the DPB are removed from the DPB when they are output and are no longer needed for encoding subsequent pictures. In ITU-TH.265, the determination of whether the picture should be removed from the DPB is called once for each picture after decoding the slice header, that is, at the beginning of decoding the picture. For example, the reference Figure 2 , Pic3 is shown as referring to Pic2. Similarly, Pic4 is shown as referring to Pic1. Figure 2, assuming that the number of pictures corresponds to the decoding order, the DPB will be filled as follows: after decoding Pic1, the DPB will include {Pic1}; at the beginning of decoding Pic2, the DPB will include {Pic1}; after decoding Pic2, the DPB will include {Pic1, Pic2}; at the beginning of decoding Pic3, the DPB will include {Pic1, Pic2}. Pic3 will then be decoded with reference to Pic2, and after decoding Pic3, the DPB will include {Pic1, Pic2, Pic3}. At the beginning of decoding Pic4, pictures Pic2 and Pic3 will be marked for removal from the DPB because they are not required for decoding Pic4 (or any subsequent pictures, not shown), and assuming that Pic2 and Pic3 have been output, the DPB will be updated to include {Pic1}. Pic4 will then be decoded with reference to Pic1. The process of marking pictures to remove them from the DPB can be called reference picture set (RPS) management.

[0029] As described above, intra prediction data or inter prediction data is used to generate reference sample values ​​for a block of sample values. The difference between the sample values ​​included in the current PB or another type of picture region structure and the associated reference samples (e.g., those generated using prediction) can be referred to as residual data. The residual data may include a corresponding difference array corresponding to each component of the video data. The residual data may be in the pixel domain. A transform such as a discrete cosine transform (DCT), a discrete sine transform (DST), an integer transform, a wavelet transform, or a conceptually similar transform may be applied to the difference array to generate transform coefficients. It should be noted that in ITU-T H.265 and JVET-N1001, a CU is associated with a transform unit (TU) structure having its root at the CU level. That is, in order to generate transform coefficients, the array of difference values ​​may be divided (e.g., four 8×8 transforms may be applied to a 16×16 residual value array). For each component of the video data, such a subdivision of the difference values ​​may be referred to as a transform block (TB). It should be noted that in some cases, a core transform and a subsequent secondary transform may be applied (in a video encoder) to generate transform coefficients. For the video decoder, the order of transforms is reversed.

[0030] The quantization process may be performed directly on the transform coefficients or residual sample values ​​(e.g., in the case of palette coded quantization). Quantization approximates the transform coefficients by limiting the amplitude to a set of specified values. Quantization essentially scales the transform coefficients in order to change the amount of data required to represent a set of transform coefficients. Quantization may include dividing the transform coefficients (or the values ​​obtained by adding the offset values ​​to the transform coefficients) by a quantization scaling factor and any associated rounding function (e.g., rounding to the nearest integer). The quantized transform coefficients may be referred to as coefficient level values. Inverse quantization (or "dequantization") may include multiplying the coefficient level values ​​by the quantization scaling factor, as well as any reciprocal rounding or offset addition operations. It should be noted that, as used herein, the term quantization process may refer to dividing by a scaling factor in some cases to generate a level value, and may refer to multiplying by a scaling factor in some cases to recover the transform coefficients. That is, the quantization process may refer to quantization in some cases, and to inverse quantization in some cases. In addition, it should be noted that although the quantization process is described in some of the examples below with respect to arithmetic operations associated with decimal notation, such description is for illustrative purposes and should not be construed as limiting. For example, the techniques described herein can be implemented in devices that use binary operations, etc. For example, the multiplication and division operations described herein can be implemented using shift operations, etc.

[0031] The quantized transform coefficients and syntax elements (e.g., syntax elements indicating the coding structure of the video block) may be entropy encoded according to an entropy coding technique. The entropy coding process includes encoding the syntax element values ​​using a lossless data compression algorithm. Examples of entropy coding techniques include content adaptive variable length coding (CAVLC), context adaptive binary arithmetic coding (CABAC), probability interval partitioning entropy coding (PIPE), etc. The entropy-coded quantized transform coefficients and the corresponding entropy-coded syntax elements may form a compliant bitstream that can be used to reproduce the video data at the video decoder. An entropy coding process, such as CABAC, may include binarizing the syntax elements. Binarization refers to the process of converting the value of a syntax element into a sequence of one or more bits. These bits may be referred to as "bins". Binarization may include one or a combination of the following coding techniques: fixed length coding, unary coding, truncated unary coding, truncated Rice coding, Golomb coding, k-order exponential Golomb coding, and Golomb-Rice coding. For example, binarization may include representing the integer value 5 of the syntax element as 00000101 using an 8-bit fixed-length binarization technique, or representing the integer value 5 as 11110 using a unary coding binarization technique. As used herein, each of the terms fixed-length coding, unary coding, truncated unary coding, truncated Rice coding, Golomb coding, k-order exponential Golomb coding, and Golomb-Rice coding may refer to general implementations of these techniques and / or more specific implementations of these coding techniques. For example, a Golomb-Rice coding implementation may be specifically defined according to a video coding standard. In the example of CABAC, for a particular bin, the context provides a maximum probability state (MPS) value of the bin (i.e., the MPS of the bin is one of 0 or 1), and a probability value that the bin is an MPS or a minimum probability state (LPS). For example, the context may indicate that the MPS of the bin is 0, and the probability that the bin is 1 is 0.3. It should be noted that the context may be determined based on the values ​​of previously encoded bins including bins in the current syntax element and previously encoded syntax elements. For example, the values ​​of syntax elements associated with neighboring video blocks may be used to determine the context of the current bin.

[0032] For the formulas used in this article, the following arithmetic operators can be used: +Addition - Subtraction * Multiplication, including matrix multiplication x y Exponentiation. Raises x to the power of y. In other contexts, such symbols are used for superscripts and are not intended to be interpreted as exponentiation. / Integer division truncates the result toward zero. For example, 7 / 4 and -7 / -4 are truncated to 1, and -7 / 4 and 7 / -4 are truncated to -1. ÷ is used to represent division in mathematical formulas when truncation or rounding is not intended. Used to represent division in mathematical formulas without the intent to truncate or round.

[0033] Additionally, the following math functions are available: Log2(x) is the logarithm of x with base 2; Ceil(x) is the smallest integer greater than or equal to x.

[0034] With respect to the exemplary syntax used herein, the following definitions of logical operators may be applied: x&&y Boolean logical "AND" of x and y x||y Boolean logical OR of x and y ! Boolean logic "NO" x? y:z If x is TRUE or not equal to 0, it evaluates to y; otherwise, it evaluates to z.

[0035] Additionally, the following relational operators can be applied: > greater than ≥ greater than or equal to < ≤Less than or equal to == is equal to ! = not equal to

[0036] Furthermore, it should be noted that among the syntax descriptors used in this document, the following descriptors may be applied: -b(8): Byte (8 bits) with any bit string pattern. The parsing of this descriptor is specified by the return value of the function read_bit(8). -f(n): A fixed-mode bit string written using n bits (from left to right) starting from the leftmost bit. The parsing process of this descriptor is specified by the return value of the function read_bit(n). -se(v): signed integer order 0 Exp-Golomb coded syntax element, starting from the leftmost bit. -tb(v): truncated binary code using at most maxVal bits, where maxVal is defined in the semantics of the syntax element. -tu(v): truncated unary code using at most maxVal bits, where maxVal is defined in the semantics of the syntax element. -u(n): Use an unsigned integer of n bits. When n is "v" in the syntax table, the number of bits varies in a way that depends on the values ​​of other syntax elements. The parsing process of this descriptor is specified by the return value of the function read_bits(n), which is interpreted as the binary representation of the unsigned integer with the most significant bit written first. -ue(v): unsigned integer O-order Exp-Golomb coded syntax element, starting from the leftmost bit.

[0037] As described above, the video content includes a video sequence consisting of a series of frames (or pictures), and each video frame or picture can be divided into one or more regions. A coded video sequence (CVS) can be encapsulated (or structured) as a series of access units, each of which includes video data configured as a network abstraction layer (NAL) unit. The bitstream can be described as a sequence of NAL units that form one or more CVSs. It should be noted that the multi-layer extension enables video presentation to include a base layer and one or more additional enhancement layers. For example, the base layer can enable video presentation with a basic quality level (e.g., high-definition presentation and / or 30Hz frame rate), and the enhancement layer can enable video presentation with an enhanced quality level (e.g., ultra-high-definition rendering and / or 60Hz frame rate). The enhancement layer can be encoded by referring to the base layer. That is, for example, the picture in the enhancement layer can be encoded (e.g., using inter-layer prediction technology) by referring to one or more pictures in the base layer (including its scaled version). Each NAL unit may include an identifier indicating the video data layer associated with the NAL unit. It should be noted that sub-bitstream extraction can refer to the process by which a device receiving a compatible or compliant bitstream forms a new compatible or compliant bitstream by discarding and / or modifying data in the received bitstream. For example, sub-bitstream extraction can be used to form a new compatible or compliant bitstream corresponding to a specific representation (e.g., a high-quality representation) of a video. Layers can also be encoded independently of each other. In this case, there may be no inter-layer prediction between the two layers.

[0038] refer to Figure 2In the example shown in , each video data slice included in Pic3 (i.e., slice 0 slice 1) is shown as being encapsulated in a NAL unit. In JVET-N1001, each of a video sequence, a GOP, a picture, a slice, and a CTU can be associated with metadata describing video coding properties. JVET-N1001 defines parameter sets that can be used to describe video data and / or video coding properties. Specifically, JVET-N1001 includes the following five types of parameter sets: decoding parameter set (DPS), video parameter set (VPS), sequence parameter set (SPS), picture parameter set (PPS), and adaptation parameter set (APS). Regarding SPS, JVET-N1001 includes the following definitions: Sequence parameter set (SPS): A syntax structure containing syntax elements that apply to zero or more complete CVSs determined by the contents of syntax elements found in the PPS referenced by syntax elements found in each slice header.

[0039] In JVET-N1001, parameter sets can be encapsulated as special types of NAL units, or can be signaled as messages. NAL units that include coded video data (e.g., slices) can be referred to as VCL (video coding layer) NAL units, and NAL units that include metadata (e.g., parameter sets) can be referred to as non-VCL NAL units. In addition, JVET-N1001 enables supplementary enhancement information (SEI) messages to be signaled. In JVET-N1001, SEI messages assist processes related to decoding, display, or other purposes, however, SEI messages may not be required to construct luminance or chrominance samples through the decoding process. In JVET-N1001, non-VCL NAL units can be used to signal SEI messages in the bitstream. In addition, SEI messages can be transmitted in some way other than by being present in the bitstream (i.e., signaled out of band).

[0040] The access unit may be referred to as a layer access unit. As described above, multi-layer extension enables a video presentation to include a base layer and one or more additional enhancement layers. It should be noted that in ITU-T H.265, temporal proper subsets of scalable layers are not referred to as layers, but rather as sublayers or temporal sublayers: That is, ITU-T H.265 provides the following definition of sublayers: Sublayer: The temporal scalable layer of a temporally scalable bitstream consists of VCL NAL units with a specific value of the TemporalId variable and associated non-VCL NAL units. The terms sublayer and temporal sublayer are used interchangeably.

[0041] Figure 3An example of a bitstream including a plurality of CVSs is shown, wherein the CVSs are represented by NAL units included in corresponding access units. Figure 3 In the example shown, the non-VCL NAL units include corresponding parameter set NAL units (i.e., sequence parameter set (SPS) and picture parameter set (PPS) NAL units), SEI message NAL units, and access unit delimiter NAL units. Figure 3 HEADER is the NAL unit header.

[0042] JVET-N1001 defines the NAL unit header semantics, which specifies the type of raw byte sequence payload (RBSP) data structure included in the NAL unit. Table 1 shows the syntax of the NAL unit header provided in JVET-N1001. nal_unit_header(){ Descriptors zero_tid_required_flag u(1) nuh_temporal_id_plus1 u(3) nal_unit_type_lsb u(4) nuh_layer_id u(7) nuh_reserved_zero_bit u(1) } Table 1

[0043] JVET-N1001 provides the following definitions for the corresponding syntax elements shown in Table 1. Zero_tid_required_flag equal to 0 specifies that zero_tid_required_flag does not impose any additional constraints on the value of nuh_temporal_id_plus1. nuh_temporal_id_plus1 minus 1 specifies the temporal identifier of the NAL unit. The value of nuh_temporal_id_plus1 shall not be equal to 0. When zero_tid_required_flag is equal to 1, the value of nuh_temporal_id_plus1 shall be equal to 1. The variable TemporalId is derived as follows: TemporalId=nuh_temporal_id_plus1-1 NOTE—NAL unit types in the range 16 to 31 (inclusive) have zero_tid_required_flag equal to 1, and therefore have TemporalId equal to 0. The value of TemporalId of all VCL NAL units of a layer access unit shall be the same. The value of TemporalId of a coded picture or layer access unit is the value of TemporalId of the VCL NAL unit of the coded picture or layer access unit. The value of TemporalId for non-VCL NAL units is constrained as follows: - If NalUnitType is equal to SPS_NUT, then TemporalId is equal to 0, and the TemporalId of the layer access unit containing the NAL unit shall be equal to 0. - Otherwise, if NalUnitType is equal to APS_NUT, TemporalId shall be equal to the TemporalId of the layer access unit containing the NAL unit. - Otherwise, when NalUnitType is not equal to EOS_NUT and is not equal to EOB_NUT, TemporalId shall be greater than or equal to the TemporalId of the layer access unit containing the NAL unit. NOTE—When the NAL unit is a non-VCL NAL unit, the value of TemporalId is equal to the minimum of the TemporalId values ​​of all layer access units to which the non-VCL NAL unit applies. When NalUnitType is equal to PPS_NUT, TemporalId may be greater than or equal to the TemporalId of the containing layer access unit, because all picture parameter sets (PPS) may be included in the beginning of the bitstream, with the first coded picture having a TemporalId equal to 0. When NalUnitType is equal to PREFIX_SEI_NUT or SUFFLX_SEI_NUT, TemporalId may be greater than or equal to the TemporalId of the containing layer access unit, because a SEI NAL unit may contain information that applies to a subset of the bitstream that includes layer access units whose TemporalId values ​​are greater than the TemporalId of the layer access unit containing the SEI NAL unit. nuh_layer_id specifies the identifier of the layer to which the VCL NAL unit belongs or the identifier of the layer to which a non-VCL NAL unit applies. The value of nuh_layer_id SHOULD be in the range of 0 to 126, inclusive. The value 127 MAY be specified by ITU-T | ISO / IEC in the future. A decoder SHOULD ignore all data following the value 127 of nuh_layer_id in a NAL unit for purposes other than determining the amount of data in a decoding unit of the bitstream. NOTE - In future extensions of this specification, a nuh_layer_id value of 127 may be used to indicate an extension layer identifier. The value of nuh_layer_id of all VCL NAL units of a coded picture shall be the same. The value of nuh_layer_id of a coded picture is the value of nuh_layer_id of the VCL NAL unit of the coded picture. nuh_reserved_zero_bit shall be equal to '0'. The value 1 of the nuh_rescrved_zero bit may be specified by ITU-T|ISO / IEC in the future. A decoder shall ignore (ie, remove from the bitstream and discard) NAL units with nuh_reserved_zero_bit equal to '1'.

[0044] Regarding the syntax element nal_unit_type_lsb, JVET-N1001 provides the following: nal_unit_type_lsb specifies the least significant bit of the NAL unit type. The variable NalUnitType specifies the NAL unit type, i.e., the type of RBSP data structure contained in the NAL unit as specified in Table 2, and is derived as follows: NalUnitType=(zero_tid_required_flag<<4)+nal_unit_type_lsb NAL units with a NalUnitType in the range of UNSPEC28 ... UNSPEC31 (inclusive), whose semantics are unspecified, shall not affect the decoding process specified in this specification. NOTE—NAL unit types within the range of UNSPEC28…UNSPEC31 may be used as determined by the application. The decoding process for these values ​​of NalUnitType is not specified in this specification. Since different applications may use these NAL unit types for different purposes, special attention must be paid when designing encoders that generate NAL units with these NalUnitType values ​​and when designing decoders that interpret the contents of NAL units with these NalUnitType values. This specification does not define any management of these values. These NalUnitType values ​​may only be used in contexts where the use of "conflicts" (i.e., the meaning of the NAL unit content of the same NalUnitType value has different definitions) is unimportant, impossible, or managed, for example, defined or managed in a control application or transport specification, or managed by controlling the environment in which the bitstream is distributed. For purposes other than determining the amount of data in a decoding unit of the bitstream, a decoder should ignore (remove from the bitstream and discard) the contents of all NAL units that use reserved values ​​of NalUnitType. NOTE—This requirement allows for the definition of compatible extensions of this specification in the future. Table 2 NOTE—A clean random access (CRA) picture may have associated RASL or RADL pictures present in the bitstream. NOTE—An Instantaneous Decoding Refresh (IDR) picture with NalUnitType equal to IDR_N_LP does not have an associated leading picture present in the bitstream. An IDR picture with NalUnitType equal to IDR_W_RADL does not have an associated RASL picture present in the bitstream, but may have an associated RADL picture in the bitstream.

[0045] It should be noted that, in general, for example, with respect to ITU-T H.265, an IRAP picture is a picture that does not reference any picture other than itself for inter-frame prediction during its decoding process. Typically, the first picture in the decoding order in the bitstream must be an IRAP picture. In ITU-T H.265, an IRAP picture can be a broken link access (BLA) picture, a clean random access (CRA) picture, or an instantaneous decoding refresh (IDR) picture. ITU-T H.265 describes the concept of a leading picture, which is a picture that precedes the associated IRAP picture in the output order. ITU-T H.265 also describes the concept of a trailing picture, which is a non-IRAP picture that follows the associated IRAP picture in the output order. Trailing pictures associated with IRAP pictures are also after the IRAP picture in the decoding order. For IDR pictures, there is no need to reference trailing pictures that are decoded before the IDR picture. ITU-T H.265 specifies that a CRA picture may have a leading picture following the CRA picture in decoding order and contain inter-picture predictions that reference pictures decoded before the CRA picture. Therefore, when a CRA picture is used as a random access point, these leading pictures may not be decodable and are identified as random access skip leading (RASL) pictures. BLA pictures may also follow RASL pictures. When these RASL pictures are not decodable, that is, when the decoder starts its decoding process at the CRA point, these RASL pictures are always discarded for BLA pictures and for CRA pictures. Another type of picture that may follow an IRAP picture in decoding order and precede an IRAP picture in output order is a random access decodable leading (RADL) picture, which may not contain a reference to any picture preceding the IRAP picture in decoding order.

[0046] As described above, in JVET-N1001, a non-VCL NAL unit includes a corresponding parameter set NAL unit. Table 3 shows the sequence parameter set syntax provided in JVET-N1001. Table 3

[0047] Regarding Table 3, JVET-N1001 provides the following semantics: When sps_decoding_parameter_set_id, g_ When parameter_set_id is greater than 0, it specifies the value of dps_decoding_parameter_set_id of the DPS referenced by the SPS. When sps_decoding_parameter_set_id is equal to 0, the SPS does not reference a DPS, and when decoding each CVS that references the SPS, no DPS is active. When sps_video_parameter_set_id is greater than 0, it specifies the value of vps_video_parameter_set_id of the VPS referenced by the SPS. When sps_video_parameter_set_id is equal to 0, the SPS does not reference a VPS, and when decoding each CVS that references the SPS, no VPS is active. sps_max_sub_layers_minus1 plus 1 specifies the maximum number of temporal sub-layers that can be present in each CVS referencing the SPS. The value of sps_max_sub_layers_minus1 shall be in the range of 0 to 6, inclusive. In bitstreams conforming to this version of this specification, sps_reserved_zero_5bits shall be equal to 0. Other values ​​for sps_reserved_zero_5bits are reserved for future use by ITU-T | ISO / IEC. gra_enabled_flag equal to 1 specifies that GRA pictures may be present in the CVS referencing the SPS. gra_enabled_flag equal to 0 specifies that GRA pictures may not be present in the CVS referencing the SPS. sps_seq_parameter_set_id provides an identifier of the SPS for reference by other syntax elements. The value of sps seq parameter set id shall be in the range of 0 to 15 (inclusive). As specified, chroma_format_idc specifies the chroma samples relative to the luma samples. The value of chroma_format_idc shall be in the range of 0 to 3 (inclusive). Separate_colour_plane_flag equal to 1 specifies that the three colour components of the 4:4:4 chroma format are coded separately. separate_colour_plane_flag equal to 0 specifies that the colour components are not coded separately. When separate_colour_plane_flag is not present, it is inferred to be equal to 0. When separate_colour_plane_flag is equal to 1, the coded picture consists of three separate components, each consisting of coded samples of one colour plane (Y, Cb or Cr) and using monochrome coding syntax. In this case, each colour plane is associated with a specific colour_plane_id value. NOTE - There is no dependency in the decoding process between colour planes with different colour_plane_id values. For example, the decoding process of a monochrome picture with one value of colour_plane_id does not use any data from a monochrome picture with a different colour plane id value for inter prediction. Depending on the value of separate_colour_plane_flag, the value of the variable ChromaArrayType is specified as follows: - If separate_colour_plane flag is equal to 0, set ChromaArrayType equal to chroma_format_idc. - Otherwise (separate_colour__plane_flag is equal to 1), set ChromaArrayType equal to 0. pic_width_in_luma_samples specifies the width of each decoded picture in luma samples. pic_width_in_luma_samples shall not be equal to 0 and shall be an integer multiple of MinCbSizeY. pic_height_in_luma_samples specifies the height of each decoded picture in luma samples. pic_height_in_luma_samples shall not be equal to 0 and shall be an integer multiple of MinCbSizeY. conformance_window_flag equal to 1 indicates that the conformance cropping window offset parameter follows immediately in the SPS. conformance_window_flag equal to 0 indicates that there is no conformance cropping window offset parameter. conf_win_left_offset, conf_win_right_offset, conf_win_top_offset, and conf_win_bottom_offset specify samples of the picture output from the decoding process in the CVS, based on the rectangular area specified in the coordinates of the picture to be output. When conformance_window_flag is equal to 0, the values ​​of conf_win_left_offset, conf_win_right_offset, conf_win_top_offset, and conf_win_bottom_offset are inferred to be equal to 0. The fitted crop window contains luma samples with horizontal picture coordinates from SubWidthC*conf_win_left_offset to pic_width_in_luma_samples - (SubWidthC*conf_win_right_offset + 1), and vertical picture coordinates from SubHeightC*conf_win_top_offset to pic_height_in_luma_samples - (SubHeightC*conf_win_bottom_offset + 1), inclusive. The value of SubWidthC*(conf_win_left_offset+conf_win_right_offset) should be less than pic_width__in_luma_samples, and the value of SubHeightC*(conf_win_top_offset+conf_win_bottom_offset) should be less than pic_height_in_luma_samples. When ChromaArrayType is not equal to 0, the corresponding specified samples of the two chroma matrices are samples with picture coordinates (x / SubWidthC, y / SubHeightC), where (x, y) are the picture coordinates of the specified luma sample. NOTE - The Fit Crop Window Offset parameters are only applied on output. All internal decoding processes are applied to the uncropped image size. bit_depth_luma_minus8 specifies the brightness array BitDepth as follows Y The sample bit depth and luma quantizer range offset QpBdOffsety value: BitDepth Y =8+bit_depth_luma_minus8 QpBdOffset Y =6*bit_depth_luna_minus8 bit_depth_luma_minus8 should be in the range of 0 to 8 (inclusive). bit_depth_chroma_minus8 specifies the chroma array BitDepth as follows C The sample bit depth and chroma quantization parameter range offset QpBdOffset C Values: BitDepth C =8+bit_depth_chroma_minus8 QpBdOffset C =6*bit_depth_chroma_minus8 bit_depth_chroma_minus8 should be in the range of 0 to 8 (inclusive). log2 max_pic_order_cnt_lsb_minus4 specifies the value of the variable MaxPicOrderCntLsb used in the decoding process of the picture sequence number as follows: MaxPicOrderCntLsb=2 (log2_max_pic_order”cnt_lsb”minus4+4) The value of log2_max_pic_order_cnt_lsb_minus4 shall be in the range of 0 to 12 (inclusive). sps_sub_layer_ordering_info_present_flag equal to 1 specifies that sps_max_dec_pic_buffering_minus1[i], sps_max_num_reorder_pics[i], and sps_max_latency_increase_plus1[i] are present for sps_max_sub_layers_minus1+1 sublayers. sps_sub_layer_orderinginfo_present_flag equal to 0 specifies that the values ​​of sps_max_dec_pic_buffering_minus1[sps_max_sub_layers_minus1], sps_max_num_reorder_pics[sps_max_sub_layers_minus1], and sps_max_latency_increase_plus1[sps_max_sub_layers_minus1] apply to all sublayers. sps_max_dec_pic_buffering_minus1[i] plus 1 specifies the maximum required size of the decoded picture buffer of the CVS when HighestTid is equal to i, in units of picture storage buffers. The value of sps_max_dec_pic_buffering_minus1[i] shall be in the range of 0 to MaxDpbSize-1, inclusive, where MaxDpbSize is as specified elsewhere. When i is greater than 0, sps_max_dec_pic_buffering_minus1[i] shall be greater than or equal to sps_max_dec_pic_buffering_minus1[i-1]. When sps_max_dec_pic_buffering_minus1[i] is not present for i in the range of 0 to sps_max_sub_layers_minus1-1 (inclusive), it is inferred to be equal to sps_max_dec_pic_buffering_minus1[sps_max_sub_layers_minus1] since sps_sub_layer_ordering_info_present_flag is equal to 0. sps_max_num_reorder_pics[i] indicates the maximum allowed number of pictures that may precede any picture in the CVS in decoding order and follow that picture in output order when HighestTid is equal to i. The value of sps_max_num_reorder_pics[i] shall be in the range of 0 to sps_max_dec_pic_buffering_minus1[i], inclusive. When i is greater than 0, sps_max_num_reorder_pics[i] shall be greater than or equal to sps_max_num_reorder_pics[i-1]. When sps_max_num_reorder_pics[i] is not present for i in the range of 0 to sps_max_sub_layers_minus1 -1, inclusive, it is inferred to be equal to sps_max_num_reorder_pics[sps_max_sub_layers_minus1] since sps_sub_layer ordering info_present_nag is equal to 0. sps_max_latency_increase_plusl[i] not equal to 0 is used to calculate the value of SpsMaxLatencyPictures[i], which specifies the maximum number of pictures that can precede any picture in the CVS in output order and follow that picture in decoding order when HighestTid is equal to i. When sps_max_latency_increase_plusl[i] is not equal to 0, the value of SpsMaxLatencyPictures[i] is specified as follows: SpsMaxLatencyPictures[i]=sps_max_num_reorder_pics[i]+sps_max_latency_increase_plus1[i]-1 When sps_max_latency_increase_plus1[i] is equal to 0, no corresponding restriction is indicated. The value of sps_max_latency_increase_plus1[i] should be between 0 and 2. 32 When sps_max_latency_increase_plus1[i] is not present for i in the range of 0 to sps_max_sub_layers_minus1-1, it is inferred to be equal to sps_max_latency_increase_plus1[sps_max_sub_layers_minus1] since sps_sub_layer_ordering_info_present_flag is equal to 0. long_term_ref_pics_flag equal to 0 indicates that no LTRP is used for inter prediction of any coded picture in the CVS. long_term_ref_pics_flag equal to 1 indicates that LTRP may be used for inter prediction of one or more coded pictures in the CVS. sps_idr_rpl_present_flag equal to 1 specifies that the reference picture list syntax element is present in the slice header of the IDR picture. sps_idr_rpl_present_flag equal to 0 specifies that the reference picture list syntax element is not present in the slice header of the IDR picture. rpl1_same_as_rp10_flag equal to 1 indicates that the syntax structures num_refjpic_lists_in_sps[1] and ref_pic_list_struct(1,rplsIdx) do not exist and the following apply: - The value of num_ref_pic_lists_in_sps[1] is inferred to be equal to the value of num_ref_pic_lists_in_sps[0]. - The value of each of the syntax elements in refjpic_list_struct(1, rplsIdx) is inferred to be equal to the value of the corresponding syntax element in ref_pic_list_struct(0, rplsIdx) for rplsIdx in the range 0 to num_ref_pic_lists_in_sps[0]-1. num_ref_pic_lists_in_sps[i] specifies the number of ref_pic_list_struct(listIdx, rplsIdx) syntax structures included in the SPS, where listIdx is equal to i. The value of num_ref_pic_lists_in_sps[i] shall be in the range of 0 to 64, inclusive. NOTE—For each value of listIdx (equal to 0 or 1), the decoder should allocate memory for a total number of num_ref_pic_lists_in_sps[i]+1 syntax structures ref_pic_list_struct(listIdx,rplsIdx), because there may be one ref_pic_list_struct(listIdx,rplsIdx) syntax structure directly signaled in the slice header of the current picture. qtbtt_dual_tree_intra_flag equal to 1 specifies that, for I slices, each CTU is partitioned into coding units with 64x64 luma samples using implicit quadtree partitioning, and these coding units are the roots of two separate coding_tree syntax structures for luma and chroma. log2_ctu_size_minus2 plus 2 specifies the luma coding tree block size for each CTU. log2_min_luma_coding_block_size minus2 plus 2 specifies the minimum luma coding block size. The variables CtbLog2SizeY, CtbSizeY, MinCbLog2SizeY, MinCbSizeY, MinTbLog2SizeY, MaxTbLog2SizeY, MinTbSizeY, MaxTbSizeY, PicWidthlnCtbsY, PicHeightlnCtbsY, PicSizelnCtbsY, PicWidthlnMinCbsY, PicHeightlnMinCbsY, PicSizelnMinCbsY, PicSizelnSamplesY, PicWidthlnSamplesC, and PicHeightlnSamplesC are derived as follows: CtbLog2SizeY = log2_ctu_size_minus2 + 2 CtbSizeY = 1 << CtbLog2SizeY MinCbLog2SizeY = log2_min_luma_coding_block_size minus2 + 2 MinCbSizeY = 1 << MinCbLog2SizeY MinTbLog2SizeY = 2 MaxTbLog2SizeY = 6 MinTbSizeY = 1 << MinTbLog2SizeY MaxTbSizeY = 1 << MaxTbLog2SizeY PicWidthlnCtbsY = Ceil(pic_width_in_luma_samples ÷ CtbSizeY) PicHeightlnCtbsY = Ceil(pic_height_in_luma_samples ÷ CtbSizeY) PicSizelnCtbsY = PicWidthlnCtbsY * PicHeightlnCtbsY PicWidthlnMinCbsY = pic_width_in_luma_samples / MinCbSizeY PicHeightlnMinCbsY = pic_height_in_luma_samples / MinCbSizeY PicSizelnMinCbsY = PicWidthlnMinCbsY * PicHeightlnMinCbsY PicSizelnSamplesY = pic_width_in_luma_samples * pic_height_in_luma_samples PicWidthlnSamplesC = pic_width_in_luma_samples / SubWidthC PicHeightlnSamplesC = pic_height_in_luma_samples / SubHeightC The variables CtbWidthC and CtbHeightC respectively specify the width and height of the array of each chroma CTB, and these two variables are derived as follows: - If chroma_format_idc is equal to 0 (monochrome) or separate_colour_plane_flag is equal to 1, then both CtbWidthC and CtbHeightC are equal to 0. - Otherwise, CtbWidthC and CtbHeightC are derived as follows: CtbWidthC = CtbSizeY / SubWidthC CtbHeightC = CtbSizeY / SubHeightC For log2BlockWidth in the range from 0 to 4 (including the end values) and log2BlockHeight in the range from 0 to 4 (including the end values), the right upper diagonal scan order array initialization process as specified is called with 1 << log2BlockWidth and 1 << log2BlockHeight as inputs, and the output is assigned to DiagScanOrder[log2BlockWidth][log2BlockHeight]. partition_constraints_override_enabled_flag being equal to 1 specifies the presence of partition_constraints_override_flag in the slice header of the slice that references the SPS. partition_constraints_override_enabled_flag being equal to 0 specifies the absence of partition_constraints_override_flag in the slice header of the slice that references the SPS. sps_log2_diff_min_qt_min_cb_intra_slice_luma specifies the default difference between the base 2 logarithm of the minimum size in luma samples of the luma leaf blocks resulting from quadtree partitioning of the CTU and the base 2 logarithm of the minimum coding block size in luma samples of the luma CU in the slice that references the SPS with slice_type equal to 2(I). This default difference can be overwritten by slice_log2_diff_min_qt_min_cb_luma present in the slice header of the slice that references the SPS when partition_constraints_override_flag is equal to 1. The value of sps_log2_diff_min_qt_min_cb_intra_slice_luma should be in the range of 0 to CtbLog2SizeY-MinCbLog2SizeY (inclusive). The base 2 logarithm of the minimum size in luma samples of the luma leaf blocks resulting from quadtree partitioning of the CTU is derived as follows: MinQtLog2SizeIntraY=sps_log2_diff_min_qt_min_cb_intra_slice_luma+MinCbLog2SizeY sps_log2 diff_min_qt_min_cb_inter_slice specifies the default difference between the base 2 logarithm of the minimum size in luma samples of the luma leaf blocks resulting from quadtree partitioning of a CTU and the base 2 logarithm of the minimum luma coding block size in luma samples of the luma CU in the slice whose slice_type references the SPS equals 0 (B) or 1 (P). This default difference can be overwritten by slice_log2 diff_mim_qt_min cb_luma present in the slice header of the slice referencing the SPS when partition_constraints_override_flag is equal to 1. The value of sps log2_diff_min_qt_min_cb_inter_slice should be in the range of 0 to CtbLog2SizeY-MinCbLog2SizeY (inclusive). The base 2 logarithm of the minimum size in luma samples of the luma leaf blocks resulting from quadtree partitioning of a CTU is derived as follows: MinQtLog2SizeInterY=sps_log2_diff_min_qt_min_cb_inter_slice+MinCbLog2SizeY sps_max_mtt_hierarchy_depth_inter_slice specifies the default maximum hierarchical depth of coding units produced by multi-type tree partitioning of quadtree leaves in slices with slice_type equal to 0 (B) or 1 (P) that reference an SPS. When partition_constraints_override_flag is equal to 1, the default maximum hierarchical depth can be overridden by slice_max_mtt_hierarchy_depth_luma in the slice header of the slice that references the SPS. The value of sps_max_mti_hierarchy_depth_inter_slice should be in the range of 0 to CtbLog2SizeY-MinCbLog2SizeY, inclusive. sps_max_mtt_hierarchy_depth_intra_slice_luma specifies the default maximum hierarchy depth of coding units produced by multi-type tree partitioning of quadtree leaves in slices with slice_type equal to 2(I) that reference the SPS. When partition_constraints_override_flag is equal to 1, the default maximum hierarchy depth can be overridden by slice_max_mtt_hierarchy_dcpth luma in the slice header of the slice that references the SPS. The value of sps_max_mtt_hierarchy_depth_intra_slice_luma should be in the range of 0 to CtbLog2SizeY-MinCbLog2SizeY, inclusive. sps_log2_diff_max_bt_min_qt_intra_slice_luma specifies the default difference between the base-2 logarithm of the maximum size (width or height) in luma samples of a luma coding block that can use binary partitioning and the base-2 logarithm of the minimum size (width or height) in luma samples of luma leaf blocks produced by quadtree partitioning of CTUs in slices that reference the SPS with slice_type equal to 2(I). This default difference can be overridden by slice_log2_diff_max_bt_min_qt_luma present in the slice header of the slice that references the SPS when partition_constraints_override_flag is equal to 1. The value of sps_log2_diff_max_bt_min_qt_intra_slice_luma should be in the range of 0 to CtbLog2SizeY-MinQtLog2SizeIntraY, inclusive. When sps_log2_diff_max_bt_min_qt_intra_slice_luma is not present, the value of sps_log2_diff_max_bt_min_qt_intra_slice_luma is inferred to be equal to 0. sps_log2_diff_max_tt_min_qt_intra_slice_luma specifies the default difference between the base-2 logarithm of the maximum size (width or height) in luma samples of a luma coding block that can use ternary partitioning and the base-2 logarithm of the minimum size (width or height) in luma samples of luma leaf blocks produced by quadtree partitioning of CTUs in slices that reference the SPS with slice_type equal to 2(I). This default difference can be overridden by slice_log2_diff_max_tt_min_qt_luma present in the slice header of the slice that references the SPS when partition_constraints_override_flag is equal to 1. The value of sps_log2_diff_max_tt_min_qt_intra_slice_luma should be in the range of 0 to CtbLog2SizeY-MinQtLog2SizelntraY, inclusive. When sps_log2_diff_max_tt_min_qt_intra_slice_luma is not present, the value of sps_log2_diff_max_tt_min_qt_intra_slice_luma is inferred to be equal to 0. sps_log2_diff_max_bt_min_qt_inter_slice specifies the default difference between the base-2 logarithm of the maximum size (width or height) in luma samples of a luma coding block that can use binary partitioning and the base-2 logarithm of the minimum size (width or height) in luma samples of luma leaf blocks produced by quadtree partitioning of CTUs in slices with slice_type equal to 0 (B) or 1 (P) that reference the SPS. This default difference can be overridden by slice_log2_diff_max_bt_min_qt_luma present in the slice header of the slice that references the SPS when partition_constraints_override_flag is equal to 1. The value of sps_log2_diff_max_bt_min_qt_inter_slice should be in the range of 0 to CtbLog2SizeY-MinQtLog2SizeInterY, inclusive. When sps_log2_diff_max_bt_min_qt_inter_slice is not present, the value of sps_log2_diff_max_bt_min_qt_inter_slice is inferred to be equal to 0. sps_log2_diff_max_tt_min_qt_inter_slice specifies the default difference between the base-2 logarithm of the maximum size (width or height) in luma samples of a luma coding block that can use ternary partitioning and the base-2 logarithm of the minimum size (width or height) in luma samples of luma leaf blocks produced by quadtree partitioning of CTUs in slices with slice_type equal to 0 (B) or 1 (P) that reference the SPS. This default difference can be overridden by slice_log2 diffmax ttmin qtluma present in the slice header of the slice that references the SPS when partition_constraints_override_flag is equal to 1. The value of sps_log2_diff_max_tt_min_qt_inter_slice should be in the range of 0 to CtbLog2SizeY-MinQtLog2SizeInterY, inclusive. When sps_log2_diff_max_tt_min_qt_inter_slice is not present, the value of sps_log2_diff_max_tt_min_qt_inter_slice is inferred to be equal to 0. sps_log2_diff_min_qt_min_cb_intra_slice_chroma specifies the default difference between the base-2 logarithm of the minimum size in luma samples of chroma leaf blocks resulting from quadtree partitioning of chroma CTUs with treeType equal to DUAL_TREE_CHROMA and the base-2 logarithm of the minimum coding block size in luma samples of chroma CUs with treeType equal to DUAL_TREE_CHROMA in slices with slice_type equal to 2(I) referencing the SPS. This default difference may be overridden by the slice Jog2_diff_min_qt_min_cb_chroma present in the slice header of the slice referencing the SPS when partition_constraints_override_flag is equal to 1. The value of sps_log2_diff_min_qt_min_cb_intra_slice_chroma shall be in the range of 0 to CtbLog2SizeY-MinCbLog2SizeY, inclusive. When not present, the value of sps_log2_diff_min_qt_min_cb_intra_slice_chroma is inferred to be equal to 0. The base-2 logarithm of the minimum size among luma samples of chroma leaf blocks resulting from quadtree partitioning of a CTU with treeType equal to DUAL_TREE_CHROMA is derived as follows: MinQtLog2SizeIntraC=sps_log2_diff_min_qt_min_cb_intra_slice_chroma+MinCbLog2SizeY sps_max_mtt_hierarchy_depth_intra_slice_chroma specifies the default maximum hierarchy depth of chroma coding units resulting from multi-type tree partitioning of chroma quadtree leaves with treeType equal to DUAL_TREE_CHROMA in slices with slice_type equal to 2(I) of the referencing SPS. When partition_constraints_override_flag is equal to 1, the default maximum hierarchy depth may be overridden by slice_max_mtt_hierarchy_depth_chroma in the slice header of the slice referencing the SPS. The value of sps_max_mtt_hierarchy_depth_intra_slice_chroma shall be in the range of 0 to CtbLog2SizeY-MinCbLog2SizeY, inclusive. When not present, the value of sps_max_mtt_hierarchy_depth_intra_slice_chroma is inferred to be equal to 0. sps_log2_diff_max_bt_min_qt_intra_slice_chroma specifies the default difference between the base-2 logarithm of the maximum size (width or height) in luma samples of a chroma coded block that can use binary partitioning and the base-2 logarithm of the minimum size (width or height) in luma samples of chroma leaf blocks resulting from quadtree partitioning of chroma CTUs with treeType equal to DUAL_TREE_CHROMA in slices with slice_type equal to 2(I) that reference the SPS. This default difference may be overridden by slice_log2 diff_max_bt min_qt_chroma present in the slice header of the slice that references the SPS when partition_constraints_override_flag is equal to 1. The value of sps_log2_diff_max_bt_min_qt_intra_slice_chroma shall be in the range of 0 to CtbLog2SizeY - MinQtLog2SizeIntraC, inclusive. When sps_log2_diff_max_bt_min_qt_intra_slice_chroma is not present, the value of sps_log2_diff_max_bt_min_qt_intra_slice_chroma is inferred to be equal to 0. sps_log2_diff_max_tt_min_qt_intra_slice_chroma specifies the default difference between the base-2 logarithm of the maximum size (width or height) in luma samples of chroma coding blocks that can use ternary partitioning and the base-2 logarithm of the minimum size (width or height) in luma samples of chroma leaf blocks resulting from quadtree partitioning of chroma CTUs with treeType equal to DUALTREECHROMA in slices with slice_type equal to 2(I) that reference the SPS. This default difference may be overridden by slice_log2_diff_max_tt_min_qt_chroma present in the slice header of the slice that references the SPS when partition_constraints_override_flag is equal to 1. The value of sps_log2_diff_max_tt_min_qt_intra_slice_chroma shall be in the range of 0 to CtbLog2SizeY - MinQtLog2SizeIntraC, inclusive. When spslog2_diff_max_tt_min_qt_intra_slice_chroma is not present, the value of spslog2_diff_max_tt_min_qt_intra_slice_chroma is inferred to be equal to 0. sps_sao_enabled_flag equal to 1 specifies that the sample adaptive offset process is applied to the reconstructed picture after the deblocking filter process. sps_sao_enabled_flag equal to 0 specifies that the sample adaptive offset process is not applied to the reconstructed picture after the deblocking filter process. sps_alf_cnablcd_flag equal to 0 specifies that the adaptive loop filter is disabled. sps_alf_cnablcd_flag equal to 1 specifies that the adaptive loop filter is enabled. sps_pcm_enabled_flag equal to 0 specifies that the PCM-related syntax (pcm_sample_bit_depth_luma_minus1, pcm_sample_bit_depth_chroma_minus1, log2_min_pcm_luma_coding_block_size_minus3, log2_diff_max_min_pcm_luma_coding_block_size, pcm_loop_filter_disabled_flag, pcm_flag, pcm_alignment_zero_bit syntax elements, and pcm_sample() syntax structure) are not present in the CVS. NOTE—When MinCbLog2SizeY is equal to 6 and sps_pcm_enabled_flag is equal to 1, there is no PCM sample data related syntax (pcm_flag, pcm_alignment_zero_bit syntax elements and pcm _sample () syntax structure), because the maximum size of a coding block that can transmit syntax related to PCM sample data is limited to less than or equal to Min(CtbLog2SizeY, 5). Therefore, MinCbLog2SizeY equal to 6 and sps_pcm_enabled_flag equal to 1 are not appropriate settings for transmitting PCM sample data in CVS. pcm_sample_bit_depth_luma_minus1 specifies the number of bits used to represent each PCM sample value of the luma component as follows: PcmBitDepth Y =pcm_sample_bit_depth_luma_minus1+1 PcmBitDepth Y The value should be less than or equal to BitDepth Y The value of . pcm_sample_bit_depth_chroma_minus1 specifies the number of bits used to represent each PCM sample value of the chroma component as follows: PcmBitDepth C =pcm_sample_bit_depth_chroma_minus1+1 PcmBitDepth C The value should be less than or equal to BitDepth C When ChromaArrayType is equal to 0, pcm_sample_bit_depth_chroma_minus1 is not used during decoding and the decoder should ignore its value. log2_min_pcm luma_coding_block_size_minus3 plus 3 specifies the minimum size of a coding block with pcm_flag equal to 1. The variable Log2MinIpcmCbSizeY is set equal to log2_min_pcm_luma_coding_block_size_minus3 + 3. The value of Log2MinIpcmCbSizeY shall be in the range of Min(MinCbLog2SizeY, 5) to Min(CtbLog2SizeY, 5), inclusive. log2_diff_max_min_pcm_luma_coding_block_size specifies the difference between the maximum and minimum size of a coding block for which pcm_flag is equal to 1. The variable Log2MaxIpcmCbSizeY is set equal to log2_diff_max_min_pcm_luma_coding_block_size + Log2MinIpcmCbSizeY. The value of Log2MaxIpcmCbSizeY should be less than or equal to Min(CtbLog2SizeY,5). pcm_loop_filter_disabled_flag specifies whether the loop filtering process is disabled on reconstructed samples in coding units where pcm_flag is equal to 1 as follows: If pcm_loop_filter_disabled_flag is equal to 1, the deblocking filter process, the sample adaptive offset filter process, and the adaptive loop filter process are disabled on the reconstructed samples in the coding unit where pcm_flag is equal to 1. Otherwise (pcm_loop_filter_disabled_flag value is equal to 0), the deblocking filter process, the sample adaptive offset filter process and the adaptive loop filter process are not disabled on the reconstructed samples in the coding unit with pcm_flag equal to 1. When pcm_loop_filter_disabled_flag is not present, it is inferred to be equal to 0. sps_ref_wraparound_enabled_flag equal to 1 specifies that horizontal wraparound motion compensation is applied in inter prediction. sps_ref_wraparound_enabled_flag equal to 0 specifies that horizontal wraparound motion compensation is not applied. When not present, the value of sps_ref_wraparound_enabled_flag is inferred to be equal to 0. sps_ref_wraparound_offset_minus1 plus 1 specifies the offset used to calculate the horizontal wrap position in MinCbSizeY luma samples. The value of ref_wraparound_offset_minus1 should be in the range of (CtbSizeY / MinCbSizeY)+1 to (pic_width_in_luma_samples / MinCbSizeY)-1, inclusive. sps_temporal_mvp_enabled_flag equal to 1 specifies that slice_temporal_mvp_enabled_flag is present in the slice header of slices in the CVS with slice_type not equal to 1. sps_temporal_mvp_enabled_flag equal to 0 specifies that slice_temporal_mvp_enabled_flag is not present in the slice header and the temporal motion vector predictor is not used in the CVS. sps_sbtmvp_enabled_flag equal to 1 specifies that a subblock based temporal motion vector predictor may be used in picture decoding in the CVS, where all slices have slice_type not equal to 1. sps_sbtmvp_enabled_flag equal to 0 specifies that a subblock based temporal motion vector predictor is not used in the CVS. When sps_sbtmvp_enabled_flag is not present, it is inferred to be equal to 0. sps_amvr_enabled_flag equal to 1 specifies that adaptive motion vector difference resolution is used in motion vector coding. amvr_enabled_flag equal to 0 specifies that adaptive motion vector difference resolution is not used in motion vector coding. sps_bdof_enabled_flag equal to 0 specifies that bidirectional optical flow inter-frame prediction is disabled. sps_bdof_enabled_flag equal to 1 specifies that bidirectional optical flow inter-frame prediction is enabled. sps_smvd_enabled_flag equal to 1 specifies that symmetric motion vector differences can be used in motion vector decoding. sps_smvd_enabled_flag equal to 0 specifies that symmetric motion vector differences are not used in motion vector encoding. sps_affine_amvr_enabled_flag equal to 1 specifies that adaptive motion vector difference resolution is used in motion vector coding in affine inter mode, and sps_affine_amvr_enabled_flag equal to 0 specifies that adaptive motion vector difference resolution is not used in motion vector coding in affine inter mode. sps_dmvr_enabled_flag equal to 1 specifies that inter bi-prediction based on decoder motion vector modification is enabled. sps_dmvr_enabled_flag equal to 0 specifies that inter bi-prediction based on decoder motion vector modification is disabled. sps_mmvd_enabled_flag equal to 1 specifies that merge mode with motion vector difference is enabled. sps_mmvd_enabled_flag equal to 0 specifies that merge mode with motion vector difference is disabled. sps_isp_enabled_flag equal to 1 specifies that intra prediction with sub-partitioning is enabled. sps_isp_enabled_flag equal to 0 specifies that intra prediction with sub-partitioning is disabled. sps_mrl_enabled_flag equal to 1 specifies that intra prediction with multiple reference lines is enabled. sps_mrl_enabled_flag equal to 0 specifies that intra prediction with multiple reference lines is disabled. sps_mip_enabled_flag equal to 1 specifies that matrix-based intra prediction is enabled. sps_mip_enabled_flag equal to 0 specifies that matrix-based intra prediction is disabled. sps_cclm_enabled_flag equal to 0 specifies that cross-component linear model intra prediction from luma components to chroma components is disabled, and sps_cclm_enabled_flag equal to 1 specifies that cross-component linear model intra prediction from luma components to chroma components is enabled. sps_cclm_colocated_chroma_flag equal to 1 specifies that the left up and down sampled luma samples in cross-component linear model intra prediction are co-located with the top left luma sample. sps_cclm_colocated_chroma_flag equal to 0 specifies that the left up and down sampled luma samples in cross-component linear model intra prediction are horizontally co-located with the top left luma sample, but vertically shifted by 0.5 luma sample units relative to the top left luma sample. sps_mts_enabled_flag equal to 1 specifies that sps_explicit_mts_intra_enabled_flag is present in the sequence parameter set RBSP syntax and sps_explicit_mts_inter_enabled_flag is present in the sequence parameter set RBSP syntax. sps_mts_enabled_flag equal to 0 specifies that sps_explicit_mts_intra_enabled_flag is not present in the sequence parameter set RBSP syntax and sps_explicit_mts_inter_enabled_flag is not present in the sequence parameter set RBSP syntax. sps_explicit_mts_intra_enabled_flag equal to 1 specifies that tu_mts_idx may be present in transform unit syntax for intra coding units. sps_explicit_mts_intra_enabled_flag equal to 0 specifies that tu_mts_idx is not present in transform unit syntax for intra coding units. When not present, the value of sps_explicit_mts_intra_enabled_flag is inferred to be equal to 0. sps_explicit_mts_inter_enabled_flag equal to 1 specifies that tu_mts_idx may be present in transform unit syntax for inter coding units. sps_explicit_mts_inter_enabled_flag equal to 0 specifies that tu_mts_idx is not present in transform unit syntax for inter coding units. When not present, the value of sps_explicit_mts_inter_enabled_flag is inferred to be equal to 0. sps_sbt_enabled_flag equal to 0 specifies that sub-block transform for inter-predicted CUs is disabled. sps_sbt_enabled_flag equal to 1 specifies that sub-block transform for inter-predicted CUs is enabled. sps_sbt_max_size_64_flag equal to 0 specifies that the maximum CU width and height for allowing sub-block transforms is 32 luma samples. sps_sbt_max_size_64_flag equal to 1 specifies that the maximum CU width and height for allowing sub-block transforms is 64 luma samples. MaxSbtSize=sps_sbt_max_size_64_flag? 64:32 sps_affine_enabled_flag specifies whether affine model-based motion compensation can be used for inter prediction. If sps_affine_enabled_flag is equal to 0, the syntax should be constrained so that affine model-based motion compensation is not used in CVS, and inter_affine_flag and cu_affine_type_flag are not present in the coding unit syntax of CVS. Otherwise (sps_affine_enabled_flag is equal to 1), affine model-based motion compensation can be used in CVS. sps_afflne_type_flag specifies whether motion compensation based on a 6-parameter affine model can be used for inter prediction. If sps_affine_type_flag is equal to 0, the syntax should be constrained so that motion compensation based on a 6-parameter affine model is not used in the CVS, and cu_affine_type_flag is not present in the coding unit syntax in the CVS. Otherwise (sps_affine_type_flag is equal to 1), motion compensation based on a 6-parameter affine model can be used in the CVS. When not present, the value of sps_affine_type_flag is inferred to be equal to 0. sps_bcw_enabled_flag specifies whether bi-prediction with CU weights can be used for inter prediction. If sps_bcw_enabled_flag is equal to 0, the syntax should be constrained so that bi-prediction with CU weights is not used in CVS, and bcw_idx is not present in the coding unit syntax of CVS. Otherwise (sps_bcw_enabled_flag is equal to 1), bi-prediction with CU weights can be used in CVS. sps_ibc_enabled_flag equal to 1 specifies that the current picture reference may be used in picture decoding in CVS. sps_ibc_enabled_flag equal to 0 specifies that the current picture reference is not used in CVS. When sps_ibc_enabled_flag is not present, it is inferred to be equal to 0. sps_ciip_enabled_flag specifies that ciip_flag may be present in the coding unit syntax for inter coding units. sps_ciip_enabled_flag equal to 0 specifies that ciip_flag is not present in the coding unit syntax for inter coding units. sps_fpel_mmvd_enabled_flag equal to 1 specifies that merge mode with motion vector differences is using integer sample precision. sps_fpel_mmvd_enabled_flag equal to 0 specifies that merge mode with motion vector differences may use fractional sample precision. sps_triangle_enabled_flag specifies whether triangle shape based motion compensation can be used for inter prediction. sps_triangle_enabled_flag equal to 0 specifies that the syntax should be constrained so that triangle shape based motion compensation is not used in CVS, and merge_triangle_split_dir, merge_triangle_idx0, and merge_triangle_idx1 are not present in the coding unit syntax of CVS. sps_triangle_enabled_flag equal to 1 specifies that triangle shape based motion compensation can be used in CVS. sps_lmcs_enabled_flag equal to 1 specifies that luma mapping with chroma scaling is used in CVS. sps_lmcs_enabled_flag equal to 0 specifies that luma mapping with chroma scaling is not used in CVS. sps_ladf_enabled_flag equal to 1 specifies that sps_num_ladf_intervals_minus2, sps_ladf_lowest_interval_qp_offset, sps_ladf_qp_offset[i], and sps_ladf_delta_threshold_minus1[i] are present in the SPS. sps_num_ladf_intervals_minus2 plus 1 specifies the number of sps_ladf_delta_threshold_minus1[i] and sps_ladf_qp_offset[i] syntax elements present in the SPS. The value of sps_num_ladf_intervals_minus2 shall be in the range of 0 to 3, inclusive. sps_ladf_lowest_interval_qp_offset specifies the offset used to derive the specified variable qP. The value of sps_ladf_lowest_interval_qp_offset should be in the range of 0 to 63 (inclusive). sps_ladf_qp_offset[i] specifies the offset array used to derive the specified variable qP. The value of sps_ladf_qp_offset[i] should be in the range of 0 to 63 (inclusive). sps_ladf_delta_threshold_minus1[i] is used to calculate the value of SpsLadfIntervalLowerBound[i], which specifies the lower bound of the i-th interval of luminance intensity levels. The value of sps_ladf_delta_threshold_minus1[i] should be between 0 and 2. BitDepthY -3 (including the end value). Set the value of SpsLadfIntervalLowerBound[0] equal to 0. For each value of i in the range of 0 to sps_num_ladf_intervals_minus2 (inclusive), the variable SpsLadfIntervalLowerBound[i+1] is derived as follows: SpsLadfIntervalLowerBound[i+1]=SpsLadfIntervalLowerBound[i] +sps_ladf_delta_threshold_minus1[i]+1 timing_info_present_flag equal to 1 specifies that the syntax elements num_units_in_tick, time_scale, and hrd_parameters_present_flag are present in the SPS RBSP syntax structure. timing_info_present_flag equal to 0 specifies that num_units_in_tick, time_scale, and hrd_parameters_present_flag are not present in the SPS RBSP syntax structure. num_units_in_tick is the number of time units of a clock operating at a frequency time_scale Hz, which corresponds to one increment of the clock tick counter (called a clock tick). num_units_in_tick should be greater than 0. The clock tick in seconds is equal to the quotient of num_units_in_tick divided by time_scale. For example, when the picture rate of the video signal is 25 Hz, time_scale may be equal to 27,000,000 and num_units_in_tick may be equal to 1,080,000, and thus the clock tick may be equal to 0.04 seconds. time_scale is the number of time units that pass in one second. For example, a time coordinate system that uses a 27MHz clock to measure time has a time_scale of 27,000,000. The value of time_scale should be greater than 0. hrd_parameters_present_flag equal to 1 specifies that the syntax structure hrd_parameters() is present in the SPS RBSP syntax structure. hrd_parameters_present_flag equal to 0 specifies that the syntax structure hrd_parameters() is not present in the SPS RBSP syntax structure. vui_parameters_present_flag equal to 1 specifies that the syntax structure vui_parameters() is present in the SPS RBSP syntax structure. vui_parameters_present_flag equal to 0 specifies that the syntax structure vui_parameters() is not present in the SPS RBSP syntax structure. sps_extension_flag equal to 0 specifies that the sps_extension_data_flag syntax structure is not present in the SPS RBSP syntax structure. sps_extension_flag equal to 1 specifies that the sps_extension_data_flag syntax structure is present in the SPS RBSP syntax structure. sps_extension_data_flag can have any value. Its presence and value do not affect whether a decoder conforms to the profile specified in this version of this specification. Decoders conforming to this version of this specification should ignore all sps_extension_data_flag syntax elements.

[0048] As shown in Table 3, hrd_parameters_present_flag specifies whether the syntax structure hrd_parameters() is present in the SPS RBSP syntax structure. The syntax structure hrd_parameters() provides the so-called hypothetical reference decoder (HRD) parameters. Typically, the video coding standard specifies an HRD that can be used to check the compliance of the bitstream and the decoder. For example, the video coding standard may specify an HRD that includes a coded picture buffer (CPB), an instantaneous decoding process, a decoded picture buffer (DPB), and output clipping and corresponding compliance tests. The HRD parameter value may specify which compliance tests to be performed / how to perform the compliance test. Therefore, the video decoder may perform a compliance test based on the received HRD parameter value to determine whether the bitstream complies. In addition, the video decoder may use the timing information provided in the HRD parameter value to determine the best time to output the decoded picture, which can provide the best video quality. HRD operation for a compliant bitstream and the HRD parameters that are signaled can ensure that when the bitstream is decoded with a specified buffer size, a specified bit rate, and a specified initial buffer delay, the video buffer will never overflow or underflow. In this context, buffer overflow may mean that the video buffer has received new data, but it does not have the ability to fill the video buffer with that data because the video buffer is already full of data to be decoded or displayed. Additionally, in this context, buffer underflow may mean that the video decoder is in a state where the video decoder can decode the next picture from the data in the video buffer (e.g., be able to present it to the user without interruption), but does not have any data in its buffer. Thus, the video decoder may decode the encoded video based on the signaled HRD parameters.

[0049] JVET-N1001 specifies HRD. Table 4 shows the syntax structure of hrd_parameters() provided in JVET-N1001. Table 4

[0050] Regarding Table 4, JVET-N1001 provides the following semantics: hrd_parameters() provides the HRD parameters used in HRD operations. vui_nal_hrd_parameters_present_flag equal to 1 specifies that NAL HRD parameters are present in the hrd_parameters() syntax structure (which is relevant for Type II bitstream compliance). vui_nal_hrd_parameters_present_flag equal to 0 specifies that NAL HRD parameters are not present in the hrd_parameters() syntax structure. NOTE—When vui_nal_hrd_paramcters_present_flag is equal to 0, without providing NAL HRD parameters and all buffering period and picture timing SEI messages, the bitstream conformance cannot be verified by some means not specified in this specification. The variable NalHrdBpPresentFlag is derived as follows: - NalHrdBpPresentFlag is set equal to 1 if one or more of the following conditions are true: - vui_nal_hrd_parameters_present_flag is present in the bitstream and is equal to 1. - The presence of a buffering period for NAL HRD operations in the bitstream, as present in the buffering period SEI message, needs to be determined by the application in some way not specified in this specification. - Otherwise, set the value of NalHrdBpPresentFlag equal to 0. vui_vcl_hrd_parameters_present_flag equal to 1 specifies that VCL HRD parameters are present in the hrd_parameters() syntax structure (which is relevant for all bitstream compliance). vui_vcl_hrd_parameters_present_flag equal to 0 specifies that VCL HRD parameters are not present in the hrd_parameters() syntax structure. NOTE—When vui_vcl_hrd_parameters_present_flag is equal to 0, without providing the VCL HRD parameters and all buffering period and picture timing SEI messages, the conformance of the bitstream cannot be verified by some means not specified in this specification. The variable VclHrdBpPresentFlag is derived as follows: - VclHrdBpPresentFlag is set equal to 1 if one or more of the following conditions are true: -vui_vcl_hrd parameters present_flag is present in the bitstream and is equal to 1. - The need for the presence of a buffering period for VCL HRD operations in the bitstream, as present in the buffering period SEI message, is determined by the application in some way not specified in this specification. - Otherwise, set the value of VclHrdBpPresentFlag equal to 0. The variable CpbDpbDelaysPresentFlag is derived as follows: - Set CpbDpbDelaysPresentFlag equal to 1 if one or more of the following conditions are true: - vui_nal_hrd_parameters_present_flag is present in the bitstream and is equal to 1. - vui_vcl_hrd_parameters_present_flag is present in the bitstream and is equal to 1. - The presence of CPB and DPB output delays in the bitstream in the picture timing SEI message needs to be determined by the application by some means not specified in this specification. - Otherwise, set the value of CpbDpbDelaysPresentFlag equal to 0. bit rate scale (together with bit rate value_minus1[i]) specifies the maximum input bit rate for the i-th CPB. Cpb_size_scale (together with cpb_size_value_minus1[i]) specifies the CPB size of the i-th CPB when the CPB operates at the access unit level. fixed_pic_rate_general_flag[i] equal to 1 indicates that when HighestTid is equal to i, the temporal distance between HRD output times of consecutive pictures in output order is constrained as described below. fixed_pic_rate_general_flag[i] equal to 0 indicates that this constraint may not apply. When fixed_pic_rate_general_flag[i] is not present, it is inferred to be equal to 0. fixed_pic_rate_within_cvs_flag[i] equal to 1 indicates that when HighestTid is equal to i, the temporal distance between HRD output times of consecutive pictures in output order is constrained as described below. fixed_pic_rate_within_cvs_flag[i] equal to 0 indicates that this constraint may not apply. When fixed_pic_rate_general_flag[i] is equal to 1, the value of fixed_pic_rate_within_cvs_flag[i] is inferred to be equal to 1. elemental_duration_in tc_minus1[i] plus 1, when present, specifies the temporal distance in clock ticks between elemental units of HRD output times of consecutive pictures in output order as specified below when HighestTid is equal to i. The value of elemental_duration_in_tc_minus1[i] shall be in the range of 0 to 2047, inclusive. For each picture n that is output other than the last picture in the output bitstream (in output order), the value of the variable DpbOutputElementalInterval[n] is specified as follows: DpbOutputElementalInterval[n]=DpbOutputInterval[n]÷DeltaToDivisor Where DpbOutputInterval[n] is specified in Formula C-13 (i.e., DpbOutputInterval[n] = DpbOutputTime[nextPicInOutputOrder] - DpbOutputTime[n]), and DeltaToDivisor is specified based on the values ​​of frame_field_info_present_flag and pic_struct of the CVS containing picture n in Table 5. Entries marked with "-" in Table 5 indicate that DeltaToDivisor does not depend on the corresponding syntax element. When HighestTid is equal to i and fixed_pic_rate_general_flag[i] is equal to 1 for the CVS containing picture n, the value calculated for DpbOutputElementalInterval[n] shall be equal to ClockTick*(elemental_duration_in_tc_minus1[i]+1), where ClockTick is specified by Formula C-1 (i.e., ClockTick=num_units_in_tick÷time_scale) (using the value of ClockTick for the CVS containing picture n) when one of the following conditions is true for the next picture in the output order nextPicInOutputOrder specified for use in Formula C-13: -Picture nextPicInOutputOrder is in the same CVS as picture n. - The picture nextPicInOutputOrder is in different CVSs, and fixed_pic_rate_general_flag[i] is equal to 1 in the CVS containing the picture nextPicInOutputOrder, the value of ClockTick is the same for both CVSs, and the value of elemental_duration_in_tc_minus1[i] is the same for both CVSs. When HighestTid is equal to i and fixed_pic_rate_within_cvs_flag[i] is equal to 1 for the CVS containing picture n, the value calculated for DpbOutputElementalInterval[n] shall be equal to ClockTick*(elemental_duration_in_tc_minus1[i]+1), where ClockTick is specified by Formula C-1 (using the value of ClockTick for the CVS containing picture n) when the next picture in the output order nextPicInOutputOrder specified for use in Formula C-13 is in the same CVS as picture n. Table 5 When HighestTid is equal to i, low_delay_hrd_flag[i] specifies the HRD operation mode. When not present, the value of low_delay_hrd_flag[i] is inferred to be equal to 0. NOTE—When low_delay_hrd_flag[i] is equal to 1, "big pictures" are allowed that violate the nominal CPB removal time due to the number of bits used by the access unit. It is expected, but not required, that such "big pictures" only occur occasionally. vui_cpb_cnt_minus1[i] plus 1 specifies the number of alternative CPB specifications in the bitstream of the CVS when HighestTid is equal to i. The value of vui_cpb_cnt_minus1[i] shall be in the range of 0 to 31, inclusive. When not present, the value of vui_cpb_cnt_minus1[i] is inferred to be equal to 0.

[0051] Table 6 shows the sublayer HRD parameter set syntax structure provided in JVET-N1001. sub_layer_hrd_parameters(subLayerId){ Descriptors for(i=0;i<=CpbCnt;i++){ bit_rate_value_minus1[i] ue(v) cpb_size_value_minus1[i] ue(v) cbr_flag[i] u(1) } } Table 6

[0052] Regarding Table 6, JVET-N1001 provides the following semantics: Set the variable CpbCnt equal to vui_cpb_cnt_minus1[subLayerId]. bit_rate_value_minus1[i] (together with bit_rate_scale) specifies the maximum input bit rate for the i-th CPB when the CPB operates at access unit level. bit_rate_value_minus1[i] should be between 0 and 2. 32 For any i>0, bit_rate_value_minus1[i] shall be greater than bit_rate_value_minus1[i-1]. The bit rate in bits per second is given by: BitRate[i]=(bit_rate_value_minus1[i]+1)*2 (6+bit-rate-scale) When the bit_rate_value_minus1[i] syntax element is not present, the value of BitRate[i] is inferred to be equal to CpbBrVclFactor*MaxBR for VCLHRD parameters and CpbBrNalFactor*MaxBR for NAL HRD parameters, where MaxBR, CpbBrVclFactor and CpbBrNalFactor are specified in subclause A.4.] cpb_size_value_minus1[i] is used together with cpb_size_scale to specify the size of the i-th CPB when the CPB operates at the access unit level. cpb_size_value_minus1[i] should be between 0 and 2. 32 For any i greater than 0, cpb_size_value_minus1[i] shall be less than or equal to cpb_size_value_minus1[i-1]. The CPB size, in bits, is given by: CpbSize[i]=(cpb_size_value_minus1[i]+1)*2 (4+cpb_size-scale ) When the cpb_size_value_minus1[i] syntax element is not present, the value of CpbSize[i] is inferred to be equal to CpbBrVclFactor*MaxCPB for VCLHRD parameters and to CpbBrNalFactor*MaxCPB for NAL HRD parameters, where MaxCPB, CpbBrVclFactor and CpbBrNalFactor are specified in subclause A.4.] cbr_flag[i] equal to 0 specifies that for decoding of the bitstream by the HRD using the i-th CPB specification, the Hypothetical Stream Scheduler (HSS) operates in intermittent bitrate mode. cbr_flag[i] equal to 1 specifies that the HSS operates in constant bitrate (CBR) mode. When not present, The value of cbr_flag[i] is inferred to be equal to 0. It should be noted that although the semantics of the syntax elements bit_rate_value_minus1 and cpb_size_value_minus1 include a reference to subclause A.4 for specifying MaxCPB, CpbBrVclFactor, and CpbBrNalFactor, JVET-N1001 does not define Annex A and therefore does not include subclause A.4 and a mechanism for specifying MaxBR, CpbBrVclFactor, and CpbBrNalFactor. Techniques for specifying MaxCPB, CpbBrVclFactor, and CpbBrNalFactor according to the techniques herein are provided below with respect to Table 10.

[0053] As described above, JVET-N1001 enables SEI messages to be signaled, which facilitates processes related to decoding, display, or other purposes. In addition, as provided above in the semantics of Table 4, the SEI message types for VCL HRD operations include buffering period SEI messages. Table 7 shows the syntax structure of buffering_period() provided in JVET-N1001. Table 7

[0054] Regarding Table 7, JVET-N1001 provides the following semantics: The buffering period SEI message provides initial CPB removal delay and initial CPB removal delay offset information for initializing HRBs at the location of the associated access unit in decoding order. When the buffering period SEI message is present, when a picture has a TemporalId equal to 0, the picture is called a notDiscardablePic picture and is not a RASL or RADL picture. When the current picture is not the first picture in decoding order in the bitstream, let prevNonDiscardablePic be the previous picture in decoding order with TemporalId equal to 0, which is not a RASL or RADL picture. The presence of the buffering period SEI message is specified as follows: - If NalHrdBpPresentFlag is equal to 1 or VclHrdBpPresentFlag is equal to 1, then for each access unit in the CVS, the following applies: - If the access unit is an IRAP access unit, a buffering period SEI message applicable to the operation point shall be associated with the access unit. - Otherwise, if the access unit contains notDiscardablePic, a buffering period SEI message applicable to the operation point may or may not be associated with the access unit. - Otherwise, the access unit shall not be associated with a buffering period SEI message applicable to the operation point. - Otherwise (both NalHrdBpPresentFlag and VclHrdBpPresentFlag are equal to 0), no access unit in the CVS shall be associated with the buffering period SEI message. NOTE—For some applications, frequent buffering period SEI messages may be desired (eg, for random access at IRAP pictures or non-IRAP pictures or for bitstream splicing). bp_seq_parameter_set_id indicates and shall be equal to sps_seq_parameter_set_id of the SPS that is active for the coded picture associated with the buffering period SEI message. The value of bp_seq_parameter_set_id shall be equal to the value of pps_seq_parameter_set_id in the PPS referenced by slice_pic_parameter_set_id of the slice header of the coded picture associated with the buffering period SEI message. The value of bp_seq_parameter_set_id shall be in the range of 0 to 15, inclusive. bp_nal_hrd_parameters_present_flag equal to 1 specifies that the list of syntax elements nal_initial_cpb_removal_delay[i] and nal_initial_cpb_removal_offset[i] is present in the buffering period SEI message. bp_nal_hrd_parameters_present_flag equal to 0 specifies that the list of syntax elements nal_initial_cpb_removal_delay[i] and nal_initial_cpb_removal_offset[i] is not present in the buffering period SEI message. Bitstream conformance requires that the value of bp_nal_hrd_parameters_present_flag in the buffering period SEI message associated with the access unit is equal to the value of vui_nal_hrd_paramctersprescnt flag in the VUI parameters of the active SPS. bp_vcl_hrd_parameters_present_flag equal to 1 specifies that the list of syntax elements vcl_initial cpb_removal_delay[i] and vcl_initial_cpb_removal_offset[i] is present in the buffering period SEI message. bp_vcl_hrd_parameters_present_flag equal to 0 specifies that the list of syntax elements vcl_initial_cpb_removal_delay[i] and vcl_initial_cpb_removal_offset[i] is not present in the buffering period SEI message. Bitstream conformance requires that the value of bp_vcl_hrd_parameters_present_flag in the buffering period SEI message associated with an access unit is equal to the value of vui_vcl_hrd_parameters_present_flag in the VUI parameters of the active SPS. initial_cpb_removal_delay_length_minus1 plus 1 specifies the length of the syntax elements nal_initial_cpb_removal_delay[i], nal_initial_cpb_removal_offset[i], vcl_initial_cpb_removal_delay[i], and vcl_initial_cpb_removal_offset[i] of the buffering period SEI message in bits. When not present, the value of initial_cpb_removal_delay_length_minus1 is inferred to be equal to 23. cpb_removal_delay_length_minus1 plus 1 specifies the length in bits of the syntax elements cpb_removal_delay_delta_minus1 and cpb_removal_delay_delta[i] in the buffering period SEI message and the syntax element cpb_removal_delay__minus1[i] in the picture timing SEI message. When not present, the value of cpb_removal_delay_length_minus1 is inferred to be equal to 23. dpb_output_delay_length_minus1 plus 1 specifies the length of the syntax element dpb_output_delay in the picture timing SEI message in bits. When not present, the value of dpb_output_delay_length_minus1 is inferred to be equal to 23. concatenation_flag indicates whether the nominal CPB removal time of the current picture is determined relative to the nominal CPB removal time of the previous picture with the buffering period SEI message or relative to the nominal CPB removal time of the picture prevNonDiscardablePic when the current picture is not the first picture in the decoding order in the bitstream. cpb_removal_delay_delta_minus1 plus 1 specifies: the CPB removal delay delta value relative to the nominal CPB removal time of picture prevNonDiscardablePic, when the current picture is not the first picture in decoding order in the bitstream. The length of this syntax element is cpb_removal_deiay_length minus1+1 bits. When the current picture contains a buffering period SEI message and concatenation_flag is equal to 0 and the current picture is not the first picture in decoding order in the bitstream, bitstream conformance requires the following constraints to apply: - If the picture prevNonDiscardablePic is not associated with a buffering period SEI message, the cpb_removal_delay_minus1 of the current picture shall be equal to the cpb_removal_delay_minus1 of prevNonDiscardablePic plus cpb_removal_delay_delta_minus1+1. Otherwise, cpb_removal_delay_minus1 shall be equal to cpb_removal_delay_delta_minus 1. NOTE—When the current picture contains a buffering period SEI message and concatenation_flag is equal to 1, cpb_removal_delay_minus1 for the current picture is not used. In some cases, the constraints specified above may make it possible to splice the bitstream by simply changing the value of concatenation_flag from 0 to 1 in the buffering period SEI message for the IRAP picture at the splice point (which uses a properly designed reference structure). When concatenation_flag is equal to 0, the constraints specified above enable the decoder to check whether the constraints are satisfied as a way to detect the loss of the picture prevNonDiscardablePic. cpb_removal_delay_deltas_present_flag equal to 1 specifies that the buffering period SEI message contains CPB removal delay deltas. cpb_removal_delay_deltas_present_flag equal to 0 specifies that no CPB removal delay deltas are present in the buffering period SEI message. num_cpb_removal_delay_deltas_minus1 plus 1 specifies the number of syntax elements cpb_removal_delay_delta[i] in the buffering period SEI message. The value of num_cpb_removal_offsets_minus1 shall be in the range of 0 to 15, inclusive. cpb_removal_delay_delta[i] specifies the i-th CPB removal delay increment. The length of this syntax element is cpb_removal_delay_length_minus1+1 bits. bp_cpb_cnt_minus1 plus 1 specifies the number of syntax element pairs nal_initial_cpb_removal_delay[i] and nal_initial_cpb_removal_offset[i] when bp_nal_hrd_parameters_present_flag is equal to 1, and specifies the number of syntax element pairs vcl_initial_cpb_removal_delay[i] and vcl_initial_cpb_removal_offset[i] when bp_vcl_hrd_parameters_present_flag is equal to 1. The value of bp_cpb_cnt_minus1 shall be in the range of 0 to 31, inclusive. nal_initial_cpb_removal_delay[i] specifies the i-th initial CPB removal delay of the NAL HRD in 90kHz clock units. The length of nal_initial_cpb_removal_delay[i] is initial_cpb_removal_delay_length_minus1+1 bits. The value of nal_initial_cpb_removal_delay[i] shall not be equal to 0 and shall be less than or equal to 90000*(CpbSize[i]4÷BitRate[i]), which is the time equivalent of the CPB size in 90kHz clock units. nal_initial_cpb_removal_offset[i] specifies the i-th initial CPB removal offset of the NAL HRD in 90kHz clock units. The length of nal_initial_cpb_removal_offset[i] is initial_cpb_removal_delay_length_minus1+1 bits. Over the entire CVS, for each value of i, the sum of nal_initial_cpb_removal_delay[i] and nal_initial_cpb_removal_offset[i] shall be constant. vcl_initial_cpb_removal_delay[i] specifies the i-th initial CPB removal delay of the VCL HRD in 90kHz clock units. The length of vcl_initial_cpb_removal_delay[i] is initial_cpb_removal_delay_length_minus1+1 bits. The value of vcl_initial_cpb_removal_delay[i] shall not be equal to 0 and shall be less than or equal to 90000*(CpbSize[i]÷BitRate[i]), which is the time equivalent of the CPB size in 90kHz clock units. vcl_initial_cpb_removal_offset[i] specifies the i-th initial CPB removal offset of the VCL HRD in 90kHz clocks. The length of vcl_initial_cpb_removal_offset[i] is initial_cpb_removal_delay_length_minus1+1 bits. The sum of vcl_initial_cpb_removal_delay[i] and vcl_initial_cpb_removal_offset[i] shall be constant for each value of i over the entire CVS.

[0055] Referring to Table 7 and the corresponding semantics, the buffering period SEI message signals the initial CPB removal delay for NAL HRD and VCL HRD. This is achieved through the syntax elements nal_initial_cpb_removal_delay[i], nal_initial_cpb_removal_offset[i] for NAL HRD and vcl_initial_cpb_removal_delay[i], vcl_initial_cpb_removal_offset[i] for VCL HRD. The number of these NAL and VCL HRD initial CPB removal delays is signaled for different HRD schedules (signaled via the syntax element bp_cpb_cnt_minus1), and the appropriate initial CPB removal delay parameters are allowed to be selected based on the selected HRD schedule index (SchedSelIdx). However, one buffering period SEI message only signals the initial CPB removal delay parameters for a specific HighestTid value. Because bitstreams can operate at different highest TemporalID values, and because the initial CPB removal delay value is different for sub-bitstreams with different numbers of temporal sub-layers, signaling is less than ideal because it is not allowed to signal the initial CPB removal delay in the buffering period SEI message for all supported different HighestTid values ​​of the bitstream. According to the techniques herein, a buffering period SEI message is described that allows the initial CPB removal delay to be signaled for all supported different HighestTid values ​​of the bitstream, and the initial CPB removal delay parameter can be appropriately selected based on the selected HRD schedule and the selected highest number of temporal sub-layers. This allows for low overhead parameters for signaling the initial CPB removal delay parameter.

[0056] In JVET-N1001, signaling HRD parameters is less than ideal. Specifically, in JVET-N1001, when HRD parameters exist, corresponding instances of HRD parameters are signaled for each time sublayer. Such signaling may be inefficient because, in some cases, time sublayers may share a common set of HRD parameters. In other cases, the need to signal HRD parameters for each time sublayer may be cumbersome because these parameters need to be calculated using relatively complex formulas. According to the techniques described herein, a mechanism is provided for conditionally signaling corresponding instances of HRD parameters for sublayers. In addition, inference rules are specified for situations where corresponding instances of HRD parameters are not signaled for each sublayer.

[0057] Figure 11 is a block diagram illustrating an example of a system that may be configured to encode (e.g., encode and / or decode) video data in accordance with one or more techniques of this disclosure. System 100 represents an example of a system that may encapsulate video data in accordance with one or more techniques of this disclosure. Figure 1 As shown, system 100 includes source device 102, communication medium 110, and target device 120. Figure 1 In the example shown, source device 102 may include any device configured to encode video data and transmit the encoded video data to communication medium 110. Target device 120 may include any device configured to receive the encoded video data via communication medium 110 and decode the encoded video data. Source device 102 and / or target device 120 may include computing devices equipped for wired and / or wireless communication, and may include, for example, set-top boxes, digital video recorders, televisions, desktop, laptop or tablet computers, game consoles, medical imaging devices, and mobile devices (including, for example, smart phones, cellular phones, personal gaming devices).

[0058] The communication medium 110 may include any combination of wireless and wired communication media and / or storage devices. The communication medium 110 may include coaxial cables, fiber optic cables, twisted pair cables, wireless transmitters and receivers, routers, switches, repeaters, base stations, or any other device that can be used to facilitate communication between various devices and sites. The communication medium 110 may include one or more networks. For example, the communication medium 110 may include a network configured to allow access to the World Wide Web, such as the Internet. The network may operate according to a combination of one or more telecommunication protocols. The telecommunication protocol may include proprietary aspects and / or may include standardized telecommunication protocols. Examples of standardized telecommunication protocols include digital video broadcasting (DVB) standards, advanced television system committee (ATSC) standards, integrated service digital broadcasting (ISDB) standards, cable data service interface specifications (DOCSIS) standards, global mobile communication systems (GSM) standards, code division multiple access (CDMA) standards, 3rd generation partnership project (3GPP) standards, European Telecommunications Standards Institute (ETSI) standards, Internet Protocol (IP) standards, wireless application protocol (WAP) standards, and Institute of Electrical and Electronics Engineers (IEEE) standards.

[0059] The storage device may include any type of device or storage medium capable of storing data. The storage medium may include a tangible or non-transitory computer-readable medium. The computer-readable medium may include an optical disk, a flash memory, a magnetic memory, or any other suitable digital storage medium. In some examples, the memory device or a portion thereof may be described as a non-volatile memory, and in other examples, a portion of the memory device may be described as a volatile memory. Examples of volatile memory may include random access memory (RAM), dynamic random access memory (DRAM), and static random access memory (SRAM). Examples of non-volatile memory may include a magnetic hard disk, an optical disk, a floppy disk, a flash memory, or an electrically programmable memory (EPROM) or an electrically erasable and programmable (EEPROM) memory. The storage device may include a memory card (e.g., a secure digital (SD) memory card), an internal / external hard disk drive, and / or an internal / external solid-state drive. Data may be stored on the storage device according to a defined file format.

[0060] Figure 4 is a conceptual diagram illustrating examples of components that may be included in a specific implementation of system 100. Figure 4 In the example implementation shown, system 100 includes one or more computing devices 402A-402N, a television service network 404, a television service provider site 406, a wide area network 408, a local area network 410, and one or more content provider sites 412A-412N. Figure 4 The specific implementations shown in represent examples of a system that can be configured to allow digital media content (such as movies, live sporting events, etc.) and data and applications associated therewith and media presentations to be distributed to and accessed by multiple computing devices (such as computing devices 402A to 402N). Figure 4 In the example shown, computing devices 402A to 402N may include any device configured to receive data from one or more of television service network 404, wide area network 408, and / or local area network 410. For example, computing devices 402A to 402N may be equipped for wired and / or wireless communications and may be configured to receive services over one or more data channels, and may include televisions, including so-called smart televisions, set-top boxes, and digital video recorders. In addition, computing devices 402A to 402N may include desktop computers, laptop or tablet computers, game consoles, mobile devices (including, for example, "smart" phones, cellular phones, and personal gaming devices).

[0061] The television service network 404 is an example of a network configured to enable distribution of digital media content that may include television services. For example, the television service network 404 may include a public air television network, a public or subscription-based satellite television service provider network, and a public or subscription-based cable television provider network and / or a cloud or Internet service provider. It should be noted that, although in some examples, the television service network 404 may be primarily used to allow the provision of television services, the television service network 404 may also allow the provision of other types of data and services according to any combination of the telecommunications protocols described herein. In addition, it should be noted that in some examples, the television service network 404 may allow two-way communication between the television service provider site 406 and one or more of the computing devices 402A to 402N. The television service network 404 may include any combination of wireless and / or wired communication media. The television service network 404 may include coaxial cables, fiber optic cables, twisted pair cables, wireless transmitters and receivers, routers, switches, repeaters, base stations, or any other device that can be used to facilitate communication between various devices and sites. The television service network 404 may operate according to a combination of one or more telecommunications protocols. The telecommunication protocol may include proprietary aspects and / or may include a standardized telecommunication protocol. Examples of standardized telecommunication protocols include the DVB standard, the ATSC standard, the ISDB standard, the DTMB standard, the DMB standard, the Data Over Cable Service Interface Specification (DOCSIS) standard, the HbbTV standard, the W3C standard, and the UPnP standard.

[0062] Reference again Figure 4 , the television service provider site 406 may be configured to distribute television services via the television service network 404. For example, the television service provider site 406 may include one or more broadcast stations, cable television providers or satellite television providers, or Internet-based television providers. For example, the television service provider site 406 may be configured to receive transmissions (including television programs) via satellite uplink / downlink. In addition, as Figure 4 As shown, television service provider site 406 may be in communication with wide area network 408 and may be configured to receive data from content provider sites 412A through 412N. It should be noted that in some examples, television service provider site 406 may include a television studio and the content may originate from the television studio.

[0063] Wide area network 408 may include a packet-based network and operate according to a combination of one or more telecommunication protocols. Telecommunication protocols may include proprietary aspects and / or may include standardized telecommunication protocols. Examples of standardized telecommunication protocols include global system mobile communications (GSM) standards, code division multiple access (CDMA) standards, 3rd generation partnership project (3GPP) standards, European Telecommunications Standards Institute (ETSI) standards, European standards (EN), IP standards, wireless application protocols (WAP) standards, and Institute of Electrical and Electronics Engineers (IEEE) standards, such as, one or more IEEE 802 standards (e.g., Wi-Fi). Wide area network 408 may include any combination of wireless and / or wired communication media. Wide area network 408 may include coaxial cables, fiber optic cables, twisted pair cables, Ethernet cables, wireless transmitters and receivers, routers, switches, repeaters, base stations, or any other equipment that can be used to facilitate communication between various devices and sites. In one example, wide area network 408 may include the Internet. Local area network 410 may include a packet-based network and operate according to a combination of one or more telecommunication protocols. The local area network 410 may be distinguished from the wide area network 408 based on access levels and / or physical infrastructure. For example, the local area network 410 may include a secure home network.

[0064] Reference again Figure 4 , content provider sites 412A to 412N represent examples of sites that can provide multimedia content to television service provider sites 406 and / or computing devices 402A to 402N. For example, a content provider site may include a studio with one or more studio content servers, which are configured to provide multimedia files and / or streams to television service provider sites 406. In one example, content provider sites 412A to 412N may be configured to provide multimedia content using an IP suite. For example, a content provider site may be configured to provide multimedia content to a receiver device according to a real-time streaming protocol (RTSP), HTTP, etc. In addition, content provider sites 412A to 412N may be configured to provide data including hypertext-based content, etc. to one or more of receiver devices 402A to 402N and / or television service provider sites 406 via a wide area network 408. Content provider sites 412A to 412N may include one or more web servers. The data provided by data provider sites 412A to 412N may be defined according to a data format.

[0065] Reference again Figure 1, source device 102 includes video source 104, video encoder 106, data encapsulator 107 and interface 108. Video source 104 may include any device configured to capture and / or store video data. For example, video source 104 may include a camera and a storage device operably coupled thereto. Video encoder 106 may include any device configured to receive video data and generate a compatible bitstream representing the video data. A compatible bitstream may refer to a bitstream from which a video decoder can receive and reproduce video data. Various aspects of a compliant bitstream may be defined according to a video coding standard. When generating a compatible bitstream, video encoder 106 may compress the video data. The compression may be lossy (perceptible or imperceptible to the viewer) or lossless. Figure 5 is a block diagram illustrating an example of a video encoder 500 that can implement the techniques described herein for encoding video data. It should be noted that although the exemplary video encoder 500 is illustrated as having different functional blocks, such illustration is intended for descriptive purposes and does not limit the video encoder 500 and / or its subcomponents to a particular hardware or software architecture. The functionality of the video encoder 500 may be implemented using any combination of hardware, firmware, and / or software implementations.

[0066] The video encoder 500 can perform intra-frame prediction encoding and inter-frame prediction encoding of a picture region and thus can be referred to as a hybrid video encoder. Figure 5 In the example shown, the video encoder 500 receives a source video block. In some examples, the source video block may include picture regions that have been divided according to a coding structure. For example, the source video data may include macroblocks, CTUs, CBs, sub-partitions thereof, and / or additional equivalent coding units. In some examples, the video encoder 500 may be configured to perform additional subdivisions of the source video block. It should be noted that the techniques described herein are generally applicable to video encoding regardless of how the source video data is divided before and / or during encoding. Figure 5 In the example shown, the video encoder 500 includes an adder 502, a transform coefficient generator 504, a coefficient quantization unit 506, an inverse quantization and transform coefficient processing unit 508, an adder 510, an intra-frame prediction processing unit 512, an inter-frame prediction processing unit 514, a filter unit 516, and an entropy coding unit 518. Figure 5 As shown, video encoder 500 receives source video blocks and outputs a bitstream.

[0067] exist Figure 5In the example shown, the video encoder 500 can generate residual data by subtracting the predicted video block from the source video block. The selection of the predicted video block is described in detail below. The summer 502 represents a component configured to perform the subtraction operation. In one example, the subtraction of the video block occurs in the pixel domain. The transform coefficient generator 504 applies a transform such as a discrete cosine transform (DCT), a discrete sine transform (DST), or a conceptually similar transform (for example, four 8×8 transforms can be applied to a 16×16 residual value array) to the residual block or its sub-partition to generate a set of residual transform coefficients. The transform coefficient generator 504 can be configured to perform any and all combinations of transforms included in the discrete triangular transform series, including its approximation. The transform coefficient generator 504 can output the transform coefficients to the coefficient quantization unit 506. The coefficient quantization unit 506 can be configured to perform quantization of the transform coefficients. The quantization process can reduce the bit depth associated with some or all coefficients. The degree of quantization can change the rate distortion (i.e., the relationship between bit rate and video quality) of the encoded video data. The degree of quantization can be modified by adjusting a quantization parameter (QP). The quantization parameter can be determined based on a slice level value and / or a CU level value (e.g., a CU delta QP value). The QP data may include any data used to determine the QP used to quantize a particular set of transform coefficients. Figure 5 As shown, the quantized transform coefficients (which may be referred to as level values) are output to the inverse quantization and transform coefficient processing unit 508. The inverse quantization and transform coefficient processing unit 508 may be configured to apply inverse quantization and inverse transformation to generate reconstructed residual data. Figure 5 As shown, at the summer 510, the reconstructed residual data can be added to the predicted video block. In this way, the encoded video block can be reconstructed, and the resulting reconstructed video block can be used to evaluate the coding quality of a given prediction, transformation and / or quantization. The video encoder 500 can be configured to perform multiple encoding rounds (e.g., performing encoding while changing one or more of the prediction, transformation parameters and quantization parameters). The rate distortion or other system parameters of the bitstream can be optimized based on the evaluation of the reconstructed video block. In addition, the reconstructed video block can be stored and used as a reference for predicting subsequent blocks.

[0068] Reference again Figure 5, the intra-frame prediction processing unit 512 may be configured to select an intra-frame prediction mode for a video block to be encoded. The intra-frame prediction processing unit 512 may be configured to evaluate a frame and determine an intra-frame prediction mode for encoding a current block. As described above, possible intra-frame prediction modes may include a planar prediction mode, a DC prediction mode, and an angular prediction mode. Furthermore, it should be noted that in some examples, the prediction mode of the chrominance component may be inferred based on the prediction mode of the luma prediction mode. The intra-frame prediction processing unit 512 may select an intra-frame prediction mode after performing one or more encoding rounds. Furthermore, in one example, the intra-frame prediction processing unit 512 may select a prediction mode based on a rate-distortion analysis. As Figure 5 As shown, the intra prediction processing unit 512 outputs intra prediction data (e.g., syntax elements) to the entropy coding unit 518 and the transform coefficient generator 504. As described above, the transform performed on the residual data can be mode-dependent (e.g., the secondary transform matrix can be determined based on the prediction mode).

[0069] Reference again Figure 5 , the inter-frame prediction processing unit 514 may be configured to perform inter-frame prediction encoding for the current video block. The inter-frame prediction processing unit 514 may be configured to receive a source video block and calculate a motion vector of a PU of the video block. The motion vector may indicate the displacement of a PU of a video block within a current video frame relative to a prediction block within a reference frame. Inter-frame prediction encoding may use one or more reference pictures. In addition, motion prediction may be unidirectional prediction (using one motion vector) or bidirectional prediction (using two motion vectors). The inter-frame prediction processing unit 514 may be configured to select a prediction block by calculating a pixel difference determined by, for example, the sum of absolute differences (SAD), the sum of square differences (SSD), or other difference metrics. As described above, a motion vector may be determined and specified based on motion vector prediction. As described above, the inter-frame prediction processing unit 514 may be configured to perform motion vector prediction. The inter-frame prediction processing unit 514 may be configured to generate a prediction block using motion prediction data. For example, the inter-frame prediction processing unit 514 may locate a prediction video block ( Figure 5 ). It should be noted that the inter-frame prediction processing unit 514 can be further configured to apply one or more interpolation filters to the reconstructed residual block to calculate sub-integer pixel values ​​for motion estimation. The inter-frame prediction processing unit 514 can output the motion prediction data of the calculated motion vector to the entropy coding unit 518.

[0070] like Figure 5As shown, filter unit 516 receives the reconstructed video blocks and encoding parameters and outputs modified reconstructed video data. Filter unit 516 may be configured to perform deblocking and / or sample adaptive offset (SAO) filtering. SAO filtering is a nonlinear amplitude mapping that can be used to improve reconstruction by adding an offset to the reconstructed video data. It should be noted that, as Figure 5 As shown, the intra prediction processing unit 512 and the inter prediction processing unit 514 can receive the modified reconstructed video block via the filter unit 216. The entropy coding unit 518 receives the quantized transform coefficients and the prediction syntax data (i.e., the intra prediction data and the motion prediction data). It should be noted that in some examples, the coefficient quantization unit 506 can perform a scan of the matrix including the quantized transform coefficients before outputting the coefficients to the entropy coding unit 518. In other examples, the entropy coding unit 518 can perform the scan. The entropy coding unit 518 can be configured to perform entropy coding according to one or more of the techniques described herein. As such, the video encoder 500 represents an example of a device configured to generate encoded video data according to one or more techniques of the present disclosure.

[0071] Reference again Figure 1 , the data encapsulator 107 can receive the encoded video data and generate a compatible bitstream according to a defined data structure, for example, a sequence of NAL units. A device receiving the compatible bitstream can reproduce the video data therefrom. In addition, as described above, sub-bitstream extraction can refer to the process by which a device receiving a bitstream compliant with ITU-T H.265 forms a new bitstream compliant with ITU-T H.265 by discarding and / or modifying data in the received bitstream. It should be noted that the term compliant bitstream can be used instead of the term compatible bitstream. In one example, the data encapsulator 107 can be configured to generate syntax according to one or more techniques described herein. It should be noted that the data encapsulator 107 is not necessarily located in the same physical device as the video encoder 106. For example, the functions described as being performed by the video encoder 106 and the data encapsulator 107 can be distributed in Figure 4 in the device shown.

[0072] As described above, in JVET-N1001, signaling HRD parameters is less than ideal. According to the techniques herein, signaling HRD parameters for time sublayers is controlled by a flag. That is, in one example, according to the techniques herein, a flag controls whether to signal a corresponding HRD parameter for each time sublayer or a single instance of the HRD parameter for all time sublayers. In the case of a single instance of the HRD parameter being signaled for all time sublayers, a specific HRD parameter is inferred for each time sublayer.

[0073] Table 8 shows an example of the syntax of a sequence parameter set that may be signaled according to the techniques herein. Table 8

[0074] With respect to Table 8, the semantics may be based on the semantics provided above with respect to Table 3 for the syntax element sub_layer_cpb_parameters_present_flag. In one example, the following semantics may be used: sub_layer_cpb_parameters_present_flag equal to 1 specifies that the syntax structure hrdjparameters() includes parameters (sub_layer_hrd_parameters(i), where i is in the range of 0 to spsmax_sub_layers_minus1, inclusive) for temporal sublayers in the range of 0 to spsmax_sub_layers_minus1, inclusive). sub_layer_cpb_parameters_present_flag equal to 0 specifies that the syntax structure hrd_parameters() includes parameters for only the sps_max_sub_layers_minus1-th temporal sublayer. In another example, the following semantics may be used: sub_layer_cpb_parameters_present_flag equal to 1 specifies that the syntax structure hrd_parameters() includes parameters for temporal sublayers in the range of 0 to sps_max_sub_layers_minus1, inclusive (sub_layer_hrd_parametcrs(i), where i is in the range of 0 to sps_max_sub_layers_minus1, inclusive). sub_layer_cpb_parameters_present_flag equal to 0 specifies that the syntax structure hrd_parameters() includes parameters for sub_jayer_hrd_parameters(sps_max_sub_layers_minnus) only, and these parameters apply to all sublayers. In one example, one or more occurrences of the words "parameters for temporal sublayers" may be changed to the words "parameters for temporal sublayer representations," and the words "parameters only for sub_layer_hrd_parameters(sps_max_sub_layers_minus)" may be changed to the words "parameters only for highest temporal sublayer representations."

[0075] Table 9 shows an example of an HRD syntax structure that may be signaled according to the techniques herein. Table 9

[0076] With respect to Table 9, the semantics may be based on the semantics provided above with respect to Table 4.

[0077] With respect to Table 6 above, in one example, according to the techniques herein, the semantics may be based on the following semantics: Set the variable CpbCnt equal to vui_cpb_cnt_minus1[subLayerId]. bit_rate_value_minus1[i] (together with bit_rate_scale) specifies the maximum input bit rate for the i-th CPB when the CPB operates at access unit level. bit_rate_value_minus1[i] should be between 0 and 2. 32 -2 (inclusive). For any i>0, bit rate value_minus1[i] shall be greater than bitjrate_value_minus1[i-1]. The bit rate in bits per second is given by: BitRate[i]=(bit_rate_value_minus 1[i]+1)*2 (6+bit_rate_scale) When the bit_ratevalue_minus1[i] syntax element is not present, If bit rate_value_minus1[sps_max_sub_layers_minus1] is present, the value of BitRate[i] is inferred to be equal to BitRate[sps_max_sub_layers_minus1]. Otherwise, the value of BitRate[i] is inferred to be equal to CpbBrVclFactor*MaxBR for VCL HRD parameters and equal to CpbBrNalFactor*MaxBR for NAL HRD parameters, where MaxBR, CpbBrVclFactor, and CpbBrNalFactor are specified as provided below.] cpb_size_value_minus1[i] is used together with cpb_size_scale to specify the i-th CPB size when CPB operates at access unit level. cpb_size_value_minus1[i] should be between 0 and 2. 32For any i greater than 0, cpb_size_value_minus1[i] shall be less than or equal to cpb_size_value_minus1[i-1]. The CPB size, in bits, is given by: CpbSize[i]=(cpb_size_value_minus1[i]+1)*2 (4+cpb_size_scale) When the cpb_size_value_minus1[i] syntax element is not present, If cpb_size_value_minus1[sps_max_sub_layers_minus1] is present, the value of CpbSize[i] is inferred to be equal to CpbSize[sps_max_sub_layers_minus1]. Otherwise, the value of CpbSize[i] is inferred to be equal to CpbBrVclFactor*MaxCPB for VCL HRD parameters and equal to CpbBrNalFactor*MaxCPB for NAL HRD parameters, where MaxCPB, CpbBrVclFactor, and CpbBrNalFactor are specified as provided below. cbr_flag[i] equal to 0 specifies that for decoding of the bitstream by the HRD using the i-th CPB specification, the Hypothetical Stream Scheduler (HSS) operates in intermittent bitrate mode. cbr_flag[i] equal to 1 specifies that the HSS operates in constant bitrate (CBR) mode. When not present, If cbr_flag[sps_max_sub_layers_minus1] is present, the value of cbr_flag[i] is inferred to be equal to cbr_flag[sps_max_sub_layers_minus1]. The value of cbr_flag[i] is inferred to be equal to 0. In another example, when not present, the value of cbr_flag[i] is inferred to be equal to 0.

[0078] As provided with respect to Tables 8 and 9 and updated semantic Table 6, according to the techniques herein, the number of iterations of the loop (i=start-NumSubLayers; i<=maxNumSubLayersMinus1; i++){...} is one or maxNumSubLayersMinus1+1. Therefore, the amount of data used to signal HRD parameters can be reduced compared to JVET-N1001.

[0079] In a variant example, according to the techniques herein, an existing flag in JVET-N1001 (rather than a new flag) may be used to control whether HRD CPB parameters are signaled for each temporal sublayer or only for all temporal sublayers. In the case where HRD CPB parameters are signaled for all temporal sublayers, HRD CPB parameters are inferred for each temporal sublayer. For example, a flag sps_sub_layer_ordering_info_present_flag signaled in the SPS may be used instead of the new flag sub_layer_cpb_parameters_present_flag. In this case, in one example, the semantics of sps_sub_layer_ordering_info_present_flag may be as follows: sps_sub_layer_ordering_info_present_flag equal to 1 specifies that sps_max_dec_pic_buffering_minus1[i], sps_max_num_reorder_pics[i], and sps_max_latency_increase_plus1[i] are present for sps__max_sub_layers_minus1+1 sublayers, and that the syntax structure hrd_parameters() includes parameters (sub_layer_hrd_parameters(i), where i is in the range of 0 to sps_max_sub_layers_minus1, inclusive) for temporal sublayers in the range of 0 to sps_max_sub_layers_minus1, inclusive. sps_sub_layer_ordering_info_present_flag equal to 0 specifies that the values ​​of sps_max_dec_pic_buffering_minus1[spsmax_sub_layers_minus1], sps_max_num_reorder_pics[sps_max_sub_layers_minus1] and sps__max_latency_increase_plus1[sps_max_sub_layers_minus1] apply to all sub-layers, and likewise the syntax structure hrd_parameters() includes parameters only for sub_layer_hrd_paramcters(sps_max_sub_layers_minus) and these parameters apply to all sub-layers.

[0080] As mentioned above, JVET-N1001 does not define Appendix A for specifying MaxBR, CpbBrVclFactor, and CpbBrNalFactor. In one example, according to the techniques of this article, MaxBR, CpbBrVclFactor, and CpbBrNalFactor may be defined as follows: - Set the variable CpbBrVclFactor equal to 1000. -Set the variable CpbBrNalFactor equal to 1100. -MaxBR can be specified as a function of level as shown in Table 10 below. Tier and level restrictions for Main profile and Main 10 profile Table 10

[0081] As described above, the buffering period SEI message in JVET-N1001 is less than ideal. In one example, according to the technology of this article, a flag is used to specify whether a signal can be sent to notify the initial CPB removal delay value associated with different highest temporal sublayer representations. In addition, in one example, if the flag is 1, the initial CPB removal delay value is signaled for different highest temporal sublayers supported by the bitstream. In addition, the operation of the coded picture buffer is modified based on HighestTid to use the appropriate initial buffering delay parameters. In addition, the BitRate and CpbSize values ​​represented by each temporal sublayer can be calculated separately.

[0082] Table 11 shows an example of a buffering period SEI message syntax structure that may be signaled according to the techniques herein. Table 11

[0083] With respect to Table 11, the semantics may be based on the semantics provided above with respect to Table 7 for the syntax elements bp_nal_hrd_parameters_present_flag, bp_vcl_hrd_parameters_present_flag, initial_cpb_removal_delay_length_minus1, bp_cpb_ent_minus1[j], nal_initial_cpb_removal_delay[i][j], nal_initial_cpb_removal_offset[i][j], vcl_initial_cpb_removal_delay[i][j], and vcl_initial_cpb_removal_offset[i][j], in one example, the semantics may be based on the following: bp_nal_hrd_parameters_resent_flag equal to 1 specifies that the list of syntax elements nal_initial_cpb_removal_delay[i][j] and nal_initial_cpb_removal_offset[i][j] is present in the buffering period SEI message. bp_nal_hrd_parameters_present_flag equal to 0 specifies that the list of syntax elements nal_initial_cpb_removal_delay[i][j] and nal_initial_cpb_removal_offset[i][j] is not present in the buffering period SEI message. Bitstream conformance requires that the value of bp_nal_hrd_parameters_present_flag in the buffering period SEI message associated with the access unit is equal to the value of vui_nal_hrd_parameters_present_flag in the VUI parameters of the active SPS. bp_vcl_hrd_parameters_present_flag equal to 1 specifies that a list of syntax elements vcl_initial_cpb_removal_delay[i][j] and vcl_initial_cpb_removal_offset[i][j] is present in the buffering period SEI message. bp_vcl_hrd_parameters_present_flag equal to 0 specifies that a list of syntax elements vcl_initial_cpb_removal_delay[i][j] and vcl_initial_cpb_removal_offset[i][j] is not present in the buffering period SEI message.

[0084] Bitstream conformance requires that the value of bp_vcl_hrd_parameters_present_flag in the buffering period SEI message associated with an access unit is equal to the value of vui_vcl_hrd_parameters_present_flag in the VUI parameters of the active SPS. initial_cpb_removal_delay_length_minus1 plus 1 specifies the length of the syntax elements nal_initial_cpb_removal_delay[i][j], nal_initial_cpb_removal_offset[i][j], vcl_initial_cpb_removal_delay[i][j] and vcl_initial_cpb_removal_offset[i][j] of the buffering period SEI message in bits. When not present, the value of initial_cpb_removal_delay_length_minus1 is inferred to be equal to 23. bp_cpb_cnt_minus1[j] plus 1 specifies the number of syntax element pairs nal_initial_cpb_removal_delay[i][j] and nal_initial_cpb_removal_offset[i][j] for the j-th temporal sub-layer representation when bp_nal_hrd_parameters_present_flag is equal to 1, and specifies the number of syntax element pairs vcl_initial_cpb_removal_delay[i][j] and vcl_initial_cpb_removal_offset[i][j] for the j-th temporal sub-layer representation when bp_vcl_hrd_paraineters_presenl_flag is equal to 1. The value of bp_cpb_cnt_minus1[j] shall be in the range of 0 to 31, inclusive. nal_initial_cpb_removal_delay[i][j] specifies the i-th initial CPB removal delay of the NALHRD for the j-th temporal sublayer representation, in units of 90kHz clock. The length of nal_initial_cpb_removal_delay[i][j] is initial_cpb_removal_delay_length_minus1+1 bits. The value of nal_initial_cpb_removal_delay[i][j] shall not be equal to 0 and shall be less than or equal to 90000*(CpbSize[i]4-BitRate[i]), which is the time equivalent of the CPB size in units of 90kHz clock, where BitRate[i], CpbSize[i], respectively, are obtained from the formulas provided above by setting the HighestTid value equal to j. nal_initial_cpb_removal_offset[i][j] specifies the i-th initial CPB removal offset of the NAL HRD for the j-th temporal sub-layer representation in 90kHz clock. The length of nal_initial_cpb_removal_offset[i][j] is initial_cpb_removal_delay_length_minus1+1 bits. Over the entire CVS, for each value of i and for each value of j, the sum of nal_initial_cpb_removal_delay[i][j] and nal_initial_cpb_removal_offset[i][j] shall be constant. vcl_initial_cpb_removal_delay[i][j] specifies the i-th initial CPB removal delay of the VCL HRD for the j-th temporal sublayer representation, in units of 90kHz clock. The length of vcl_initial_cpb_removal_delay[i][j] is initial_cpb_removal_delay_length_minus1+1 bits. The value of vcl_initial_cpb_removal_delay[i][j] shall not be equal to 0 and shall be less than or equal to 90000*(CpbSize[i]+BitRate[i]), which is the temporal equivalent of the CPB size in units of 90kHz clock, where BitRate[i], CpbSize[i] are obtained from the formulas provided above by setting the HighestTid value equal to j, respectively. vcl_initial_cpb_removal_offset[i][j] specifies the i-th initial CPB removal offset of the VCL HRD for the j-th temporal sub-layer representation in units of 90kHz clock. The length of vcl_initial_cpb_removal_offset[i][j] is initial_cpb_removal_delay_length_minus1+1 bits. Over the entire CVS, for each value of i and for each value of j, the sum of vcl_initial_cpb_removal_delay[i][j] and vcl_initial_cpb_removal_offset[i][j] shall be constant.

[0085] It should be noted that, in one example, the term "for the j-th temporal sub-layer representation" may be replaced with the term "applied to HighestTid equal to j".

[0086] In addition, in one example, a flag is signaled, and depending on the flag value, the initial CPB removal delay is signaled only for the entire bitstream / CVS, or the initial CPB removal delay is signaled for each temporal sublayer representation of the bitstream / CVS. Table 12 shows an example of a buffering period SEI message syntax structure that can be signaled according to this example. Table 12

[0087] With respect to Table 12, the semantics may be based on the semantics provided above with respect to Table 11 for the syntax element sub_layer_initial_cpb_removal_delay_present_flag. In one example, the semantics may be based on the following: sub_layer_initial_cpb_removal_delay_present_flag equal to 1 specifies that for temporal sub-layer representations in the range of 0 to sps_max_sub_layers_minus1, inclusive, the initial CPB removal delay-related syntax elements are present (i.e., nal_initial_cpb_removal_delay[i][j], nal_initial_cpb_removal_offset[i][j] are present when bp_nal_hrd_parameters_present_flag is equal to 1, and vcl_initial_cpb_removal_delay[i][j], vcl_initial_cpb_removal_offset[i][j] are present when bp_nal_hrd_parameters_present_flag is equal to 1). sub_layer_initial_cpb_removal_delay_present_flag equal to 0 specifies that the initial CPB removal delay-related syntax elements are present only for the sps_max_sub_layers_minus1-th temporal sub-layer representation (i.e., nal_initial_cpb_removal_delay[i][j], nal_initial_cpb_removal_offset[i][j] are present when bp_nal_hrd_parameters_present_flag is equal to 1, and vcl_initial_cpb_removal_delay[i][j], vcl_imtial_cpb_removal_offset[i][j] are present when bp_vcl_hrd_parameters_present_flag is equal to 1). In one example, the term "temporal sub-layer representation in the range of 0 to sps_max_sub_layers_minus1 (inclusive)" may be replaced with the term "temporal sub-layer representation in the range of 0 to HighestTid (inclusive)", or "temporal sub-layer representation in the range of TemporalId to sps_max_sub_layers_minus1 (inclusive)", or "temporal sub-layer representation in the range of TemporalId to HighestTid (inclusive)".

[0088] In addition, with respect to Table 11 and Table 12, in one example, one or more of the following modifications may be made to the syntax: cycle for(j=0;j<=sps_max_sub_layers_minus1;j++) Can be replaced by a loop for(j=0;j<=HighestTid;j++); cycle for(j=(sub_layer_initial_cpb_removal_delay_present_flag? 0: sps_max_sub_layers_minus1);j<=sps_max_mb_layers_minus1;j++) Can be replaced by a loop for(j=(sub_layer_initial_cpb_removal_delay_present_flag? 0:HighestTid); j<=HighestTid; j++); cycle for(j=0;j<=sps_max_sub_layers_minus1;j++) Can be replaced by a loop for(j=TemporalId;j<=sps_max_sub_layers_minus1;j++); cycle for(j=(sub_layer_initial_cpb_removal_delay_present_flag?0:sps_max_sub_layers_minus1); j<=sps_max_sub_layers_minus1; J++) can be replaced by a loop for(j=(sub_layer_imtial_cpb_removal_delay_present_flag?TemporalId: sps_max__sub_layers_minus1); j<=HighestTid; j++); cycle for(j=0;j<=sps_max_sub_layers_minus1;j++) Can be replaced by a loop for(j=TemporalId; j<=HighestTid; j++); and / or cycle for(j=(sub_layer_initial_cpb_removal_delay_present_flag?0:sps_max_sub_layers_minus1); j<=sps_max_sub_layers_minus1; J++) can be replaced by a loop for(j=(sub_layer_initial_cpb_removal_delay_prescnt_flag?TemporalId: HighestTid);j<=HighestTid;j++)

[0089] In one example, according to the techniques herein, the variables InitCpbRemovalDelay[SchedSelIdx] and InitCpbRemovalDelayOffset[SchedSelIdx] that provide an initial CPB removal delay and an initial CPB removal delay offset may be derived as follows: - InitCpbRemovaIDelay[SchedSelIdx] and InitCpbRemovalDelayOffset[SchedSelIdx] are set to the values ​​of the buffering period SEI message syntax elements nal_initial_cpb_removal_delay[SchedSelIdx][HighestTid] and nal_initial_cpb_removal_offset[SchedSelIdx][HighestTid], respectively, when NalHrdModeFlag is equal to 1, or to the values ​​of vcl_initial_cpb__removal_delay[SchedSelIdx][HighestTid] and vcl_initial_cpb_removal_offset[SchedSelIdx][HighestTid], respectively, when NalHrdModeFlag is equal to 0, The buffering period SEI message syntax elements are selected as follows: For each access unit in BitstreamToDecode starting from access unit 0, select the buffering period SEI message associated with the access unit and applied to TargetOp (present in BitstreamToDecode or available by external means not specified in this specification), and select the picture timing SEI message associated with the access unit and applied to TargetOp (present in BitstreamToDecode or available by external means not specified in this specification). The selected buffering period and picture timing SEI messages shall be SEI messages or provided by external means.

[0090] In one example, according to the techniques herein, the variables InitCpbRemovalDelay[SchedSelIdx] and InitCpbRemovalDelayOffset[SchedSelIdx] that provide an initial CPB removal delay and an initial CPB removal delay offset may be derived as follows: - InitCpbRemovalDelay[SchedSelIdx] and InitCpbRemovalDelayOffset[SchedSelIdx] are set to the values ​​of the buffering period SEI message syntax elements nal_initial_cpb_removal_delay[SchedSelIdx][HighestTid] and nal_initial_cpb_removal_offset[SchedSelIdx][HighestTid], respectively, when NalHrdModeFlag is equal to 1, or to the values ​​of vcl_initial_cpb_removal_delay[SchedSelIdx][HighestTid] and vcl_initial_cpb_removal_offset[SchedSelIdx][HighestTid], respectively, when NalHrdModeFlag is equal to 0, The buffering period SEI message containing the syntax elements is selected as follows: For each access unit in BitstreamToDecode starting from access unit 0, select the buffering period SEI message associated with the access unit and applied to TargetOp (present in BitstreamToDecode or available by external means not specified in this specification), and select the picture timing SEI message associated with the access unit and applied to TargetOp (present in BitstreamToDecode or available by external means not specified in this specification). The selected buffering period and picture timing SEI messages shall be SEI messages or provided by external means.

[0091] Furthermore, in one example, according to the techniques herein, the BitRate and CpbSize values ​​for each temporal sublayer representation may be calculated separately and kept track of separately, and the appropriate variable values ​​used during operation. Currently, without this change in JVET-N1001, there is a degree of ambiguity when the BitRate and CpbSize values ​​are used for different HighestTid values. It is asserted that the changes proposed below make the specification text clear.

[0092] Table 13 shows an example of a sub_layer_hrd_parameters() syntax structure that may be signaled according to this example. sub_layer_hrd_parameters(subLayerId){ Descriptors for(i=0;i<=CpbCnt;i++){ bit_rate_value_minus1[i][subLayerId] ue(v) cpb_size_value_minus1[i][subLayerId] ue(v) cbr_flag[i][subLayerId] u(1) } } Table 13

[0093] With respect to Table 13, in one example, the following semantics may be used for the sub_layer_hrd_parameters() syntax structure: Set the variable CpbCnt equal to vui_cpb_cnt_minus1[subLayerId]. bit_rate_value_minus1[i][subLayerId] (together with bit_rate_scale) specifies the maximum input bit rate for the i-th CPB when the CPB operates at access unit level, where HighestTid is equal to subLayerId. bit_rate_value_minus1[i][subLayerId] shall be between 0 and 2. 32 For any i>0, bit_rate_value_minus1[i][subLayerId] shall be greater than bit_rate_value_minus1[i-1][subLayerId]. The bit rate in bits per second is given by: BitRate[i][subLayerId]=(bit_rate_value_minus1[i][subLayerId]+1)*2 (6 +bit_rate_scale) When the bit_rate_value_minus1[i][subLayerId] syntax element is not present, the value of BitRate[i][subLayerId] is inferred to be equal to CpbBrVclFactor*MaxBR for VCL HRD parameters and to CpbBrNalFactor*MaxBR for NAL HRD parameters, where MaxBR, CpbBrVclFactor and CpbBrNalFactor are specified, for example, as provided above]. cpb_size_value_minus1[i][subLayerId] is used together with cpb_size_scale to specify the i-th CPB size when the CPB operates at access unit level, where HighestTid is equal to subLayerId. cpb_size_value_minus1[i][subLayerId] shall be between 0 and 2. 32 For any i greater than 0, cpb_size_value_minus1[i][subLayerId] shall be less than or equal to cpb_size__value_minus1[i-1][subLayerId]. The CPB size, in bits, is given by: CpbSize[i][subLayerId]=(cpb_size_value_minus1[i][subLayerId]+1)*2 (4 +cpb_size_scale) When the cpb_size_value__minus1[i][subLayerId] syntax element is not present, the value of CpbSize[i][subLayerId] is inferred to be equal to CpbBrVclFactor*MaxCPB for VCL HRD parameters and to CpbBrNalFactor*MaxCPB for NAL HRD parameters, where MaxCPB, CpbBrVclFactor, and CpbBrNalFactor are specified, for example, as provided above.] cbr_flag[i][subLayerId] equal to 0 specifies that for decoding of this bitstream by HRD using the i-th CPB specification, the Hypothetical Stream Scheduler (HSS) operates in intermittent bitrate mode with HighestTid equal to subLayerId. cbr_flag[i][subLayerId] equal to 1 specifies that the HSS operates in constant bitrate (CBR) mode with HighestTid equal to subLayerId. When not present, the value of cbr_flag[i][subLayerId] is inferred to be equal to 0. Furthermore, for the syntax elements bit_rate_scale and cpb_size_scale in the hrd_parameters() syntax structure, in one example, the semantics may be based on the following: Bit_rate_scale (together with bit_rate_value_minus1[i][subLayerId]) specifies the maximum input bit rate for the i-th CPB. Cpb_size_scale (together with cpb_size_value_minus1[i][subLayerId]) specifies the CPB size of the i-th CPB when the CPB operates at the access unit level. Furthermore, for the syntax elements nal_initial_cpb_removal_delay[i][j], nal_initial_cpb_removal_offset[i][j], and vcl_initial_cpb_removal_delay[i][j] in the buffering period SEI message syntax structure, in one example, the semantics may be based on the following: nal_initial_cpb_removal_delay[i][j] specifies the i-th initial CPB removal delay of the NAL HRB for the j-th temporal sublayer representation, in units of 90 kHz clock. The length of nal_initial_cpb_removal_delay[i][j] is initial_cpb_removal_delay_length_minus1+1 bits. The value of nal_initial_cpb_removal_delay[i][j] shall not be equal to 0 and shall be less than or equal to 90000*(CpbSize[i][j]+BitRate[i][j]), which is the time equivalent of the CPB size in 90 kHz clock units. nal_initial_cpb_removal_offset[i][j] specifies the i-th initial CPB removal offset of the NAL HRD for the j-th temporal sub-layer representation in 90kHz clock. The length of nal_initial_cpb_removal_offset[i][j] is initial_cpb_removal_delay_length_minus1+1 bits. Over the entire CVS, for each value of i and for each value of j, the sum of nal_initial_cpb_removal_delay[i][j] and nal_initial_cpb_removal_offset[i][j] shall be constant. vcl_initial_cpb_removal_delay[i][j] specifies the i-th initial CPB removal delay of the VCL HRD for the j-th temporal sub-layer representation, in units of 90kHz clock. The length of vcl_initial_cpb_removal_delay[i][j] is initial_cpb_removal_delay_length_minus1+1 bits. The value of vcl_initial_cpb_removal_delay[i][j] shall not be equal to 0 and shall be less than or equal to 90000*(CpbSize[i][j]÷BitRate[i][j]), which is the temporal equivalent of the CPB size in 90kHz clock units.

[0094] In another example, for the syntax elements nal_initial_cpb_removal_delay[i][j], nal_initial_cpb_removal_offset[i][j], and vcl_initial_cpb_removal_delay[i][j] in the buffering period SEI message syntax structure, the semantics may be based on the following: bp_cpb_cnt_minus1 plus 1 specifies the number of syntax element pairs nal_initial_cpb_removal_delay[i] and nal_initial_cpb_removal_offset[i] when bp_nal_hrd_parameters_present_flag is equal to 1, and specifies the number of syntax element pairs vcl_initial_cpb_removal_delay[i] and vcl_initial_cpb_removal_offset[i] when bp_vcl_hrd_parameters_present_flag is equal to 1. The value of bp_cpb_cnt_minus1 shall be in the range of 0 to 31, inclusive. nal_initial_cpb_removal_delay[i] specifies the i-th initial CPB removal delay of the NAL HRD in 90kHz clock units. The length of nal_initial_cpb_removal_delay[i] is initial_cpb_removal_delay_length_minus1+1 bits. The value of nal_initial_cpb_removal_delay[i] shall not be equal to 0 and shall be less than or equal to 90000*(CpbSize[i][HighestTid]÷BitRate[i][HighestTid]), which is the time equivalent of the CPB size in 90kHz clock units. nal_initial_cpb_removal_offset[i] specifies the i-th initial CPB removal offset of the NAL HRD in 90kHz clock units. The length of nal_initial_cpb_removal_offset[i] is initial_cpb_removal_delay_length_minus1+1 bits. Over the entire CVS, for each value of i, the sum of nal_initial_cpb_removal_delay[i] and nal_initial_cpb_removal_offset[i] shall be constant. vcl_initial_cpb_removal_delay[i] specifies the i-th initial CPB removal delay of the VCL HRD in 90kHz clock units. The length of vcl_initial_cpb_removal_delay[i] is initialcpb_removal_delay_length_minus1+1 bits. The value of vcl_initial_cpb_removal_delay[i] shall not be equal to 0 and shall be less than or equal to 90000*(CpbSize[i][HighestTid]÷BitRate[i][HighestTid]), which is the time equivalent of the CPB size in 90kHz clock units. vcl_initial_cpb_removal_offset[i] specifies the i-th initial CPB removal offset of the VCL HRD in 90kHz clocks. The length of vcl_initial_cpb_removal_offset[i] is vcl_initial_cpb_removal_delay_length_minus1+1 bits. The sum of vcl_initial_cpb_removal_delay[i] and vcl_initial_cpb_removal_offset[i] shall be constant for each value of i over the entire CVS.

[0095] Furthermore, in one example, according to the techniques of this document, the final arrival time of access unit m can be derived as follows: FinalArrivalTime[m]=initArrivalTime[m]+sizelnbits[m]÷BitRate[SchedSelIdx][HighestTid] where sizelnbits[m] is the size in bits of access unit m, counting the bits of VCL NAL units and padding data NAL units for a Type I conformance point or all bits of the Type II bitstream for a Type II conformance point.

[0096] Furthermore, in one example, according to the techniques herein, the values ​​of SchedSelIdx, BitRate[SchedSelIdx][HighestTid], and CpbSize[SchedSelIdx][HighestTid] may be constrained as follows: - If the access unit containing access unit m has a different selected hrd_parameters() syntax structure from the previous access unit, the HSS selects a value of SchedSelIdx SchedSelIdx 1 from the SchedSelIdx values ​​provided in the selected hrd_parameters() syntax structure of the access unit containing access unit m that produces BitRate[SchedSelIdx1][HighestTid] or CpbSize[SchedSelIdx1][HighestTid] for the access unit containing access unit m. The value of BitRate[SchedSelIdx1][HighestTid] or CpbSize[SchedSelIdx1][HighestTid] may be different from the value of BitRate[SchedSelIdx0][HighestTid] or CpbSize[SchedSelIdx0][HighestTid] for the previous access unit having a value of SchedSelIdx SchedSelIdx0. - Otherwise, HSS continues to operate with the previous values ​​of SchedSelIdx, BitRate[SchedSelIdx][HighestTid] and CpbSize[SchedSelIdx][HighestTid]. When the HSS selects an address for BitRate[SchedSelIdx][HighestTid] or CpbSize[SchedSelIdx][HighestTid] that is different from these values ​​for the previous access unit, the following applies: -The variable BitRate[SchedSelIdx][HighestTid] takes effect at the initial CPB arrival time of the current access unit. - The variable CpbSize[SchedSelIdx][HighestTid] takes effect as follows: - If the new value of CpbSize[SchedSelIdx][HighestTid] is greater than the old CPB size, this variable takes effect at the initial CPB arrival time of the current access unit. - Otherwise, the new value of CpbSize[SchedSelIdx][HighestTid] takes effect at the CPB removal time of the current access unit.

[0097] Furthermore, in one example, according to the techniques herein, when the HRD parameters and buffering period SEI messages are present when vui_cpb_cnt_minus1[HighestTid] is greater than 0, the decoder should be able to decode the bitstream delivered from the HSS operation using an “interpolated” delivery schedule specified with a peak bitrate r, a CPB size c(r), and an initial CPB removal delay (f(r) ÷ r) as follows: α={r-BitRate[SchedSelIdx-1][HighestTid])÷(BitRate[SchedSelIdx][HighestTid]-BitRate[SchedSelIdx-1][HighestTid]), c(r)=α*CpbSize[SchedSelIdx][HighestTid]+(1-α)*CpbSize[SchedSelIdx-1][HighestTid], f(r)=α*InitCpbRemovalDelay[SchedSelIdx]*BitRate[SchedSelIdx][HighestTid]+(1-α)*InitCpbRemovalDelay[SchedSelIdx-1]*BitRate[SchedSelIdx-1][HighestTid] For any SchedSelIdx>0 and r such that BitRate[SchedSelIdx-1][HighestTid]<=r<=BitRate[SchedSelIdx][HighestTid], such that for the maximum bit rate and buffer size for the specified profile, layer, and level, r and c(r) are within the limits specified, for example, with respect to Table 10.

[0098] Referring to Table 3, it should be noted that the syntax elements num_units_in_tick and time_scale are used to derive the variable ClockTick for HRD operation. Therefore, it does not make sense to signal these two syntax elements without signaling the HRD parameters (hrd_parameters). In one example, according to the techniques herein, the syntax elements num_units_in_tick and time_scale may be conditionally signaled based on hrd_parameters_present_flag, which provides simplification of syntax and improves coding efficiency. Table 14 shows an example of the syntax of a sequence parameter set that may be signaled according to the techniques herein. Table 14

[0099] With respect to Table 14, the semantics may be based on the semantics provided above with respect to Table 3 for the syntax element hrd_parameters_present_flag. In one example, the following semantics may be used: hrd_parameters_present_flag equal to 1 specifies that the syntax elements num_units_in_tick, time_scale and the syntax structure hrd_parameters() are present in the SPS RBSP syntax structure. hrd_parameters_present_flag equal to 0 specifies that the syntax elements num_units_in_tick, time_scale and the syntax structure hrd_parameters() are not present in the SPS RBSP syntax structure.

[0100] It should be noted that, in one example, the flag hrd_parameters_present_flag may be referred to as hrd_and_timing_info_parameters_present_flag or hrd_timing_parameters_present_flag or hrd_and_timing_parameters_present_flag.

[0101] Furthermore, it should be noted that for each of the syntax elements num_units_in_tick and time_scale, a value of 0 is meaningless and is not allowed. According to the techniques herein, in one example, one or both of the syntax elements num_units_in_tick and time_scale may be encoded using minus-one encoding. According to the techniques herein, instances of num_units_in_tick and / or time_scale in the table above may be replaced with syntax elements num_units_in_tick_minus1 and / or time_scale_minus1 having semantics based on the following: Num_units_in_tick_minus1 plus 1 is the number of time units of a clock operating at a frequency time_scale Hz, which corresponds to one increment of the clock tick counter (called a clock tick). The clock tick in seconds is equal to the quotient of num_units_in_tick divided by time_scale. For example, when the picture rate of the video signal is 25 Hz, time_scale may be equal to 27,000,000 and num_units_in_tick may be equal to 1,080,000, and thus the clock tick may be equal to 0.04 seconds. Time_scale_minus1 plus 1 is the number of time units that pass in one second. For example, a time coordinate system that measures time using a 27MHz clock has a time_scale of 27,000,000.

[0102] In one example, using the above technology, the syntax of the relevant parameters can be as shown in Table 15. Table 15

[0103] Thus, source device 102 represents an example of a device that is configured to: send a signal notification mark, wherein the value of the mark indicates whether to signal a corresponding instance of the HRD parameter for each temporal sublayer of the video data or to signal a single instance of the HRD parameter for all temporal sublayers of the video data; and send a plurality of signal notification instances of the HRD parameter based on the number of temporal sublayers of the video data and the value of the mark.

[0104] Reference again Figure 1 , interface 108 may include any device configured to receive data generated by data encapsulator 107 and transmit and / or store the data to a communication medium. Interface 108 may include a network interface card such as an Ethernet card, and may include an optical transceiver, a radio frequency transceiver, or any other type of device that can send and / or receive information. In addition, interface 108 may include a computer system interface that enables files to be stored on a storage device. For example, interface 108 may include support for peripheral component interconnect (PCI) and high-speed peripheral component interconnect (PCIe) bus protocols, proprietary bus protocols, universal serial bus (USB) protocols, I 2 C's chipset, or any other logical and physical structure that can be used to interconnect peer devices.

[0105] Reference again Figure 1, target device 120 includes interface 122, data decapsulator 123, video decoder 124, and display 126. Interface 122 may include any device configured to receive data from a communication medium. Interface 122 may include a network interface card such as an Ethernet card, and may include an optical transceiver, a radio frequency transceiver, or any other type of device that can receive and / or send information. In addition, interface 122 may include a computer system interface that allows a compatible video bitstream to be retrieved from a storage device. For example, interface 122 may include a computer system that supports PCI and PCIe bus protocols, proprietary bus protocols, USB protocols, I 2 The data decapsulator 123 may be configured to receive and parse any of the example grammatical structures described herein.

[0106] Video decoder 124 may include any device configured to receive a bitstream (e.g., sub-bitstream extraction) and / or acceptable variants thereof and reproduce video data therefrom. Display 126 may include any device configured to display video data. Display 126 may include one of a variety of display devices such as a liquid crystal display (LCD), a plasma display, an organic light emitting diode (OLED) display, or another type of display. Display 126 may include a high definition display or an ultra high definition display. It should be noted that although in Figure 1 In the example shown, the video decoder 124 is described as outputting data to a display 126, but the video decoder 124 can be configured to output video data to various types of devices and / or subcomponents thereof. For example, the video decoder 124 can be configured to output video data to any communication medium, as described herein.

[0107] Figure 6 600 is a block diagram illustrating an example of a video decoder that may be configured to decode video data in accordance with one or more techniques of this disclosure (e.g., a decoding process for reference picture list construction described above). In one example, the video decoder 600 may be configured to decode transform data and reconstruct residual data from transform coefficients based on the decoded transform data. The video decoder 600 may be configured to perform intra-prediction decoding and inter-prediction decoding, and may therefore be referred to as a hybrid decoder. The video decoder 600 may be configured to parse any combination of the syntax elements described above in Tables 1 to 15. The video decoder 600 may decode a picture based on or according to the process described above.

[0108] exist Figure 6In the example shown, the video decoder 600 includes an entropy decoding unit 602, an inverse quantization unit and a transform coefficient processing unit 604, an intra-frame prediction processing unit 606, an inter-frame prediction processing unit 608, an adder 610, a post-filter unit 612, and a reference buffer 614. The video decoder 600 can be configured to decode video data in a manner consistent with a video encoding system. It should be noted that although the exemplary video decoder 600 shown has different functional blocks, such illustrations are intended for descriptive purposes and do not limit the video decoder 600 and / or its subcomponents to a specific hardware or software architecture. The functionality of the video decoder 600 can be implemented using any combination of hardware, firmware, and / or software implementations.

[0109] like Figure 6 As shown, the entropy decoding unit 602 receives an entropy-encoded bitstream. The entropy decoding unit 602 may be configured to decode syntax elements and quantized coefficients from the bitstream according to a process that is inverse to the entropy encoding process. The entropy decoding unit 602 may be configured to perform entropy decoding according to any of the entropy encoding techniques described above. The entropy decoding unit 602 may determine the values ​​of syntax elements in the encoded bitstream in a manner consistent with the video coding standard. Figure 6 As shown, the entropy decoding unit 602 can determine the quantization parameter, the quantization coefficient value, the transformation data and the prediction data from the bit stream. Figure 6 As shown, the inverse quantization unit and transform coefficient processing unit 604 receives the quantization parameters, quantization coefficient values, transform data and prediction data from the entropy decoding unit 602, and outputs reconstructed residual data.

[0110] Reference again Figure 6, the reconstructed residual data may be provided to the adder 610. The adder 610 may add the reconstructed residual data to the predicted video block and generate the reconstructed video data. The predicted video block may be determined according to a predictive video technique, namely intra prediction and inter prediction. The intra prediction processing unit 606 may be configured to receive an intra prediction syntax element and retrieve the predicted video block from a reference buffer 614. The reference buffer 614 may include a memory device configured to store one or more frames of video data. The intra prediction syntax element may identify an intra prediction mode, such as the intra prediction mode described above. The inter prediction processing unit 608 may receive the inter prediction syntax element and generate a motion vector to identify a predicted block in one or more reference frames stored in the reference buffer 616. The inter prediction processing unit 608 may generate a motion compensation block, possibly performing interpolation based on an interpolation filter. An identifier of an interpolation filter for motion estimation with sub-pixel precision may be included in the syntax element. The inter prediction processing unit 608 may use an interpolation filter to calculate interpolated values ​​of sub-integer pixels of a reference block. The post filter unit 614 may be configured to perform filtering on the reconstructed video data. For example, the post filter unit 614 may be configured to perform deblocking and / or sample adaptive offset (SAO) filtering, for example based on parameters specified in the bitstream. Furthermore, it should be noted that in some examples, the post filter unit 614 may be configured to perform dedicated arbitrary filtering (e.g., visual enhancement, such as mosquito noise removal). Figure 6 As shown, the video decoder 600 can output the reconstructed video block. Thus, the video decoder 600 represents an example of a device that is configured to: parse a flag indicating whether a corresponding instance of the HRD parameter is signaled for each temporal sub-layer of the video data or a single instance of the HRD parameter is signaled for all temporal sub-layers of the video data; and parse multiple instances of the HRD parameter according to the number of temporal sub-layers of the video data and the value of the flag.

[0111] In one or more examples, the functions may be implemented by hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored as one or more instructions or codes on or transmitted via a computer-readable medium and executed by a hardware-based processing unit. Computer-readable media may include computer-readable storage media corresponding to tangible media such as data storage media, or propagation media including any medium that facilitates the transmission of a computer program from one place to another, for example, according to a communication protocol. Thus, computer-readable media may generally correspond to: (1) a non-transitory tangible computer-readable storage medium, or (2) a communication medium such as a signal or carrier wave. The data storage medium may be any available medium that can be accessed by one or more computers or one or more processors to retrieve instructions, codes, and / or data structures for implementing the techniques described in the present disclosure. A computer program product may include a computer-readable medium.

[0112] By way of example and not limitation, such computer-readable storage media may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage devices, disk storage devices or other magnetic storage devices, flash memory, or any other medium that can be used to store the required program code in the form of instructions or data structures and can be accessed by a computer. Moreover, any connection is appropriately referred to as a computer-readable medium. For example, if a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL) or wireless technology such as infrared, radio and microwave are used to transmit instructions from a website, server or other remote source, then coaxial cable, fiber optic cable, twisted pair, DSL or wireless technology such as infrared, radio and microwave are included in the definition of medium. However, it should be understood that computer-readable storage media and data storage media do not include connections, carriers, signals or other transient media, but are directed to non-transient tangible storage media. As used herein, disks and optical disks include compact disks (CDs), laser disks, optical optical disks, digital versatile disks (DVDs), floppy disks and Blu-ray disks, wherein disks usually copy data magnetically, while optical disks use lasers to copy data optically. The above combination should also be included in the scope of computer-readable media.

[0113] Instructions may be executed by one or more processors such as one or more digital signal processors (DSPs), general-purpose microprocessors, application-specific integrated circuits (ASICs), field programmable logic arrays (FPGAs), or other equivalent integrated or discrete logic circuits. Therefore, the term "processor" as used herein may refer to any of the aforementioned structures or any other structure suitable for implementing the techniques described herein. In addition, in some aspects, the functions described herein may be provided within dedicated hardware and / or software modules configured for encoding and decoding, or incorporated into a combined codec. Moreover, these techniques may be implemented entirely in one or more circuits or logic elements.

[0114] The techniques of the present disclosure may be implemented in a variety of devices or apparatuses including wireless handsets, integrated circuits (ICs) or IC sets (e.g., chipsets). Various components, modules, or units are described in the present disclosure to emphasize functional aspects of devices configured to perform the disclosed techniques, but do not necessarily need to be implemented by different hardware units. Instead, as described above, the various units may be combined in a codec hardware unit, or provided by a collection of interoperable hardware units including one or more processors as described above, in combination with appropriate software and / or firmware.

[0115] In addition, each functional block or various features of the base station equipment and terminal equipment used in each of the above-mentioned embodiments can be realized or executed by a circuit (usually an integrated circuit or multiple integrated circuits). The circuit designed to perform the functions described in this specification may include a general-purpose processor, a digital signal processor (DSP), a dedicated or general-purpose integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices or discrete hardware components or a combination thereof. The general-purpose processor may be a microprocessor, or alternatively, the processor may be a conventional processor, a controller, a microcontroller or a state machine. The general-purpose processor or each of the above-mentioned circuits may be configured by a digital circuit, or may be configured by an analog circuit. In addition, when the technology of making an integrated circuit that replaces the current integrated circuit appears due to the progress of semiconductor technology, the integrated circuit produced by the technology can also be used.

[0116] Various examples have been described. These and other examples are within the scope of the following claims. <Disclosure of the Invention>

[0117] In one example, a method for signaling hypothetical reference decoder (HRD) parameters for decoding video data includes: signaling a flag, wherein a value of the flag indicates whether to signal a corresponding instance of the HRD parameter for each temporal sublayer of the video data or to signal a single instance of the HRD parameter for all temporal sublayers of the video data; and signaling multiple instances of the HRD parameter based on the number of temporal sublayers of the video data and the value of the flag.

[0118] In one example, a method for decoding video data includes: parsing a tag that indicates whether a corresponding instance of an HRD parameter is signaled for each temporal sublayer of the video data or a single instance of the HRD parameter is signaled for all temporal sublayers of the video data; and parsing multiple instances of the HRD parameter based on the number of temporal sublayers of the video data and the value of the tag.

[0119] In one example, a method is provided wherein a tag is included in a sequence parameter set.

[0120] In one example, a device includes one or more processors configured to perform any and all combinations of steps.

[0121] In one example, a device is provided, wherein the device includes a video encoder.

[0122] In one example, a device is provided, wherein the device includes a video decoder.

[0123] In one example, a system includes: a device including a video encoder; and the device including a video decoder.

[0124] In one example, an apparatus includes means for performing any and all combinations of the steps.

[0125] In one example, a non-transitory computer-readable storage medium includes instructions stored thereon that, when executed, cause one or more processors of a device to perform any and all combinations of steps.

[0126] In one example, a method for decoding video data, the method comprising: decoding a first syntax element in a sequence parameter set, wherein the first syntax element specifies whether a hypothetical reference decoder parameter syntax structure includes parameters of temporal sublayers whose identifiers are in the range of 0 to a maximum number of temporal sublayers minus one or whether a hypothetical reference decoder parameter syntax structure includes parameters of temporal sublayers whose identifiers are only equal to a maximum number of temporal sublayers minus one; defining a starting sublayer of the hypothetical reference decoder parameter syntax structure according to a value of the first syntax element; and decoding each of the parameters in the hypothetical reference decoder parameter syntax structure by using the starting sublayer and the maximum number of sublayers.

[0127] In one example, a method is provided wherein the first syntax element is a sub-layer coded picture buffer parameter present flag.

[0128] In one example, a method is provided wherein a hypothetical reference decoder parameter syntax structure comprises a sub-layer hypothetical reference decoder parameter syntax structure.

[0129] In one example, a method is provided wherein a sub-layer HRD parameter syntax structure includes a second syntax element that specifies a maximum input bit rate for a coded picture buffer, wherein a highest temporal identifier is equal to a corresponding identifier when the coded picture buffer operates at an access unit level.

[0130] In one example, a method is provided wherein the second syntax element is a bit rate value minus one syntax element.

[0131] In one example, a method is provided wherein a sub-layer HRD parameter syntax structure includes a third syntax element specifying a coded picture buffer size, wherein a highest temporal identifier is equal to a corresponding identifier when the coded picture buffer operates at an access unit level.

[0132] In one example, a method is provided wherein the third syntax element is a coded picture buffer size value minus one syntax element.

[0133] In one example, a method for encoding image data, the method comprising: encoding a first syntax element in a sequence parameter set, wherein the first syntax element specifies whether a hypothetical reference decoder parameter syntax structure includes parameters of temporal sublayers whose identifiers are in the range of 0 to a maximum number of temporal sublayers minus one or whether a hypothetical reference decoder parameter syntax structure includes parameters of temporal sublayers whose identifiers are only equal to a maximum number of temporal sublayers minus one; defining a starting sublayer of the hypothetical reference decoder parameter syntax structure according to a value of the first syntax element; and encoding each of these parameters in the hypothetical reference decoder parameter syntax structure by using the starting sublayer and the maximum number of sublayers.

[0134] In one example, a decoder for decoding video data comprises: a processor, and a memory associated with the processor; wherein the processor is configured to perform the following steps: decode a first syntax element in a sequence parameter set, wherein the first syntax element specifies whether a hypothetical reference decoder parameter syntax structure includes parameters of temporal sublayers whose identifiers are in the range of 0 to a maximum number of temporal sublayers minus one or whether a hypothetical reference decoder parameter syntax structure includes parameters of temporal sublayers whose identifiers are only equal to the maximum number of temporal sublayers minus one; define a starting sublayer of the hypothetical reference decoder parameter syntax structure according to the value of the first syntax element; and decode each of these parameters in the hypothetical reference decoder parameter syntax structure by using the starting sublayer and the maximum number of sublayers.

[0135] In one example, an encoder for encoding video data comprises: a processor, and a memory associated with the processor; wherein the processor is configured to perform the following steps: encode a first syntax element in a sequence parameter set, wherein the first syntax element specifies whether a hypothetical reference decoder parameter syntax structure includes parameters of temporal sublayers whose identifiers are in the range of 0 to a maximum number of temporal sublayers minus one or whether a hypothetical reference decoder parameter syntax structure includes parameters of temporal sublayers whose identifiers are only equal to a maximum number of temporal sublayers minus one; define a starting sublayer of the hypothetical reference decoder parameter syntax structure according to the value of the first syntax element; and encode each of these parameters in the hypothetical reference decoder parameter syntax structure by using the starting sublayer and the maximum number of sublayers. <Cross Reference>

[0136] This non-provisional patent application claims priority under 35 U.S.C. §119 to provisional application 62 / 854,921 filed on May 30, 2019, provisional application 62 / 864,398 filed on June 20, 2019, and provisional application 62 / 865,733 filed on June 24, 2019, the entire contents of which are hereby incorporated by reference.

Claims

1. A decoder for decoding coded data, the decoder comprising: processor; and a memory associated with the processor; wherein the processor is configured to decoding a first syntax element in a sequence parameter set, wherein the first syntax element specifies whether the sequence parameter set includes a second syntax element and a third syntax element; decoding the second syntax element based on the value of the first syntax element, wherein the second syntax element is a number of time units of a clock operating at a frequency, the frequency corresponding to a clock tick; and The third syntax element is decoded based on the value of the first syntax element, wherein the third syntax element is the number of time units that have passed in one second.

2. An encoder for encoding image data, the encoder comprising: processor; and a memory associated with the processor; wherein the processor is configured to encoding a first syntax element in a sequence parameter set, wherein the first syntax element specifies whether the sequence parameter set includes a second syntax element and a third syntax element; encoding the second syntax element based on the value of the first syntax element, wherein the second syntax element is a number of time units of a clock operating at a frequency, the frequency corresponding to a clock tick; and The third syntax element is encoded based on the value of the first syntax element, wherein the third syntax element is the number of time units that have passed in one second.

3. A non-transitory computer-readable storage medium storing a bit stream generated by encoding video data, the bit stream being decoded by: decoding a first syntax element in a sequence parameter set from the bitstream, wherein: The first syntax element specifies whether the sequence parameter set includes a second syntax element and a third syntax element; decoding the second syntax element based on the value of the first syntax element, wherein the second syntax element is a number of time units of a clock operating at a frequency, the frequency corresponding to a clock tick; and The third syntax element is decoded based on the value of the first syntax element, wherein the third syntax element is the number of time units that have passed in one second.

Images