Signaling notifies sub-picture level and buffer information

Abstract

Examples of a video encoding method and apparatus and a video decoding method and apparatus are described. An example method of video processing includes performing a conversion between a video and a bitstream of the video, wherein according to a rule the bitstream includes one or more output layer sets, the output layer set includes one or more video layers, and wherein the rule specifies a particular decoding order between an SLI SEI message and a BP SEI message applicable to a particular output layer set in response to a condition that an SLI SEI message and a BP SEI message are included in an access unit supplemented with sub-picture level information (SLI).

Description

[0001] Cross - Reference to Related Applications

[0002] This application is a continuation of International Patent Application No. PCT / US2021 / 036491, filed on June 8, 2021, and claims the priority and benefit of U.S. Provisional Patent Application No. 63 / 036,808, filed on June 9, 2020. The entire disclosure of the foregoing patent applications is incorporated herein by reference. Technical Field

[0003] This patent document relates to image and video encoding and decoding. Background Art

[0004] In the Internet and other digital communication networks, digital video occupies the largest bandwidth usage. With the increase in the number of connected user devices capable of receiving and displaying video, the bandwidth demand for digital video is expected to continue to grow. Summary of the Invention

[0005] This document discloses techniques that video encoders and decoders can use to perform video encoding or decoding.

[0006] In one example aspect, a video processing method is disclosed. The method includes performing a conversion between a video and a bitstream of the video, wherein according to format rules, the bitstream includes one or more output layer sets, the output layer set includes one or more video layers, and the format rules specify that a non - scalable nested supplementary enhancement information (SEI) message including information about a Hypothetical Reference Decoder (HRD) is applicable to all output layer sets including the same video layers as the bitstream.

[0007] In another example aspect, another video processing method is disclosed. The method includes performing a conversion between a video and a bitstream of the video, wherein according to format rules, the bitstream includes one or more output layer sets, the output layer set includes one or more video layers, and the format rules specify omitting a non - scalable nested Hypothetical Reference Decoder (HRD) - related Supplementary Enhancement Information (SEI) message in response to a condition that there is no output layer set including the same layer set as the bitstream.

[0008] In another example aspect, another video processing method is disclosed. The method includes performing a conversion between a video and a bitstream of the video, wherein according to format rules, the bitstream includes one or more output layer sets, the output layer set includes one or more video layers, and the format rules specify that the value of the layer identifier for a Supplementary Enhancement Information (SEI) Network Abstraction Layer (NAL) unit including a non - scalable nested SEI message is not constrained.

[0009] In another exemplary aspect, another video processing method is disclosed. The method includes performing a conversion between a video and a bitstream of the video, wherein according to format rules, the bitstream includes one or more output layer sets, the output layer set includes one or more video layers, and the format rules specify that a list of allowed supplementary enhancement information (SEI) payload type values including SEI messages related to a non-hypothetical reference decoder (HRD) does not allow a specific payload type value corresponding to sub-picture level information.

[0010] In another exemplary aspect, another video processing method is disclosed. The method includes performing a conversion between a video and a bitstream of the video, wherein according to format rules, the bitstream includes one or more output layer sets, the output layer set includes one or more video layers, and the format rules specify that unscalable nested supplementary enhancement information SEI messages including information unrelated to a hypothetical reference decoder (HRD) apply to all layers in the bitstream.

[0011] In another exemplary aspect, another video processing method is disclosed. The method includes performing a conversion between a video and a bitstream of the video including one or more output layer sets according to rules, wherein the rules specify that since the same picture timing is used in all output layer sets in the bitstream, SEI network abstraction layer (NAL) units containing scalable nested supplementary enhancement information (SEI) messages carrying picture timing information are not included.

[0012] In another exemplary aspect, another video processing method is disclosed. The method includes performing a conversion between a video and a bitstream of the video, wherein according to rules, the bitstream includes one or more supplementary enhancement information SEI network abstraction layer (NAL) units, and the rules specify that in response to the SEI NAL unit including an unscalable nested SEI message of a first payload type, the SEI NAL unit is not allowed to include another SEI message of a second payload type.

[0013] In another exemplary aspect, another video processing method is disclosed. The method includes performing a conversion between a video and a bitstream of the video according to rules, wherein according to rules, the bitstream includes one or more supplementary enhancement information SEI network abstraction layer (NAL) units, and the rules specify that in response to the SEI NAL unit including a scalable nested SEI message of a first payload type, the SEI NAL unit is not allowed to include another SEI message of a second payload type.

[0014] In another example aspect, another video processing method is disclosed. The method includes performing a conversion between a video and a bitstream of the video, wherein according to a rule the bitstream includes one or more Supplemental Enhancement Information (SEI) Network Abstraction Layer (NAL) units, and the rule specifies that in response to the SEI NAL unit including an SEI message of a first payload type, the SEI NAL unit is not allowed to include another SEI message that is not equal to the first payload type or a second payload type.

[0015] In another example aspect, another video processing method is disclosed. The method includes performing a conversion between a video and a bitstream of the video, wherein according to a rule the bitstream includes one or more output layer sets, and the output layer set includes one or more video layers, and in response to a condition that Sub-Picture Level Information (SLI) Supplemental Enhancement Information (SEI) messages and Buffering Period (BP) SEI messages are included in an access unit, the rule specifies a specific decoding order between the SLI SEI messages and the BP SEI messages applicable to a specific output layer set.

[0016] In another example aspect, another video processing method is disclosed. The method includes performing a conversion between a video and a bitstream of the video according to a rule, wherein in response to a specific value of a second syntax field indicating a maximum number of time sub-layers of an initial Coding Picture Buffer (CPB) removal delay indicated in a Buffering Period Supplemental Enhancement Information (SEI) message, the rule specifies omitting a first syntax field of sub-layer representation information indicating the existence of a syntax element related to the initial CPB removal delay.

[0017] In yet another example aspect, a video encoder device is disclosed. The video encoder includes a processor configured to implement the above method.

[0018] In yet another example aspect, a video decoder device is disclosed. The video decoder includes a processor configured to implement the above method.

[0019] In yet another example aspect, a computer-readable medium storing code is disclosed. The code embodies one of the methods described herein in a form of processor-executable code.

[0020] These and other features are described throughout this document. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 is a block diagram showing a video codec system according to some embodiments of the disclosed technology.

[0022] Figure 2 is a block diagram of an example hardware platform for video processing.

[0023] Figure 3It is a flowchart of an example method for video processing.

[0024] Figure 4 It is a block diagram showing an example video codec system.

[0025] Figure 5 It is a block diagram showing an encoder according to some embodiments of the disclosed technology.

[0026] Figure 6 It is a block diagram showing a decoder according to some embodiments of the disclosed technology.

[0027] Figures 7A to 7E It is a flowchart of an example method for video processing based on some embodiments of the disclosed technology.

[0028] Figure 8 It is a flowchart of an example method for video processing based on some embodiments of the disclosed technology.

[0029] Figures 9A to 9C It is a flowchart of an example method for video processing based on some embodiments of the disclosed technology.

[0030] Figure 10A and Figure 10B It is a flowchart of an example method for video processing based on some embodiments of the disclosed technology. Detailed Description

[0031] The use of section headings in this document is for ease of understanding and does not limit the applicability of the technologies and embodiments disclosed in each section to that section. Additionally, the use of H.266 terms in some descriptions is for ease of understanding only and does not limit the scope of the disclosed technology. Thus, the technologies described herein are also applicable to other video codec protocols and designs.

[0032] 1. Introduction

[0033] This document relates to video codec technology. Specifically, it is about defining the levels and bitstream conformances of video codecs that support single-layer video coding and decoding and multi-layer video coding and decoding. It can be applied to any video codec standard or non-standard video codec that supports single-layer video coding and decoding and multi-layer video coding and decoding (such as the General Video Coding (VVC) under development).

[0034] 2. Abbreviations

[0035] APS Adaptive Parameter Set

[0036] AU Access Unit

[0037] AUD Access Unit Delimiter

[0038] AVC Advanced Video Coding

[0039] BP Buffering Period

[0040] CLVS Coding Layer Video Sequence

[0041] CPB Coding Picture Buffer

[0042] CRA Complete Random Access

[0043] CTU Coding Tree Unit

[0044] CVS Coding Video Sequence

[0045] DPB Decoded Picture Buffer

[0046] DPS Decoding Parameter Set

[0047] DUI Decoding Unit Information

[0048] EOB End of Bitstream

[0049] EOS End of Sequence

[0050] GCI General Constraint Information

[0051] GDR Gradual Decoding Refresh

[0052] HEVC High Efficiency Video Coding

[0053] HDR Hypothetical Reference Decoder

[0054] IDR Instantaneous Decoding Refresh

[0055] JEM Joint Exploration Model

[0056] MCTS Motion Constraint Tile Set

[0057] NAL Network Abstraction Layer

[0058] OLS Output Layer Set

[0059] PH Picture Header

[0060] PPS Picture Parameter Set

[0061] PT Picture Timing

[0062] PTL Profile, Tier and Level

[0063] PU Picture Unit

[0064] RRP Reference Picture Resampling

[0065] RBSP Raw Byte Sequence Payload

[0066] SEI supplementary enhancement information

[0067] SH slice header

[0068] SLI sub - picture level information

[0069] SPS sequence parameter set

[0070] SVC scalable video coding

[0071] VCL video coding layer

[0072] VPS video parameter set

[0073] VTM VVC test model

[0074] VUI video usability information

[0075] VVC versatile video coding

[0076] 3. Preliminary discussion

[0077] Video coding standards have mainly evolved through the development of well - known ITU - T and ISO / IEC standards. ITU - T produced H.261 and H.263, ISO / IEC produced MPEG - 1 and MPEG - 4 Visual, and the two organizations jointly produced H.262 / MPEG - 2 video, H.264 / MPEG - 4 Advanced Video Coding (AVC), and H.265 / HEVC standards. Since H.262, video coding standards have been based on a hybrid video coding structure, where temporal prediction plus transform coding is used. To explore future video coding technologies beyond HEVC, VCEG and MPEG jointly established the Joint Video Exploration Team (JVET) in 2015. Since then, JVET has adopted many new methods and applied them to a reference software called the Joint Exploration Model (JEM). JVET meetings are held quarterly simultaneously. The goal of the new coding standard is to reduce the bitrate by 50% compared to HEVC. The new video coding standard was officially named Versatile Video Coding (VVC) at the JVET meeting in April 2018, when the first version of the VVC Test Model (VTM) was released. With continuous efforts towards VVC standardization, new coding technologies have been adopted into the VVC standard at each JVET meeting. Then the VVC working draft and the test model VTM are updated after each meeting. The VVC project now aims for Feature Complete (FDIS) at the meeting in July 2020.

[0078] 3.1. Parameter sets

[0079] AVC, HEVC, and VVC specified parameter sets. The types of parameter sets include SPS, PPS, APS, and VPS. All AVC, HEVC, and VVC support SPS and PPS. VPS was introduced starting from HEVC and is included in HEVC and VVC. APS is not included in AVC or HEVC but is included in the latest VVC draft text.

[0080] SPS is designed to carry sequence-level header information, while PPS is designed to carry picture-level header information that does not change frequently. Using SPS and PPS, it is not necessary to repeat the information that does not change frequently for each sequence or picture, thus avoiding redundant signaling of this information. In addition, the use of SPS and PPS enables out-of-band transmission of important header information, thus not only avoiding the need for redundant transmission but also improving error recovery capabilities.

[0081] VPS was introduced to carry sequence-level header information common to all layers in a multi-layer bitstream.

[0082] APS was introduced to carry such picture-level or slice-level information that requires a significant number of bits to encode and decode, can be shared by multiple pictures, and can have a significant number of different variations in a sequence.

[0083] 3.2. Picture Resolution Change within a Sequence

[0084] In AVC and HEVC, the spatial resolution of a picture cannot change unless a new sequence with a new SPS starts with an IRAP picture. VVC can change the picture resolution at a certain position within a sequence without encoding an IRAP picture, and IRAP images are always intra-coded and decoded. This feature is sometimes referred to as Reference Picture Resampling (RPR) because when the reference picture used for inter prediction has a different resolution from the current picture being decoded, this feature requires resampling of the reference picture.

[0085] The scaling ratio is limited to be greater than or equal to 1 / 2 (2x downsampling from the reference picture to the current picture) and less than or equal to 8 (8x upsampling). Three resampling filter sets with different cut-off frequencies are specified to handle various scaling ratios between the reference picture and the current picture. The three resampling filter sets are applied to scaling ratios of 1 / 2 to 1 / 1.75, 1 / 1.75 to 1 / 1.25, and 1 / 1.25 to 8 respectively. Each resampling filter set has 16 phases for luminance and 32 phases for chrominance, the same as in the case of motion compensation interpolation filters. The normal MC interpolation process is actually a special case of the resampling process with a scaling ratio from 1 / 1.25 to 8. The horizontal and vertical scaling ratios are derived from the picture width and height and the left, right, top, and bottom scaling offsets specified for the reference picture and the current picture.

[0086] Other aspects of the VVC design that support this feature and are different from HEVC include: i) Signaling the picture resolution and the corresponding consistency window in the PPS instead of the SPS, while signaling the maximum picture resolution in the SPS. ii) For a single-layer bitstream, each picture memory (the time slot in the DPB for storing a decoded picture) occupies the buffer size required to store a decoded picture with the maximum picture resolution.

[0087] 3.3. General and Scalable Video Coding (SVC) in VVC

[0088] Scalable Video Coding (SVC, sometimes also simply referred to as scalability in video coding) refers to video coding in which a base layer (BL) (sometimes called a reference layer (RL)) and one or more scalable enhancement layers (ELs) are used. In SVC, the base layer can carry video data with a base quality level. One or more enhancement layers can carry additional video data to support, for example, higher spatial, temporal, and / or signal-to-noise ratio (SNR) levels. The enhancement layers can be defined relative to previously encoded layers. For example, the bottom layer can be used as the BL, and the top layer can be used as the EL. Intermediate layers can be used as ELs or RLs, or both. For example, an intermediate layer (e.g., a layer that is neither the lowest nor the highest layer) can be an EL of the layer below it, such as the base layer or any intermediate enhancement layer, and at the same time be used as an RL for one or more enhancement layers above it. Similarly, in the multi-view or 3D extension of the HEVC standard, there can be multiple views, and the information of one view can be used to code (e.g., encode or decode) the information of another view (e.g., motion estimation, motion vector prediction, and / or other redundancies).

[0089] In SVC, the parameters used by the encoder or decoder are grouped into parameter sets based on the coding levels they may use (e.g., video level, sequence level, picture level, slice level, etc.). For example, the parameters that can be used by one or more coded video sequences in different layers in the bitstream can be included in the Video Parameter Set (VPS), and the parameters used by one or more pictures in a coded video sequence can be included in the Sequence Parameter Set (SPS). Similarly, the parameters used by one or more slices in a picture can be included in the Picture Parameter Set (PPS), and other parameters specific to a single slice can be included in the slice header. Similarly, an indication of which (which) parameter set a given layer uses at a given time can be provided at various coding levels.

[0090] Since VVC supports Reference Picture Resampling (RPR), support for bitstreams containing multiple layers (e.g., two layers with SD and HD resolutions in VVC) can be designed without any additional signal processing stage codec tools. For example, the upsampling required to support spatial scalability only needs to use the RPR upsampling filter. However, to support scalability, advanced syntax changes are required (compared to not supporting scalability). Scalability support is specified in VVC version 1. Different from the scalability support in any early video codec standards, including the extensions of AVC and HEVC, the design of VVC scalability has been made as friendly as possible to the single-layer decoder design. The decoding capabilities of the multi-layer bitstream are specified in the same way as if there were only a single layer in the bitstream. For example, decoding capabilities such as the DPB size are specified in a way that is independent of the number of layers in the bitstream to be decoded. Basically, a decoder designed for a single-layer bitstream does not need much modification to be able to decode a multi-layer bitstream. Compared with the design of the multi-layer extensions of AVC and HEVC, the HLS aspect has been significantly simplified at the expense of some flexibility. For example, an IRAP AU needs to contain pictures of each layer present in the CVS.

[0091] 3.4. SEI Messages and General SEI Semantics and Constraints

[0092] Appendix D of VVC specifies the syntax and semantics of the SEI message payloads of certain SEI messages, and specifies the use of the SEI messages and VUI parameters whose syntax and semantics are specified in ITU-T H.SEI-ISO / IEC 23002-7.

[0093] SEI messages assist in processes related to decoding, display, or other purposes. However, SEI messages are not required to construct luminance or chrominance samples through the decoding process. A conforming decoder does not need to process this information to achieve output order consistency. Some SEI messages are required to check bitstream consistency and output timing decoder consistency. Other SEI messages are not required to check bitstream consistency.

[0094] In the latest VVC draft text, general SEI is required.

[0095] The syntax and semantics of the scalable nested SEI messages in the latest VVC draft text are as follows.

[0096] D.2.1 General SEI Message Syntax

[0097]

[0098]

[0099]

[0100] D.2.2 General SEI Payload Semantics

[0101] The reserved_payload_extension_data shall not appear in bitstreams compliant with this version of the specification. However, decoders compliant with this version of the specification shall ignore the presence and value of reserved_payload_extension_data. When present, the length of reserved_payload_extension_data, in bits, is equal to 8 * payloadSize - nEarlierBits - nPayloadZeroBits - 1, where nEarlierBits is the number of bits in the sei_payload() syntax structure that precede the reserved_payload_extension_data syntax element, and nPayloadZeroBits is the number of payload_bit_equal_to_zero syntax elements at the end of the sei_payload() syntax structure.

[0102] payload_bit_equal_to_one shall be equal to 1.

[0103] payload_bit_equal_to_zero shall be equal to 0.

[0104] Note 1—SEI messages with the same payloadType value are conceptually the same SEI message, regardless of whether they are contained in a prefix or suffix SEI NAL unit.

[0105] Note 2—For SEI messages with payloadType values in the range 0 to 47 (inclusive) as specified in this specification, the payloadType values are consistent with similar SEI messages specified in Rec. ITU-T H.264|ISO / IEC 14496-10.

[0106] The semantics and persistence scope of each SEI message are specified in the semantic specification of each particular SEI message.

[0107] Note 3—The persistence information for SEI messages is summarized informatively in Table D.1.

[0108] Table D.1 - Persistence Scope of SEI Messages (Informative)

[0109]

[0110] The list VclAssociatedSeiList is set to consist of payloadType values 3, 19, 45, 129, 132, 137, 144, 145, 147 to 150 (inclusive), 153 to 156 (inclusive), 168, 203, and 204.

[0111] The list PicUnitRepConSeiList is set to consist of payloadType values 0, 1, 19, 45, 129, 132, 133, 137, 147 to 150 (inclusive), 153 to 156 (inclusive), 168, 203, and 204.

[0112] Note 4 - The VclAssociatedSeiList consists of payloadType values of SEI messages that, when non - scalable nested and contained in an SEI NAL unit, infer constraints on the NAL unit header of the SEI NAL unit based on the NAL unit header of the associated VCL NAL unit. The PicUnitRepConSeiList consists of payloadType values of SEI messages that are restricted to 4 repetitions per PU.

[0113] The requirements for bit - stream conformance are that the following restrictions apply when an SEI message is contained in an SEI NAL unit:

[0114] — When an SEI NAL unit contains a non - scalable nested BP SEI message, a non - scalable nested PT SEI message, or a non - scalable nested DUI SEI message, the SEI NAL unit shall not contain any other SEI message with a payloadType not equal to 0 (BP), 1 (PT), or 130 (DUI).

[0115] — When an SEI NAL unit contains a scalable nested BP SEI message, a scalable nested PT SEI message, or a scalable nested DUI SEI message, the SEI NAL unit shall not contain any other SEI message with a payloadType not equal to 0 (BP), 1 (PT), 130 (DUI), or 133 (scalable nested).

[0116] The following applies to the applicable OLS or non - scalable nested SEI message layer:

[0117] — For a non - scalable nested SEI message, when the payloadType is equal to 0 (BP), 1 (PT), or 130 (DUI), the non - scalable nested SEI message only applies to the 0th OLS.

[0118] — For non-scalable nested SEI messages, when the payloadType is equal to any value in VclAssociatedSeiList, the non-scalable nested SEI message only applies to the layer where the nuh_layer_id of the VCL NAL unit is equal to the nuh_layer_id of the SEI NAL unit containing the SEI message.

[0119] The requirement for bitstream consistency is that the following restrictions apply to the value of nuh_layer_id of the SEI NAL unit:

[0120] — When the payloadType of the non-scalable nested SEI message is equal to 0 (BP), 1 (PT), or 130 (DUI), the nuh_layer_id of the SEI NAL unit containing the non-scalable nested SEI message shall be equal to vps_layer_id[0].

[0121] — When the payloadType of the non-scalable nested SEI message is equal to any value in VclAssociatedSeiList, the nuh_layer_id of the SEI NAL unit containing the non-scalable nested SEI message shall be equal to the value of the nuh_layer_id of the VCL NAL unit associated with the SEI NAL unit.

[0122] — The nuh_layer_id of the SEI NAL unit containing the scalable nested SEI message shall be equal to the lowest value of the nuh_layer_id of all layers to which the scalable nested SEI message is applied (when the sn_ols_flag of the scalable nested SEI message is equal to 0) or the lowest value of the nuh_layer_id of all layers in the OLS to which the scalable nested SEI message is applied (when the sn_ols_flag of the scalable nested SEI message is equal to 1).

[0123] The consistency requirement of the bitstream is that the following restrictions apply to the repetition of SEI messages:

[0124] — For each payloadType value included in PicUnitRepConSeiList, there should be less than or equal to 4 identical sei_payload() syntax structures in a PU.

[0125] — In a DU, the number of identical sei_payload() syntax structures with payloadType equal to 130 should be less than or equal to 4.

[0126] The following applies to the order of BP, PT, and DUI SEI messages:

[0127] — When BP SEI messages and PT SEI messages applicable to a specific OP are present within an AU, the BP SEI messages shall precede the PT SEI messages in decoding order.

[0128] — When BP SEI messages and DUI SEI messages applicable to a specific OP are present within an AU, the BP SEI messages shall precede the DUI SEI messages in decoding order.

[0129] — When PT SEI messages and DUI SEI messages applicable to a specific OP are present within an AU, the PT SEI messages shall precede the DUI SEI messages in decoding order.

[0130] 4. Technical problems solved by the disclosed technical solution

[0131] The existing general SEI payload semantics (including general SEI constraints) have the following problems:

[0132] 1) It should be specified that non-scalable nested HRD-related SEI messages are applicable to the OLS that includes the same set of layers as the entire bitstream (instead of only applicable to the 0th OLS).

[0133] 2) The entire bitstream can include multiple layers, and there is no specified OLS that includes all layers. In this case, there cannot be any non-scalable nested HRD-related SEI messages because they would be applicable to the OLS that includes the same set of layers as the entire bitstream.

[0134] 3) Currently specified, non-scalable nested non-HRD-related SEI messages are only applicable to the layer where the nuh_layer_id is equal to the nuh_layer_id of the SEI NAL unit. However, to be consistent with non-scalable nested HRD-related SEI messages, it should be specified that non-scalable nested non-HRD-related SEI messages are applicable to all layers in the entire bitstream.

[0135] 4) Currently specified, the value of nuh_layer_id of the SEI NAL unit containing the HRD-related SEI message with non-scalable nesting shall be equal to vps_layer_id[0], and the value of nuh_layer_id of the SEI NAL unit containing the non-HRD-related SEI message with non-scalable nesting shall be equal to the nuh_layer_id of the VCL NAL unit associated with the SEI NAL unit. However, when the non-scalable nested SEI message is applied to the entire bitstream, these constraints on the value of nuh_layer_id shall be removed, so that the value of nuh_layer_id of the SEI NAL unit containing the non-scalable nested SEI message is unconstrained and the same as the nuh_layer_id of the DCI, VPS, AUD, and EOS NAL units.

[0136] 5) The list variable VclAssociatedSeiList currently consists of the SEIpayloadType values of the non-HRD-related SEI messages. However, for the SLI SEI message, the value 203 is also an HRD-related SEI message. Therefore, the payloadType value 203 (SLI) shall be removed from the list.

[0137] 6) There is a lack of such a constraint that when general_same_pic_timing_in_all_ols_flag is equal to 1, there shall be no SEI NAL unit containing a scalable nested SEI message with payloadType equal to 1 (PT). This is because when general_same_pic_timing_in_all_ols_flag is equal to 1, it is not necessary to include the PT SEI message in the scalable nested SEI message.

[0138] 7) There is a lack of such a constraint that when the SEI NAL unit contains a non-scalable nested SEI message with payloadType equal to 0 (BP), 1 (PT), 130 (DUI), or 203 (SLI), the SEI NAL unit shall not contain any other SEI message with payloadType not equal to 0, 1, 130, or 203. Only in this way, during the sub-bitstream extraction process, the removal of these four HRD-related SEI messages from the output bitstream can be performed by simply removing the SEI NAL unit containing one or more of these SEI messages.

[0139] 8) There is a lack of a constraint that when an SEI NAL unit contains a scalable nested SEI message with payloadType equal to 0 (BP), 1 (PT), 130 (DUI), or 203 (SLI), the SEI NAL unit shall not contain any other SEI message (scalable nested) with payloadType not equal to 0, 1, 130, 203, or 133. Only in this way can the sn_ols_flag value of the scalable nested SEI message be set without problems for both scalable nested HRD-related and non-HRD-related SEI messages.

[0140] 9) There is a lack of a constraint that when an SEI NAL unit contains an SEI message with payloadType equal to 3 (padding payload), the SEI NAL unit shall not contain any other SEI message with payloadType not equal to 3. Only in this way can the removal of padding payload SEI messages from the output bitstream be performed during the sub-bitstream extraction process by simply removing the SEI NAL units containing one or more padding payload SEI messages.

[0141] 10) There is a lack of a constraint that when SLI SEI messages and BP SEI messages applicable to a specific OLS exist within an AU, the SLI SEI messages shall be before the BP SEI messages in decoding order. This is necessary because SLI SEI messages provide sequence-level information such as VPS and SPS, which are also before the BP SEI messages when present in the same AU as the BP SEI messages.

[0142] 11) In the BP SEI message, even when bp_max_sublayers_minus1 is equal to 0, the bp_sublayer_initial_cpb_removal_delay_present_flag is signaled. However, when bp_max_sublayers_minus1 is equal to 0, the value of bp_sublayer_initial_cpb_removal_delay_present_flag is known to be 0.

[0143] 5. Solutions and Example Implementations

[0144] To solve the above problems and other problems, methods summarized as follows are disclosed. These solution items should be regarded as examples explaining general concepts and should not be interpreted in a narrow way. In addition, these items can be applied individually or in any combination.

[0145] 1) To solve the first problem, it is specified that non-scalable nested HRD-related SEI messages apply to the OLSs that include the same set of layers as the entire bitstream (instead of only applying to the 0th OLS).

[0146] a. In one example, the HRD-related SEI messages refer to the SEI messages with payloadType equal to 0 (BP), 1 (PT), 130 (DUI), or 203 (SLI).

[0147] 2) To solve the second problem, a constraint is added such that when there is no OLS that includes the same set of layers as the entire bitstream, there should be no non-scalable nested HRD-related SEI messages.

[0148] a. In one example, the HRD-related SEI messages refer to the SEI messages with payloadType equal to 0 (BP), 1 (PT), 130 (DUI), or 203 (SLI).

[0149] 3) To solve the third problem, it is specified that non-scalable nested non-HRD-related SEI messages apply to all layers in the entire bitstream to be consistent with non-scalable nested HRD-related SEI messages.

[0150] a. In one example, the non-HRD-related SEI messages refer to the SEI messages with payloadType not equal to 0 (BP), 1 (PT), 130 (DUI), or 203 (SLI).

[0151] 4) To solve the fourth problem, the constraint on the value of nuh_layer_id of the SEI NAL unit containing non-scalable nested SEI messages is removed, so that the value of nuh_layer_id of the SEI NAL unit containing non-scalable nested SEI messages is not constrained and is the same as that of DCI, VPS, AUD, and EOS NAL units.

[0152] 5) To solve the fifth problem, rename VclAssociatedSeiList to NestingForLayersSeiList and remove the payloadType value 203 (SLI) from the list.

[0153] 6) To solve the sixth problem, a constraint is added such that when general_same_pic_timing_in_all_ols_flag is equal to 1, there should be no SEI NAL unit containing scalable nested SEI messages with payloadType equal to 1 (PT).

[0154] a. Additionally, in one example, it is specified that when general_same_pic_timing_in_all_ols_flag is equal to 1, the non-scalable nested PT SEI message applies to all OLSs and the sub-picture sequences that can be extracted from the bitstream of the OLSs.

[0155] b. In one example, alternatively, a constraint is added such that when general_same_pic_timing_in_all_ols_flag is equal to 1, there should be no SEI NAL unit containing a scalable nested SEI message with payloadType equal to 1 (PT) and its sn_subpic_flag equal to 0.

[0156] 7) To solve the seventh problem, it is specified that when an SEI NAL unit contains a non-scalable nested SEI message with payloadType equal to 0 (BP), 1 (PT), 130 (DUI), or 203 (SLI), the SEI NAL unit should not contain any other SEI message with payloadType not equal to 0, 1, 130, or 203.

[0157] 8) To solve the eighth problem, it is specified that when an SEI NAL unit contains a scalable nested SEI message with payloadType equal to 0 (BP), 1 (PT), 130 (DUI), or 203 (SLI), the SEI NAL unit should not contain any other SEI message (scalable nested) with payloadType not equal to 0, 1, 130, 203, or 133.

[0158] 9) To solve the ninth problem, a constraint is added such that when an SEI NAL unit contains an SEI message with payloadType equal to 3 (padding payload), the SEI NAL unit should not contain any other SEI message with payloadType not equal to 3.

[0159] a. Additionally, in one example, it is specified that the padding data SEI message should not be scalable nested, i.e., it should not be contained in a scalable nested SEI message.

[0160] b. In one example, alternatively, a constraint is added such that when an SEI NAL unit contains an SEI message with payloadType equal to 3 (padding payload), the SEI NAL unit should not contain any other SEI message (scalable nested) with payloadType not equal to 3 or 133.

[0161] 10) To solve the tenth problem, add a constraint such that when SLI SEI messages and BP SEI messages applicable to a particular OLS exist within an AU, the SLI SEI messages will be before the BP SEI messages in decoding order.

[0162] 11) To solve the eleventh problem, specify that when bp_max_sublayers_minus1 is equal to 0, bp_sublayer_initial_cpb_removal_delay_present_flag is skipped (i.e., not signaled in the BP SEI message).

[0163] a. Additionally, in one example, when bp_max_sublayers_minus1 is equal to 0, infer that the value of bp_sublayer_initial_cpb_removal_delay_present_flag is equal to 0.

[0164] 6. Embodiments

[0165] The following are some example embodiments of some aspects of the present invention summarized above in this section, which can be applied to the VVC specification. The changed text is based on the latest VVC text in JVET - S0152 - v5. Most of the relevant parts that have been added or modified are in bold and italic, and some deleted parts are marked with double brackets (e.g., [[a]] indicates deletion of the character 'a').

[0166] 6.1. Embodiment 1

[0167] This embodiment is for items 1 to 11 and some of their sub - items.

[0168] D.2.2 General SEI Payload Semantics ...

[0169] List [[VclAssociated]] SeiList is set to be composed of payloadType values 3, 19, 45, 129, 132, 137, 144, 145, 147 to 150 (inclusive), 153 to 156 (inclusive), 168, [

[203] ], and 204.

[0170] List PicUnitRepConSeiList is set to be composed of payloadType values 0, 1, 19, 45, 129, 132, 133, 137, 147 to 150 (inclusive), 153 to 156 (inclusive), 168, 203, and 204.

[0171] Note 4—The [[VclAssociatedSeiList]] consists of payloadType values of SEI messages which, when non-scalable nested and contained in an SEI NAL unit, infer constraints on the NAL unit header of the SEI NAL unit based on the NAL unit header of the associated VCL NAL unit. The [[PicUnitRepConSeiList]] consists of payloadType values of SEI messages which are restricted to 4 repetitions per PU.

[0172] The requirement for bitstream conformance is that the following restrictions apply when an SEI message is contained in an SEI NAL unit:

[0173] —

[0174] —When an SEI NAL unit contains a non-scalable nested SEI message with payloadType equal to 0 (BP), 1 (PT) [[or]] 130 (DUI) or 203 (SLI), the SEI NAL unit shall not contain any other SEI message with payloadType not equal to 0, 1 [[or]] 130 or 203.

[0175] —When an SEI NAL unit contains a scalable nested SEI message with payloadType equal to 0 (BP), 1 (PT) [[or]] 130 (DUI) or 203 (SLI), the SEI NAL unit shall not contain any other SEI message with payloadType not equal to 0, 1, 130, 203 or 133 (scalable nesting).

[0176] —When an SEI NAL unit contains an SEI message with payloadType equal to 3 (Padding Payload), the SEI NAL unit shall not contain any other SEI message with payloadType not equal to 3.

[0177] The following applies to the applicable OLS or non-scalable nested SEI message layer:

[0178] —For non-scalable nested SEI messages, when payloadType is equal to 0 (BP), 1 (PT), [[or]] 130 (DUI) or 203 (SLI), the non-scalable nested SEI message applies [[only to the 0th OLS]].

[0179] —For non-scalable nested SEI messages, when the payloadType is equal to [[VclAssociated]] any value in the SeiList, the non-scalable nested SEI message applies [[only applicable to the layer where the nuh_layer_id of the VCL NAL unit is equal to the nuh_layer_id of the SEI NAL unit containing the SEI message]].

[0180] The requirement for bitstream consistency is that the following restrictions apply to the value of nuh_layer_id of the SEI NAL unit:

[0181] —[[When the payloadType of the non-scalable nested SEI message is equal to 0 (BP), 1 (PT), or 130 (DUI), the nuh_layer_id of the SEI NAL unit containing the non-scalable nested SEI message shall be equal to vps_layer_id[0].

[0182] —When the payloadType of the non-scalable nested SEI message is equal to any value in the VclAssociatedSeiList, the nuh_layer_id of the SEI NAL unit containing the non-scalable nested SEI message shall be equal to the value of the nuh_layer_id of the VCL NAL unit associated with the SEI NAL unit.]]

[0183] —The nuh_layer_id of the SEI NAL unit containing the scalable nested SEI message shall be equal to the lowest value of the nuh_layer_id of all layers to which the scalable nested SEI message is applied (when the sn_ols_flag of the scalable nested SEI message is equal to 0) or the lowest value of the nuh_layer_id of all layers in the OLS to which the scalable nested SEI message is applied (when the sn_ols_flag of the scalable nested SEI message is equal to 1).

[0184]

[0185] The requirement for bitstream consistency is that the following restrictions apply to the repetition of SEI messages:

[0186] —For each payloadType value included in the PicUnitRepConSeiList, there should be less than or equal to 4 identical sei_payload() syntax structures in a PU.

[0187] — In a DU, the number of identical sei_payload() syntax structures with payloadType equal to 130 shall be less than or equal to 4.

[0188] The following applies to the order of SLI, BP, PT, and DUI SEI messages:

[0189] —

[0190] — When BP SEI messages and PT SEI messages applicable to a specific [[OP]] exist within an AU, the BP SEI message shall be before the PT SEI message in decoding order.

[0191] — When BP SEI messages and DUI SEI messages applicable to a specific [[OP]] exist within an AU, the BP SEI message shall be before the DUI SEI message in decoding order.

[0192] — When PT SEI messages and DUI SEI messages applicable to a specific [[OP]] exist within an AU, the PT SEI message shall be before the DUI SEI message in decoding order.

[0193] D.3.1 Buffering period SEI message syntax

[0194]

[0195] D.3.2 Buffering period SEI message semantics ...

[0196] The bp_sublayer_initial_cpb_removal_delay_present_flag equal to 1 indicates that there are syntax elements related to the initial CPB removal delay in the (multiple) sublayer representations within the range of 0 to bp_max_sublayers_minus1 (inclusive). The bp_sublayer_initial_cpb_removal_delay_present_flag equal to 0 indicates that there are syntax elements related to the initial CPB removal delay in the bp_max_sublayers_minus1 sublayer representation. ...

[0197] Figure 1is a block diagram showing an example video processing system 1900 in which various techniques disclosed herein can be implemented. Various embodiments may include some or all of the components of system 1900. System 1900 may include an input 1902 for receiving video content. The video content may be received in a raw or uncompressed format (e.g., 8 or 10-bit multi-component pixel values), or may be received in a compressed or encoded format. Input 1902 may represent a network interface, a peripheral bus interface, or a storage interface. Examples of network interfaces include wired interfaces such as Ethernet, Passive Optical Network (PON), etc., and wireless interfaces such as Wi-Fi or cellular interfaces.

[0198] System 1900 may include a codec component 1904, which may implement various codec or encoding methods described in this document. The codec component 1904 may reduce the average bit rate of the video from the input 1902 to the output of the codec component 1904 to produce a codec representation of the video. Thus, codec techniques are sometimes referred to as video compression or video transcoding techniques. The output of the codec component 1904 may be stored or transmitted via a connected communication represented by component 1906. Component 1908 may use the stored or communicated bitstream (or codec) representation of the video received at the input 1902 to generate pixel values or a displayable video to be sent to the display interface 1910. The process of generating a user-visible video from the bitstream representation is sometimes referred to as video decompression. Additionally, although certain video processing operations are referred to as "codec" operations or tools, it should be understood that codec tools or operations are used at the encoder, and the decoder will perform the corresponding decoding tools or operations that reverse the results of the codec.

[0199] Examples of peripheral bus interfaces or display interfaces may include Universal Serial Bus (USB), High-Definition Multimedia Interface (HDMI), or DisplayPort, etc. Examples of storage interfaces include SATA (Serial Advanced Technology Attachment), PCI, IDE interfaces, etc. The techniques described in this document may be embodied in various electronic devices such as mobile phones, laptops, smartphones, or other devices capable of performing digital data processing and / or video display.

[0200] Figure 2is a block diagram of a video processing apparatus 3600. The apparatus 3600 can be used to implement one or more of the methods described herein. The apparatus 3600 can be embodied in a smart phone, a tablet computer, a computer, an Internet of Things (IoT) receiver, etc. The apparatus 3600 can include one or more processors 3602, one or more memories 3604, and video processing hardware 3606. The processor 3602 can be configured to implement one or more of the methods described in this document. The memory 3604 can be used to store data and code for implementing the methods and techniques described herein. The video processing hardware 3606 can be used to implement some of the techniques described in this document in hardware circuitry.

[0201] Figure 4 is a block diagram showing an example video codec system 100 that can utilize the techniques of the present disclosure.

[0202] As Figure 4 shown, the video codec system 100 can include a source device 110 and a destination device 120. The source device 110 generates encoded video data, and the source device 110 can be referred to as a video encoding device. The destination device 120 can decode the encoded video data generated by the source device 110, and the destination device 120 can be referred to as a video decoding device.

[0203] The source device 110 can include a video source 112, a video encoder 114, and an input / output (I / O) interface 116.

[0204] The video source 112 can include sources such as a video capture device, an interface for receiving video data from a video content provider, and / or a computer graphics system for generating video data, or a combination of these sources. The video data can include one or more pictures. The video encoder 114 encodes the video data from the video source 112 to generate a bitstream. The bitstream can include a sequence of bits that form a codec representation of the video data. The bitstream can include coded pictures and associated data. A coded picture is a codec representation of a picture. The associated data can include a sequence parameter set, a picture parameter set, and other syntax structures. The I / O interface 116 can include a modulator / demodulator (modem) and / or a transmitter. The encoded video data can be sent directly to the destination device 120 via the I / O interface 116 over a network 130a. The encoded video data can also be stored on a storage medium / server 130b for access by the destination device 120.

[0205] The destination device 120 can include an I / O interface 126, a video decoder 124, and a display device 122.

[0206] The I / O interface 126 may include a receiver and / or a modem. The I / O interface 126 may obtain encoded video data from the source device 110 or the storage medium / server 130b. The video decoder 124 may decode the encoded video data. The display device 122 may display the decoded video data to the user. The display device 122 may be integrated with the target device 120 or may be external to the target device 120, which is configured to interface with an external display device.

[0207] The video encoder 114 and the video decoder 124 may operate according to video compression standards, such as the High Efficiency Video Coding (HEVC) standard, the Versatile Video Coding (VVC) standard, and other current and / or further standards.

[0208] Figure 5 is a block diagram showing an example of a video encoder 200 that may be the video encoder 114 in the system 100 shown. Figure 4 of the system 100 shown.

[0209] The video encoder 200 may be configured to perform any or all of the techniques of the present disclosure. In Figure 5 an example, the video encoder 200 includes a plurality of functional components. The techniques described in the present disclosure may be shared among the various components of the video encoder 200. In some examples, a processor may be configured to perform any or all of the techniques described in the present disclosure.

[0210] The functional components of the video encoder 200 may include a splitting unit 201, a prediction unit 202, and the prediction unit 1602 may include a mode selection unit 203, a motion estimation unit 204, a motion compensation unit 205, and an intra prediction unit 206, a residual generation unit 207, a transformation unit 208, a quantization unit 209, an inverse quantization unit 210, an inverse transformation unit 211, a reconstruction unit 212, a buffer 213, and an entropy coding unit 214.

[0211] In other examples, the video encoder 200 may include more, fewer, or different functional components. In one example, the prediction unit 202 may include an Intra Block Copy (IBC) unit. The IBC unit may perform prediction in the IBC mode, where at least one reference picture is the picture in which the current video block is located.

[0212] In addition, some components, such as the motion estimation unit 204 and the motion compensation unit 205, may be highly integrated, but are shown separately in Figure 5 an example for the purpose of explanation.

[0213] The splitting unit 201 may split a picture into one or more video blocks. The video encoder 200 and the video decoder 300 may support various video block sizes.

[0214] The mode selection unit 203 can select, for example, an encoding / decoding mode (intra or inter) based on an error result, and provide the resulting intra- or inter-coded block to the residual generation unit 207 to generate residual block data, and provide it to the reconstruction unit 212 to reconstruct the coded block for use as a reference picture. In some examples, the mode selection unit 203 can select a combined intra and inter prediction (CIIP) mode, where the prediction is based on an inter prediction signal and an intra prediction signal. The mode selection unit 203 can also select the resolution of the motion vector for a block in the case of inter prediction (e.g., sub-pixel or integer pixel precision).

[0215] To perform inter prediction on a current video block, the motion estimation unit 204 can generate motion information for the current video block by comparing one or more reference frames from the buffer 213 with the current video block. The motion compensation unit 205 can determine a predicted video block for the current video block based on the motion information and decoded samples of pictures from the buffer 213 (except for the picture associated with the current video block).

[0216] For example, the motion estimation unit 204 and the motion compensation unit 205 can perform different operations on the current video block depending on whether the current video block is in an I-slice, a P-slice, or a B-slice.

[0217] In some examples, the motion estimation unit 204 can perform uni-directional prediction on the current video block, and the motion estimation unit 204 can search for a reference video block of the current video block in the reference pictures of list 0 or list 1. Then, the motion estimation unit 204 can generate a reference index that indicates the reference picture in list 0 or list 1 that contains the reference video block and a motion vector that indicates the spatial displacement between the current video block and the reference video block. The motion estimation unit 204 can output the reference index, a prediction direction indicator, and the motion vector as the motion information of the current video block. The motion compensation unit 205 can generate a predicted video block for the current block based on the reference video block indicated by the motion information of the current video block.

[0218] In other examples, the motion estimation unit 204 may perform bidirectional prediction on a current video block. The motion estimation unit 204 may search for a reference video block of the current video block in the reference pictures in list 0, and may also search for another reference video block of the current video block in the reference pictures in list 1. Then, the motion estimation unit 204 may generate a reference index indicating the reference pictures in list 0 and list 1 that contain the reference video blocks and a motion vector indicating the spatial displacement between the reference video block and the current video block. The motion estimation unit 204 may output the reference index and the motion vector of the current video block as the motion information of the current video block. The motion compensation unit 205 may generate a predicted video block of the current video block based on the reference video block indicated by the motion information of the current video block.

[0219] In some examples, the motion estimation unit 204 may output a complete set of motion information for the decoding process of the decoder.

[0220] In some examples, the motion estimation unit 204 may not output a complete set of motion information of the current video. Instead, the motion estimation unit 204 may signal the motion information of the current video block by referring to the motion information of another video block. For example, the motion estimation unit 204 may determine that the motion information of the current video block is similar enough to the motion information of an adjacent video block.

[0221] In one example, the motion estimation unit 204 may indicate in a syntax structure associated with the current video block a value that indicates to the video decoder 300 that the current video block has the same motion information as another video block.

[0222] In another example, the motion estimation unit 204 may identify another video block and a motion vector difference (MVD) in a syntax structure associated with the current video block. The motion vector difference indicates the difference between the motion vector of the current video block and the motion vector of the indicated video block. The video decoder 300 may use the motion vector of the indicated video block and the motion vector difference to determine the motion vector of the current video block.

[0223] As described above, the video encoder 200 may predictively signal motion vectors. Two examples of predictive signaling techniques that may be implemented by the video encoder 200 include advanced motion vector prediction (AMVP) and merge mode signaling.

[0224] The intra prediction unit 206 may perform intra prediction on a current video block. When the intra prediction unit 206 performs intra prediction on the current video block, the intra prediction unit 206 may generate prediction data of the current video block based on the decoded samples of other video blocks in the same picture. The prediction data of the current video block may include a predicted video block and various syntax elements.

[0225] The residual generation unit 207 can generate residual data for a current video block by subtracting (e.g., indicated by a minus sign) a (plurality of) predicted video blocks of the current video block from the current video block. The residual data for the current video block can include residual video blocks corresponding to different sample components of the samples in the current video block.

[0226] In other examples, such as in the skip mode, for a current video block, there may be no residual data for the current video block, and the residual generation unit 207 may not perform a subtraction operation.

[0227] The transform processing unit 208 can generate one or more transform coefficient video blocks for a current video block by applying one or more transforms to the residual video block associated with the current video block.

[0228] After the transform processing unit 208 generates the transform coefficient video block associated with the current video block, the quantization unit 209 can quantize the transform coefficient video block associated with the current video block based on one or more quantization parameter (QP) values associated with the current video block.

[0229] The inverse quantization unit 210 and the inverse transform unit 211 can apply inverse quantization and inverse transform to the transform coefficient video block, respectively, to reconstruct the residual video block from the transform coefficient video block. The reconstruction unit 212 can add the reconstructed residual video block to the corresponding samples of one or more predicted video blocks generated by the prediction unit 202 to produce a reconstructed video block associated with the current block for storage in the buffer 213.

[0230] After the reconstruction unit 212 reconstructs the video block, a loop filtering operation can be performed to reduce video block artifacts in the video block.

[0231] The entropy coding unit 214 can receive data from other functional components of the video encoder 200. When the entropy coding unit 214 receives the data, the entropy coding unit 214 can perform one or more entropy coding operations to generate entropy coded data and output a bitstream including the entropy coded data.

[0232] Figure 6 is shown to be Figure 4 A block diagram of an example of a video decoder 300 that can be the video decoder 124 in the system 100 shown.

[0233] The video decoder 300 can be configured to perform any or all of the techniques of the present disclosure. In Figure 5 the example, the video decoder 300 includes a plurality of functional components. The techniques described in the present disclosure can be shared among the various components of the video decoder 300. In some examples, a processor can be configured to perform any or all of the techniques described in the present disclosure.

[0234] In Figure 6 the example of Figure 6 , the video decoder 300 includes an entropy decoding unit 301, a motion compensation unit 302, an intra prediction unit 303, an inverse quantization unit 304, an inverse transform unit 305, a reconstruction unit 306, and a buffer 307. In some examples, the video decoder 300 may perform a decoding process that is generally the reverse of the encoding process described with respect to the video encoder 200 ( Figure 5 ).

[0235] The entropy decoding unit 301 may retrieve the encoded bitstream. The encoded bitstream may include entropy-coded video data (e.g., encoded blocks of video data). The entropy decoding unit 301 may decode the entropy-coded video data, and the motion compensation unit 302 may determine motion information including motion vectors, motion vector precision, reference picture list indices, and other motion information from the entropy-decoded video data. For example, the motion compensation unit 302 may determine such information by performing AMVP and merge mode.

[0236] The motion compensation unit 302 may generate a motion-compensated block, possibly performing interpolation based on an interpolation filter. An identifier of the interpolation filter used with sub-pixel precision may be included in the syntax element.

[0237] The motion compensation unit 302 may use the interpolation filter used by the video encoder 20 during video block encoding to calculate the interpolation of sub-integer pixels of a reference block. The motion compensation unit 302 may determine the interpolation filter used by the video encoder 200 based on the received syntax information and use the interpolation filter to generate a prediction block.

[0238] The motion compensation unit 302 may use some syntax information to determine the size of the blocks for encoding the (multiple) frames and / or (multiple) slices of the encoded video sequence, partition information describing how each macroblock of a picture of the encoded video sequence is partitioned, the mode indicating how each partition is encoded, one or more reference frames (and reference frame lists) for each inter-coded block, and other information for decoding the encoded video sequence.

[0239] The intra prediction unit 303 may form a prediction block from spatially adjacent blocks using, for example, an intra prediction mode received in the bitstream. The inverse quantization unit 304 inverse quantizes (i.e., de-quantizes) the quantized video block coefficients provided in the bitstream and decoded by the entropy decoding unit 301. The inverse transform unit 305 applies an inverse transform.

[0240] The reconstruction unit 306 can add the residual block to the corresponding prediction block generated by the motion compensation unit 302 or the intra prediction unit 303 to form a decoded block. If necessary, a deblocking filter can also be applied to filter the decoded block to remove blocking artifacts. The decoded video block is then stored in the buffer 307, which provides reference blocks for subsequent motion compensation / intra prediction and also generates the decoded video for presentation on a display device.

[0241] The list of solutions describes some embodiments of the disclosed technology.

[0242] A first set of solutions is provided below. The following solutions illustrate example embodiments of the techniques discussed in the previous section (e.g., items 1 - 3).

[0243] 1. A video processing method (e.g., Figure 3 the method 600 shown), comprising: performing (602) a conversion between a video including one or more video layers and a codec representation of a video including one or more output layer sets, wherein the codec representation complies with format rules related to whether and how one or more syntax elements are related to supplementary enhancement information (SEI) associated with a non-scalable nested hypothetical reference decoder (HRD).

[0244] 2. The method according to solution 1, wherein the format rules specify that messages related to SEI associated with a non-scalable nested HRD apply to an output layer set including the same layer set as the entire codec representation.

[0245] 3. The method according to any one of solutions 1 - 2, wherein the format rules specify that one or more syntax elements are omitted in the absence of an output layer set including the same layer set as the entire codec representation.

[0246] 4. The method according to solution 1, wherein the format rules specify that one or more syntax elements apply to all layers in the codec representation.

[0247] The following solutions illustrate example embodiments of the techniques discussed in the previous section (e.g., items 4 - 10).

[0248] 5. A video processing method, comprising: performing a conversion between a video including one or more video layers and a codec representation of a video including one or more output layer sets, wherein the codec representation complies with format rules related to whether and how one or more syntax elements are included in a supplementary enhancement information (SEI) network abstraction layer (NAL) unit.

[0249] 6. A method according to solution 5, wherein the format rule specifies that in the case where the SEI NAL unit includes a non-scalable nested SEI message, the value of the layer identifier is not constrained.

[0250] 7. A method according to any one of solutions 5 - 6, wherein due to the signaling of the same picture timing being used in all output layer sets in the codec representation, the format rule prohibits SEI NAL units that include scalable nested SEI messages containing a specific payload type.

[0251] 8. A method according to any one of solutions 5 - 7, wherein the format rule specifies that SEI NAL units that include non-scalable nested SEI messages of a first specific payload type are not allowed to include another SEI message of a second specific type.

[0252] 9. A method according to solution 8, wherein the first specific payload type is equal to 0, 1, 130, or 203.

[0253] 10. A method according to solution 8 or 9, wherein the second specific payload type is equal to 0, 1, 130, 203, or 133.

[0254] 11. A method according to solutions 8 to 10, wherein the first specific payload type and the second specific payload type are 3.

[0255] 12. A method according to any one of solutions 1 - 11, wherein performing the conversion includes encoding the video to generate a codec representation.

[0256] 13. A method according to any one of solutions 1 - 11, wherein performing the conversion includes parsing and decoding the codec representation to generate the video.

[0257] 14. A video decoding device, comprising a processor configured to implement the method according to any one or more of solutions 1 to 13.

[0258] 15. A video encoding device, comprising a processor configured to implement the method according to any one or more of solutions 1 to 13.

[0259] 16. A computer program product having computer code stored thereon, which when executed by a processor causes the processor to implement the method according to any one of solutions 1 to 13.

[0260] 17. A method, device, or system described in this document.

[0261] The second set of solutions shows example embodiments of the techniques discussed in the previous section (e.g., items 1-5).

[0262] 1. A method for processing video data (e.g., method 700 as shown in Figure 7A ), comprising: performing 702 a conversion between a video and a bitstream of the video, wherein according to format rules the bitstream includes one or more output layer sets, the output layer set including one or more video layers, and wherein the format rules specify that a non-scalable nested supplementary enhancement information (SEI) message including information about a hypothetical reference decoder (HRD) is applicable to all output layer sets including the same video layers as the bitstream.

[0263] 2. The method according to solution 1, wherein the non-scalable nested SEI message is an SEI message not included in a scalable nested SEI message.

[0264] 3. The method according to solution 1 or 2, wherein the non-scalable nested SEI message including information about the HRD is a buffering period (BP) SEI message, a picture timing (PT) SEI message, a decoding unit information (DUI) SEI message, or a sub-picture level information (SLI) SEI message.

[0265] 4. The method according to solution 1 or 2, wherein the non-scalable nested SEI message including information about the HRD has a payload type equal to 0, 1, 130, or 203.

[0266] 5. A method for processing video data (e.g., method 710 as shown in Figure 7B ), comprising: performing 712 a conversion between a video and a bitstream of the video, wherein, according to format rules the bitstream includes one or more output layer sets, the output layer set including one or more video layers, and wherein the format rules specify that a non-scalable nested hypothetical reference decoder (HRD)-related supplementary enhancement information (SEI) message is omitted in response to a condition that there is no output layer set including the same layer set as the bitstream.

[0267] 6. The method according to solution 5, wherein the non-scalable nested SEI message is an SEI message not included in a scalable nested SEI message.

[0268] 7. The method according to solution 5 or 6, wherein the non-scalable nested HRD-related SEI message is a buffering period (BP) SEI message, a picture timing (PT) SEI message, a decoding unit information (DUI) SEI message, or a sub-picture level information (SLI) SEI message.

[0269] 8. A method according to solution 5 or 6, wherein the non-scalable nested HRD-related SEI message has a payload type equal to 0, 1, 130, or 203.

[0270] 9. A method of processing video data (e.g., a method 720 as Figure 7C shown), comprising: performing 722 a conversion between a video and a bitstream of the video, wherein according to format rules the bitstream includes one or more output layer sets, the output layer set including one or more video layers, and wherein the format rules specify that the value of the layer identifier for a SEI network abstraction layer (NAL) unit including a non-scalable nested supplementary enhancement information (SEI) message is not constrained.

[0271] 10. A method according to solution 9, wherein the non-scalable nested SEI message is an SEI message not included in a scalable nested SEI message.

[0272] 11. A method according to solution 9, wherein the non-scalable nested SEI message is a buffering period (BP) SEI message, a picture timing (PT) SEI message, a decoded unit information (DUI) SEI message, or a sub-picture level information (SLI) SEI message.

[0273] 12. A method according to solution 9, wherein the non-scalable nested SEI message has a payload type equal to 0, 1, 130, or 203.

[0274] 13. A method of processing video data (e.g., a method 730 as Figure 7D shown), comprising: performing 732 a conversion between a video and a bitstream of the video, wherein, according to format rules the bitstream includes one or more output layer sets, the output layer set including one or more video layers, and wherein the format rules specify that a particular payload type value corresponding to sub-picture level information is not allowed in a list of allowed supplementary enhancement information (SEI) payload type values for a non-hypothetical reference decoder (HRD)-related supplementary enhancement information (SEI) message.

[0275] 14. A method according to solution 13, wherein the particular payload type value is 203.

[0276] 15. A method according to solution 13, wherein the allowed SEI payload type values include padding payload, film grain characteristic, frame packaging arrangement, parameter set inclusion indication, primary display color volume, content light level information, dependent rap indication, alternative transmission characteristic, surrounding viewing environment, content color volume, equirectangular projection, generalized cube map projection, sphere rotation, region packaging, omnidirectional viewport, frame field information, and sample aspect ratio information.

[0277] 16. The method according to solution 13, wherein the allowed SEI payload type values include 3, 19, 45, 129, 137, 144, 145, 147 to 150, 153 to 156, 168, and 204.

[0278] 17. A method for processing video data, comprising: performing a conversion between a video and a bitstream of the video, wherein, according to formatting rules, the bitstream includes one or more output layer sets, the output layer set includes one or more video layers, and the formatting rules specify that non-scalable nested supplementary enhancement information SEI messages including information independent of a hypothetical reference decoder (HRD) are applicable to all layers in the bitstream.

[0279] 18. The method according to solution 17, wherein the non-scalable nested SEI message is an SEI message not included in a scalable nested SEI message.

[0280] 19. The method according to solution 17 or 18, wherein the non-scalable nested SEI message including information independent of HRD has a payload type not equal to 0, 1, 130, or 203.

[0281] 20. The method according to solution 17 or 18, wherein the non-scalable nested SEI message including information independent of HRD does not correspond to a buffering period (BP) SEI message, a picture timing (PT) SEI message, a decoding unit information (DUI) SEI message, or a sub-picture level information (SLI) SEI message.

[0282] 21. The method according to any one of solutions 1 to 20, wherein the conversion includes encoding the video into a bitstream.

[0283] 22. The method according to any one of solutions 1 to 20, wherein the conversion includes decoding the video from the bitstream.

[0284] 23. The method according to any one of solutions 1 to 20, wherein the conversion includes generating a bitstream from the video, and the method further includes: storing the bitstream in a non-transitory computer-readable recording medium.

[0285] 24. A video processing apparatus, comprising a processor configured to implement the method according to any one or more of solutions 1 to 23.

[0286] 25. A method for storing a bitstream of a video, comprising the method according to any one of solutions 1 to 23, and further comprising storing the bitstream in a non-transitory computer-readable recording medium.

[0287] 26. A computer-readable medium storing program code which, when executed, causes a processor to implement the method according to any one or more of Solutions 1 to 23.

[0288] 27. A computer-readable medium storing a bitstream generated according to any of the above methods.

[0289] 28. A video processing apparatus for storing a bitstream, wherein the video processing apparatus is configured to implement the method according to any one or more of Solutions 1 to 23.

[0290] The third set of solutions shows example embodiments of the techniques discussed in the previous section (e.g., item 6).

[0291] 1. A method for processing video data (e.g., the method 800 as Figure 8 shown), comprising: performing a conversion 802 between a video and a bitstream of the video including one or more output layer sets according to a rule, where the rule specifies that since the same picture timing is used in all output layer sets in the bitstream, SEI network abstraction layer (NAL) units containing scalable nested supplementary enhancement information (SEI) messages carrying picture timing information are not included.

[0292] 2. The method according to Solution 1, wherein the scalable nested SEI message is an SEI message contained in the scalable nested SEI message.

[0293] 3. The method according to Solution 1 or 2, wherein the scalable nested SEI message carrying picture timing information corresponds to a picture timing (PT) SEI message.

[0294] 4. The method according to Solution 1, wherein the scalable nested SEI message carrying picture timing information has a payload type equal to 1.

[0295] 5. The method according to any one of Solutions 1 to 4, wherein the rule further specifies that non-scalable nested SEI messages carrying picture timing information are applicable to all output layer sets and sub-picture sequences allowed to be extracted from the bitstream.

[0296] 6. The method according to Solution 5, wherein the non-scalable nested SEI message is an SEI message not contained in the scalable nested SEI message.

[0297] 7. A method according to any one of Solutions 1 to 4, wherein the rule further specifies that, in a case where a syntax field has a scalable nested SEI message applicable to all values of sub-pictures of a specified output layer set or layer and the SEI message specifies values applicable to a specified output layer set or layer, an SEI NAL unit including the scalable nested SEI message carrying picture timing information is excluded.

[0298] 8. A method according to any one of Solutions 1 to 7, wherein the conversion includes encoding video into a bitstream.

[0299] 9. A method according to any one of Solutions 1 to 7, wherein the conversion includes decoding video from the bitstream.

[0300] 10. A method according to any one of Solutions 1 to 7, wherein the conversion includes generating a bitstream from video, and the method further includes: storing the bitstream in a non-transitory computer-readable recording medium.

[0301] 11. A video processing apparatus, including a processor configured to implement the method according to any one or more of Solutions 1 to 10.

[0302] 12. A method of storing a bitstream of video, including the method according to any one of Solutions 1 to 10, and further including storing the bitstream in a non-transitory computer-readable recording medium.

[0303] 13. A computer-readable medium storing program code that, when executed, causes a processor to implement the method according to any one or more of Solutions 1 to 10.

[0304] 14. A computer-readable medium storing a bitstream generated according to any of the above methods.

[0305] 15. A video processing apparatus for storing a bitstream, wherein the video processing apparatus is configured to implement the method according to any one or more of Solutions 1 to 10.

[0306] The fourth set of solutions shows example embodiments of the techniques discussed in the previous section (e.g., Items 7 - 9).

[0307] 1. A method of processing video data (e.g., such as the method 900 shown Figure 9A ), including: performing 902 a conversion between video and a bitstream of the video, wherein according to a rule the bitstream includes one or more supplementary enhancement information SEI network abstraction layer (NAL) units, and the rule specifies that in response to an SEI NAL unit including a non-scalable nested SEI message of a first payload type, the SEI NAL unit is not allowed to include another SEI message of a second payload type.

[0308] 2. According to the method of Solution 1, an unscalable nested SEI message is an SEI message that is not included in a scalable nested SEI message.

[0309] 3. According to the method of Solution 1 or 2, an unscalable nested SEI message of the first payload type corresponds to a buffering period (BP) SEI message, a picture timing (PT) SEI message, a decoded unit information (DUI) SEI message, or a sub-picture level information (SLI) SEI message.

[0310] 4. According to the method of Solution 1 or 2, the first payload type is equal to 0, 1, 130, or 203.

[0311] 5. According to the method of any one of Solutions 1 to 4, an unscalable nested SEI message of the second payload type does not correspond to any of a buffering period (BP) SEI message, a picture timing (PT) SEI message, a decoded unit information (DUI) SEI message, and a sub-picture level information (SLI) SEI message.

[0312] 6. According to the method of any one of Solutions 1 to 4, the second payload type is not equal to any of 0, 1, 130, and 203.

[0313] 7. A method for processing video data (e.g., the method 910 as shown in Figure 9B ) includes: performing a conversion between a video and a bitstream of the video according to a rule, where the bitstream includes one or more supplementary enhancement information SEI network abstraction layer (NAL) units according to the rule, and the rule specifies that in response to an SEI NAL unit including a scalable nested SEI message of the first payload type, it is not allowed for the SEI NAL unit to include another SEI message of the second payload type.

[0314] 8. According to the method of Solution 7, a scalable nested SEI message is an SEI message included in a scalable nested SEI message.

[0315] 9. According to the method of Solution 7 or 8, a scalable nested SEI message of the first payload type corresponds to a buffering period (BP) SEI message, a picture timing (PT) SEI message, a decoded unit information (DUI) SEI message, or a sub-picture level information (SLI) SEI message.

[0316] 10. According to the method of Solution 7 or 8, the first payload type is equal to 0, 1, 130, or 203.

[0317] 11. The method according to any one of Solutions 7 to 10, wherein the scalable nested SEI message of the second payload type does not correspond to any of the buffering period (BP) SEI message, picture timing (PT) SEI message, decoding unit information (DUI) SEI message, sub-picture level information (SLI) SEI message, and scalable nested information SEI message.

[0318] 12. The method according to any one of Solutions 7 to 10, wherein the second payload type is not equal to any of 0, 1, 130, 203, and 133.

[0319] 13. A method of processing video data (e.g., the method 920 as Figure 9C shown), comprising: performing 922 a conversion between a video and a bitstream of the video, wherein according to a rule the bitstream includes one or more supplementary enhancement information SEI network abstraction layer (NAL) units, and the rule specifies that in response to the SEI NAL unit including an SEI message of a first payload type, it is not allowed for the SEI NAL unit to include another SEI message that is not equal to the first payload type or the second payload type.

[0320] 14. The method according to Solution 13, wherein the SEI message of the first payload type corresponds to a padding payload SEI message.

[0321] 15. The method according to Solution 13 or 14, wherein the first payload type is equal to 3.

[0322] 16. The method according to any one of Solutions 13 to 15, wherein the rule further specifies that it is not allowed to include the SEI message of the first payload type in a scalable nested SEI message.

[0323] 17. The method according to any one of Solutions 13 to 16, wherein the rule further specifies that the SEI message of the second payload type is a scalable nested SEI message.

[0324] 18. The method according to any one of Solutions 13 to 16, wherein the second payload type is equal to 133.

[0325] 19. The method according to any one of Solutions 1 to 18, wherein the conversion includes encoding the video into a bitstream.

[0326] 20. The method according to any one of Solutions 1 to 18, wherein the conversion includes decoding the video from the bitstream.

[0327] 21. The method according to any one of Solutions 1 to 18, wherein the conversion includes generating a bitstream from a video, and the method further includes: storing the bitstream in a non-transitory computer-readable recording medium.

[0328] 22. A video processing apparatus, including a processor configured to implement the method according to any one or more of Solutions 1 to 21.

[0329] 23. A method for storing a bitstream of a video, including the method according to any one of Solutions 1 to 21, and further including storing the bitstream in a non-transitory computer-readable recording medium.

[0330] 24. A computer-readable medium storing program code which, when executed, causes a processor to implement the method according to any one or more of Solutions 1 to 21.

[0331] 25. A computer-readable medium storing a bitstream generated according to any of the above methods.

[0332] 26. A video processing apparatus for storing a bitstream, wherein the video processing apparatus is configured to implement the method according to any one or more of Solutions 1 to 21.

[0333] The fifth group of solutions shows example embodiments of the techniques discussed in the previous section (e.g., Items 10 and 11).

[0334] 1. A method of processing video data (e.g., such as Figure 10A shown in Method 1000), including: performing 1002 a conversion between a video and a bitstream of the video, wherein according to a rule the bitstream includes one or more output layer sets, the output layer set including one or more video layers, and wherein in response to the condition that a sub-picture level information (SLI) supplementary enhancement information (SEI) message and a buffering period (BP) SEI message are included in an access unit, the rule specifies a specific decoding order between the SLI SEI message and the BP SEI message applicable to a specific output layer set.

[0335] 2. The method according to Solution 1, wherein the specific decoding order is that the SLI SEI message is before the BP SEI message.

[0336] 3. A method of processing video data (e.g., such as Figure 10BThe method 1010) shown includes: performing 1012 the conversion between a video and a bitstream of the video according to a rule, where, in response to a specific value of a second syntax field indicating the maximum number of temporal sublayers of an enhancement information (SEI) message indicating an initial coded picture buffer (CPB) removal delay during a buffering period, the rule specifies omitting a first syntax field of sublayer representation information indicating the existence of a syntax element related to the initial CPB removal delay.

[0337] 4. The method according to solution 3, wherein the specific value is 0.

[0338] 5. The method according to solution 3 or 4, wherein the second syntax field having the specific value specifies that the maximum number of temporal sublayers is 1.

[0339] 6. The method according to any one of solutions 3 to 5, wherein the first syntax field equal to another specific value specifies that a syntax element related to the initial CPB removal delay exists in a sublayer representation within a range from 0 to the second syntax field (inclusive).

[0340] 7. The method according to solution 6, wherein the another specific value is 1.

[0341] 8. The method according to any one of solutions 3 to 7, wherein the syntax field equal to another specific value specifies that a syntax element related to the initial CPB removal delay exists in a sublayer representation corresponding to the specific value of the second syntax field.

[0342] 9. The method according to solution 8, wherein the another specific value is 0.

[0343] 10. The method according to any one of solutions 3 to 9, wherein the rule also specifies inferring that the value of the first syntax field is equal to 0 in response to the specific value of the second syntax field.

[0344] 11. The method according to any one of solutions 1 to 10, wherein the conversion includes encoding the video into a bitstream.

[0345] 12. The method according to any one of solutions 1 to 10, wherein the conversion includes decoding the video from the bitstream.

[0346] 13. The method according to any one of solutions 1 to 10, wherein the conversion includes generating a bitstream from the video, and the method further includes: storing the bitstream in a non-transitory computer-readable recording medium.

[0347] 14. A video processing apparatus, including a processor configured to implement the method according to any one or more of solutions 1 to 13.

[0348] 15. A method for storing a bitstream of a video, including the method according to any one of Solutions 1 to 13, and further including storing the bitstream into a non-transitory computer-readable recording medium.

[0349] 16. A computer-readable medium storing program code that, when executed, causes a processor to implement the method according to any one or more of Solutions 1 to 13.

[0350] 17. A computer-readable medium storing a bitstream generated according to any of the above methods.

[0351] 18. A video processing apparatus for storing a bitstream, wherein the video processing apparatus is configured to implement the method according to any one or more of Solutions 1 to 13.

[0352] The disclosures and other solutions, examples, embodiments, modules, and functional operations described in this document can be implemented in digital electronic circuits or in computer software, firmware, or hardware (including the structures disclosed in this document and their structural equivalents, or in a combination of one or more of them). The disclosed embodiments and other embodiments can be implemented as one or more computer program products, that is, one or more modules of computer program instructions encoded on a computer-readable medium for execution or control of the operation by a data processing apparatus. The computer-readable medium can be a machine-readable storage device, a machine-readable storage substrate, a storage device, a composition of matter affecting a machine-readable propagated signal, or a combination of one or more of them. The term "data processing apparatus" includes all apparatuses, devices, and machines for processing data, such as including programmable processors, computers, or multiple processors or computers. In addition to hardware, the apparatus may further include code for creating an execution environment for the computer program under discussion, for example, code constituting processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of them. A propagated signal is a human-generated signal, for example, a machine-generated electrical, optical, or electromagnetic signal generated for encoding information to be transmitted to a suitable receiver device.

[0353] A computer program (also known as a program, software, software application, script, or code) can be written in any form of programming language, including compiled or interpreted languages, and can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A computer program does not necessarily correspond to a file in a file system. The program can be stored in a part of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files that store one or more modules, subroutines, or portions of code). A computer program can be deployed to execute on one computer or on multiple computers located at one site or distributed across multiple sites and interconnected by a communication network.

[0354] The processes and logical flows described in this document can be performed by one or more programmable processors that execute one or more computer programs to perform functions by operating on input data and generating output. The processing and logical flows can also be performed by, and the apparatus can also be implemented as, special purpose logic circuitry, such as an FPGA (Field Programmable Gate Array) or an ASIC (Application Specific Integrated Circuit).

[0355] For example, processors suitable for executing a computer program include general and special purpose microprocessors, and any one or more processors of any type of digital computer. Generally, a processor will receive instructions and data from a read only memory or a random access memory or both. The essential elements of a computer are a processor that executes the instructions and one or more storage devices that store the instructions and data. Generally, a computer will also include, or be operatively coupled to, receive data from and / or transfer data to one or more mass storage devices (such as, for example, magnetic disks, magneto-optical disks, or optical disks) for storing the data. However, a computer need not have such devices. Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, including by way of example semiconductor memory devices, such as EPROM, EEPROM, and flash memory devices; magnetic disks, such as internal hard disks or removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks. The processor and the memory can be supplemented by, or incorporated in, special purpose logic circuitry.

[0356] Although this patent document contains many details, these details should not be construed as limiting the scope of any subject matter or of what may be claimed, but rather as descriptions of features specific to particular embodiments that may be specific to a particular technology. Certain features that are described in the context of separate embodiments in this patent document may also be implemented in combination in a single embodiment. Conversely, the various features that are described in the context of a single embodiment may also be implemented separately or in any suitable sub-combination in multiple embodiments. Additionally, although the above features may be described as acting in certain combinations and even initially claimed as such, in some cases, one or more features from the claimed combination may be deleted from the combination, and the claimed combination may be directed to a sub-combination or a variant of a sub-combination.

[0357] Similarly, although operations are depicted in the drawings in a particular order, this should not be construed as requiring that such operations be performed in the particular order shown or in sequential order, or that all illustrated operations be performed to achieve a desired result. Additionally, the separation of various system components in the embodiments described in this patent document should not be construed as requiring such separation in all embodiments.

[0358] Only some embodiments and examples have been described, and other embodiments, enhancements, and variations may be made based on what is described and illustrated in this patent document.

Claims

1. A method for processing video data, comprising: Perform a conversion between a video and a bitstream of the video, wherein, according to the rules, the bitstream includes one or more output layer sets, and the output layer sets include one or more video layers, wherein the rules specify that when sub-picture level information (SLI) supplementary enhancement information (SEI) messages and buffering period (BP) SEI messages applied to a specific output layer set exist within an access unit, the SLI SEI messages are in decoding order before the BP SEI messages.

2. The method according to claim 1, wherein, The rules specify, in response to a specific value of a second syntax field indicating the maximum number of time sub-layers indicating an initial codec picture buffer (CPB) removal delay in the buffering period SEI message, to omit a first syntax field indicating sub-layer representation information of syntax elements related to the existence of the initial CPB removal delay.

3. The method according to claim 2, wherein, The specific value is 0.

4. The method according to claim 2, wherein, The second syntax field having the specific value specifies that the maximum number of time sub-layers is 1.

5. The method according to claim 2, wherein, The first syntax field equal to a first value specifies that the syntax elements related to the initial CPB removal delay exist in the sub-layer representation within a range from 0 to the second syntax field and including the second syntax field; 6. The method according to claim 5, wherein, wherein the first value is 1.

7. The method according to claim 2, wherein, The first syntax field equal to a second value specifies that the syntax elements related to the initial CPB removal delay exist in the sub-layer representation corresponding to the specific value of the second syntax field; 8. The method according to claim 7, wherein, wherein the second value is 0.

9. The method according to claim 2, wherein, The rules also specify that in response to the specific value of the second syntax field, it is inferred that the value of the first syntax field is equal to 0.

10. The method according to claim 1, wherein, The conversion includes encoding the video into the bitstream.

11. The method according to claim 1, wherein, The conversion includes decoding the video from the bitstream.

12. An apparatus for processing video data, comprising a processor and a non-transitory memory having instructions thereon, wherein, When executed by the processor, the instructions cause the processor to: Perform a conversion between a video and a bitstream of the video, wherein, according to the rules, the bitstream includes one or more output layer sets, and the output layer sets include one or more video layers, wherein the rules specify that when sub-picture level information (SLI) supplementary enhancement information (SEI) messages and buffering period (BP) SEI messages applied to a specific output layer set exist within an access unit, the SLI SEI messages are in decoding order before the BP SEI messages.

13. The apparatus according to claim 12, wherein, The rules specify, in response to a specific value of a second syntax field indicating the maximum number of time sub-layers indicating an initial codec picture buffer (CPB) removal delay in the buffering period SEI message, to omit a first syntax field indicating sub-layer representation information of syntax elements related to the existence of the initial CPB removal delay.

14. The apparatus according to claim 13, wherein, The specific value is 0.

15. The apparatus according to claim 13, wherein, The second syntax field having the specific value specifies that the maximum number of time sub-layers is 1.

16. The apparatus according to claim 13, wherein, The first syntax field equal to a first value specifies that the syntax elements related to the initial CPB removal delay exist in the sub-layer representation within a range from 0 to the second syntax field and including the second syntax field; wherein the first value is 1.

17. The apparatus according to claim 13, wherein, The first syntax field equal to a second value specifies that the syntax elements related to the initial CPB removal delay exist in the sub-layer representation corresponding to the specific value of the second syntax field; Wherein, the second value is 0.

18. The apparatus according to claim 13, wherein,The rule further specifies that, in response to the specific value of the second syntax field, it is inferred that the value of the first syntax field is equal to 0.

19. A non - transitory computer - readable storage medium stores instructions that cause a processor to: Perform a conversion between a video and a bitstream of the video, wherein, According to the rule, the bitstream includes one or more output layer sets, and the output layer set includes one or more video layers. Wherein, the rule specifies that when the sub-picture level information (SLI) supplementary enhancement information (SEI) message and the buffering period (BP) SEI message applied to a specific output layer set exist within an access unit, the SLI SEI message is before the BP SEI message in decoding order.

20. A non - transitory computer - readable recording medium stores a bitstream of a video, and the bitstream of the video is generated by a method executed by a video processing device, wherein, The method includes: Generating the bitstream of the video. Wherein, according to the rule, the bitstream includes one or more output layer sets, and the output layer set includes one or more video layers. Wherein, the rule specifies that when the sub-picture level information (SLI) supplementary enhancement information (SEI) message and the buffering period (BP) SEI message applied to a specific output layer set exist within an access unit, the SLI SEI message is before the BP SEI message in decoding order.

21. A method for storing a bitstream of a video, comprising: Generating the bitstream of the video. Storing the bitstream in a non-transitory computer-readable recording medium. Wherein, according to the rule, the bitstream includes one or more output layer sets, and the output layer set includes one or more video layers. Wherein, the rule specifies that when the sub-picture level information (SLI) supplementary enhancement information (SEI) message and the buffering period (BP) SEI message applied to a specific output layer set exist within an access unit, the SLI SEI message is before the BP SEI message in decoding order.

Images