Profile, layer and layer indication in video coding
By modifying VVC specifications to optimize parameter signaling and slice type constraints, the encoding and decoding of multiple scalable video layers are enhanced, improving error resilience and bandwidth efficiency in video coding.
Patent Information
- Application Number
- JP2023200299
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Priority Date
- 2019-12-30
- Filing Date
- 2023-11-28
- Publication Date
- 2024-12-09
- Estimated Expiration
- 2040-12-24
AI Technical Summary
Existing scalability designs in Versatile Video Coding (VVC) have constraints that limit the inclusion of independent layers in a multilayer bitstream, leading to inefficiencies and unnecessary signaling of parameters, which affect error resilience and bandwidth utilization.
Modifications to the VVC specifications, including updating constraints on slice types, optimizing parameter signaling, and adjusting HRD parameter handling, to allow flexible and efficient encoding and decoding of multiple scalable video layers without redundant information.
Enhances error resilience and bandwidth efficiency by allowing flexible encoding and decoding of multiple scalable video layers, reducing unnecessary parameter signaling and improving compatibility with various video coding standards.
Smart Images

Figure 0007596495000022 
Figure 0007596495000023 
Figure 0007596495000024
Abstract
Description
[Technical field]
[0001] [CROSS-REFERENCE TO RELATED APPLICATIONS] This application is a divisional application of Japanese Patent Application No. 2022-539112 filed on January 24, 2022, which is based on International Patent Application No. PCT / US2020 / 067015 filed on December 24, 2020, which claims priority and benefit of U.S. Provisional Application No. 62 / 953,854 filed on December 26, 2019 and U.S. Provisional Application No. 62 / 954,907 filed on December 30, 2019. The entire disclosures of all of the above patent applications are incorporated by reference.
[0002] [Technical field] This patent document relates to image and video coding and decoding. [Background technology]
[0003] Digital video accounts for the largest bandwidth usage on the Internet and other digital communications networks, and it is expected that the bandwidth demands for digital video usage will continue to increase as the number of connected user devices capable of receiving and displaying video increases. Summary of the Invention
[0004] This document discloses techniques that can be used by video encoders and decoders to perform video encoding or decoding.
[0005] In one exemplary aspect, a video processing method is disclosed. The method includes performing a conversion between a video including one or more scalable video layers and a bitstream of the video. The video includes one or more video pictures including one or more slices. The bitstream conforms to a format rule. The format rule specifies that if a corresponding network abstraction layer unit type is within a predetermined range and a corresponding video layer flag indicates that a video layer corresponding to the slice does not use inter-layer prediction, a value of a field indicating a slice type of the slice is set to indicate a type of intra slice.
[0006] In another example aspect, a video processing method is disclosed that includes converting between a video including a plurality of video layers and a bitstream of the video, the bitstream including a plurality of output layer sets (OLS), each including one or more of a plurality of scalable video layers, the bitstream conforming to a format rule that specifies that for an OLS having a single layer, a profile-tier-level (PTL) syntax structure indicating a profile, tier and level of the OLS is included in a video parameter set of the bitstream, and a PTL syntax structure of the OLS is also included in a sequence parameter set coded into the bitstream.
[0007] In another example aspect, a video processing method is disclosed that includes converting between a video including a plurality of video layers and a bitstream of the video, the bitstream including a plurality of output layer sets (OLS), each including one or more of the plurality of video layers, the bitstream conforming to a format rule, the format rule specifying a relationship between a number of occurrences of profile-tier-level (PTL) syntax structures in a video parameter set of the bitstream and byte alignment syntax fields in the video parameter set, each PTL syntax structure indicating a profile, tier, and level of one or more of the plurality of OLS.
[0008] In another example aspect, a video processing method is disclosed that includes converting between a video including a plurality of scalable video layers and a bitstream of the video, the bitstream including a plurality of output layer sets (OLS), each including one or more of the plurality of scalable video layers, the bitstream conforming to a format rule that specifies that during encoding, a syntax element indicating an index into a syntax structure describing a profile, layer, and level of one or more of the plurality of OLS is omitted from a video parameter set of the bitstream if the value of the index is zero, or that during decoding, the syntax element is inferred to have a value of zero if not present in the bitstream.
[0009] In another example aspect, a video processing method is disclosed that includes converting between a video including a plurality of video layers and a bitstream of the video, the bitstream including a plurality of output layer sets (OLS), each including one or more of the plurality of video layers, the bitstream conforming to a format rule that specifies that for layer i, where i is an integer, the bitstream includes a set of first syntax elements indicating a first variable indicating whether layer i is included in at least one of the plurality of OLSs.
[0010] In another example aspect, a video processing method is disclosed, the method including performing a conversion between a video and a video bitstream, the bitstream including one or more output layer sets each including one or more video layers, the bitstream conforming to a format rule, the format rule specifying that a number of syntax structures for decoded picture buffer parameters included in a video parameter set of the bitstream is equal to zero if each output layer set includes a single video layer, or equal to one plus a value of a syntax element if it is not true that each output layer set includes a single layer.
[0011] In another example aspect, a video processing method is disclosed, the method including performing a conversion between a video and a video bitstream, the bitstream including a coded video sequence (CVS) including one or more coded video pictures of one or more video layers, the bitstream conforming to a format rule specifying that one or more sequence parameter sets (SPS) indicating conversion parameters referenced by one or more coded pictures of the CVS have the same reference video parameter set (VPS) identifier indicating a reference video parameter set (VPS).
[0012] In another example aspect, a video processing method is disclosed that includes performing a conversion between video and a video bitstream, the bitstream including one or more output layer sets (OLS), each including one or more video layers, the bitstream conforming to a format rule, the format rule specifying whether or how a first syntax element indicating whether a first syntax structure describing parameters of a hypothetical reference decoder (HRD) used for the conversion is included in a video parameter set (VPS) of the bitstream.
[0013] In another example aspect, a video processing method is disclosed that includes performing a conversion between video and a video bitstream, the bitstream including one or more output layer sets (OLS), each including one or more video layers, the bitstream conforming to a format rule that specifies whether or how a first syntax structure describing general hypothetical reference decoder (HRD) parameters and a second syntax structure describing OLS-specific HRD parameters are included in a video parameter set (VPS) of the bitstream.
[0014] In yet another exemplary aspect, a video encoder apparatus is disclosed, the video encoder including a processor configured to implement the above-described method.
[0015] In yet another exemplary aspect, a video decoder apparatus is disclosed, the video decoder including a processor configured to implement the above-described method.
[0016] In yet another exemplary aspect, a computer readable medium is disclosed having code stored thereon, the code embodying one of the methods described herein in the form of processor executable code.
[0017] In yet another exemplary aspect, a method for writing a bitstream generated according to one of the above methods to a computer-readable medium is disclosed.
[0018] In another exemplary aspect, a computer readable medium storing a video bitstream generated according to the above-described method is disclosed.
[0019] These and other features are described throughout this document. [Brief description of the drawings]
[0020] [Figure 1] 1 is a block diagram illustrating a video coding system according to some embodiments of the present disclosure. [Diagram 2] FIG. 1 is a block diagram of an exemplary hardware platform used for video processing. [Diagram 3] 4 is a flow chart for an exemplary method of video processing. [Figure 4] 1 is a block diagram illustrating an example video coding system. [Diagram 5] 1 is a block diagram illustrating an encoder according to some embodiments of the present disclosure. [Figure 6] 2 is a block diagram illustrating a decoder according to some embodiments of the present disclosure. [Figure 7A] 1 is a flowchart of various example video processing methods. [Figure 7B] 1 is a flowchart of various example video processing methods. [Figure 7C] 1 is a flowchart of various example video processing methods. [Figure 7D]1 is a flowchart of various example video processing methods. [Figure 7E] 1 is a flowchart of various example video processing methods. [Figure 7F] 1 is a flowchart of various example video processing methods. [Figure 7G] 1 is a flowchart of various example video processing methods. [Figure 7H] 1 is a flowchart of various example video processing methods. [Figure 7I] 1 is a flowchart of various example video processing methods. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0021] Section headings are used in this document for ease of understanding and are not intended to limit the applicability of the techniques and embodiments disclosed in each section to only that section. Furthermore, H.266 terminology is used in some descriptions for ease of understanding only and is not intended to limit the scope of the disclosed techniques. Thus, the techniques described herein are applicable to other video codec protocols and designs.
[0022] 1. Summary This document relates to video coding technology, and in particular to various improvements in scalable video coding, where a video bitstream can contain more than one layer. The concepts may be applied, individually or in various combinations, to any video coding standard or non-standard video codec that supports multi-layer video coding, such as the currently under development Versatile Video Coding (VVC).
[0023] 2. Abbreviations APS Adaptation Parameter Set AU Access Unit AUD Access Unit Delimiter AVC Advanced Video Coding CLVS Coded Layer Video Sequence CPB Coded Picture Buffer CRA Clean Random Access CTU Coding Tree Unit CVS Coded Video Sequence DPB Decoded Picture Buffer DPS Decoding Parameter Set EOB End Of Bitstream EOS End Of Sequence GDR Gradual Decoding Refresh HEVC High Efficiency Video Coding HRD Hypothetical Reference Decoder IDR Instantaneous Decoding Refresh JEM Joint Exploration Model MCTS Motion-Constrained Tile Sets NAL Network Abstraction Layer OLS Output Layer Set PH Picture Header PPS Picture Parameter Set PTL Profile, Tier and Level PU Picture Unit RBSP Raw Byte Sequence Payload SEI Supplemental Enhancement Information SPS Sequence Parameter Set SVC Scalable Video Coding VCL Video Coding Layer VPS Video Parameter Set VTM VVC Test Model VUI Video Usability Information VVC Versatile Video Coding
[0024] 3. Initial discussion Video coding standards have evolved primarily through the development of well-known ITU-T and ISO / IEC standards. ITU-T created H.261 and H.263, ISO / IEC created MPEG-1 and MPEG-4 Visual, and the two organizations jointly created the H.262 / MPEG-2 Video and H.264 / MPEG-4 Advanced Video Coding (AVC) and H.265 / HEVC[1] standards. Starting with H.262, video coding standards are based on a hybrid video coding structure in which transform coding is utilized in addition to temporal prediction. In 2015, the Joint Video Exploration Team (JVET) was jointly established by VCEG and MPEG to explore future video coding technologies beyond HEVC. Since then, many new methods have been adopted by JVET and put into a reference software named the Joint Exploration Model (JEM)[2]. Currently, JVET meetings are held quarterly, and the new coding standard aims to achieve a 50% bitrate reduction compared to HEVC. The new video coding standard was officially named Versatile Video Coding (VVC) at the April 2018 JVET meeting, when the first version of the VVC test model (VTM) was released. As continuous efforts are being made to contribute to VVC standardization, new coding techniques are adopted into the VVC standard at each JVET meeting. Since then, the VVC working draft and test model VTM are updated after each meeting. The VVC project is currently aiming for technical completion (FDIS) at the July 2020 meeting.
[0025] 3.1. General scalable video coding (SVC) Scalable video coding (SVC) refers to video coding in which a base layer (BL), sometimes called a reference layer (RL), and one or more scalable enhancement layers (EL) are used. In SVC, the base layer can carry video data at a base level of quality. One or more enhancement layers can carry additional video data, for example, to support higher spatial, temporal, and / or signal-to-noise (SNR) levels. Enhancement layers may be defined relative to previously coded layers. For example, a lower layer may function as a BL and an upper layer may function as an EL. An intermediate layer may function as either an EL or an RL, or both. For example, an intermediate layer (e.g., a layer that is neither the lowest nor the highest layer) may be an EL for a layer below the intermediate layer, e.g., the base layer or any intervening enhancement layer, and may simultaneously function as a RL for one or more enhancement layers above the intermediate layer. Similarly, in multiview or 3D extensions of the HEVC standard, there may be multiple views and information of one view may be utilized to code (e.g., encode or decode) information of other views (e.g., motion estimation, motion vector prediction, and / or other redundancy).
[0026] In SVC, parameters used by an encoder or decoder are grouped into parameter sets based on the coding level at which they may be utilized (e.g., video level, sequence level, picture level, slice level, etc.). For example, parameters that may be utilized by one or more coding video sequences of different layers in a bitstream may be included in a video parameter set (VPS), and parameters utilized by one or more pictures in a coding video sequence may be included in a sequence parameter set (SPS). Similarly, parameters utilized by one or more slices in a picture may be included in a picture parameter set (PPS), and other parameters specific to a single slice may be included in a slice header. Similarly, an indication of which parameter set a particular layer is using at a given time may be provided at various coding levels.
[0027] 3.2. Parameter Set AVC, HEVC and VVC specify parameter sets. Parameter set types include SPS, PPS, APS, VPS and DPS. SPS and PPS are all supported by AVC, HEVC and VVC. VPS was introduced by HEVC and is included in both HEVC and VVC. APS and DPS are not included in AVC or HEVC but are included in the latest VVC draft text.
[0028] The SPS is designed to carry sequence-level header information, and the PPS is designed to carry picture-level header information that changes infrequently. With the SPS and PPS, redundant signaling of infrequently changing information does not need to be repeated for every sequence or picture, thus avoiding the need for redundant signaling of such information. Furthermore, the use of the SPS and PPS allows out-of-band transmission of important header information, which not only avoids the need for redundant transmission, but also improves error resilience.
[0029] The VPS is introduced to carry sequence-level header information that is common to all layers of a multi-layer bitstream.
[0030] APS is introduced to carry picture or slice level information that requires significant bits to code, can be shared by multiple pictures, and can exist in many different variations in a sequence.
[0031] DPS is introduced to carry bitstream-level information that indicates the maximum capacity required to decode the entire bitstream.
[0032] 3.3. VPS Syntax and Semantics in VVC VVC supports scalability, also known as scalable video coding, where multiple layers can be encoded in one coding video bitstream.
[0033] In the latest VVC text, scalability information is signaled in the VPS, with the following syntax and semantics: 7.3.2.2 Video parameter set syntax [Table 1] TIFF0007596495000002.tif240168TIFF0007596495000003.tif221667.4.3.2 Semantics of video parameter set RBSP The VPS RBSP shall be available to the decoding process before it is referenced and shall either be contained in at least one AU with TemporalId equal to 0 or be provided through external means. All VPS NAL units with a particular value of vps_video_parameter_set_id in the CVS shall have the same content. The vps_video_parameter_set_id provides an identifier for the VPS for reference by other syntax elements. The value of vps_video_parameter_set_id shall be greater than 0. vps_max_layers_minus1 plus 1 specifies the maximum number of layers allowed in each CVS that references the VPS. vps_max_sublayers_minus1 plus 1 specifies the maximum number of temporal sublayers that may exist in each CVS that references the VPS. The value of vps_max_sublayers_minus1 shall be in the range 0 to 6, inclusive. vps_all_layers_same_num_sublayers_flag equal to 1 specifies that the number of temporal sublayers is the same for all layers in each CVS that references the VPS. vps_all_layers_same_num_sublayers_flag equal to 0 specifies that layers in each CVS that references the VPS may or may not have the same number of temporal sublayers. If not present, the value of vps_all_layers_same_num_sublayers_flag is inferred to be equal to 1. vps_all_independent_layers_flag equal to 1 specifies that all layers in the CVS are coded independently, without using inter-layer prediction. vps_all_independent_layers_flag equal to 0 specifies that one or more of the layers in the CVS may use inter-layer prediction. If not present, the value of vps_all_independent_layers_flag is inferred to be equal to 1. If vps_all_independent_layers_flag is equal to 1, the value of vps_independent_layer_flag[i] is inferred to be equal to 1. If vps_all_independent_layers_flag is equal to 0, the value of vps_independent_layer_flag[0] is inferred to be equal to 1. vps_layer_id[i] specifies the nuh_layer_id value of the i-th layer. For any two non-negative integer values of m and n, if m is less than n, then the value of vps_layer_id[m] shall be less than vps_layer_id[n]. vps_independent_layer_flag[i] equal to 1 specifies that the layer with index i does not use inter-layer prediction. vps_independent_layer_flag[i] equal to 0 specifies that the layer with index i may use inter-layer prediction and that the syntax element vps_direct_ref_layer_flag[i][j], for j in the range 0 to i-1, is present in the VPS. If not present, the value of vps_independent_layer_flag[i] is inferred to be equal to 1. vps_direct_ref_layer_flag[i][j] equal to 0 specifies that the layer with index j is not a direct reference layer of the layer with index i. vps_direct_ref_layer_flag[i][j] equal to 1 specifies that the layer with index j is a direct reference layer of the layer with index i. For i and j in the range 0 to vps_max_layers_minus1, inclusive, if vps_direct_ref_layer_flag[i][j] is not present, it is inferred to be equal to 0. If vps_independent_layer_flag[i] is equal to 0, then there shall be at least one value of j in the range 0 to i-1, inclusive, such that vps_direct_ref_layer_flag[i][j] is equal to 1. The variables NumDirectRefLayers[i], DirectRefLayerIdx[i][d], NumRefLayers[i], RefLayerIdx[i][r], and LayerUsedAsRefLayerFlag[j] are derived as follows:
number
number
number
number
number
number
[0034] 3.4. SPS Syntax and Semantics in VVC In the latest VVC draft text in JVET-P2001-v14, the SPS syntax and semantics most relevant to the invention herein are as follows: 7.3.2.3 Sequence parameter set RBSP syntax [Table 2] 7.4.3.3 Semantics of the sequence parameter set RBSP The SPS RBSP shall be available to the decoding process before it is referenced and shall either be contained in at least one AU with TemporalId equal to 0 or be provided through external means. All SPS NAL units with a particular value of sps_seq_parameter_set_id in the CVS shall have the same content. When sps_decoding_parameter_set_id is greater than 0, it specifies the value of dps_decoding_parameter_set_id of the DPS referenced by the SPS. When sps_decoding_parameter_set_id is equal to 0, the SPS does not reference a DPS, and the DPS is not referenced when decoding each CLVS with reference to the SPS. The value of sps_decoding_parameter_set_id shall be the same in all SPSs referenced by coding pictures in the bitstream. If sps_video_parameter_set_id is greater than 0, it specifies the value of vps_video_parameter_set_id of the VPS referenced by the SPS. If sps_video_parameter_set_id is equal to 0, the following applies: -SPS does not refer to VPS. When each CLVS is decoded by referring to the -SPS, the VPS is not referred to. The value of -vps_max_layers_minus1 is inferred to be equal to 0. - A CVS shall contain only one layer (i.e., all VCL NAL units in the CVS shall have the same value of nuh_layer_id). -The value of GeneralLayerIdx[nuh_layer_id] is inferred to be equal to 0. The value of -vps_independent_layer_flag[GeneralLayerIdx[nuh_layer_id]] is inferred to be equal to 1. If vps_independent_layer_flag[GeneralLayerIdx[nuh_layer_id]] is equal to 1, then any SPS referenced by a CLVS with a particular nuh_layer_id value nuhLayerId shall have nuh_layer_id equal to nuhLayerId. sps_max_sublayers_minus1 plus 1 specifies the maximum number of temporal sublayers that may exist in each CLVS that references an SPS. The value of sps_max_sublayers_minus1 shall be in the range 0 to vps_max_sublayers_minus1, inclusive. sps_reserved_zero_4bits shall be equal to 0 in bitstreams conforming to this version of this specification. Other values of sps_reserved_zero_4bits are reserved for future use by ITU-T|ISO / IEC. sps_ptl_dpb_hrd_params_present_flag equal to 1 specifies that the profile_tier_level() and dpb_parameters() syntax structures are present in the SPS, and that the general_hrd_parameters() and ols_hrd_parameters() syntax structures may also be present in the SPS. sps_ptl_dpb_hrd_params_present_flag equal to 0 specifies that none of these syntax structures are present in the SPS. The value of sps_ptl_dpb_hrd_params_present_flag shall be equal to vps_independent_layer_flag[nuh_layer_id]. If vps_independent_layer_flag[GeneralLayerIdx[nuh_layer_id]] is equal to 1, then the variable MaxDecPicBuffMinus1 is set equal to max_dec_pic_buffering_minus1[sps_max_sublayers_minus1] in the dpb_parameters() syntax structure in the SPS. Otherwise, MaxDecPicBuffMinus1 is set equal to max_dec_pic_buffering_minus1[sps_max_sublayers_minus1] in the dpb_parameters() syntax structure for layer_nonoutput_dpb_params_idx[GeneralLayerIdx[nuh_layer_id]] in the VPS.
[0035] 3.5. Slice Header Syntax and Semantics in VVC In the latest VVC draft text in JVET-P2001-v14, the slice header syntax and semantics most relevant to the invention herein are as follows: 7.3.7.1 General slice header syntax [Table 3] 7.4.8.1 General slice header semantics slice_type specifies the coding type of the slice according to Table 9. [Table 4] If nal_unit_type is a value of nal_unit_type in the range from IDR_W_RADL to CRA_NUT, inclusive, and the current picture is the first picture in the access unit, slice_type shall be equal to 2.
[0036] 4. Technical problem solved by the described technical solution The existing scalability design in VVC has the following problems. 1) The latest VVC draft text includes the following constraints on slice_type: If nal_unit_type is a value of nal_unit_type in the range from IDR_W_RADL to CRA_NUT, inclusive, and the current picture is the first picture in the AU, slice_type shall be equal to 2. slice_type equal to 2 for a slice means that the slice is intra-coded without using inter prediction from a reference picture. However, in an AU, not only must the first picture in the AU and be an IRAP picture contain only intra-coded slices, but all IRAP pictures in all independent layers must contain only intra-coded slices. Therefore, the above constraints need to be updated. 2) If the syntax element ols_ptl_idx[i] is not present, its value must still be used, but if it is not present, there is no guessing of the value of ols_ptl_idx[i]. 3) Signaling of the syntax element layer_output_dpb_params_idx[i] is not required if the i layer is not used as any output layer of the OLS. 4) The value of the syntax element vps_num_dpb_params can be equal to 0. However, if vps_all_independent_layers_flag is equal to 0, there must be at least one dpb_parameters() syntax structure in the VPS. 5) In the latest VVC draft text, the PTL information of an OLS that contains only one layer is signaled only in the SPS, since it is an independently coded layer without reference to other layers. However, for the purpose of session negotiation, it is desirable to signal the PTL information of all OLSs in the bitstream in the VPS. 6) The vps_ptl_byte_alignment_zero_bit syntax element is signaled unnecessarily if the number of PTL syntax structures signaled in the VPS is zero. 7) The semantics of sps_video_parameter_set_id are subject to the following constraints: If sps_video_parameter_set_id is equal to 0, the CVS shall contain only one layer (i.e., all VCL NAL units in the CVS shall have the same value of nuh_layer_id). However, this constraint does not allow independent layers that do not reference a VPS to be included in the multi-layer bitstream. sps_video_parameter_set_id equal to 0 means that no SPS (and therefore no layers) reference a VPS. 8) The value of vps_general_hrd_params_present_flag MAY be equal to 1 if each_layer_is_an_ols_flag is equal to 1. However, if each_layer_is_an_ols_flag is equal to 1, HRD parameters are only signaled in the SPS, and therefore the value of vps_general_hrd_params_present_flag shall not be equal to 1. In some cases, it may not make sense to signal a general_hrd_parameter() syntax structure while signaling a zero old_hrd_parameters() syntax structure in the VPS. 9) The HRD parameters of an OLS that includes only one layer are signaled in both the VPS and the SPS, but it is not useful to repeat the HRD parameters in the VPS of an OLS that includes only one layer.
[0037] 5. Exemplary embodiments and techniques In order to solve the above problems and others, the methods summarized below are disclosed. The inventions should be considered as examples to illustrate the general concept and should not be interpreted in a narrow sense. Moreover, these inventions can be applied alone or in any combination. 1) To solve the first problem, the following constraints are specified: If nal_unit_type is in the range IDR_W_RADL <= CRA_NUT <= and vps_independent_layer_flag[GeneralLayerIdx[nuh_layer_id]] is equal to 1, then slice_type shall be equal to 2. 2) To solve the second problem, for each possible value of i, the value of ols_ptl_idx[i] is presumed to be equal to 0 if the syntax element is not present. 3) To solve the third problem, the variable LayerUsedAsOutputLayerFlag[i] is specified to indicate whether the i layer is used as an output layer in any OLS, and if the variable is equal to 0, the signaling of the variable layer_output_dpb_params_idx[i] is avoided. Additionally, the following constraint may be further specified: For each value of i in the range 0 to vps_max_layers_minus1, inclusive, the values of LayerUsedAsRefLayerFlag[i] and LayerUsedAsOutputLayerFlag[i] shall not both be equal to 0. In other words, there shall be no layer that is neither a direct reference layer of any other layer nor an output layer of at least one OLS. 4) To solve the fourth problem, change vps_num_dpb_params to vps_num_dpb_params_minus1 and specify VpsNumDpbParams as follows:
number
[0038] 6. Embodiments Below are some example embodiments of the aspects summarized above in Section 5, as applicable to the VVC specification. The modified text is based on the latest VVC text in JVET-P2001-v14. Most relevant parts that have been added or changed are highlighted in bold underline (or underline), and some parts that have been removed are highlighted in bold italics (or [[]]). There are some other changes that are not highlighted, as they are editorial in nature. 6.1. First embodiment 6.1.1. VPS Syntax and Semantics 7.3.2.2 Video parameter set syntax [Table 5] TIFF0007596495000015.tif243167TIFF0007596495000016.tif751687.4.3.2 Semantics of video parameter set RBSP ols_output_layer_flag[i][j] equal to 1 specifies that the layer with nuh_layer_id equal to vps_layer_id[j] is the output layer of the i-th OLS when ols_mode_idc is equal to 2. ols_output_layer_flag[i][j] equal to 0 specifies that the layer with nuh_layer_id equal to vps_layer_id[j] is not the output layer of the i-th OLS when ols_mode_idc is equal to 2. The variable NumOutputLayersInOls[i], which specifies the number of output layers in the i-th OLS, and the variable OutputLayerIdInOls[i][j], which specifies the nuh_layer_id value of the j-th output layer in the i-th OLS, are derived as follows:
number
number
number
number
[0039] FIG. 1 is a block diagram illustrating an example video processing system 1900 in which various techniques disclosed herein may be implemented. Various implementations may include some or all of the components of system 1900. System 1900 may include an input 1902 for receiving video content. The video content may be received in raw or uncompressed format, for example, 8-bit or 10-bit multi-component pixel values, or may be in a compressed or encoded format. Input 1902 may represent a network interface, a peripheral bus interface, or a storage interface. Examples of network interfaces include wired interfaces such as Ethernet, passive optical network (PON), etc., and wireless interfaces such as Wi-Fi or cellular interfaces.
[0040] The system 1900 may include a coding component 1904 that may implement various coding or encoding methods described herein. The coding component 1904 may reduce the average bit rate of the video from the input 1902 to the output of the coding component 1904 and generate a coded representation of the video. Thus, the coding techniques are sometimes referred to as video compression or video transcoding techniques. The output of the coding component 1904 may be stored or transmitted via a communication connected as represented by component 1906. The stored or communicated bitstream (or coded) representation of the video received at the input 1902 may be used by component 1908 to generate pixel values or displayable video that are transmitted to a display interface 1910. The process of generating user-viewable video from a bitstream is sometimes referred to as video decompression. Additionally, although certain video processing operations are referred to as "coding" operations or tools, it is recognized that the coding tools or operations are used in an encoder and a corresponding decoding tool or operation that is the inverse of the result of the coding is performed in a decoder.
[0041] Examples of peripheral bus interfaces or display interfaces may include universal serial bus (USB) or high definition multimedia interface (HDMI) or Displayport, etc. Examples of storage interfaces include serial advanced technology attachment (SATA), PCI, IDE interfaces, etc. The techniques described in this document may be embodied in various electronic devices such as mobile phones, laptops, smartphones, or other devices capable of digital data processing and / or video display.
[0042] FIG. 2 is a block diagram of a video processing device 3600. The device 3600 may be used to implement one or more of the methods described herein. The device 3600 may be embodied in a smartphone, tablet, computer, or Internet of Things (IoT) receiver. The device 3600 may include one or more processors 3602, one or more memories 3604, and video processing hardware 3606. The processor 3602 may be configured to execute one or more of the methods described herein. The memory 3604 may be used to store data and code used to implement the methods and techniques described herein. The video processing hardware 3606 may be used to implement some of the techniques described herein in hardware circuitry. In some implementations, the hardware 3606 may be partially or completely located within the processor 3602, such as a graphics processor.
[0043] FIG. 4 is a block diagram illustrating an example video coding system 100 that may utilize the techniques of this disclosure.
[0044] 4, video coding system 100 may include a source device 110 and a destination device 120. Source device 110 may generate encoded video data and may be referred to as a video encoding device. Destination device 120 may decode the encoded video data generated by source device 110 and may be referred to as a video decoding device.
[0045] The source device 110 may include a video source 112, a video encoder 114, and an input / output (I / O) interface 116.
[0046] The video source 112 may include sources such as a video capture device, an interface for receiving video data from a video content provider, and / or a computer graphics system for generating video data, or a combination of such sources. The video data may include one or more pictures. The video encoder 114 encodes the video data from the video source 112 to generate a bitstream. The bitstream may include a sequence of bits that form a coded representation of the video data. The bitstream may include coding pictures and associated data. A coding picture is a coded representation of a picture. The associated data may include sequence parameter sets, picture parameter sets, and other syntax structures. The I / O interface 116 may include a modulator / demodulator (modem) and / or a transmitter. The coded video data may be transmitted via the I / O interface 116 directly to the destination device 120 over the network 130a. The coded video data may also be stored in a storage medium / server 130b for access by the destination device 120.
[0047] The destination device 120 may include an I / O interface 126 , a video decoder 124 , and a display device 122 .
[0048] I / O interface 126 may include a receiver and / or a modem. I / O interface 126 may obtain encoded video data from source device 110 or storage medium / server 130b. Video decoder 124 may decode the encoded video data. Display device 122 may display the decoded video data to a user. Display device 122 may be integrated with destination device 120 or may be external to destination device 120 configured to interface with an external display device.
[0049] Video encoder 114 and video decoder 124 may operate according to a video compression standard, such as the High Efficiency Video Coding (HEVC) standard, the Versatile Video Coding (VVC) standard, and other current and / or future standards.
[0050] FIG. 5 is a block diagram illustrating an example of a video encoder 200, which may be the video encoder 114 in the system 100 shown in FIG.
[0051] Video encoder 200 may be configured to perform any or all of the techniques described in this disclosure. In the example of FIG. 5, video encoder 200 includes multiple functional components. The techniques described in this disclosure may be shared among various components of video encoder 200. In some examples, a processor may be configured to perform any or all of the techniques described in this disclosure.
[0052] The functional components of the video encoder 200 may include a division unit 201, a prediction unit 202 which may include a mode selection unit 203, a motion estimation unit 204, a motion compensation unit 205, and an intra prediction unit 206, a residual generation unit 207, a transform unit 208, a quantization unit 209, an inverse quantization unit 210, an inverse transform unit 211, a reconstruction unit 212, a buffer 213, and an entropy coding unit 214.
[0053] In other examples, video encoder 200 may include more, fewer, or different functional components. In one example, prediction unit 202 may include an intra block copy (IBC) unit. The IBC unit may perform prediction in an IBC mode, where at least one reference picture is the picture in which the current video block is located.
[0054] Furthermore, some components, such as the motion estimation unit 204 and the motion compensation unit 205, although depicted separately for illustrative purposes in the example of FIG. 5, may be highly integrated.
[0055] Division unit 201 may divide a picture into one or more video blocks. Video encoder 200 and video decoder 300 may support a variety of video block sizes.
[0056] The mode selection unit 203 may select one of the coding modes, such as intra or inter, for example, based on the error result, and provide the resulting intra- or inter-coded block to the residual generation unit 207 for generating residual block data and to the reconstruction unit 212 for reconstructing the coding block for use as a reference picture. In some examples, the mode selection unit 203 may select a combination of intra and inter prediction (CIIP) mode, in which prediction is based on an inter prediction signal and an intra prediction signal. The mode selection unit 203 may also select the resolution of the motion vector for the block (e.g., sub-pixel or integer pixel precision) in the case of inter prediction.
[0057] To perform inter prediction on the current video block, motion estimation unit 204 may generate motion information for the current video block by comparing one or more reference frames from buffer 213 to the current video block. Motion compensation unit 205 may determine a prediction video block for the current video block based on the motion information and decoded samples of pictures from buffer 213 other than the picture associated with the current video block.
[0058] Motion estimation unit 204 and motion compensation unit 205 may perform different operations on a current video block depending on whether the current video block is in an I slice, a P slice, or a B slice, for example.
[0059] In some examples, motion estimation unit 204 may perform unidirectional prediction for the current video block, and motion estimation unit 204 may search reference pictures of list 0 or list 1 for a reference video block for the current video block. Motion estimation unit 204 may then generate a reference index indicating a reference picture in list 0 or list 1 that contains the reference video block, and a motion vector indicating a spatial displacement between the current video block and the reference video block. Motion estimation unit 204 may output the reference index, the prediction direction indicator, and the motion vector as motion information for the current video block. Motion compensation unit 205 may generate a predictive video block for the current block based on the reference video block indicated by the motion information of the current video block.
[0060] In other examples, motion estimation unit 204 may perform bidirectional prediction for the current video block, and motion estimation unit 204 may search reference pictures in list 0 for reference video blocks for the current video block and may also search reference pictures in list 1 for other reference video blocks for the current video block. Motion estimation unit 204 may then generate reference indexes indicating reference pictures in list 0 and list 1 that contain reference video blocks, and motion vectors indicating spatial displacement between the reference video blocks and the current video block. Motion estimation unit 204 may output the reference indexes and the motion vector for the current video block as motion information for the current video block. Motion compensation unit 205 may generate a predictive video block for the current block based on the reference video blocks indicated by the motion information of the current video block.
[0061] In some examples, the motion estimation unit 204 may output a complete set of motion information for the decoding process of the decoder.
[0062] In some examples, motion estimation unit 204 may not output a complete set of motion information for the current video. Rather, motion estimation unit 204 may signal the motion information of the current video block with reference to motion information of other video blocks. For example, motion estimation unit 204 may determine that the motion information of the current video block is sufficiently similar to the motion information of a neighboring video block.
[0063] In one example, motion estimation unit 204 may indicate in a syntax structure associated with the current video block a value that indicates to video decoder 300 that the current video block has the same motion information as other video blocks.
[0064] In other examples, motion estimation unit 204 may identify in a syntax structure associated with the current video block another video block and a motion vector difference (MVD). The motion vector difference indicates a difference between the motion vector of the current video block and the motion vector of the indicated video block. Video decoder 300 may use the motion vector of the indicated video block and the motion vector difference to determine the motion vector of the current video block.
[0065] As mentioned above, video encoder 200 may predictively signal motion vectors. Two examples of predictive signaling techniques that may be implemented by video encoder 200 include advanced motion vector predication (AMVP) and merge mode signaling.
[0066] Intra prediction unit 206 may perform intra prediction on the current video block. When intra prediction unit 206 performs intra prediction on the current video block, intra prediction unit 206 may generate predictive data for the current video block based on decoded samples of other video blocks in the same picture. The predictive data for the current video block may include a predictive video block and various syntax elements.
[0067] Residual generation unit 207 may generate residual data for the current video block by subtracting (e.g., as indicated by a minus sign) the prediction video block of the current video block from the current video block. The residual data for the current video block may include residual video blocks that correspond to different sample components of the samples in the current video block.
[0068] In other examples, for the current video block, e.g., in skip mode, residual data for the current video block may not be present and residual generation unit 207 may not perform a subtraction operation.
[0069] Transform processing unit 208 may generate one or more transform coefficient video blocks for the current video block by applying one or more transforms to a residual video block associated with the current video block.
[0070] After transform processing unit 208 generates a transform coefficient video block associated with the current video block, quantization unit 209 may quantize the transform coefficient video block associated with the current video block based on one or more quantization parameter (QP) values associated with the current video block.
[0071] Inverse quantization unit 210 and inverse transform unit 211 may apply inverse quantization and inverse transformation, respectively, to the transform coefficient video block to reconstruct a residual video block from the transform coefficient video block. Reconstruction unit 212 may add the reconstructed residual video block to corresponding samples from one or more prediction video blocks generated by prediction unit 202 to generate a reconstructed video block related to the current block, which is stored in buffer 213.
[0072] After reconstruction unit 212 reconstructs the video blocks, a loop filtering operation may be performed to reduce video blocking artifacts in the video blocks.
[0073] The entropy encoding unit 214 may receive data from other functional components of the video encoder 200. If the entropy encoding unit 214 receives the data, the entropy encoding unit 214 may perform one or more entropy encoding operations to generate entropy-coded data and output a bitstream that includes the entropy-coded data.
[0074] FIG. 6 is a block diagram illustrating an example of a video decoder 300, which may be the video decoder 114 in the system 100 shown in FIG.
[0075] The video decoder 300 may be configured to perform any or all of the techniques described in this disclosure. In the example of FIG. 5, the video decoder 300 includes multiple functional components. The techniques described in this disclosure may be shared among various components of the video decoder 300. In some examples, a processor may be configured to perform any or all of the techniques described in this disclosure.
[0076] 6, the video decoder 300 includes an entropy decoding unit 301, a motion compensation unit 302, an intra prediction unit 303, an inverse quantization unit 304, an inverse transform unit 305, a reconstruction unit 306, and a buffer 307. The video decoder 300 may, in some examples, perform a decoding pass that is generally inverse to the encoding pass described with respect to the video encoder 200 (FIG. 5).
[0077] The entropy decoding unit 305 may obtain an encoded bitstream. The encoded bitstream may include entropy coded video data (e.g., coded blocks of video data). The entropy decoding unit 301 may decode the entropy coded video data, and from the entropy decoded video data, the motion compensation unit 302 may determine motion information including motion vectors, motion vector precision, reference picture list indexes, and other motion information. The motion compensation unit 302 may determine such information, for example, by implementing AMVP and merge mode.
[0078] The motion compensation unit 302 may optionally perform interpolation based on an interpolation filter to generate the motion compensated blocks. An identifier for the interpolation filter used with sub-pixel precision may be included in the syntax element.
[0079] Motion compensation unit 302 may calculate interpolated values for sub-integer pixels of the reference block using the interpolation filters used by video encoder 200 during encoding of the video block. Motion compensation unit 302 may determine the interpolation filters used by video encoder 200 according to received syntax information and use the interpolation filters to generate the predictive block.
[0080] The motion compensation unit 302 may use some of the syntax information to determine the size of the blocks used to code the frames and / or slices of the coded video sequence, partition information describing how each macroblock of a picture of the coded video sequence is divided, a mode indicating how each partition is coded, one or more reference frames (and reference frame lists) for each inter-coded block, and other information for decoding the coded video sequence.
[0081] The intra prediction unit 303 may use, for example, an intra prediction mode received in the bitstream to form a prediction block from spatially neighboring blocks. The inverse quantization unit 303 inverse quantizes, i.e., dequantizes, the quantized video block coefficients provided in the bitstream and decoded by the entropy decoding unit 301. The inverse transform unit 303 applies an inverse transform.
[0082] Reconstruction unit 306 may add the residual blocks with corresponding prediction blocks generated by motion compensation unit 202 or intra prediction unit 303 to form decoded blocks. If desired, a deblocking filter may also be applied to filter the decoded blocks to remove block artifacts. The decoded video blocks are then stored in buffer 307, which provides reference blocks for later motion compensation / intra prediction and also generates decoded video for presentation on a display device.
[0083] Next, we provide a list of solutions chosen according to some embodiments.
[0084] The following solutions represent exemplary embodiments of the techniques discussed in the previous section (eg, item 1).
[0085] 1. A method of video processing (e.g., method 600 in FIG. 3), comprising: A method comprising: performing a conversion between a video including one or more scalable video layers and a coding representation of the video, the coding representation conforming to a format rule, the format rule specifying that if a corresponding network abstraction layer unit type is within a predetermined range and a corresponding video layer flag indicates that a video layer corresponding to the slice does not use inter-layer prediction, a value of a field indicating a slice type of the slice is set to indicate a type of intra slice.
[0086] 2. The slice type of the slice is set to 2, as in Solution 1.
[0087] 3. The method according to Solution 1, wherein the predetermined range is greater than or equal to IDR_W_RADL and less than or equal to CRA_NUT.
[0088] 4. A method of video processing comprising: 1. A method comprising: performing a conversion between a video including one or more scalable video layers and a coding representation of the video, the coding representation conforming to a format rule, the format rule specifying that if an index is excluded from the coding representation, an indication of the index into a syntax structure describing a profile and layer level of the one or more scalable video layers is presumed to be equal to 0.
[0089] 5. The method according to solution 4, wherein the instruction includes an ols_ptl_idx[i] syntax element, where i is an integer.
[0090] 6. A method of video processing comprising: 1. A method comprising: performing a conversion between a video including one or more scalable video layers and a coding representation of the video, the coding representation including a plurality of output layer sets (OLS), each OLS being a set of layers in the coding representation that are specified to be output, the coding representation conforming to format rules, the format rules specifying that if the OLS includes a single layer, a video parameter set of the coding representation is allowed to have repeated information regarding a profile, layer and level of the OLS.
[0091] 7. The method according to solution 6, wherein the format rule specifies signaling a number field indicating the total number of sets of information on multiple OLS profiles, layers and levels, the total number being at least 1.
[0092] 8. A method according to any of solutions 1 to 7, wherein the step of performing a transformation comprises a step of encoding the video to generate a coding representation.
[0093] 9. A method according to any of solutions 1 to 7, wherein the step of performing the conversion comprises parsing and decoding the coding representation to generate the video.
[0094] 10. A video decoding device comprising a processor configured to implement the methods according to one or more of solutions 1 to 9.
[0095] 11. A video encoding device comprising a processor configured to implement the methods according to one or more of solutions 1 to 9.
[0096] 12. A computer program product having computer code stored therein, A computer program product, the code being executed by a processor, causing the processor to perform a method according to any one of solutions 1 to 9.
[0097] 13. A method, apparatus or system as described in this document.
[0098] With respect to FIGS. 7A-7I, the following listed solutions may be preferably implemented in some embodiments.
[0099] For example, the following solution may be implemented in accordance with item 1 of the previous section.
[0100] 1. A method of video processing (e.g., method 710 shown in FIG. 7A), comprising: A method comprising: performing (712) a conversion between a video including one or more scalable video layers and a bitstream of the video, the video including one or more video pictures including one or more slices, the bitstream conforming to a format rule, the format rule specifying that if a corresponding network abstraction layer unit type is within a predetermined range and a corresponding video layer flag indicates that the video layer corresponding to the slice does not use inter-layer prediction, then a value of a field indicating a slice type of the slice is set to indicate a type of intra slice.
[0101] 2. The slice type of the slice is set to 2, as in Solution 1.
[0102] 3. The method according to Solution 1, wherein the predetermined range is equal to or greater than IDR_W_RADL and equal to or less than CRA_NUT, where IDR_W_RADL indicates a slice having an instantaneous decoding refresh type and CRA_NUT indicates a slice having a clean random access type.
[0103] 4. A method according to any one of Solutions 1 to 3, wherein the corresponding video layer flag corresponds to vps_independent_layer_flag[GeneralLayerIdx[nuh_layer_id]], where nuh_layer_id corresponds to an identifier of the video layer, GeneralLayerIdx[] is an index of the video layer, and vps_independent_layer_flag corresponds to the video layer flag.
[0104] 5. A method according to any of solutions 1 to 4, wherein the step of performing a conversion comprises a step of encoding the video into a bitstream.
[0105] 6. A method according to any of Solutions 1 to 4, wherein the step of performing the conversion comprises parsing and decoding the video from the bitstream.
[0106] 7. A method for storing a video bitstream, comprising: A method comprising the steps of generating a bitstream from a video including one or more scalable video layers and storing the bitstream on a non-transitory computer-readable recording medium, wherein the video includes one or more video pictures including one or more slices, and the bitstream conforms to a format rule, the format rule specifying that if a corresponding network abstraction layer unit type is within a predetermined range and a corresponding video layer flag indicates that the video layer corresponding to the slice does not use inter-layer prediction, then a value of a field indicating a slice type of the slice is set to indicate a type of I slice.
[0107] For example, the following solution may be implemented in accordance with item 5 of the previous section.
[0108] 1. A method of video processing (e.g., method 720 shown in FIG. 7B), comprising: 11. A method comprising: converting (722) between a video including a plurality of video layers and a bitstream of the video, the bitstream including a plurality of output layer sets (OLS), each including one or more of a plurality of scalable video layers, the bitstream conforming to a format rule, the format rule specifying that for an OLS having a single layer, a profile-tier-level (PTL) syntax structure indicating a profile, tier and level of the OLS is included in a video parameter set of the bitstream, and a PTL syntax structure of the OLS is also included in a sequence parameter set coded into the bitstream.
[0109] 2. The method according to Solution 1, wherein the format rule specifies that the video parameter set includes a syntax element indicating the number of PTL syntax structures coded in the video parameter set.
[0110] 3. The method according to solution 2, wherein the number of PTL syntax structures coded in the video parameter set is equal to the value of the syntax element plus one.
[0111] 4. The method according to solution 2, wherein the number of PTL syntax structures coded in the video parameter set is equal to the value of the syntax element.
[0112] 5. The method of Solution 4, wherein the format rules specify that a syntax element is to be coded into a video parameter set during encoding or parsed out of a video parameter set during decoding if one or more of the multiple OLSs is determined to contain more than one video layer.
[0113] 6. The method of Solution 4, wherein the format rule specifies that the syntax element is coded into a video parameter set during encoding or parsed from a video parameter set during decoding if the number of video layers is greater than 0 and it is determined that one or more of the video layers uses inter-layer prediction.
[0114] 7. A method as described in any of Solutions 1 to 6, wherein the format rules further specify that for an OLS having a single layer, a decoded picture buffer size parameter for the OLS is included in the video parameter set of the bitstream, and a decoded picture buffer size parameter for the OLS is also included in the sequence parameter set coded in the bitstream.
[0115] 8. A method according to any of Solutions 1 to 6, wherein the format rules further specify that for an OLS having a single layer, parameter information of the decoded picture buffer for the OLS is included in a video parameter set of the bitstream, and parameter information of the decoded picture buffer for the OLS is also included in a sequence parameter set coded in the bitstream.
[0116] For example, the following solution may be implemented in accordance with item 6 of the previous section.
[0117] 9. A method of video processing (e.g., method 730 shown in FIG. 7C), comprising: 1. A method comprising: converting (732) between a video including a plurality of video layers and a bitstream of the video, the bitstream including a plurality of output layer sets (OLS), each including one or more of the plurality of video layers, the bitstream conforming to a format rule, the format rule specifying a relationship between a number of occurrences of profile-tier-level (PTL) syntax structures in a video parameter set of the bitstream and byte alignment syntax fields in the video parameter set, each PTL syntax structure indicating a profile, tier, and level of one or more of the plurality of OLS.
[0118] 10. The method of solution 9, wherein the format rule specifies that at least one PTL syntax structure is included in the video parameter set, and that in order for the at least one PTL syntax structure to be included, one or more instances of a byte alignment syntax field are included in the video parameter set.
[0119] 11. The method according to solution 10, wherein the byte alignment syntax field is 1 bit.
[0120] 12. The method according to solution 11, wherein each value of the one or more instances of the byte alignment syntax field has a value of 0.
[0121] For example, the following solution may be implemented in accordance with item 2 of the previous section.
[0122] 13. A method of video processing (e.g., method 740 shown in FIG. 7D), comprising: 11. A method comprising: converting (742) between a video including a plurality of scalable video layers and a video bitstream, the bitstream including a plurality of output layer sets (OLS), each including one or more of the plurality of scalable video layers, the bitstream conforming to a format rule, the format rule specifying that, during encoding, a syntax element indicating an index into a syntax structure describing a profile, layer, and level of one or more of the plurality of OLSs is excluded from a video parameter set of the bitstream if the value of the index is zero, or, during decoding, the syntax element is inferred to have a value of zero if not present in the bitstream.
[0123] 14. The method according to solution 13, wherein the index is for a syntax structure indicating at least one profile, stratum and level of the OLS.
[0124] 15. A method for storing a video bitstream, comprising: 1. A method comprising: generating a bitstream from a video including a plurality of video layers; and storing the bitstream on a non-transitory computer-readable recording medium, the bitstream including a plurality of output layer sets (OLS), each including one or more of the plurality of video layers, the bitstream conforming to a format rule, the format rule specifying that for an OLS having a single layer, a profile-tier-level (PTL) syntax structure indicating a profile, tier and level of the OLS is included in a video parameter set of the bitstream, and a PTL syntax structure for the OLS is also included in a sequence parameter set coded in the bitstream.
[0125] 16. A method according to any of Solutions 1 to 14, wherein the step of performing the conversion includes a step of encoding the video into a bitstream, and the method further includes a step of storing the bitstream in a non-transitory computer-readable recording medium.
[0126] For example, the following solution may be implemented in accordance with item 3 of the previous section.
[0127] 1. A method of video processing (e.g., method 750 shown in FIG. 7E), comprising: 1. A method comprising: converting (752) between a video including a plurality of video layers and a bitstream of the video, the bitstream including a plurality of output layer sets (OLS), each including one or more of the plurality of video layers, the bitstream conforming to a format rule, the format rule specifying that for layer i, where i is an integer, the bitstream includes a set of first syntax elements indicating a first variable indicating whether layer i is included in at least one of the plurality of OLSs.
[0128] 2. The method according to Solution 1, wherein the format rule specifies that if a first variable for layer i is equal to zero, meaning that layer i is not included in any of the multiple OLSs, then the bitstream excludes a second set of syntax elements indicating decoded picture buffer parameters for layer i.
[0129] 3. A method as described in any of Solutions 1 to 2, wherein the format rules further specify that the bitstream includes a set of third syntax elements indicating a second variable indicating whether layer i is used as a reference layer for at least one of the multiple video layers, and the format rules do not allow the first variable and the second variable to have a value of zero.
[0130] 4. The method according to Solution 3, wherein the formatting rules do not allow the values of the first and second variables to both be equal to 0, indicating that there is no layer that is neither a direct reference layer of any other layer nor an output layer of at least one OLS layer.
[0131] 5. The method according to any one of solutions 1 to 4, wherein the first variable is a one-bit flag denoted as LayerUsedAsOutputLayerFlag.
[0132] 6. The method of solution 5, wherein the first variable is determined based on repeatedly examining, for each of the plurality of video layers, values of a third variable indicative of a relationship between the number of layers included in the plurality of OLS.
[0133] 7. The method according to solution 6, wherein the third variable indicating the relationship between the number of layers included in the multiple OLS can have the values 0, 1 or 2.
[0134] 8. A method according to any of Solutions 1 to 7, wherein the step of performing the conversion comprises a step of encoding the video into a bitstream, the method further comprising a step of storing the bitstream in a non-transitory computer-readable recording medium.
[0135] 9. A method for storing a video bitstream, comprising: 1. A method comprising: generating a bitstream from a video including a plurality of video layers; and storing the bitstream on a non-transitory computer-readable recording medium, the bitstream including a plurality of output layer sets (OLS), each including one or more of the plurality of video layers, the bitstream conforming to a formatting rule, the formatting rule specifying that, for layer i, where i is an integer, the bitstream includes a set of first syntax elements indicating a first variable indicating whether layer i is included in at least one of the plurality of OLSs.
[0136] For example, the following solution may be implemented in accordance with item 4 of the previous section.
[0137] 1. A method of video processing (e.g., method 760 shown in FIG. 7F), comprising: A method comprising: performing (762) a conversion between video and a video bitstream, the bitstream including one or more output layer sets each including one or more video layers, the bitstream conforming to a format rule, the format rule specifying that a number of syntax structures for decoded picture buffer parameters included in a video parameter set of the bitstream is equal to zero if each output layer set includes a single video layer, or equal to one plus the value of the syntax element if it is not true that each output layer set includes a single layer.
[0138] 2. The method according to Solution 1, wherein the syntax element corresponds to the vps_num_dpb_params_minus1 syntax element.
[0139] 3. A method as described in any of Solutions 1 to 2, wherein the format rule specifies that if other syntax elements indicating whether the same size is used to indicate the syntax structure of the decoded picture buffer for video layers that are included and not included in one or more output layer sets are not present in the video parameter set, the value of the other syntax elements is presumed to be equal to 1.
[0140] 4. A method according to any of Solutions 1 to 3, wherein the step of performing the conversion comprises a step of encoding the video into a bitstream, the method further comprising a step of storing the bitstream in a non-transitory computer-readable recording medium.
[0141] 5. A method for storing a video bitstream, comprising: 11. A method comprising the steps of generating a bitstream from video and storing the bitstream on a non-transitory computer-readable recording medium, the bitstream including one or more output layer sets each including one or more video layers, the bitstream conforming to a format rule, the format rule specifying that a number of syntax structures for decoded picture buffer parameters included in a video parameter set of the bitstream is equal to zero if each output layer set includes a single video layer, or equal to one plus the value of the syntax element if it is not true that each output layer set includes a single layer.
[0142] For example, the following solution may be implemented in accordance with item 7 of the previous section.
[0143] 1. A method of video processing (e.g., method 770 shown in FIG. 7G), comprising: 1. A method comprising: performing (772) a conversion between video and a video bitstream, the bitstream including a coded video sequence (CVS) including one or more coded video pictures of one or more video layers, the bitstream conforming to a format rule specifying that one or more sequence parameter sets (SPS) indicating transformation parameters referenced by one or more coded pictures of the CVS have the same reference video parameter set (VPS) identifier indicating a reference video parameter set.
[0144] 2. The method according to solution 1, wherein the format rule further specifies that the same reference VPS identifier has a value greater than 0.
[0145] 3. The method according to any of Solutions 1 to 2, wherein the format rules further specify that the SPS identifier value zero is used accordingly only if the CVS contains a single video layer.
[0146] 4. A method according to any of Solutions 1 to 3, wherein the step of performing the conversion comprises a step of encoding the video into a bitstream, the method further comprising a step of storing the bitstream in a non-transitory computer-readable recording medium.
[0147] 5. A method for storing a video bitstream, comprising: 1. A method comprising: generating a bitstream from a video; and storing the bitstream on a non-transitory computer-readable recording medium, wherein the video includes one or more video pictures including one or more slices, and the bitstream includes a coded video sequence (CVS) including one or more coded video pictures of one or more video layers, and wherein the bitstream complies with a format rule specifying that one or more sequence parameter sets (SPSs) indicating transformation parameters referenced by one or more coded pictures of the CVS have the same reference video parameter set (VPS) identifier indicating a reference video parameter set (VPS).
[0148] For example, the following solution may be implemented in accordance with item 8 of the previous section.
[0149] 1. A method of video processing (e.g., method 780 shown in FIG. 7H), comprising: 1. A method comprising: performing (782) a conversion between video and a video bitstream, the bitstream including one or more output layer sets (OLS), each including one or more video layers, the bitstream conforming to a format rule, the format rule specifying whether or how a first syntax element indicating whether a first syntax structure describing parameters of a hypothetical reference decoder (HRD) used for the conversion is included in a video parameter set (VPS) of the bitstream.
[0150] 2. The method according to solution 1, wherein the first syntax structure includes a set of general HRD parameters.
[0151] 3. A method according to any of Solutions 1 to 2, wherein the format rule specifies that the first syntax element is omitted from the VPS and is presumed to have a value of zero if not present in the VPS because each of the one or more video layers is included in an OLS among the one or more OLSs, and each of the one or more OLSs includes a single video layer.
[0152] 4. A method according to any of Solutions 1 to 2, wherein the format rule specifies that if present in the VPS each of one or more video layers is included in an OLS of one or more OLSs, the first syntax element has a value of zero in the VPS, and each of the one or more OLSs contains a single video layer.
[0153] 5. The method according to any of Solutions 1 to 4, wherein the formatting rules further specify whether or how a second syntax element indicating a number of syntax structures describing OLS-specific HRD parameters is included in the VPS.
[0154] 6. The method of Solution 5, wherein the formatting rule further specifies that if the first syntax element has a value of 1, the second syntax element is included in the VPS, regardless of whether the total number of OLSs in the one or more OLSs is greater than 1.
[0155] For example, the following solution may be implemented in accordance with item 9 of the previous section.
[0156] 7. A method of video processing (e.g., method 790 shown in FIG. 7I), comprising: 1. A method comprising: performing (792) a conversion between video and a video bitstream, the bitstream including one or more output layer sets (OLS), each including one or more video layers, the bitstream conforming to a format rule, the format rule specifying whether or how a first syntax structure describing general hypothetical reference decoder (HRD) parameters and a second syntax structure describing OLS-specific HRD parameters are included in a video parameter set (VPS) of the bitstream.
[0157] 8. The method of solution 7, wherein the formatting rule specifies that if the second syntax structure is not included in the VPS, then the first syntax structure is to be omitted from the VPS.
[0158] 9. The method of Solution 7, wherein the formatting rules exclude the inclusion of the first syntax structure and the second syntax structure in the VPS for an OLS that includes only one video layer, and the formatting rules allow the inclusion of the first syntax structure and the second syntax structure for an OLS that includes only one video layer.
[0159] 10. A method for storing a video bitstream, comprising: 1. A method comprising: generating a bitstream from video; and storing the bitstream on a non-transitory computer-readable recording medium, the bitstream including one or more output layer sets (OLS), each including one or more video layers, the bitstream conforming to a format rule, the format rule specifying whether or how a first syntax element indicating whether a first syntax structure describing parameters of a hypothetical reference decoder (HRD) used for the conversion is included in a video parameter set (VPS) of the bitstream.
[0160] The above solution may further include:
[0161] In some embodiments, in the above solution, performing the conversion comprises encoding the video into a bitstream.
[0162] In some embodiments, in the above solution, performing the conversion comprises parsing and decoding the video from the bitstream.
[0163] In some embodiments, in the above solution, the step of performing the conversion includes a step of encoding the video into a bitstream, and the method further includes a step of storing the bitstream in a non-transitory computer-readable recording medium.
[0164] In some embodiments, a video decoding apparatus includes a processor configured to implement the methods described in one or more of the above solutions.
[0165] In some embodiments, a video encoding apparatus includes a processor configured to implement the methods described in one or more of the above solutions.
[0166] In some embodiments, a non-transitory computer-readable storage medium may store instructions that cause a processor to perform a method according to one or more of the above solutions.
[0167] In some embodiments, a non-transitory computer-readable recording medium stores, in a processor, a bitstream of video generated by a method according to one or more of the above solutions.
[0168] In some embodiments, the above encoding method may be performed by an apparatus, which may further write the bitstream generated by performing the method to a computer-readable medium.
[0169] The disclosed and other solutions, examples, modules and functional operations described in this document can be implemented in digital electronic circuitry, or computer software, firmware or hardware, including the structures disclosed in this document and their structural equivalents, or any combination of one or more of these. The disclosed and other embodiments can be implemented as one or more computer program products, i.e., as one or more modules of computer program instructions encoded on a computer-readable medium for execution by or for controlling the operation of a data processing apparatus. The computer-readable medium can be a machine-readable storage device, a machine-readable storage substrate, a memory device, a composition of matter producing a machine-readable propagated signal, or a combination of one or more of these. The term "data processing apparatus" encompasses all apparatus, devices and machines for processing data, including, by way of example, a programmable processor, a computer or multiple processors or computers. An apparatus can include, in addition to hardware, code that creates an execution environment for the computer program in question, such as code that constitutes a processor firmware, a protocol stack, a database management system, an operating system, or any combination of one or more of these. A propagated signal is an artificially generated signal, such as a machine-generated electrical, optical or electromagnetic signal, that is generated to encode information for transmission to an appropriate receiver device.
[0170] A computer program (also known as a program, software, software application, script, or code) can be written in any type of programming language, including compiled or interpreted languages, and can be deployed in any form, either as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A computer program does not necessarily correspond to a file in a file system. A program can be stored as part of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), or in a single file dedicated to the program in question, or in multiple associated files (e.g., a file that stores one or more modules, subprograms, or portions of code). A computer program can be deployed to be executed on one computer, or to be executed on multiple computers that are located at one site or distributed across multiple sites and interconnected by a communication network.
[0171] The processes and logic flows described in this document may be performed by one or more programmable processors executing one or more computer programs to perform functions by operating on input data and generating output. The processes and logic flows may also be performed by, and an apparatus may be implemented as, special purpose logic circuitry, such as a field programmable gate array (FPGA) or application specific integrated circuit (ASIC).
[0172] Processors suitable for executing computer programs include, by way of example, both general purpose and special purpose microprocessors, and any one or more processors of any kind of digital computer. Typically, a processor receives instructions and data from a read-only memory or a random access memory, or both. The essential elements of a computer are a processor for executing instructions and one or more memory devices for storing instructions and data. Typically, a computer also includes one or more mass storage devices, such as magnetic disks, magneto-optical disks, or optical disks, for storing data, or is operatively coupled to receive data from or transfer data to the mass storage devices. However, a computer need not have such devices. Computer-readable media suitable for storing computer program instructions and data include, by way of example, semiconductor memory devices, such as EPROM, EEPROM, and flash memory devices, and all types of non-volatile memory, media, and memory devices. The processor and memory may be supplemented by, or incorporated in, dedicated logic circuitry.
[0173] Although this patent document contains many details, these should not be construed as limitations on any subject matter or the scope of what may be claimed, but rather as descriptions of features that may be specific to particular embodiments of a particular technology. Certain features described in this patent document with respect to separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features described with respect to a single embodiment can be implemented in multiple embodiments separately or in any suitable subcombination. Furthermore, although features may be described above as acting in a particular combination, and even initially claimed as such, in some cases one or more features from the claimed combination can be excluded from the combination, and the claimed combination may be led to a subcombination or a variation of the subcombination.
[0174] Similarly, although the figures show acts in a particular order, this should not be understood as requiring that such acts be performed in the particular order or sequence shown, or that all of the acts shown be performed, to achieve desired results.Furthermore, the separation of various system components in the embodiments described in this patent document should not be understood as requiring such separation in all embodiments.
[0175] Only a few implementations and examples have been described; other implementations, extensions and variations can be made based on what is described and illustrated in this patent document.
Claims
1. 1. A method of video processing, comprising: performing a first conversion between a video including a plurality of video layers and a bitstream of the video; the bitstream includes a plurality of output layer sets (OLS), each OLS including one or more of the plurality of video layers, the bitstream conforming to a format rule; The format rule specifies that a number syntax element indicating a number of profile, layer and level (PTL) syntax structures coded in a video parameter set is included in the video parameter set of the bitstream; The format rule is: During encoding, if the value of the number syntax element is equal to zero, an index syntax element indicating an index into the PTL syntax structure is excluded from the video parameter set of the bitstream; During decoding, if the index syntax element is not present in the bitstream, the value of the index syntax element is inferred to be zero. The method further specifies that.
2. 2. The method of claim 1, wherein the format rules further specify that for an OLS having a single layer, the PTL syntax structure indicating a profile, tier and level of the OLS is included in the video parameter set of the bitstream, and the PTL syntax structure of the OLS is also included in a sequence parameter set coded to the bitstream.
3. The method of claim 2 , wherein the number of PTL syntax structures is equal to the value of the number syntax element plus one.
4. The method of claim 1 , wherein the format rules further specify a relationship between a number of occurrences of the PTL syntax structures in the video parameter set of the bitstream and a byte alignment syntax field in the video parameter set.
5. 5. The method of claim 4, wherein the formatting rules specify that at least one PTL syntax structure is included in the video parameter set, and that one or more instances of the byte alignment syntax field are included in the video parameter set in order for the at least one PTL syntax structure to be included.
6. the byte alignment syntax field is 1 bit; 6. The method of claim 5, wherein a value of each of the one or more instances of the byte alignment syntax field has a value of 0.
7. the formatting rules further specify that for layer i, where i is an integer, the bitstream includes a first set of syntax elements used to derive a first variable indicating whether the layer i is used as an output layer in at least one of the plurality of OLSs; the formatting rules further specify that the bitstream includes a third set of syntax elements used to derive a second variable indicating whether the layer i is used as a reference layer of at least one of the plurality of video layers, the formatting rules disallowing values of both the first variable and the second variable to be equal to 0; 2. The method of claim 1, wherein not allowing the values of both the first variable and the second variable to be equal to 0 indicates that no layer is neither a direct reference layer of any other layer nor an output layer of at least one OLS layer.
8. 8. The method of claim 7, wherein the formatting rule specifies that the first variable for the layer i is equal to zero, which means that the layer i is not used as an output layer of any of the multiple OLSs.
9. 9. The method of claim 8, wherein the format rule specifies that if the first variable for the layer i is equal to zero, then the bitstream excludes a second set of syntax elements indicating decoded picture buffer parameters for the layer i.
10. The first variable is a 1-bit flag denoted as LayerUsedAsOutputLayerFlag, 8. The method of claim 7, wherein the first variable is determined based on iteratively examining, for each of the plurality of video layers, a value of a third variable indicative of a relationship between the plurality of video layers and the plurality of OLSs.
11. performing a second conversion between a video including one or more scalable video layers and the bitstream of the video; the video includes one or more video pictures, each of which includes one or more slices; The format rule further specifies that if a corresponding network abstraction layer unit type is within a predetermined range and a corresponding video layer flag indicates that a video layer corresponding to a slice does not use inter-layer prediction, a value of a field indicating a slice type of the slice is set to indicate a type of intra slice; the predetermined range is from IDR_W_RADL to CRA_NUT, the IDR_W_RADL indicating a slice having an instantaneous decoding refresh type, and the CRA_NUT indicating a slice having a clean random access type; 2. The method of claim 1 , wherein the corresponding video layer flag corresponds to vps_independent_layer_flag[GeneralLayerIdx[nuh_layer_id]], where nuh_layer_id corresponds to an identifier of the video layer, GeneralLayerIdx[ ] is an index of the video layer, and where the vps_independent_layer_flag corresponds to the corresponding video layer flag.
12. The method of claim 11 , wherein the value of the field indicating the slice type of the slice is set to 2.
13. 13. The method of claim 1, wherein the step of performing a first conversion comprises encoding the video into the bitstream.
14. The method of claim 1 , wherein the step of performing a first conversion comprises decoding the video from the bitstream.
15. 1. An apparatus for processing video data, comprising a processor and a non-transitory memory having instructions, the apparatus comprising: The instructions, when executed by the processor, cause the processor to: performing a first conversion between a video including a plurality of video layers and a bitstream of the video; the bitstream includes a plurality of output layer sets (OLS), each OLS including one or more of the plurality of video layers, the bitstream conforming to a format rule; The format rule specifies that a number syntax element indicating a number of profile, layer and level (PTL) syntax structures coded in a video parameter set is included in the video parameter set of the bitstream; The format rule is: During encoding, if the value of the number syntax element is equal to zero, an index syntax element indicating an index into the PTL syntax structure is excluded from the video parameter set of the bitstream; During decoding, if the index syntax element is not present in the bitstream, the value of the index syntax element is inferred to be zero. The apparatus further specifies that
16. A non-transitory computer-readable storage medium having instructions stored thereon, comprising: The instructions cause a processor to: performing a first conversion between a video including a plurality of video layers and a bitstream of the video; the bitstream includes a plurality of output layer sets (OLS), each OLS including one or more of the plurality of video layers, the bitstream conforming to a format rule; The format rule specifies that a number syntax element indicating a number of profile, layer and level (PTL) syntax structures coded in a video parameter set is included in the video parameter set of the bitstream; The format rule is: During encoding, if the value of the number syntax element is equal to zero, an index syntax element indicating an index into the PTL syntax structure is excluded from the video parameter set of the bitstream; During decoding, if the index syntax element is not present in the bitstream, the value of the index syntax element is inferred to be zero.
20. A non-transitory computer readable storage medium, further specifying:
17. 1. A method for storing a video bitstream, comprising: For a video including multiple video layers, generating the bitstream for the video; storing the bitstream on a non-transitory computer-readable recording medium; Including, the bitstream includes a plurality of output layer sets (OLS), each OLS including one or more of the plurality of video layers, the bitstream conforming to a format rule; The format rule specifies that a number syntax element indicating a number of profile, layer and level (PTL) syntax structures coded in a video parameter set is included in the video parameter set of the bitstream; The format rule is: The method of claim 1, further specifying that during encoding, if the value of the number syntax element is equal to zero, then an index syntax element indicating an index into the PTL syntax structure is excluded from the video parameter set of the bitstream.
Citation Information
Patent Citations
Conformance parameters for bitstream segmentation
JP2017522779A
Conformance parameters for bitstream partitions
US20150358640A1
Coding device, decoding device, coding method, and decoding method
WO2021117813A1
Cited By
Processing of filler data units in video streams
US12506903B2
Signaling of decoded picture buffer parameters in layered video
US12634521B2
