Semantics for Constraint Handling and Conformance Testing in Video Coding

Abstract

A method, system, and bitstream syntax for constrained processing in video coding is provided. [Solution] The encoder uses one or more syntax elements and either explicit or implicit signaling to signal to a compliant decoder that certain features of the Main Profile, such as subpictures and multi-layered scalable coding, are disabled.

Description

[Technical Field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims the benefit of priority to U.S. Provisional Patent Application No. 63 / 028,214, filed May 21, 2020, U.S. Provisional Patent Application No. 63 / 013,713, filed April 22, 2020, and U.S. Provisional Patent Application No. 63 / 013,474, filed April 21, 2020, all of which are incorporated herein by reference in their entireties.

[0002]

[0002] Technical field This disclosure relates generally to images. More particularly, embodiments of the present invention relate to semantics and syntax elements for constraint processing and conformance testing in video coding. [Background technology]

[0003]

[0003] In 2013, the MPEG Group of the International Organization for Standardization (ISO), in collaboration with the International Telecommunication Union (ITU), published the first draft of the HEVC (also known as H.265) video coding standard (Ref. [1]). More recently, the same group has been working on developing a next-generation coding standard (referred to as the Versatile Video Coding, or VVC, standard (Ref. [2])) that will provide improved coding performance over existing video coding techniques.

[0004] To facilitate their deployment, video coding standards such as HEVC may define profiles, tiers, levels, and other syntax elements that specify constraints on a bitstream and thus describe limitations on the capabilities required to decode the bitstream. Profiles, tiers, levels, and other syntax elements may also be used to indicate interoperability points between individual decoder implementations.

[0005]

[0005] As understood by the inventors herein, improved techniques are described herein for defining constraints on VVC-compliant bitstreams and improving conformance testing while providing access to all of the versatile features.

[0006] The approaches described herein are approaches that could be pursued, but not necessarily approaches that have been previously conceived or pursued. Accordingly, unless otherwise specified, it should not be assumed that any approach described in this section qualifies as prior art merely by virtue of its inclusion in this section. Similarly, it should not be assumed, based on this section, that matter identified with one or more approaches is recognized in any prior art unless otherwise specified. [Brief explanation of the drawings]

[0007]

[0007] Embodiments of the present invention are illustrated by way of example and not by way of limitation in the figures of the accompanying drawings, in which like reference numerals refer to similar elements. [Figure 1]

[0008] FIG. 1 illustrates an exemplary process for a video delivery pipeline. [Figure 2A]

[0009] FIG. 2A illustrates an exemplary encoding video process according to an embodiment. [Figure 2B]

[0010] FIG. 2B illustrates an exemplary decoding video process according to an embodiment. [Figure 3]

[0011] FIG. 3 illustrates an exemplary process for constraint hierarchy processing according to an embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0008]

[0012] Exemplary embodiments relating to semantics for constrained processing and conformance testing in the VVC coding specification are described herein. In the following description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of various embodiments of the present invention. However, it will be apparent that various embodiments of the present invention may be practiced without these specific details. In other instances, well-known structures and devices are not described in exhaustive detail to avoid unnecessarily narrowing, obscuring, or obscuring embodiments of the present invention.

[0009]

[0013] overview The embodiments described herein relate to semantics for constraint processing and conformance testing in video coding. In an encoder, a processor receives a sequence of video pictures to be encoded into a coded bitstream along with constraint processing.

[0010] The processor is: determining a set of tools that are not required for decoding the coded bitstream by a decoder; determining one or more constraint flags associated with the set of tools; Grouping one or more constraint flags into one or more tool constraint information syntax structures; Integrating one or more tool constraint information syntax structures into a general constraint syntax structure to generate a coded bitstream, the coded bitstream including coded pictures of the sequence of video pictures and the general constraint syntax structure.

[0011]

[0014] In another embodiment, in a decoder, a processor receives a coded bitstream including coded pictures of a sequence of video pictures and a general constraint syntax structure, the general constraint syntax structure including syntax elements associated with a set of tools that are not required for decoding the coded bitstream by the decoder.

[0012] The processor then parses the general constraint syntax structure to identify one or more tool constraint information syntax structures, each tool constraint information syntax structure including one or more constraint flags associated with a particular coding tool; The processor parses each of the one or more tool constraint information syntax structures to generate one or more constraint flags associated with the set of tools; and Decoding coded pictures in the coded bitstream according to one or more constraint flags to generate a sequence of video pictures.

[0013]

[0015] In another embodiment, the processor receives a coded video bitstream including coded pictures and syntax parameters for the coded pictures, and detects whether layered processing is enabled, wherein detecting whether layered processing is enabled comprises setting a flag: A flag indicating whether scalability or layered coding is enabled; A flag indicating whether the video parameter set (VPS) is constrained; A flag indicating whether the total number of output layer sets (OLS) is constrained; a flag indicating whether the layer to which the NAL unit belongs is constrained; A flag indicating whether inter-layer prediction is constrained; detecting whether one or more of The processor determines that layering processing is disabled if one or more of these flags are set to 1, and determines that layering processing is enabled otherwise.

[0014]

[0016] In another embodiment, the processor receives a coded bitstream including coded pictures and syntax parameters for the coded pictures, and the processor receives a coded bitstream including the ... and the processor A flag indicating whether the video parameter set (VPS) is constrained; A flag indicating whether the total number of output layer sets (OLS) is constrained; a flag indicating whether the layer to which the NAL unit belongs is constrained; A flag indicating whether the NAL unit type is constrained; A flag indicating whether the number of slices in a picture is constrained; A flag indicating whether the number of tiles in the picture is constrained; A flag indicating whether the number of subpictures in a picture is constrained; A flag indicating whether the dual-tree partition is constrained; Flag indicating whether the palette mode is constrained; A flag indicating whether intra-block copies are constrained; Flag indicating whether intra prediction is constrained; A flag indicating whether inter prediction is constrained; A flag indicating whether the quantization is constrained; A flag indicating whether the transformation is constrained; a flag indicating whether residual coding is constrained; A flag indicating whether the loop filter is constrained; Flag indicating whether the sample bit depth is constrained; Flag indicating whether the chroma sampling format is constrained; Flag indicating whether 360 video coding is constrained; Flag indicating whether the screen content coding is constrained; A flag indicating whether the presence of SEI messages is constrained; A flag indicating whether inter-layer prediction is constrained; Detect whether one or more of If one or more of these flags are set to 1, bitstream compliance is determined.

[0015]

[0017] Exemplary Video Delivery Processing Pipeline FIG. 1 illustrates an exemplary process of a conventional video distribution pipeline (100), showing various stages from video capture to video content display. A sequence of video frames (102) is captured or generated using an image generation block (105). The video frames (102) can be captured digitally (e.g., by a digital camera) or generated by a computer (e.g., using computer animation) to provide video data (107). Alternatively, the video frames (102) can be captured on film using a film camera. The film is converted to a digital format to provide video data (107). In a production phase (110), the video data (107) is edited to provide a video production stream (112).

[0016]

[0018] The video data in the production stream (112) is then provided to a processor for post-production editing in block (115). Post-production editing in block (115) may involve adjusting or modifying the color or brightness in specific areas of the image to enhance image quality or achieve a particular look for the image according to the video creator's creative intent. This is sometimes called "color timing" or "color grading." Other editing (e.g., scene selection and sequencing, image cropping, addition of computer-generated visual special effects, judder or blur control, frame rate control, etc.) can be performed in block (115) to generate the final production stream (117) for distribution. During post-production editing (115), the video image is viewed on a reference display (125).

[0017]

[0019] After post-production (115), the final production video data (117) is delivered to a coding block (120) for downstream distribution to decoding and playback devices such as television sets, set-top boxes, movie theaters, etc. In some embodiments, the coding block (120) may include audio and video encoders, such as those defined by ATSC, DVB, DVD, Blu-Ray, and other distribution formats, to generate a coded bitstream (122). At the receiver, the coded bitstream (122) is decoded by a decoding unit (130) to generate a decoded signal (132) that represents the same or a similar representation of the signal (117). The receiver may be attached to a target display (140), which may have characteristics quite different from the reference display (125). In this case, a display management block (135) may be used to map the dynamic range of the decoded signal (132) to the characteristics of the target display (140) by generating a display mapping signal (137).

[0018]

[0020] Constraint handling in VVC The current working draft text of the VVC specification (Ref. [2]) defines a set of constraint flags as a means to allow an encoder to inform a decoder that certain coding tools are not required when decoding a coded bitstream, and as an alternative to encouraging sub-profiles other than the existing official profiles, e.g., Main 10 and Main 4:4:4. For example, each of these profiles limits a conforming bitstream to a particular chroma format or bit depth, and requires that certain layer or level constraints be met, but all coding tools specified in VVC are allowed as specified in a conforming bitstream.

[0019]

[0021] Some applications and uses of VVC may not require all coding tools and features specified in VVC. Such applications would benefit if the corresponding decoding process implemented a subset of the coding tools, and a conforming decoder could still decode the conforming bitstream. One possible way to accommodate such decoding processes would be to define additional profiles for VVC. For example, a "Simplified Main 10" profile could be specified that is identical to the Main 10 profile; however, one or more specific coding tools and associated syntax would not be allowed to be signaled in the bitstream. Alternatively, if they are identified in the bitstream, a conforming decoder could safely ignore them. For example, a "Simple Main 10" profile could not allow scalability or subpictures.

[0020]

[0022] One drawback of defining multiple profiles is that it may limit the market accessible to device and application providers by promoting market fragmentation. Another drawback of multiple profiles is that it may make conformance testing and verification more difficult, affecting quality and interoperability between bitstream creators and consumers.

[0021]

[0023] An alternative to profile fragmentation proposed by the embodiments herein is to package syntax constraints into a small number of general constraint information syntax elements that apply to the Main 10 and Main 4:4:4 10 profiles, as well as any equivalent still image profile in VVC. For example, a general constraint information syntax element, no_subpicture_constraint_flag, may be signaled in a bitstream to indicate that no analysis and decoding process is required to decode subpictures in the bitstream, yet the bitstream conforms to a profile, e.g., the Main 10 profile. One advantage of the general constraint method proposed herein is that it facilitates conformance testing and verification.

[0022]

[0024] The use of the general constraint information syntax elements can also facilitate the specification of domain-specific sub-profiles by other organizations, such as Standards Developing Organizations (SDOs) and industry forums. In this sense, the use of the general constraint information syntax elements facilitates the specification of a kind of "soft profile" that is easier to specify and validate.

[0023]

[0025] The current VVC draft specification includes 62 general constraint flags that specify restrictions on coding tool behavior and syntax element values. Existing general constraint flags relate to network abstraction layer (NAL) unit types, prediction modes, inter and intra prediction, transforms, quantization, loop filters, layers, and supplemental enhancement information (SEI) messages and formats. Currently, all 62 general constraint flags are signaled in the general_contstraint_info( ) syntax structure.

[0024]

[0026] As currently specified, the list of general constraint flags can be confusing because there is no logical order to the list. This lack of order can discourage the use of general constraint flags and limit their benefits. Furthermore, the syntactic structure of general_constraint_info( ) as currently specified is likely to make the list of general constraint flags even more confusing if additional general constraint flags are added in the future. To maintain backward compatibility, new flags will be added to the end of the list, regardless of the function of the new flag.

[0025]

[0027] As understood by the inventors, the structure of general_constraint_info( ) can be made easier to use and less prone to user error by signaling separate, categorized syntax structures, along with limited but greater flexibility. For example, the syntax structure of general_transform_constraint_info( ) for tools related to transform coding can be signaled as part of a call to general_constraint_info( ). For example, the syntax structure of general_transform_constraint_info( ) can signal only general constraint flags for the transform. Similarly, calls such as general_quantization_constraint_info( ), general_inter_constraint_info( ), and general_loop_filter_info( ) can be signaled with general_constraint_info( ) to group general constraint flags for quantization, inter prediction, and loop filter, respectively.

[0026]

[0028] A further advantage of the categorized, scope-limited, general constraint syntax structure is that such a structure facilitates the efficient addition of new general constraint flags in a backwards-compatible manner: new flags added to the end of a shorter list of relevant flags preserves ease of use and reduces the tendency for user error.

[0027]

[0029] In another embodiment, several new general constraint flags are also added. The new flags can be grouped into three general classes: 1) coding tool flags; 2) feature restriction flags; and 3) supplemental enhancement information (SEI) flags.

[0028]

[0030] The coding tool general constraint flags will be consistent with the currently specified general constraint flags in that they specify restrictions on coding tools and syntax element values. For example, the proposed new coding tool general constraint flags are: no_virtual_boundary_constraint_flag; no_weighted_prediction_constraint_flag; no_weighted_bipred_constraint_flag; no_explicit_scaling_list_constraint_flag; and Includes no_vps_constraint_flag.

[0029]

[0031] The feature-limited general constraint flags provide new capabilities for specifying constraints on groups of coding tools, other constraint flags, and syntax element values. The feature-limited flags also facilitate conformance testing related to use cases, services, and application types. For example, the feature-limited general constraint flags proposed in this specification are: no_scalability_constraint_flag, which specifies that scalable layered coding is disabled; no_360Video_constraint_flag, which specifies that 360 video coding is disabled; and Include noSCC_constraint_flag to specify that screen content coding is disabled.

[0032] The SEI general constraints flag extends the ability to specify that no SEI messages are generally specified outside the core VVC standard. By incorporating SEI messages, information indicating how the coded video may be used or intended for use, display, or other processing may be signaled to a decoder or other process. As an example, the proposed new SEI general constraints flag is: no_scalable_nesting_SEI_constraint_flag; no_subpic_level_SEI_constraint_flag; no_filler_payload_SEI_constraint_flag; no_user_data_reg_SEI_constraint_flag; no_user_data_unreg_SEI_constraint_flag; no_film_grain_SEI_constraint_flag; no_parameter_set_incl_SEI_constraint_flag; no_decoded_picture_hash_SEI_constraint_flag; no_mdcv_SEI_constraint_flag; no_cll_SEI_constraint_flag; no_DRAP_constraint_SEI_flag; no_alt_transfer_char_SEI_constraint_flag; no_ambient_view_envir_SEI_constraint_flag; no_ccv_SEI_constraint_flag; no_omni_video_specific_SEI_constraint_flag; no_field_frame_info_SEI_constraint_flag; and Includes no_sar_SEI_constraint_flag.

[0030]

[0033] In another embodiment, to improve clarity of the currently specified general constraint flags, slightly modified names are proposed as follows: Change general_non_packed_constraint_flag to general_non_packed_SEI_constraint_flag Change general_non_projected_constraint_flag to general_non_projected_SEI_constraint_flag Change the max_bitdepth_constraint_idc variable to max_bitdepth_minus8_constraint_idc Correct the semantics of no_aps_constraint_flag to indicate sps_explicit_scaling_list_enabled_flag Splitting single_layer_constraint_flag into two general constraint flags: a) no_vps_constraint flag (when equal to 1, indicates that sps_video_parameter_set_id shall be equal to 0) and b) single_layer_constraint_flag (when equal to 1, indicates that vps_max_layers_minus1 shall be equal to 0).

[0031]

[0034] It should be noted that the syntax, semantics, methods, and advantages of the embodiments presented herein may also be applied to other means of signaling the general constraint information flag, for example, signaling general_constraint_info( ) in NAL units of type GCI_NUT; and signaling general_constraint_info( ) in decoding_capability_information_rbsp( ).

[0032]

[0035] In an embodiment, Table 1 shows exemplary syntax for a proposed new structure for the "General Constraint Information Syntax" in VVC, replacing Section 7.3.3.2 of Ref. [2]. As shown in Table 1, all existing flags are replaced with 12 general_xxx_constraint_info() structures, where "xxx" indicates a VVC coding aspect, such as partitioning, intra-coding, or loop filtering. Each of these 12 syntax structures is described in further detail in Tables 2-13. Proposed new flags are depicted in italicized font and may be explicitly listed after each table. Those skilled in the art of video coding will understand that fewer or more than 12 general_xxx_constraint_info() structures may be used to group existing and newly proposed constraint flags and syntax parameters. Furthermore, every effort has been made to group these flags into the most appropriate "xxx" groupings, and one or more of these flags may be assigned to alternative groups with minimal, if any, impact on overall functionality.

[0033]

[0036] Note that the order of the general_xxx_constraint_info() syntax structures in Table 1 can affect syntax validation and overall decoding performance testing. As an example, general_format_constraint_info() includes max_chroma_format_constraint_idc, which is referenced by the semantics of no_qtbtt_dual_tree_intra_constraint_flag (signaled by general_partition_constraint_info()) and by the semantics of no_cclm_constraint_flag (signaled by general_intra_constraint_info()). Therefore, signaling general_format_constraint_info() before signaling general_partition_contraint_info() and general_intra_constraint_info() simplifies syntax validation. As another example, general_functionality_constraint_info( ) includes general_one_picture_only_constraint_flag, which is referenced by the semantics of single_layer_constraint_flag (signaled in general_layer_constraint_info( )). Thus, signaling general_functionality_constraint_info( ) before signaling general_layer_contraint_info( ) again simplifies syntax checking. In another embodiment, instead of ordering the general_xxx_constraint_info() structures according to ease of syntax checking, their order can be determined based on other criteria, such as importance to coding tools, decoder flow, etc.

[0034]

[0037] The proposed category groups may also be rearranged, subdivided, or combined. For example, in one embodiment, the quantization and transform groups may be combined into a larger group, such as "tqr," for transform, quantization, and residual coding, as described in the table below. In another embodiment, the pred_mod, intra, and inter groups may be combined into a larger "prediction_tools" group. In another embodiment, the quantization groups may be split into smaller quantization and residual coding groups.

[0035]

[0038] The tools in each category can also be organized depending on, for example, the emphasis of the tool's importance. For example, IBC and palette modes have advantages mainly for intra picture coding, and therefore they may be included in the intra constraint group. As another example, as described in the general_tqr_constraint_info( ) structure, it may be useful to signal transform-related constraint flags before quantization-related constraint flags, because, for example, no_transform_skip_constraint_flag is referenced by no_bdpcm_constraint_flag.

[0036] Table 1: Examples of proposed "general constraint information syntax"

[0037] [Table 1]

[0039] Table 2: Example of general format constraint information syntax

[0038] [Table 2] The proposed semantics of the new flag are as follows: no_separate_colour_plane_constraint_flag equal to 1 indicates that sps_separate_colour_plane_flag, if present, shall be equal to 0. no_separate_colour_plane_constraint_flag equal to 0 imposes no such constraint.

[0039]

[0040] Table 3: General Feature Information Syntax Examples

[0040] [Table 3] The proposed semantics of the new flag are as follows: no_scalability_constraint_flag equal to 1 indicates that scalable layered coding is disabled for CVS. no_scalability_constraint_flag equal to 0 imposes no such constraint. The value of no_scalability_constraint_flag shall be equal to the value of the variable noScalabilityConstraint. The value of noScalabilityConstraint is derived as follows:

[0041]

number

[0042] Table 4: Example of proposed NAL constraint information syntax

[0043] [Table 4] Table 5: Example of proposed partition constraint information syntax

[0044] [Table 5] The proposed semantics of the new flag are as follows: no_virtual_boundary_constraint_flag equal to 1 indicates that sps_virtual_boundaries_enabled_flag shall be equal to 0. no_virtual_boundary_constraint_flag equal to 0 imposes no such constraint.

[0045] Table 6: Example of proposed syntax for prediction mode constraint information

[0046] [Table 6] Table 7: Example of general intra-prediction constraint information syntax

[0047] [Table 7] Table 8: Example of general inter prediction constraint information syntax

[0048] [Table 8] The proposed semantics of the new flag are as follows: no_weighted_pred_constraint_flag equal to 1 indicates that sps_weighted_pred_flag shall be equal to 0. no_weighted_pred_constraint_flag equal to 0 imposes no such constraint. no_weighted_bipred_constraint_flag equal to 1 indicates that sps_weighted_bipred_flag shall be equal to 0. no_weighted_bipred_constraint_flag equal to 0 imposes no such constraint.

[0049] Table 9: Example of general transformation constraint information syntax

[0050] [Table 9] Table 10: Example of general quantization constraint information syntax

[0051] [Table 10] The proposed semantics of the new flag are as follows: no_explicit_scaling_list_constraint_flag equal to 1 indicates that sps_explicit_scaling_list_enabled_flag shall be equal to 0. no_explicit_scaling_list_constraint_flag equal to 0 imposes no such constraint. If no_aps_constraint_flag is equal to 1, the value of no_explicit_scaling_list_constraint_flag shall be equal to 1.

[0052] Table 11: Example of General Loop Filter Constraint Information Syntax

[0053] [Table 11] Table 12: Example of General Layer Constraint Information Syntax

[0054] [Table 12] The proposed semantics of the new flag are as follows: no_vps_constraint_flag equal to 1 indicates that sps_video_parameter_set_id shall be equal to 0. no_vps_constraint_flag equal to 0 imposes no such constraint. If general_one_picture_only_constraint_flag is equal to 1, the value of no_vps_constraint_flag shall be equal to 1. no_ilrp_constraint_flag equal to 1 indicates that the value of sps_inter_layer_ref_pics_present_flag shall be equal to 0. no_ilrp_constraint_flag equal to 0 imposes no such constraint. no_mnli_constraint_flag equal to 1 indicates that all VCL NAL units in the CVS shall have the same value of nuh_layer_id. no_mnli_constraint equal to 0 imposes no such constraint. If the value of no_mnli_constraint is equal to 1, one or more of the following applies: - The value of vps_max_layers_minus1 shall be equal to 0. - The value of each_layer_is_an_ols_flag is assumed to be equal to 1. no_mols_constraint_flag equal to 1 indicates that the total number of output layer sets (OLS) specified in the VPS is equal to 1. no_mols_constraint_flag equal to 0 imposes no such constraint. If the value of no_mols_constraint_flag is equal to 1, one or more of the following apply: - The value of no_vps_constraint_flag shall be equal to 0. - The value of sps_video_parameter_set shall not be equal to 0. - If the value of vps_max_layers_minus1 is greater than 0, the value of each_layer_is_an_ols_flag shall be equal to 0 and the value of vps_all_independent_layers_flag shall be equal to 0. - if the value of ols_mode_idc is equal to 2, the value of num_output_layer_sets_minus1 shall be equal to 0. Otherwise, the value of ols_mode_idc shall be equal to 0 or 1 and the value of vps_max_layers_minus1 shall be equal to 0.

[0055] Table 13: Example of General Supplemental Enhancement Information Message Constraint Information Syntax

[0056] [Table 13] The proposed semantics of the new flag are as follows: no_scalable_nesting_SEI_constraint_flag equal to 1 indicates that there shall be no scalable nesting SEI messages included in the OlsInScope bitstream. no_scalable_nesting_SEI_constraint_flag equal to 0 imposes no such constraint. no_subpic_level_SEI_constraint_flag equal to 1 indicates that there shall be no subpicture level SEI messages included in the OlsInScope bitstream. no_subpic_level_SEI_constraint_flag equal to 0 imposes no such constraint. no_filler_payload_SEI_constraint_flag equal to 1 indicates that there shall be no filler payload SEI messages included in the OlsInScope bitstream. no_filler_payload_SEI_constraint_flag equal to 0 imposes no such constraint. no_user_data_reg_SEI_constraint_flag equal to 1 indicates that there shall be no user data registered by Recommendation ITU T.35 SEI messages included in the OlsInScope bitstream. no_user_data_reg_SEI_constraint_flag equal to 0 imposes no such constraint. no_user_data_unreg_SEI_constraint_flag equal to 1 indicates that there shall be no User Data Unregistered SEI messages included in the OlsInScope bitstream. no_user_data_unreg_SEI_constraint_flag equal to 0 imposes no such constraint. no_film_grain_SEI_constraint_flag equal to 1 indicates that there shall be no film grain characteristics SEI messages included in the OlsInScope bitstream. no_film_grain_SEI_constraint_flag equal to 0 imposes no such constraint. no_parameter_set_incl_SEI_constraint_flag equal to 1 indicates that there shall be no parameter set inclusion indication SEI messages included in the OlsInScope bitstream. no_parameter_set_incl_SEI_constraint_flag equal to 0 imposes no such constraint. no_decoded_picture_hash_SEI_constraint_flag equal to 1 indicates that there shall be no decoded picture hash SEI messages included in the OlsInScope bitstream. no_decoded_picture_hash_SEI_constraint_flag equal to 0 imposes no such constraint. no_mcdv_SEI_constraint_flag equal to 1 indicates that there shall be no Mastering Display Color Volume SEI messages included in the OlsInScope bitstream. no_mcdv_SEI_constraint_flag equal to 0 imposes no such constraint. no_cll_SEI_constraint_flag equal to 1 indicates that there shall be no content light level SEI messages included in the OlsInScope bitstream. no_cll_SEI_constraint_flag equal to 0 imposes no such constraint. no_DRAP_SEI_constraint_flag equal to 1 indicates that there shall be no dependency random access point indication SEI messages included in the OlsInScope bitstream. no_DRAP_SEI_constraint_flag equal to 0 imposes no such constraint. no_alt_transfer_char_SEI_constraint_flag equal to 1 indicates that there shall be no alternate transfer characteristics SEI messages included in the OlsInScope bitstream. no_alt_transfer_char_SEI_constraint_flag equal to 0 imposes no such constraint. no_ambient_view_envir_SEI_constraint_flag equal to 1 indicates that there shall be no ambient viewing environment SEI messages included in the OlsInScope bitstream. no_ambient_view_envir_SEI_constraint_flag equal to 0 imposes no such constraint. no_ccv_SEI_constraint_flag equal to 1 indicates that there shall be no content, color, or volume SEI messages included in the OlsInScope bitstream. no_ccv_SEI_constraint_flag equal to 0 imposes no such constraint. no_omni_video_specific_SEI_constraint_flag equal to 1 indicates that there shall be no omnidirectional video specific SEI messages included in the OlsInScope bitstream. no_omni_video_specific_SEI_constraint_flag equal to 0 imposes no such constraint. no_field_frame_SEI_constraint_flag equal to 1 indicates that there shall be no Frame Field Information SEI messages included in the OlsInScope bitstream. no_field_frame_SEI_constraint_flag equal to 0 imposes no such constraint. no_sar_SEI_constraint_flag equal to 1 indicates that no sample aspect ratio SEI message shall be present. For completeness, the following section restates the semantics of the flags in Table 2-13, including both existing and newly proposed ones.

[0057] General Format Constraint Information Semantics general_frame_only_constraint_flag equal to 1 indicates that OlsInScope carries a picture that represents a frame. general_frame_only_constraint_flag equal to 0 indicates that OlsInScope carries a picture that may or may not represent a frame. If general_frame_only_constraint_flag is equal to 1, the value of sps_field_seq_flag shall be equal to 0.

[0058] NOTE - A decoder may ignore the value of general_frame_only_constraint_flag because there are no decoding process requirements associated with it. max_bitdepth_minus8_constraint_idc indicates that sps_bit_depth_minus8 shall be in the range of 0 to max_bitdepth_minus8_constraint_idc, inclusive. max_chroma_format_constraint_idc indicates that sps_chroma_format_idc shall be in the range of 0 to max_chroma_format_constraint_idc, inclusive.

[0059]

[0052] General Functional Information Semantics general_one_picture_only_constraint_flag equal to 1 indicates that there is only one coded picture in the bitstream. general_one_picture_only_constraint_flag equal to 0 imposes no such constraint. no_scalability_constraint_flag equal to 1 indicates that scalable layered coding is disabled for CVS. no_scalability_constraint_flag equal to 0 imposes no such constraints. The value of no_scalability_constraint_flag shall be equal to the value of the variable noScalabilityConstraint, which is derived as follows:

[0060]

number

[0061] General Partition Constraint Information Semantics one_tile_per_pic_constraint_flag equal to 1 indicates that each picture shall contain only one tile. one_tile_per_pic_constraint_flag equal to 0 imposes no such constraint. pic_header_in_slice_header_constraint_flag equal to 1 indicates that each picture shall contain only one slice and the value of sh_picture_header_in_slice_header_flag in each slice shall be equal to 1. pic_header_in_slice_header_constraint_flag equal to 0 imposes no such constraint. one_slice_per_pic_constraint_flag equal to 1 indicates that each picture shall contain only one slice. one_slice_per_pic_constraint_flag equal to 0 imposes no such constraint. If pic_header_in_slice_header_constraint_flag is equal to 1, the value of one_slice_per_pic_constraint_flag shall be equal to 1. one_subpic_per_pic_constraint_flag equal to 1 indicates that each picture shall contain only one subpicture and the value of sps_subpic_info_present_flag shall be equal to 0. one_subpic_per_pic_constraint_flag equal to 0 imposes no such constraint. If one_slice_per_pic_constraint_flag is equal to 1, the value of one_subpic_per_pic_constraint_flag shall be equal to 1. no_qtbtt_dual_tree_intra_constraint_flag equal to 1 indicates that sps_qtbtt_dual_tree_intra_flag shall be equal to 0. no_qtbtt_dual_tree_intra_constraint_flag equal to 0 imposes no such constraint. If max_chroma_format_constraint_idc is equal to 0, the value of no_qtbtt_dual_tree_intra_constraint_flag shall be equal to 1. no_partition_constraints_override_constraint_flag equal to 1 indicates that sps_partition_constraints_override_enabled_flag shall be equal to 0. no_partition_constraints_override_constraint_flag equal to 0 imposes no such constraint. no_virtual_boundary_constraint_flag equal to 1 indicates that sps_virtual_boundaries_enabled_flag shall be equal to 0. no_virtual_boundary_constraint_flag equal to 0 imposes no such constraint. General Prediction Mode Constraint Information Semantics An intra_only_constraint_flag equal to 1 indicates that sh_slice_type shall be equal to 1. An intra_only_constraint_flag equal to 0 imposes no such constraint. If general_one_picture_only_constraint_flag is equal to 1, the value of intra_only_constraint_flag shall be equal to 1. no_palette_constraint_flag equal to 1 indicates that sps_palette_enabled_flag shall be equal to 0. no_palette_constraint_flag equal to 0 imposes no such constraint. no_ibc_constraint_flag equal to 1 indicates that sps_ibc_enabled_flag shall be equal to 0. no_ibc_constraint_flag equal to 0 imposes no such constraint. General Intra-Constraint Information Semantics no_mrl_constraint_flag equal to 1 indicates that sps_mrl_enabled_flag shall be equal to 0. no_mrl_constraint_flag equal to 0 imposes no such constraint. no_isp_constraint_flag equal to 1 indicates that sps_isp_enabled_flag shall be equal to 0. no_isp_constraint_flag equal to 0 imposes no such constraint. no_mip_constraint_flag equal to 1 indicates that sps_mip_enabled_flag s shall be equal to 0. no_mip_constraint_flag equal to 0 imposes no such constraint. no_cclm_constraint_flag equal to 1 indicates that sps_cclm_enabled_flag shall be equal to 0. no_cclm_constraint_flag equal to 0 imposes no such constraint. If max_chroma_format_constraint_idc is equal to 0, the value of no_cclm_constraint_flag shall be equal to 1.

[0062] General Inter-Constraint Information Semantics no_ref_pic_resampling_constraint_flag equal to 1 indicates that sps_ref_pic_resampling_enabled_flag shall be equal to 0. no_ref_pic_resampling_constraint_flag equal to 0 imposes no such constraint. no_res_change_in_clvs_constraint_flag equal to 1 indicates that sps_res_change_in_clvs_allowed_flag shall be equal to 0. no_res_change_in_clvs_constraint_flag equal to 0 imposes no such constraint. If no_ref_pic_resampling_constraint_flag is equal to 1, no_res_change_in_clvs_constraint_flag shall be equal to 1. no_ref_wraparound_constraint_flag equal to 1 indicates that sps_ref_wraparound_enabled_flag shall be equal to 0. no_ref_wraparound_constraint_flag equal to 0 imposes no such constraint. If intra_only_constraint_flag is equal to 1, the value of no_ref_wraparound_constraint_flag shall be equal to 1. no_temporal_mvp_constraint_flag equal to 1 indicates that sps_temporal_mvp_enabled_flag shall be equal to 0. no_temporal_mvp_constraint_flag equal to 0 imposes no such constraint. If intra_only_constraint_flag is equal to 1, the value of no_temporal_mvp_constraint_flag shall be equal to 1. no_sbtmvp_constraint_flag equal to 1 indicates that sps_sbtmvp_enabled_flag shall be equal to 0. no_sbtmvp_constraint_flag equal to 0 imposes no such constraint. If no_temporal_mvp_constraint_flag is equal to 1, the value of no_sbtmvp_constraint_flag shall be equal to 1. no_amvr_constraint_flag equal to 1 indicates that sps_amvr_enabled_flag shall be equal to 0. no_amvr_constraint_flag equal to 0 imposes no such constraint. If intra_only_constraint_flag is equal to 1, the value of no_amvr_constraint_flag shall be equal to 1. no_bdof_constraint_flag equal to 1 indicates that sps_bdof_enabled_flag shall be equal to 0. no_bdof_constraint_flag equal to 0 imposes no such constraint. If intra_only_constraint_flag is equal to 1, the value of no_bdof_constraint_flag shall be equal to 1. no_dmvr_constraint_flag equal to 1 indicates that sps_dmvr_enabled_flag shall be equal to 0. no_dmvr_constraint_flag equal to 0 imposes no such constraint. If intra_only_constraint_flag is equal to 1, the value of no_dmvr_constraint_flag shall be equal to 1. no_affine_motion_constraint_flag equal to 1 indicates that sps_affine_enabled_flag shall be equal to 0. no_affine_motion_constraint_flag equal to 0 imposes no such constraint. If intra_only_constraint_flag is equal to 1, the value of no_affine_motion_constraint_flag shall be equal to 1. no_mmvd_constraint_flag equal to 1 indicates that sps_mmvd_enabled_flag shall be equal to 0. no_mmvd_constraint_flag equal to 0 imposes no such constraint. If intra_only_constraint_flag is equal to 1, the value of no_mmvd_constraint_flag shall be equal to 1. no_smvd_constraint_flag equal to 1 indicates that sps_smvd_enabled_flag shall be equal to 0. no_smvd_constraint_flag equal to 0 imposes no such constraint. If intra_only_constraint_flag is equal to 1, the value of no_smvd_constraint_flag shall be equal to 1. no_prof_constraint_flag equal to 1 indicates that sps_affine_prof_enabled_flag shall be equal to 0. no_prof_constraint_flag equal to 0 imposes no such constraint. If intra_only_constraint_flag is equal to 1, the value of no_prof_constraint_flag shall be equal to 1. no_bcw_constraint_flag equal to 1 indicates that sps_bcw_enabled_flag shall be equal to 0. no_bcw_constraint_flag equal to 0 imposes no such constraint. If intra_only_constraint_flag is equal to 1, the value of no_bcw_constraint_flag shall be equal to 1. no_ciip_constraint_flag equal to 1 indicates that sps_ciip_enabled_flag shall be equal to 0. no_cipp_constraint_flag equal to 0 imposes no such constraint. If intra_only_constraint_flag is equal to 1, the value of no_cipp_constraint_flag shall be equal to 1. no_gpm_constraint_flag equal to 1 indicates that sps_gpm_enabled_flag shall be equal to 0. no_gpm_constraint_flag equal to 0 imposes no such constraint. If intra_only_constraint_flag is equal to 1, the value of no_gpm_constraint_flag shall be equal to 1. no_weighted_pred_constraint_flag equal to 1 indicates that sps_weighted_pred_flag shall be equal to 0. no_weighted_pred_constraint_flag equal to 0 imposes no such constraint. no_weighted_bipred_constraint_flag equal to 1 indicates that sps_weighted_bipred_flag shall be equal to 0. no_weighted_bipred_constraint_flag equal to 0 imposes no such constraint.

[0063] General transformation constraint information semantics no_mts_constraint_flag equal to 1 indicates that sps_mts_enabled_flag shall be equal to 0. no_mts_constraint_flag equal to 0 imposes no such constraint. no_sbt_constraint_flag equal to 1 indicates that sps_sbt_enabled_flag shall be equal to 0. no_sbt_constraint_flag equal to 0 imposes no such constraint. no_lfnst_constraint_flag equal to 1 indicates that sps_lfnst_enabled_flag shall be equal to 0. no_lfnst_constraint_flag equal to 0 imposes no such constraint. no_transform_skip_constraint_flag equal to 1 indicates that sps_transform_skip_enabled_flag shall be equal to 0. no_transform_skip_constraint_flag equal to 0 imposes no such constraint. no_act_constraint_flag equal to 1 indicates that sps_act_enabled_flag shall be equal to 0. no_act_constraint_flag equal to 0 imposes no such constraint. no_tsrc_constraint_flag equal to 1 indicates that sh_ts_residual_coding_disabled_flag shall be equal to 1. no_tsrc_constraint_flag equal to 0 imposes no such constraint. If no_transform_skip_constraint_flag is equal to 1, the value of no_tsrc_constraint_flag shall be equal to 1.

[0064] General quantization constraint information semantics no_joint_cbcr_constraint_flag equal to 1 indicates that sps_joint_cbcr_enabled_flag shall be equal to 0. no_joint_cbcr_constraint_flag equal to 0 imposes no such constraint. If max_chroma_format_constraint_idc is equal to 0, the value of no_joint_cbcr_constraint_flag shall be equal to 1. no_bdpcm_constraint_flag equal to 1 indicates that sps_bdpcm_enabled_flag shall be equal to 0. no_bdpcm_constraint_flag equal to 0 imposes no such constraint. If no_transform_skip_constraint_flag is equal to 1, then no_bdpcm_constraint_flag shall be equal to 1. no_cu_qp_delta_constraint_flag equal to 1 indicates that pps_cu_qp_delta_enabled_flag shall be equal to 0. no_cu_qp_delta_constraint_flag equal to 0 imposes no such constraint. no_chroma_qp_offset_constraint_flag equal to 1 indicates that pps_cu_chroma_qp_offset_list_enabled_flag shall be equal to 0. no_chroma_qp_offset_constraint_flag equal to 0 imposes no such constraint. no_dep_quant_constraint_flag equal to 1 indicates that sps_dep_quant_enabled_flag shall be equal to 0. no_dep_quant_constraint_flag equal to 0 imposes no such constraint. no_sign_data_hiding_constraint_flag equal to 1 indicates that sps_sign_data_hiding_enabled_flag shall be equal to 0. no_sign_data_hiding_constraint_flag equal to 0 imposes no such constraint. no_explicit_scaling_list_constraint_flag equal to 1 indicates that sps_explicit_scaling_list_enabled_flag shall be equal to 0. no_explicit_scaling_list_constraint_flag equal to 0 imposes no such constraint. If no_aps_constraint_flag is equal to 1, the value of no_explicit_scaling_list_constraint_flag shall be equal to 1.

[0065] General Loop Filter Constraint Information Semantics no_sao_constraint_flag equal to 1 indicates that sps_sao_enabled_flag shall be equal to 0. no_sao_constraint_flag equal to 0 imposes no such constraint. no_alf_constraint_flag equal to 1 indicates that sps_alf_enabled_flag shall be equal to 0. no_alf_constraint_flag equal to 0 imposes no such constraint. no_ccalf_constraint_flag equal to 1 indicates that sps_ccalf_enabled_flag shall be equal to 0. no_ccalf_constraint_flag equal to 0 imposes no such constraint. If max_chroma_format_constraint_idc is equal to 0 or no_alf_constraint_flag is equal to 1, the value of no_ccalf_constraint_flag shall be equal to 1. no_ladf_constraint_flag equal to 1 indicates that sps_ladf_enabled_flag shall be equal to 0. no_ladf_constraint_flag equal to 0 imposes no such constraint. no_lmcs_constraint_flag equal to 1 indicates that sps_lmcs_enabled_flag shall be equal to 0. no_lmcs_constraint_flag equal to 0 imposes no such constraint. If no_aps_constraint_flag is equal to 1, the value of no_lmcs_constraint_flag shall be equal to 1.

[0066] General Hierarchical Constraint Information Semantics no_vps_constraint_flag equal to 1 indicates that sps_video_parameter_set_id shall be equal to 0. no_vps_constraint_flag equal to 0 imposes no such constraint. If general_one_picture_only_constraint_flag is equal to 1, the value of no_vps_constraint_flag shall be equal to 1. single_layer_constraint_flag equal to 1 indicates that vps_max_layers_minus1 shall be equal to 0. single_layer_constraint_flag equal to 0 imposes no such constraint. If general_one_picture_only_constraint_flag is equal to 1 or no_vps_constraint_flag is equal to 1, the value of single_layer_constraint_flag shall be equal to 1. all_layers_independent_constraint_flag equal to 1 indicates that vps_all_independent_layers_flag, if present, shall be equal to 1, and, if absent, shall be inferred to be equal to 1. all_layers_independent_constraint_flag equal to 0 imposes no such constraint. General Supplemental Enhancement Information Constraint Information Semantics NOTE 1 - Decoders may ignore the value of the General Supplemental Enhancement Information Constraint Information flag, since there are no decoding process requirements related to the presence or interpretation of the corresponding SEI message. no_scalable_nesting_SEI_constraint_flag equal to 1 indicates that there shall be no scalable nesting SEI messages included in the OlsInScope bitstream. no_scalable_nesting_SEI_constraint_flag equal to 0 imposes no such constraint. no_subpic_level_SEI_constraint_flag equal to 1 indicates that there shall be no subpicture level SEI messages included in the OlsInScope bitstream. no_subpic_level_SEI_constraint_flag equal to 0 imposes no such constraint. no_filler_payload_SEI_constraint_flag equal to 1 indicates that there shall be no filter payload SEI messages included in the OlsInScope bitstream. no_filler_payload_SEI_constraint_flag equal to 0 imposes no such constraint. no_user_data_reg_SEI_constraint_flag equal to 1 indicates that there shall be no user data registered by Recommendation ITU T.35 SEI messages included in the OlsInScope bitstream. no_user_data_reg_SEI_constraint_flag equal to 0 imposes no such constraint. no_user_data_unreg_SEI_constraint_flag equal to 1 indicates that there shall be no User Data Unregistered SEI messages included in the OlsInScope bitstream. no_user_data_unreg_SEI_constraint_flag equal to 0 imposes no such constraint. no_film_grain_SEI_constraint_flag equal to 1 indicates that there shall be no film grain characteristics SEI messages included in the OlsInScope bitstream. no_film_grain_SEI_constraint_flag equal to 0 imposes no such constraint. general_non_packed_SEI_constraint_flag equal to 1 indicates that there shall be no frame packing alignment SEI messages included in the OlsInScope bitstream. general_non_packed_SEI_constraint_flag equal to 0 imposes no such constraint.

[0067] NOTE 2 - Decoders may ignore the value of general_non_packed_constraint_flag because there are no decoding process requirements related to the presence or interpretation of the Frame Packing Alignment SEI message. general_non_projected_SEI_constraint_flag equal to 1 indicates that there shall be no equirectangular projection SEI messages or generalized cubemap projection SEI messages included in the OlsInScope bitstream. general_non_projected_SEI_constraint_flag equal to 0 imposes no such constraints. no_parameter_set_incl_SEI_constraint_flag equal to 1 indicates that there shall be no parameter set inclusion indication SEI messages included in the OlsInScope bitstream. no_parameter_set_incl_SEI_constraint_flag equal to 0 imposes no such constraint. no_decoded_picture_hash_SEI_constraint_flag equal to 1 indicates that there shall be no decoded hash SEI messages included in the OlsInScope bitstream. no_decoded_picture_hash_SEI_constraint_flag equal to 0 imposes no such constraint. no_mcdv_SEI_constraint_flag equal to 1 indicates that there shall be no Mastering Display Color Volume SEI messages included in the OlsInScope bitstream. No_mcdv_SEI_constraint_flag equal to 0 imposes no such constraint. no_cll_SEI_constraint_flag equal to 1 indicates that there shall be no content light level SEI messages included in the OlsInScope bitstream. no_cll_SEI_constraint_flag equal to 0 imposes no such constraint. no_DRAP_SEI_constraint_flag equal to 1 indicates that there shall be no dependency random access point indication SEI messages included in the OlsInScope bitstream. no_DRAP_SEI_constraint_flag equal to 0 imposes no such constraint. no_alt_transfer_char_SEI_constraint_flag equal to 1 indicates that there shall be no alternate transfer characteristics SEI messages included in the OlsInScope bitstream. no_alt_transfer_char_SEI_constraint_flag equal to 0 imposes no such constraint. no_ambient_view_envir_SEI_constraint_flag equal to 1 indicates that there shall be no ambient viewing environment SEI messages included in the OlsInScope bitstream. no_ambient_view_envir_SEI_constraint_flag equal to 0 imposes no such constraint. no_ccv_SEI_constraint_flag equal to 1 indicates that there shall be no content, color, or volume SEI messages included in the OlsInScope bitstream. no_ccv_SEI_constraint_flag equal to 0 imposes no such constraint. no_omni_video_specific_SEI_constraint_flag equal to 1 indicates that there shall be no omni-video-specific SEI messages included in the OlsInScope bitstream. no_omni_video_specific_SEI_constraint_flag equal to 0 imposes no such constraint. no_field_frame_SEI_constraint_flag equal to 1 indicates that there shall be no Frame Field Information SEI messages included in the OlsInScope bitstream. no_field_frame_SEI_constraint_flag equal to 0 imposes no such constraint. no_sar_SEI_constraint_flag equal to 1 indicates that there shall be no sample aspect ratio SEI message included.

[0063] Alternative Representations In an embodiment, instead of expressing the general constraint information structure according to Table 1, it is possible to apply an alternative syntax format used in Table 14, where the 12 general_xxx_constraint_info() structures are replaced with corresponding general_xxx_constraint_info syntax elements, where again "xxx" indicates an aspect of VVC coding such as partitioning, intra-coding, loop filtering, etc.

[0068] Table 14: Examples of alternative general constraint information syntax

[0069] [Table 14] Tables 15 and 16 provide example syntax and semantics for general_format_constraint_info, which may use 7 bits of total information, and are intended to illustrate how the syntax and semantics of all other general constraint flags are specified.

[0070] Table 15: Example of general format constraint information syntax (variant 1)

[0071] [Table 15] Table 16: Example of general format constraint information syntax (variant 2)

[0072] [Table 16]

[0065] In an embodiment, as an example, the semantics of Table 16 may be explained as follows: GeneralFrameOnlyConstraintFlag equal to 1 indicates that OlsInScope carries a picture that represents a frame. GeneralFrameOnlyConstraintFlag equal to 0 indicates that OlsInScope carries a picture that may or may not represent a frame. If GeneralFrameOnlyConstraintFlag equal to 1 is equal to 1, the value of sps_field_seq_flag shall be equal to 0.

[0073] NOTE - A decoder may ignore the value of general_frame_only_constraint_flag, since there are no decoding process requirements associated with it. MaxBitdepthMinus8ConstraintIdc indicates that sps_bit_depth_minus8 shall be in the range of 0 to MaxBitdepthMinus8ConstraintIdc, inclusive. MaxChromaFormatConstraintIdc indicates that sps_chroma_format_idc shall be in the range of 0 to MaxChromaFormatConstraintIdc, inclusive. NoSeparateColourPlaneConstraintFlag indicates that sps_separate_colour_plane_flag, if present, shall be equal to 0. NoSeparateColourPlaneConstraintFlag equal to 0 imposes no such constraint. In another embodiment, instead of expressing the general constraint information structure according to Table 1, it is possible to apply an alternative syntax format used in Table 17, where the 12 general_xxx_constraint_info() structures are replaced with corresponding general_xxx_constraint_info_present_flag syntax elements, where again, "xxx" indicates an aspect of VVC coding such as partitioning, intra-coding, loop filtering, etc. Each general_xxx_constraint_info_present_flag conditions the signaling of the corresponding general_xxx_constraint_info( ) syntax structure. Conditioning the presence of general_xxx_constraint_info( ) syntax structures in general_xxx_constraint_info_present_flag facilitates parsing (the decoder only needs to parse those general_xxx_constraint_info() structures that have their corresponding flag enabled) and provides an additional measure for determining that the bitstream is conforming.

[0074] Table 17: Example of another embodiment of the proposed "General Constraint Information Syntax"

[0075] [Table 17] general_xxx_constraint_info_present_flag equal to 1 indicates that the value of at least one general constraint flag signaled in general_xxx_constraint_info( ) shall not be equal to 0. general_xxx_constraint_info_present_flag equal to 0 indicates that the values ​​of all general constraint flags signaled in general_xxx_constraint_info( ) shall be equal to 0.

[0076]

[0068] In another embodiment, instead of expressing the general transform constraint information structure and the general quantization constraint information structure according to Tables 9 and 10, it is possible to apply an alternative syntax format used in Table 18, in which the general_transform_constraint_info() structure and the general_quantization_constraint_info() structure in the 12 general_xxx_constraint_info() structures are replaced with a new general_tqr_constraint_info() structure, which groups general constraint flags related to transform, quantization, and residual coding that are signaled together as a means of reducing confusion when classifying a particular general constraint flag as transform, quantization, or residual coding when there is uncertainty.

[0077] Table 18: Example of another embodiment of the general transform, quantization, and residual constraint information syntax

[0078] [Table 18] In another embodiment, to make future extensions easier, the syntax of the reserved bytes (gci_tqr_num_reserved_bytes) can be expressed as an "extension flag" followed by the corresponding extension data that is read if the extension flag is true.

[0079]

[0069] Figure 2A shows an exemplary process for video encoding according to an embodiment. As shown in Figure 2A, in step 205, the encoder analyzes the content and coding requirements (e.g., picture resolution, frame rate, available bandwidth, acceptable delay, processing power, etc.) to determine which tools to use and which tools can be skipped. In multi-layer coding, these decisions can be made independently for each layer. In steps 210 and 215, for each previously determined tool, iterates through all of the general_xxx_constraint_info() functions defined under general_constraint_info(), e.g., as defined in Table 1, to specify specific constraint flags for each tool, where "xxx" indicates the associated tool and decision category. Next, in step 220, the encoder sets corresponding syntax values ​​in various syntax layers (e.g., sequence parameter level (SPS), picture parameter level (PPS), picture header (PH), slice header, etc.). Finally, in step 225, the input video is encoded to produce a coded bitstream 230 that conforms to all constraint parameters previously defined (e.g., as in steps 210 and 215). Such a bitstream also includes a general_constraint_info() syntax structure for the decoder.

[0080]

[0070] Figure 2B shows an exemplary process for video decoding according to an embodiment under the encoding process of Figure 2A. In step 240, under the coded bitstream 230, the decoder parses the bitstream to identify bitstream constraints defined as part of the general_constraint_info() structure. In steps 245 and 250, the decoder iterates through all general_xxx_constraint_info() structures defined under general_constraint_info(), for example, as defined in Table 1, and reads all constraint flags for each category "xxx" of coding tools. In step 255, previously extracted parameters can be used to check conformance by comparing them for consistency with other flags and parameters that are part of the coded bitstream. Finally, in step 260, the coded bitstream is decoded to generate a sequence of video frames.

[0081]

[0071] Examples of Constrained Coding and Conformance Testing for Subpictures and Scalability The VVC specification (Ref. [2]) defines a subpicture as any rectangular region of one or more slices within a picture. Broadly speaking, a subpicture allows for separately coded picture sub-streams or views that can be reconstructed into a single picture by a decoder.

[0082] In VVC, a slice can be defined in terms of "tile" and "coding tree unit", which, together with some other VVC syntax elements, are explained as follows:

[0073] A slice is an integer number of complete tiles, or an integer number of consecutive complete coding tree unit (CTU) rows within a tile of a picture, that are contained exclusively in a single Network Abstraction Layer (NAL) unit.

[0083]

[0074] A tile is a rectangular region of a CTU within a particular tile column and a particular tile row in a picture.

[0084]

[0075] A tile column is a rectangular region of a CTU with a width specified by a syntax element in the picture parameter set and a height equal to the height of the picture.

[0085]

[0076] A tile row is a rectangular region of a CTU with a width equal to the width of the picture and a height specified by a syntax element in the picture parameter set.

[0086]

[0077] A coding block is an MxN block of samples for some values ​​of M and N, and as a result, the division of the CTB into coding blocks is a partitioning.

[0087]

[0078] A coding tree block (CTB) is an NxN block of samples for some value of N, so that the division of components into CTBs is a partitioning.

[0088]

[0079] A coding tree unit (CTU) is a CTB of luma samples, two corresponding CTBs of chroma samples in a picture with three sample arrays, or a CTB of samples in a monochrome picture or a picture that is coded using three separate color planes and a syntax structure used to code the samples.

[0089] Scalability In previous coding standards, such as HEVC, scalability support was defined using different profiles, such as the Scalable Profile. The current version of VVC supports layered scalability in the Main Profile. As defined in VVC, a layer contains a set of Video Coding Layer (VCL) NAL units and associated non-VCL NAL units, all with a specific value of nuh_layer_id.

[0090] In a recent contribution to the VVC specification (Ref. [3]), the authors propose two profiles for 10-bit, 4:2:0 video support in Version 1 of the VVC specification, allowing relevant markets to choose the profile that best suits their needs: "Main 10" profile with subpicture and scalability "Constrained Main 10" profile without subpicture and scalability As an example, in an embodiment, to disable subpictures, it is possible to use the existing VVC syntax element one_subpic_per_pic_constraint_flag, defined below: one_subpic_per_pic_constraint_flag equal to 1 indicates that each picture shall contain only one subpicture. one_subpic_per_pic_constraint_flag equal to 0 imposes no such constraint. If one_slice_per_pic_constraint_flag is equal to 1, the value of one_subpic_per_pic_constraint_flag shall be equal to 1.

[0091] To disable scalable / layered coding, Ref. [3] proposes to apply the following constraints to the proposed VVC Constrained Main10 profile: The Sequence Parameter Set (SPS) syntax element inter_layer_ref_pics_present_flag shall be equal to 0. This disables inter-layer prediction, which is used for SNR and spatial scalability.

[0092] The Video Parameter Set (VPS) syntax element vps_max_layers_minus1 shall be equal to 0. This disables multiple output layer sets and eliminates the need for decoders to manage different output layer sets.

[0093] A single value of the NAL unit header syntax element nuh_layer_id shall be used across the entire coded video sequence (CVS), disabling multiple pictures in each access unit (AU).

[0094] In an embodiment, instead of creating additional profiles as suggested in, for example, Ref. [3], a similar functionality can be achieved by using the following general constraint information syntax element to prohibit scalable or layered coding: · no_scalability_constraint_flag, introduced as part of the syntax elements in Table 3, which, when set to 1, indicates to the decoder that scalable layered coding is disabled.

[0095]

[0085] In other embodiments, the value of no_scalability_constraint_flag may be conditioned or suggested based on the values ​​of the following suggested additional general constraint information syntax elements:

[0096] · no_vps_constraint_flag (Ref.[3], related to single_layer_constraint_flag) indicates that no VPS exists.

[0097] no_mols_constraint_flag disables multiple output layer sets, eliminating the need for the decoder to manage different output layer sets ('mols' stands for 'multiple output layer sets') (see also Table 12).

[0098] no_mnli_constraint_flag disables multiple pictures (multiple layers) in any access unit (AU) ('mnli' stands for 'multiple nuh_layer_id') (called no_mixed_nalu_types_in_pic_constraint_flag in Ref. [3]).

[0099] no_ilrp_constraint_flag disables inter-layer prediction ('ilrp' stands for 'inter-layer reference picture') used for SNR and spatial scalability (in Ref. [3] this appears as the equivalent of all_layers_independent_constraint_flag).

[0100]

[0086] Exemplary definitions of these syntax elements include the following: no_vps_constraint_flag equal to 1 indicates that the SPS does not reference a VPS. no_vps_constraint_flag equal to 0 indicates that a VPS may exist in the CVS. When no_vps_constraint_flag is equal to 1, one or more of the following apply: - The value of no_mnli_constraint_flag shall be equal to 1. - The value of no_ilrp_constraint_flag shall be equal to 1. - The value of sps_video_parameter_set_id shall be equal to 0. - The value of vps_max_layers_minus1 is assumed to be equal to 0. - The value of each_layer_is_an_ols_flag is assumed to be equal to 1. - The value of inter_layer_ref_pics_present_flag is inferred to be equal to 0.

[0087] no_mols_constraint_flag equal to 1 indicates that the total number of output layer sets (OLS) specified in the VPS is equal to 1. no_mols_constraint_flag equal to 0 imposes no such constraint. When no_mols_constraint_flag is equal to 1, one or more of the following apply: - The value of no_vps_constraint_flag shall be equal to 0. - The value of sps_video_parameter_set shall be equal to 0. - If the value of vps_max_layers_minus1 is greater than 0, the value of each_layer_is_an_ols_flag shall be equal to 0 and the value of vps_all_independent_layers_flag shall be 0. - If the value of ols_mode_idc is equal to 2, the value of num_output_layer_sets_minus1 shall be equal to 0. Otherwise, the value of ols_mode_idc shall be equal to 0 or 1 and the value of vps_max_layers_minus1 shall be equal to 0.

[0101]

[0088] no_mnli_constraint_flag equal to 1 indicates that all VCL NAL units in the CVS shall have the same value of nuh_layer_id. no_mnli_constraint equal to 0 imposes no such constraint. If the value of no_mnli_constraint is equal to 1, one or more of the following applies: - The value of vps_max_layers_minus1 shall be equal to 0. - The value of each_layer_is_an_ols_flag is assumed to be equal to 1. no_ilrp_constraint_flag equal to 1 indicates that the inter layer reference picture (ILRP) is not used for inter prediction of any coded picture in the coded layer video sequence (CLVS). no_ilrp_constraint_flag equal to 1 imposes no such constraint. If the value of no_ilrp_constraint_flag is equal to 1, one or more of the following apply: - The value of inter_layer_ref_pics_present_flag shall be equal to 0. - the value of vps_all_independent_layers_flag shall be equal to 1. In an embodiment, the variable noScalabilityConstraint specifies the value of no_scalability_constraint_flag, which may be derived as follows:

[0102]

number

[0103]

[0091] Figure 3 shows an exemplary process for determining whether scalability is enabled or disabled in a bitstream. As shown in Figure 3, the decoder can perform the logical operations shown in step 305, similar to those described above, and if it is true, scalability is disabled (310); otherwise, scalability is enabled (315).

[0104] Those skilled in the art will understand that the proposed new flags can also be applied to constrain or detect other features of the VVC codec beyond scalability and layered coding. The following examples are provided without limitation.

[0105] In one example, if no_vps_constraint_flag is equal to 1, the flag may be used to detect one or more of the following conditions of bitstream conformance or constraint: - The value of no_mnli_constraint_flag is equal to 1, which indicates that the CVS contains only one layer. - The value of no_ilrp_constraint_flag is equal to 1, which indicates that no inter-layer reference picture (ILRP) is used for inter-prediction prediction of any coded picture in CLVS. - The value of sps_video_parameter_set_id is equal to 0, which indicates that the SPS does not reference a VPS, the value of GeneralLayerIdx[ nuh_layer_id ] is inferred to be equal to 0, and the value of GeneralLayerIdx[ nuh_layer_id ] is inferred to be 0, and the value of vps_independent_layer_flag[ GeneralLayerIdx[ nuh_layer_id ] ] is inferred to be equal to 1. - The value of vps_max_layers_minus1 is assumed to be equal to 0. - The value of each_layer_is_an_ols_flag is assumed to be equal to 1. - The value of inter_layer_ref_pics_present_flag is inferred to be equal to 0. In another example, if no_mols_constraint_flag is equal to 1, it can be used to detect one or more of the following: The value of no_vps_constraint_flag is equal to 0, which indicates that the SPS may reference a VPS with a value of vps_video_parameter_set_id equal to the value of sps_video_parameter_set_id.

[0106]

[0095] In another example, if the value of no_mnli_contraints_flag is equal to 0 and the value of no_mols_constraint_flag is equal to 1, the CVS may contain multiple layers, but there is only one output layer set.

[0107] In another example, if the SPS does not reference the VPS, Check whether no_vps_constraint_flag is 1; check whether sps_video_parameter_set_id is 0, and independently, It is also possible to detect bitstream conformance by checking whether no_vps_constraint_flag is 1; and checking whether no_mnli_constraint_flag is 1.

[0108] In another example, if we are constraining that there is only one OLS, Check whether no_mols_constraint_flag is 1; check whether no_vps_constraint_flag is 0, and independently, Compliance can be detected by checking whether no_vps_constraint_flag is 1; and checking that sps_video_paramter_set is not 0.

[0109] Each of the references listed herein is incorporated by reference in its entirety. [Prior art documents] [Non-patent literature]

[0110] [Non-Patent Document 1] [1] High efficiency video coding, H.265, Series H, Coding of moving video, ITU, (02 / 2018). [Non-patent document 2] [2] B. Bross, J. Chen, S. Liu, and YK. Wang “Versatile Video Coding (Draft 9),” JVET output document, JVET-R2001, vA (10), JVET 18-th meeting, by teleconference, 15-24 April, 2020. [Non-patent document 3] [3] W. Wan et al., “VVC Version 1 Profiles,” JVET-R0392 (v.6), JVET 18th Meeting, April 15-24, 2020.

[0098] Computer System Implementation Example: Embodiments of the present invention may be implemented using a computer system, a system configured in electronic circuits and components, an integrated circuit such as a microcontroller, a field programmable gate array (FPGA) or other configurable or programmable logic device (PLD), a discrete-time or digital signal processor (DSP), an application-specific integrated circuit (ASIC), and / or an apparatus including one or more of these systems, devices, or components. The computer and / or IC may execute, control, or implement instructions related to constrained processing in video coding, such as those described herein. The computer and / or IC may compute any of the various parameters and values ​​related to constrained processing in video coding, such as those described herein. Image and video embodiments may be implemented in hardware, software, firmware, and various combinations thereof.

[0111]

[0099] Certain implementations of the present invention include a computer processor executing software instructions that cause the processor to perform the methods of the present invention. For example, one or more processors in a display, encoder, set-top box, transcoder, etc., can implement methods related to constrained layered processing in video coding, as described above, by executing software instructions in program memory accessible to the processor. Embodiments of the present invention may also be provided in the form of a program product. A program product may include any non-transitory, tangible medium carrying a set of computer-readable signals that, when executed by a data processor, cause the data processor to perform the methods of the present invention. A program product according to the present invention may be in any of a wide variety of non-transitory, tangible forms. A program product may include, for example, physical media such as magnetic data storage media including floppy diskettes, hard disk drives, optical data storage media including CD-ROMs and DVDs, ROMs, electronic data storage media including flash RAM, etc. The computer-readable signals in the program product may optionally be compressed or encrypted. Where a component (e.g., a software module, processor, assembly, device, circuit, etc.) is referred to above, unless otherwise indicated, references to such a component (including references to "means") should be interpreted as including equivalents (e.g., functional equivalents) of that component that perform the function of the described component, including any component that is not structurally equivalent to the disclosed structure but performs the function in the described exemplary embodiment of the present invention.

[0112]

[0100] Equivalents, extensions, substitutions and others Exemplary embodiments relating to constrained processing in video coding have been described. In the foregoing specification, embodiments of the invention have been described with reference to numerous specific details that may vary from implementation to implementation. The exclusive indicator of what is, and what is intended by the applicant to be, the invention is the set of claims issued from this application in a particular form, including any subsequent amendments. Any definitions expressly set forth herein for terms contained in such claims shall govern the meaning of such terms as used in the claims. Accordingly, no limitations, elements, properties, characteristics, effects, or attributes not expressly recited in a claim should in any way limit the scope of such claim. Accordingly, the specification and drawings should be interpreted in an illustrative, and not restrictive, sense.

[0101] Various aspects of the present invention may be appreciated from the enumerated example embodiments (EEE) that follow.

[0113] EEE 1. A method for decoding a video sequence coded with constraint layering (CVS) by a processor, comprising: receiving a coded video sequence (CVS) including coded pictures and syntax parameters, and detecting whether layering processing is enabled, wherein detecting whether layering processing is enabled includes receiving a coded video sequence (CVS) including coded pictures and syntax parameters including the following syntax parameters: a first flag indicating whether scalability or layered coding is enabled; A second flag indicating whether the video parameter set (VPS) is constrained; A third flag indicating whether the total number of output layer sets (OLS) is constrained; a fourth flag indicating whether the layer to which the Network Abstraction Layer (NAL) unit belongs is constrained; A fifth flag indicating whether inter-layer prediction is constrained; detecting whether one or more of determining that tiering processing is disabled if one or more of these flags are set to 1, and determining that tiering processing is enabled otherwise; A method comprising:

[0114] EEE 2. The method of EEE 1, wherein if the second flag is set to 1: the third flag is presumed to be set to 1; the fourth flag is presumed to be set to 1; the fifth flag is presumed to be set to 1; and The parameter vps_max_layers_minus1 is estimated to be equal to 1, method.

[0115] EE 3. In the method according to IEEE 1 or 2, when the third flag is set to 1: If the value of the ols_mode_idc parameter is equal to 2, the value of num_output_layer_sets_minus1 shall be equal to 0; Otherwise, if the value of ols_mode_idc is equal to 0 or 1, the value of vps_max_layers_minus1 shall be equal to 0.

[0116] EE 4. In the method of any one of IEEE 1-3, if the fourth flag is set to 1, it indicates that all video coding layer (VCL) NAL units in the coded video sequence (CVS) shall have the same value of nuh_layer_id and vps_max_layers_minus1 = 0.

[0117] EEE 5. A method according to any one of IEEE 1-4, wherein when the fifth flag is set to 1, it indicates that no inter-layer reference pictures (ILRPs) are used for inter-prediction of any coded picture in a coded layer video sequence (CLVS), and inter_layer_ref_pics_present_flag = 0.

[0118] EEE 6. A method for decoding a coded video sequence (CVS) by a processor, comprising: Receives a Coded Video Sequence (CVS) containing coded pictures and syntax parameters, including the following syntax parameters: The first flag indicates whether the video parameter set (VPS) is constrained; A second flag indicating whether the total number of output layer sets (OLS) is constrained; a third flag indicating whether the layer to which the Network Abstraction Layer (NAL) unit belongs is constrained; A fourth flag indicating whether inter-layer prediction is constrained; detecting whether one or more of determining bitstream compliance if one or more of these flags are set to 1; A method comprising:

[0119] EEE 7. IEEE 6. The method according to claim 6, wherein, upon detecting that the first flag is set to 1, one or more of the following apply: the third flag is set to 1; the fourth flag is set to 1; sps_video_parameter_set_id = 0; vps_max_layers_minus1 is assumed to be equal to 0; each_layer_is_an_ols_flag is inferred to be equal to 1; or inter_layer_ref_pics_present_flag is inferred to be equal to 0.

[0120] EEE 8. A method for encoding a sequence of video pictures with processing constraints in a processor, comprising: receiving a sequence of video pictures to be encoded into a coded bitstream; determining a set of tools that are not required for decoding the coded bitstream by a decoder; determining one or more constraint flags associated with the set of tools; grouping the one or more constraint flags into one or more tool constraint information syntax structures; integrating the one or more tool constraint information syntax structures into a general constraint syntax structure to generate the coded bitstream, the coded bitstream including coded pictures of the sequence of video pictures and the general constraint syntax structure; A method comprising:

[0121] EE 9. The method according to IEEE 8, wherein the one or more tool constraint information syntax structures include: Constraint flags associated with Network Abstraction Layer (NAL) units; Constraint flags associated with the partition, Constraint flags associated with the prediction mode; Constraint flags related to intra prediction, Constraint flags related to inter prediction, Constraint flags associated with the transformation, quantization-related constraint flags, Constraint flags related to loop filtering, Constraint flags related to loop formats, Constraint flags related to general features, or Constraint flags related to Supplemental Enhancement Information (SEI) messaging; The method includes syntax elements relating to one or more of:

[0122] EEE 10. A method as defined in IEEE 9, wherein the syntax elements associated with the general feature related constraint flags include a non-scalability constraint flag which, when set equal to 1, indicates that scalable and layered coding is disabled for the coded bitstream.

[0123] EEE 11. A method for decoding, by a processor, a bitstream coded with constraint processing, comprising: receiving at a decoder a coded bitstream including coded pictures of a sequence of video pictures and a general constraint syntax structure, the general constraint syntax structure including syntax elements associated with a set of tools that are not necessary for decoding the coded bitstream by the decoder; parsing the general constraint syntax structure to identify one or more tool constraint information syntax structures, each tool constraint information syntax structure including one or more constraint flags associated with a particular coding tool; parsing each of the one or more tool constraint information syntax structures to generate one or more constraint flags associated with the set of tools; and decoding the coded pictures in the coded bitstream in accordance with the one or more constraint flags to generate the sequence of video pictures; A method comprising:

[0124] EE 12. The method according to IEEE 11, wherein the one or more tool constraint information syntax structures include: Constraint flags associated with Network Abstraction Layer (NAL) units; Constraint flags associated with the partition, Constraint flags associated with the prediction mode; Constraint flags related to intra prediction, Constraint flags related to inter prediction, Constraint flags associated with the transformation, quantization-related constraint flags, Constraint flags related to loop filtering, Constraint flags related to loop formats, Constraint flags related to general features, or Constraint flags related to Supplemental Enhancement Information (SEI) messaging; The method includes syntax elements relating to one or more of:

[0125] EEE 13. The method according to IEEE 12, wherein the syntax elements related to "constraint flags associated with the general features" include a non-scalability constraint flag, which, when set equal to 1, indicates that scalable and layered coding is disabled for the coded bitstream.

[0126] EEE 14. A non-transitory computer-readable storage medium storing computer-executable instructions for performing the method set forth in any one of IEEE 1-13 on the one or more processors.

[0127] EEE 15. An apparatus including a processor and configured to perform a method according to any one of IEEE 1-13.

Claims

1. 1. An apparatus for encoding a sequence of video pictures with constrained processing, comprising: a processor, the processor comprising: receiving a sequence of video pictures to be encoded into a coded bitstream; determining a set of constraints on values ​​of syntax elements and coding tools of the coded bitstream; determining one or more constraint flags for the set of restrictions; aggregating the one or more constraint flags into a general constraint syntax structure based on a predetermined number of categories, the general constraint syntax structure comprising: the one or more constraint flags belonging to a first category of the predetermined number of categories include a first set of constraint flags related to loop filtering, the first set of constraint flags including a constraint flag indicating whether a virtual bound enabled flag is constrained to have a value equal to 0; and generating the coded bitstream, the coded bitstream including coded pictures of the sequence of video pictures and the general constraint syntax structure; A device that performs the above.

2. 2. The apparatus of claim 1, wherein a value of 1 for a constraint flag included in the first set indicates that the virtual boundaries enabled flag, sps_virtual_boundaries_enabled_flag, shall be equal to 0; otherwise, a value of 0 for a constraint flag included in the first set imposes no such constraint.

3. 1. An apparatus for decoding a coded bitstream with constrained processing, comprising: an input unit for receiving a coded bitstream including coded pictures of a sequence of video pictures and a general constraint syntax structure, the general constraint syntax structure including syntax elements related to values ​​of syntax elements of the coded bitstream and a set of restrictions on coding tools; and processor; wherein the processor: analyzing the general constraint syntax structure based on a predetermined number of categories to identify one or more constraint flags for the set of restrictions; the one or more constraint flags belonging to a first category of the predetermined number of categories include a first set of constraint flags related to loop filtering, the first set of constraint flags including a constraint flag indicating whether a virtual bound enabled flag is constrained to have a value equal to 0; and decoding the coded pictures in the coded bitstream in accordance with the one or more constraint flags to generate the sequence of video pictures; A device that performs the above.

4. 4. The apparatus of claim 3, wherein a value of 1 for a constraint flag included in the first set indicates that the virtual boundaries enabled flag, sps_virtual_boundaries_enabled_flag, shall be equal to 0, and a value of 0 otherwise imposes no such constraint.

5. 1. An apparatus for transmitting a coded bitstream from an encoder to a decoder, the apparatus comprising: receiving a sequence of video pictures to be encoded into the coded bitstream; determining a set of constraints on values ​​of syntax elements and coding tools of the coded bitstream; determining one or more constraint flags for the set of restrictions; aggregating the one or more constraint flags into a general constraint syntax structure based on a predetermined number of categories, the general constraint syntax structure comprising: the one or more constraint flags belonging to a first category of the predetermined number of categories include a first set of constraint flags related to loop filtering, the first set of constraint flags including a constraint flag indicating whether a virtual bound enabled flag is constrained to have a value equal to 0; and transmitting the coded bitstream, the coded bitstream including coded pictures of the sequence of video pictures and the general constraint syntax structure; A device that performs the above.

Images