Video and image encoding and decoding method and related device

Abstract

A method for signaling high-level information in image and video coding is provided. A video decoder receives data from a bitstream to decode a current picture of a video. The video decoder parses a picture header of the current picture, the picture header including a set of one or more slice information syntax elements in the picture header to indicate the presence of one or more slice types present in the current picture. The video decoder reconstructs the slices of the current picture by using the set of slice information syntax elements.

Description

[0001] Cross-references

[0002] The present invention claims priority to U.S. Provisional Patent Applications Nos. 62 / 930,086, 62 / 930,088, 62 / 945,282, 62 / 954,023, 62 / 957,759, 62 / 958,624, and 63 / 000,537, filed on November 4, 2019, November 4, 2019, December 9, 2019, December 27, 2019, and January 6, 2020, respectively. The U.S. Provisional Patent Applications are hereby incorporated by reference in their entirety. Technical Field

[0003] The present disclosure relates generally to the transmission of video codecs and, more particularly, to improving compression efficiency of signaling of high-level information used to encode image and video data. Background Art

[0004] Unless otherwise indicated herein, the approaches described in this subsection are not prior art to the claims listed below and are not admitted to be prior art by inclusion in this subsection.

[0005] The High Efficiency Video Coding (HEVC) standard is the latest international video codec standard developed by the Joint Collaborative Team on Video Coding (JCT-VC). The input video signal is predicted from the reconstructed signal, which is derived from the coded picture area. The prediction residual signal is processed by a linear transformation. The transform coefficients are quantized and entropy encoded in the bitstream along with other side information. After inverse transformation of the dequantized transform coefficients, the reconstructed signal is generated from the prediction signal and the reconstructed residual signal. The reconstructed signal is further processed by a loop filter to remove codec artifacts. The decoded picture is stored in a frame buffer for prediction of subsequent pictures in the input video signal.

[0006] In HEVC, a coded picture is partitioned into non-overlapping square block regions represented by associated coding tree units (CTUs). A coded picture can be represented by a set of slices, each picture consisting of an integer number of CTUs. Multiple individual CTUs in a slice are processed in a progressive scanning order. A bi-predictive (B) slice can be decoded using intra prediction or inter prediction, which uses up to two motion vectors and reference indices to predict the sample values ​​of each block. A predictive (P) slice can be decoded using intra prediction or inter prediction, which uses up to one motion vector and reference index to predict the sample values ​​of each block. An intra (I) slice is decoded using only intra prediction.

[0007] Using a recursive quadtree (QT) structure, a CTU can be partitioned into multiple non-overlapping codec units (CUs) to adapt to various local motion and organizational structure characteristics. For each CU, one or more prediction units (PUs) are specified. The PU (with associated CU syntax) is used as a basic unit for signaling prediction sub-information. The values ​​of the associated pixel samples in the PU are then predicted using the specified prediction procedure. Using a residual quadtree (RQT) structure, a CU can be further partitioned to represent the associated prediction residual signals. The leaf nodes of the RQT structure correspond to transform units (TUs). A transform unit contains a transform block (TB) of a number of luma samples (size 8x8, 16x16 or 32x32) or four transform blocks of a number of luma samples (size 4x4), and two corresponding transform blocks of a number of chroma samples (a picture in 4:2:0 color format). An integer transform is applied in a transform block and the values ​​of the quantization coefficient levels are entropy encoded into the bitstream along with other side information. Figure 2 An example of a block partition (left) and its corresponding quadtree QT representation (right) is shown. Solid lines represent CU boundaries and dashed lines represent TU boundaries.

[0008] The terms coding tree unit (CTU), coding block (CB), prediction block (PB), and transform block (TB) are defined to refer to a two-dimensional array of samples of a color component associated with a CTU, CU, PU, ​​and TU, respectively. Thus a CTU consists of one luma CTB, two chroma CTBs, and associated syntax elements. Similar relationships apply to CUs, PUs, and TUs. Tree partitioning usually applies to both luma and chroma, although there are some exceptions when chroma reaches certain minimum size limits.

[0009] In Versatile Video Coding (VVC), a coded picture can also be partitioned into non-overlapping square block areas represented by CTUs. Each CTU can be partitioned into one or more smaller codec units (CUs) using binary and ternary tree partitioning using a quadtree with nested multi-type trees. The generated CU partitions can be square or rectangular in shape. Summary of the invention

[0010] The following summary of the invention is illustrative only and is not intended to be limiting in any way. That is, the following summary of the invention is provided to introduce the concepts, highlights, benefits and advantages of the new and non-obvious technology described herein. Selective but not all embodiments are further described in the detailed description below. Therefore, the following summary of the invention is not used to determine the essential characteristics of the claimed subject matter, nor is it used to determine the scope of the claimed subject matter.

[0011] A video decoder receives data from a bitstream to decode as a current picture of a video. The video decoder parses a picture header of the current picture, the picture header including a set of one or more slice information syntax elements in the picture header to indicate the presence of one or more slice types present in the current picture. The set of slice information syntax elements may indicate whether a slice of a particular type is present in the current picture. The set of slice information syntax elements may include one or more syntax elements: to indicate (i) whether a first slice type is present in the current picture, and (ii) whether a second and different slice type is present in the current picture; wherein a slice of the first slice type does not reference information of pictures other than the current picture, and a slice of the second slice type has information that references pictures other than the current picture.

[0012] When parsing a picture header, the video decoder may bypass parsing picture header syntax elements associated with a codec that is not associated with one or more slice types present in the current picture identified by the set of slice information syntax elements.

[0013] The video decoder reconstructs slices of the current picture by using the set of slice information syntax elements. In some embodiments, the set of slice information syntax elements includes a multi-slice-type syntax element to indicate whether the current picture includes slices of more than one slice type. In some embodiments, when the multi-slice-type syntax element indicates that the current picture includes slices of more than one slice type, all slice headers of all slices of the current picture indicate a slice type. In some embodiments, the current picture references a picture parameter set (PPS), the PPS including a PPS syntax element to indicate whether slices of different slice types are allowed in the current picture. When the PPS syntax element indicates that slices of more than one slice type are allowed, the picture header of the current picture includes the multi-slice-type syntax element. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] The following drawings are used to provide a further understanding of the present invention and are incorporated into and constitute a part of the present invention. These drawings illustrate embodiments of the present invention and are used together with the description to explain the principles of the present invention. In order to clearly illustrate the concept of the present invention, some elements may be shown out of scale compared to the size in the actual embodiment, and these drawings do not need to be drawn in scale.

[0015] Figure 1 A portion of an encoded video is conceptually illustrated, where syntax elements in a picture header are used to indicate whether one or more slice types are present in the corresponding picture.

[0016] Figure 2 An example video encoder is depicted.

[0017] Figure 3 A process is conceptually illustrated for signaling one or more syntax elements in a picture header to indicate the slice types that may be present in the current picture.

[0018] Figure 4 An example video decoder is shown.

[0019] Figure 5 A process is conceptually illustrated for using one or more syntax elements in a picture header to indicate the types of slices that may be present in the current picture.

[0020] Figure 6 An electronic system is conceptually depicted in which some embodiments of the present disclosure may be implemented. DETAILED DESCRIPTION

[0021] In the following detailed description, a large number of specific details are explained by way of example in order to thoroughly understand the relevant teachings. Any changes, derivations and / or extensions based on the teachings described herein are within the scope of protection of the present invention. In order to avoid unnecessary confusion of aspects of the teachings of the present invention, known methods, procedures, components and / or circuits in one or more exemplary embodiments disclosed herein are sometimes described at a relatively high level without further explanation.

[0022] I. Indicate slice types in high-level syntax

[0023] Some embodiments of the present disclosure provide methods for signaling high-level information for encoding and decoding image and video data to improve compression efficiency. In some embodiments, one or more high-level syntax (HLS) groups, such as video parameter set (VPS), sequence parameter set (SPS), picture parameter set (PPS), and picture header group, may further include multiple syntax elements to indicate slice types that may be present in an associated data structure. In this way, a video codec may skip encoding or decoding syntax elements in a current HLS group when the syntax elements in the current HLS group are related to codec tools that are not associated with any slice type present in the associated data structure. For example, in some embodiments, a video codec may further include one or more syntax elements in a current SPS to indicate slice types that may be present in an associated codec video sequence referenced by the current SPS. When one or more syntax elements in the SPS indicate that I slice is the only slice type present in the associated codec video sequence, a video codec may skip encoding and decoding all SPS syntax elements not related to I slices (e.g., inter prediction tools and other tools not applicable in an I slice).

[0024] a) Signal the presence of slice type(s)

[0025] In some embodiments, a picture header is signaled for each codec picture. The picture header may include one or more syntax elements to indicate the slice types that may be present in the current picture. Thus, a video codec may skip encoding or decoding the following picture header syntax elements: picture header syntax elements associated with codec tools that are not associated with any slice type present in the current picture. For example, when the picture header indicates that I slices are the only slice type present in the current codec picture, a video codec may skip encoding or decoding all picture header syntax elements that are not associated with I slices (e.g., inter prediction tools and other tools that are not applicable in an I slice).

[0026] In some embodiments, the picture header may further include one or more syntax elements to indicate a collection of slice types that may be present in a current picture. In some embodiments, a video codec signals a syntax element ph_slice_types in the picture header. Tables 1 and 2 provide two exemplary tables to explain the syntax values ​​of ph_slice_types. In particular, these tables show the mapping of the syntax values ​​of ph_slice_types that may be present in the associated data structures.

[0027] Table 1:

[0028]

[0029] Table 2:

[0030]

[0031] In some embodiments, the picture header may include multiple syntax elements to indicate certain slice types that may be present in the current picture. In some embodiments, the SPS may include three syntax flags sps_slice_B_present, sps_slice_P_present, and sps_slice_I_present to indicate whether B slices, P slices, and I slices are respectively present in a codec picture from a codec video sequence referenced by the current SPS. Similarly, the PPS may include three syntax flags pps_slice_B_present, pps_slice_P_present, and pps_slice_I_present to indicate whether B slices, P slices, and I slices are respectively present in a codec picture from a codec video sequence referenced by the current SPS. In some embodiments, the picture header may include three syntax flags in the picture header: ph_slice_B_present, ph_slice_P_present, and ph_slice_I_present to indicate whether B slices, P slices, and I slices are respectively present in a codec picture from a codec video sequence referenced by the current SPS.

[0032] In some embodiments, a video codec signals information in the picture header to derive the variable phPicType to indicate the slice type that may be present in the current picture. Tables 3 and 4 provide two exemplary tables to map or interpret the syntax value of phPicType.

[0033] Table 3:

[0034]

[0035] Table 4:

[0036]

[0037] Table 4 is an extension of Table 3 to explain the syntax element pic_type in an access unit delimiter (AUD). Therefore, the issuance of information to derive the variable phPicType may be conditioned on the value of phPicType signaled in the associated AUD (when present). In some embodiments, the AUD is constrained to always be signaled for each access unit. The issuance of information to derive the variable phPicType may be conditioned on the value of phPicType signaled in the associated AUD. For example, when phPicType signaled in the associated AUD is equal to 0, a video codec may skip encoding or decoding information used to derive the variable phPicType in the current picture header, and phPicType is inferred to be equal to 0.

[0038] In some embodiments, a video codec may have more than one high-level syntax group (including multiple syntax elements) for signaling slice types, and the aforementioned slice types may be present in respectively associated data structures with the same syntax element numerical interpretation. For example, a video codec may include both SPS and AUD, and both SPS and AUD have a syntax element for indicating slice types, and the aforementioned slice types may be present in respectively associated data structures with the same syntax element numerical interpretation (such as Tables 1-4).

[0039] In some embodiments, a syntax element intra_only_constraint_flag is signaled to specify constraints on allowed slice types. In some embodiments, signaling of information to derive syntax elements to signal slice types that may be present in respectively associated data structures may be further conditioned on the value of the syntax element intra_only_constraint_flag. In some embodiments, when intra_only_constraint_flag is equal to 1, a video codec may skip the following syntax elements from being encoded or decoded: syntax elements for codecs not associated with an I slice type.

[0040] In some embodiments, one or more high-level syntax (HLS) groups, such as a video parameter set (VPS), a sequence parameter set (SPS), a picture parameter set (PPS), and / or a picture header (PH), may include one or more syntax elements to indicate a network abstraction layer (NAL) unit type that may be present in a current picture. Thus, a video codec may skip encoding or decoding picture header syntax elements that are associated with codec tools that are not associated with a NAL unit type present in an associated data structure. For example, a video codec may skip encoding or decoding reference picture list syntax elements in a current picture when the picture header indicates the presence of an instantaneous decoding refresh (IDR) unit type.

[0041] In some embodiments, a video codec may signal one or more signaling syntax elements regarding the use of gradual decoding refresh (GDR) in one or more high level syntax (HLS) groups (e.g., VPS, SPS, PPS, and / or picture header). In some embodiments, syntax elements regarding GDR are signaled in a picture header conditional on the values ​​of GDR-related syntax elements in the HLS group referenced by the current picture. In some embodiments, in the picture header, the syntax element ph_pic_parameter_set_id indicating the index of a selected PPS is signaled before the syntax element gdr_pic_flag indicating whether the current picture is a GDR picture. When the syntax element gdr_enabled_flag is equal to 0 in the SPS referenced by the selected PPS in the current picture header, the encoding and decoding of gdr_pic_flag is skipped in the current picture header. The syntax element gdr_pic_flag is inferred to be equal to 0 when it is not present. Table 5 below shows an exemplary syntax table of a picture header to show syntax elements related to GDR:

[0042] Table 5:

[0043]

[0044] In some embodiments, an HLS group (e.g., VPS, SPS, and PPS) may further include one or more syntax elements to indicate slice types that may be present in the associated data structure. In this way, encoding and decoding of the following HLS elements can be skipped: HLS elements related to codec tools that are not associated with any slice type present in the associated data structure. For example, a video codec may encode or decode one or more syntax elements in a current PPS to indicate slice types that may be present in an associated codec picture (referred to as the current PPS). When one or more added syntax elements in the PPS indicate that an I slice is the only slice type present in the associated codec picture, a video codec may skip encoding and decoding all PPS syntax elements that are not related to an I slice (e.g., inter-frame prediction tools and other tools that are not applicable in an I slice). In some embodiments, the PPS may further include a syntax element represented by pps_pic_type. The interpretation of the value of pps_pic_type may be indicated by one of the exemplary mappings in Tables 1-4 above.

[0045] In some embodiments, subject to relevant information in a high-level syntax group referenced by a current picture header (when present), a video codec may encode or decode picture header syntax elements to indicate slice types that may be present in a current picture. In one example, a video codec may encode or decode three syntax flags (denoted ph_slice_B_present_flag, ph_slice_P_present_flag, and ph_slice_I_present_flag) in a picture header to indicate whether B slices, P slices, and I slices are present in the current picture, respectively. When the PPS referenced by a current picture header indicates that only I slice types are present in the current picture, the video codec may infer that

[0046] ph_slice_I_present_flag is equal to 1, ph_slice_B_present_flag and ph_slice_P_present_flag are both equal to 0, and these three syntax flags are skipped without encoding or decoding. The video codec may further skip the syntax elements related to inter-frame prediction in the current picture header without encoding or decoding. Alternatively, the PPS may include three syntax flags (denoted as pps_slice_B_present, pps_slice_P_present, and pps_slice_I_present) to indicate whether B slices, P slices, and I slices are respectively present in the coded pictures referring to the current PPS.

[0047] In some embodiments, a video codec may signal or derive information in a picture header to indicate whether more than one slice type is present in a current picture. When only one slice type is indicated to be present in the current picture, the video codec may further encode or decode the slice type (B, P, or I) present in the current picture. The video codec may further skip encoding or decoding information about a slice type in a slice associated with the current picture, wherein the slice type in the slice associated with the current picture is inferred to be the same as the coded slice type in the picture header. When more than one slice type is indicated to be present in the current picture, the video codec may further encode or decode information for deriving possible slice types that may be present in the current picture. The video codec may further skip encoding or decoding syntax elements of codec tools applicable only to unused slice types and their values.

[0048] In some embodiments, the video codec may signal one or more syntax elements in one or more HLS groups (e.g., SPS and PPS) to indicate whether the following is allowed: there are more than one slice type in a picture for all associated pictures that reference the one or more HLS groups. When the signaled HLS information indicates that the existence of more than one slice type in each associated picture is not allowed, the video codec may infer that there is only one slice type in a current picture for all associated pictures that reference the one or more HLS groups; and further skip the syntax elements in the picture header used to indicate whether there are more than one slice type in a current picture, and do not encode or decode them. The video codec may further signal another one or more syntax elements in the one or more HLS groups to indicate that each picture that references the one or more HLS groups is further divided into more than one slice. This one or more syntax elements are signaled before (in decoding order) the aforementioned one or more syntax elements used to indicate whether there may be more than one slice type for each associated picture. When another one or more codec syntax elements indicate that each associated picture referencing the one or more HLS groups contains only one slice, the video codec may infer that there is only one slice type in the associated picture and skip encoding or decoding the one or more syntax elements used to indicate whether there may be more than one slice type for each associated picture.

[0049] In some embodiments, a video codec may signal or derive information in a picture header or other HLS group to indicate whether more than one VCL NAL type exists in a current picture. When the current picture containing only one slice is first signaled, the video codec may infer that only one VCL NAL type exists in the current picture. For example, a video codec may signal the syntax element mixed_nalu_types_in_pic_flag to indicate whether each picture that references a PPS has the same nal_unit_type value. In some embodiments, the syntax element mixed_nalu_types_in_pic_flag is moved to (placed after) the syntax element no_pic_partition_flag and is only encoded and decoded when the syntax element no_pic_partition_flag is equal to 0, as shown in Table 6 below. Since mixed_nalu_types_in_pic_flag is about using sub-picture partitioning in rectangular slice partitioning mode, encoding and decoding of the syntax element mixed_nalu_types_in_pic_flag may be further conditioned on whether rectangular slice partitioning is in use and the number of slices in the current picture.

[0050] For some embodiments, Tables 6-8 below provide exemplary HLS syntax tables for PPS, picture header, and slice header, respectively.

[0051] Table 6: An exemplary high-level syntax table for a PPS

[0052]

[0053] Table 7: Example high-level syntax table for a picture header

[0054]

[0055]

[0056] Table 8: Example high-level syntax table for a slice header

[0057]

[0058] The syntax element pps_mixed_slice_types_in_pic_flag in a PPS indicates whether the presence of more than one slice type in a picture for all pictures referencing the PPS is allowed. The syntax element ph_mixed_slice_types_in_pic_flag in a current picture header indicates whether the presence of more than one slice type in a current picture is allowed. When the value of pps_mixed_slice_types_in_pic_flag is equal to 0, the presence of more than one slice type in a current picture referencing the PPS is not allowed. The value of ph_mixed_slice_types_in_pic_flag is not coded and is inferred to be equal to 0. When the value of ph_mixed_slice_types_in_pic_flag is equal to 0, the value of the syntax element slice_type in the slice header is not coded and is inferred to be equal to the value of ph_slice_type. The values ​​of syntax elements in the slice header for codec tools applicable only to unused slice types may be further skipped. When ph_mixed_slice_types_in_pic_flag is equal to 1, the possible slice types in the current picture are signaled by syntax elements denoted ph_slice_B_present_flag, ph_slice_P_present_flag, and ph_slice_I_present_flag, to indicate whether B slices, P slices, and I slices may be present in the current picture, respectively.

[0059] (b) Sending multiple slice types

[0060] In some embodiments, the syntax element mixed_slice_types_in_pic_flag in a PPS indicates whether the presence of more than one slice type in a picture for all pictures referencing the PPS is allowed. When pps_mixed_slice_types_in_pic_flag is equal to 0, the syntax element ph_slice_type indicates that only one slice type is present in the current header. The values ​​of the syntax elements in the slice header for codecs that are applicable only to unused slice types are further skipped. The value of the syntax element slice_type in the slice header can be inferred from the codec value of ph_slice_type. Tables 9-11 show the syntax tables for PPS, picture header, and slice header, respectively, using pps_mixed_slice_types_in_pic_flag.

[0061] Table 9:

[0062]

[0063]

[0064] Table 10:

[0065]

[0066]

[0067]

[0068] Table 11:

[0069]

[0070] In the PPS RBSP (as shown in Table 9), the syntax element mixed_nalu_types_in_pic_flag equal to 1 indicates that each picture referencing the PPS has more than one VCL NAL unit, and that the VCL NAL units do not have the same value as nal_unit_type, and that the picture is not an IRAP picture. The value of mixed_nalu_types_in_pic_flag equal to 0 indicates that each picture referencing the PPS has one or more VCL NAL units, and that the VCL NAL units of each picture referencing the PPS have the same value as nal_unit_type. When not present, the value of mixed_nalu_types_in_pic_flag is inferred to be equal to 0. When no_mixed_nalu_types_in_pic_constraint_flag is equal to 1, the value of mixed_nalu_types_in_pic_flag must be equal to 0.

[0071] For each slice with a nal_unit_type value nalUnitTypeA in the range IDR_W_RADL to CRA_NUT (inclusive), in a picture picA that also contains one or more slices with another nal_unit_type value (i.e., for picture picA, the value of mixed_nalu_types_in_pic_flag is equal to 1), the following applies:

[0072] – The slice must belong to the primary picture subpicA, where the value of the corresponding subpic_treated_as_pic_flag[i] is equal to 1.

[0073] – A slice may not belong to a primary picture of picA, where the secondary picture contains VCL NAL units with nal_unit_type not equal to nalUnitTypeA.

[0074] – For all subsequent PUs in decoding order in the CLVS, neither the reference picture list RefPicList[0] nor RefPicList[1] of any slice in subpicA may include any picture that is earlier in decoding order than picA in the current entry (active entry).

[0075] The value of mixed_slice_types_in_pic_flag equal to 1 indicates that more than one slice type may exist in the current picture for each picture referencing this PPS. The value of mixed_slice_types_in_pic_flag equal to 0 indicates that one type exists in the current picture for each picture referencing this PPS. When not present, the value of mixed_slice_types_in_pic_flag is inferred to be equal to 0.

[0076] In the picture header RBSP (as shown in Table 10), when mixed_slice_types_in_pic_flag is equal to 0, the syntax element ph_slice_type specifies the value of slice_type in all slice headers of the current picture.

[0077] In the slice header (as shown in Table 11), the syntax element slice_type indicates the codec type of the slice. When not present, the value of slice_types is inferred to be equal to the value of ph_slice_type. (The values ​​of slice_type associated with different slice types are shown in Table 23 below).

[0078] (c) Sending a picture header for a mix of different slice types

[0079] In some embodiments, the video codec indicates whether inter slice types (B or P) and intra slice types (I) may be present in a current picture. A syntax element ph_mixed_slice_types_in_pic_flag is further added to the picture header to indicate whether more than one slice type may be present in a current picture. When ph_mixed_slice_types_in_pic_flag is equal to 1, a syntax element ph_intra_slice_present_flag is further signaled to indicate whether slice type I may be present in the current picture. The variables InterSlicePresent and IntraSlicePresent indicate whether there are any inter slices (slice type B or P) and any intra slices (slice type I), respectively, in a current picture. The variables InterSlicePresent and IntraSlicePresent can be derived from the values ​​of ph_mixed_slice_types_in_pic_flag, ph_intra_slice_present_flag, and ph_slice_type. In the slice header, the signaling of information on slice_type depends on the values ​​of ph_mixed_slice_types_in_pic_flag and IntraSlicePresent. Tables 12 and 13 below show a picture header and a slice header, respectively, where the syntax elements indicate whether inter slices and intra slices are present:

[0080] Table 12:

[0081]

[0082]

[0083]

[0084] Table 13:

[0085]

[0086]

[0087] The value of ph_mixed_slice_types_in_pic_flag equals 1 to indicate that more than one slice type may be present in the current picture. The value of ph_mixed_slice_types_in_pic_flag equals 0 to indicate that all slices in a current picture must have the same slice_type value. When ph_mixed_slice_types_in_pic_flag is not present, the value of ph_mixed_slice_types_in_pic_flag is inferred to be equal to 0.

[0088] The value of ph_intra_slice_present_flag equal to 1 indicates that slice type I may be present in a current picture. The value of ph_intra_slice_present_flag equal to 0 indicates that slice type I is not present in a current picture. When ph_mixed_slice_types_in_pic_flag is equal to 0, the syntax element ph_slice_type specifies the value of slice_type in all slice headers of the current picture.

[0089] The variables InterSlicePresent and IntraSlicePresent are derived as follows:

[0090] – If ph_mixed_slice_types_in_pic_flag is equal to 1, InterSlicePresent is set equal to 1 and IntraSlicePresent is set equal to ph_intra_slice_present_flag

[0091] – Otherwise, InterSlicePresent is set equal to (ph_slice_type=I) and IntraSlicePresent is set equal to (ph_slice_type==I).

[0092] The syntax element slice_type indicates the codec type of the slice according to Table 23 below. When slice_type is not present, the value of slice_types is derived as follows:

[0093] – If ph_mixed_slice_types_in_pic_flag is equal to 1, the value of slice_type is set equal to P_slice_flag.

[0094] – Otherwise, the value of slice_type is set equal to ph_slice_type.

[0095] When ph_mixed_slice_types_in_pic_flag is equal to 1 and IntraSlicePresent is equal to 0, the syntax element P_slice_flag is 1 to specify that the value of slice_type is equal to P. When ph_mixed_slice_types_in_pic_flag is equal to 1 and IntraSlicePresent is equal to 0, P_slice_flag is equal to 0 to specify that the value of slice_type is equal to B.

[0096] (d) Signaling the existence of individual slice types

[0097] In some embodiments, the video codec indicates whether each slice type (B, P, and I) may be present in a current picture. The syntax element ph_mixed_slice_types_in_pic_flag in a current picture header indicates whether more than one slice type may be present in a current picture. When ph_mixed_slice_types_in_pic_flag is equal to 1, a syntax element ph_present_slice_types is further signaled to indicate the slice types that may be present in a current picture. In particular, the variables BSlicePresent, PSlicePresent, and ISlicePresent indicate whether slice types B, P, and I may be present in the current picture, respectively. The variables BSlicePresent, PSlicePresent, and ISlicePresent may be derived from the values ​​of ph_mixed_slice_types_in_pic_flag, ph_slice_type, and ph_present_slice_types. In the slice header, the signaling of information on slice_type depends on the values ​​of ph_mixed_slice_types_in_pic_flag and ph_present_slice_types. Tables 14 and 15 below show a picture header and a slice header, where the syntax includes signaling for indicating whether each slice type may be present.

[0098] Table 14:

[0099]

[0100]

[0101]

[0102]

[0103] Table 15:

[0104]

[0105] The value of ph_mixed_slice_types_in_pic_flag equal to 1 specifies that more than one slice type may be present in the current picture. The value of ph_mixed_slice_types_in_pic_flag equal to 0 specifies that all slices in a current picture must have the same slice_type value. When ph_mixed_slice_types_in_pic_flag is not present, the value of ph_mixed_slice_types_in_pic_flag is inferred to be equal to 0.

[0106] When ph_mixed_slice_types_in_pic_flag is equal to 1, the syntax element ph_present_slice_types indicates the slice types that may be present in a current picture. When ph_mixed_slice_types_in_pic_flag is equal to 0, the syntax element ph_slice_type specifies the value of slice_type in all slice headers of the current picture.

[0107] The variables BSlicePresent, PSlicePresent, and ISlicePresent are derived as follows:

[0108] If ph_mixed_slice_types_in_pic_flag is equal to 1, BSLicePresent is set equal to (ph_slice_type == B), PSlicePresent is set equal to (ph_slice_type == P), and ISlicePresent is set equal to (ph_slice_type == I)

[0109] – Otherwise, BSlicePresent is set equal to (ph_present_slice_types=0), PSlicePresent is set equal to (ph_present_slice_types=1), and ISlicePresent is set equal to (ph_present_slice_types=2).

[0110] The syntax element slice_type indicates the codec type of the slice according to Table 23 below. When the syntax element slice_type is not present, its value is derived as follows:

[0111] – If ph_mixed_slice_types_in_pic_flag is equal to 1, the value of slice_type is set equal to (slice_type_flag>=ph_present_slice_types?slice_type_flag+1:slice_type_flag).

[0112] Otherwise, the value of slice_type is set equal to ph_slice_type.

[0113] When ph_mixed_slice_types_in_pic_flag is equal to 1 and the value of ph_present_slice_types is not equal to 3, the syntax element slice_type_flag indicates the value of slice_type.

[0114] In some embodiments, the syntax elements ph_inter_slice_allowed_flag and ph_intra_slice_allowed_flag are signaled to indicate whether inter slices and intra slices, respectively, are allowed to exist in the picture. When ph_inter_slice_allowed_flag is equal to 0, only syntax elements related to intra codec tools are not signaled. When ph_intra_slice_allowed_flag is equal to 0, only syntax elements related to inter codec tools are not signaled. When ph_inter_slice_allowed_flag is equal to 0, the value of the syntax element slice_type in the slice header is inferred to be equal to 2 and is not encoded into the bitstream.

[0115] (e) Signaling whether inter-frame slices and intra-frame slices are possible

[0116] In some embodiments, the video codec signals information to deduce whether inter slice types (B or P) and intra slice types (I) may be present in a codec picture, respectively. In some embodiments, a syntax element ph_multiple_slice_types_in_pic_flag is further added to the picture header to indicate whether more than one slice type may be present in the codec slices of the picture. A value of ph_multiple_slice_types_in_pic_flag equal to 1 indicates that codec slices in the picture may have different slice_type values. A value of ph_multiple_slice_types_in_pic_flag equal to 0 indicates that all codec slices in the picture have the same slice_type value. When ph_multiple_slice_types_in_pic_flag is equal to 1, a syntax element ph_intra_slice_allowed_flag is further signaled to indicate whether slice type I may be allowed in the current picture. When ph_multiple_slice_types_in_pic_flag is equal to 0, a syntax element ph_slice_type is further signaled to indicate the slice_type value to be used for all slices in the picture. The variables InterSliceAllowed and IntraSliceAllowed indicate whether inter slice types (slice type B or P) and intra slice types (slice type I) are allowed in the picture, respectively. The values ​​of the variables InterSliceAllowed and IntraSliceAllowed can be derived from the values ​​of ph_multiple_slice_types_in_pic_flag, ph_intra_slice_allowed_flag, and ph_slice_type. Values ​​of PH syntax elements for codecs that apply only to unused slice types are not encoded. When no_pic_partition_flag is equal to 1 or rect_slice_flag is equal to 1 and num_slices_in_pic_minus1 is equal to 1, ph_multiple_slice_types_in_pic_flag is inferred to be equal to 0 and is not explicitly encoded in the bitstream. Alternatively, a new syntax element pps_multiple_slice_types_in_pic_flag may be further signaled in the PPS to indicate whether more than one slice type may exist in each picture referencing the PPS.When pps_multiple_slice_types_in_pic_flag is equal to 0, ph_multiple_slice_types_in_pic_flag is inferred to be equal to 0 and is not encoded into the bitstream. Tables 16 and 17 below show a picture header and a slice header, where the syntax includes signaling for indicating whether inter slices and intra slices exist.

[0117] Table 16:

[0118]

[0119]

[0120]

[0121] Table 17:

[0122]

[0123] The value of ph_multiple_slice_types_in_pic_flag equals 1 to indicate that codec slices in a picture may have different slice_type values. The value of ph_multiple_slice_types_in_pic_flag equals 0 to indicate that all codec slices in a picture have the same slice_type value. When ph_multiple_slice_types_in_pic_flag is not present, its value is inferred to be 0.

[0124] The value of ph_intra_slice_allowed_flag equal to 0 specifies that all codec slices in the picture have slice_type equal to 0 or 1. The value of ph_intra_slice_allowed_flag equal to 1 specifies that there may or may not be one or more codec slices with slice_type equal to 2 in the picture. When ph_multiple_slice_types_in_pic_flag is equal to 0, the syntax element ph_slice_type specifies the slice_type value of all slices in the picture.

[0125] The variables InterSliceAllowed and IntraSliceAllowed are derived as follows:

[0126] – If ph_multiple_slice_types_in_pic_flag is equal to 1, InterSliceAllowed is set equal to 1 and IntraSliceAllowed is set equal to ph_intra_slice_present_flag

[0127] – Otherwise, InterSliceAllowed is set equal to (ph_slice_type=I) and IntraSliceAllowed is set equal to (ph_slice_type==I).

[0128] In some embodiments, for a bitstream that supports subpicture-based bitstream merging without changing the PH NAL unit, the encoder sets the values ​​of both InterSliceAllowed and IntraSliceAllowed to 1.

[0129] (f) Signal whether each slice type may exist

[0130] In some embodiments, the video codec uses information to deduce whether each slice type (B, P, and I) may be present in the codec picture. In some embodiments, a syntax element ph_multiple_slice_types_in_pic_flag is further added to the picture header to indicate whether more than one slice type may be present in the codec slices of the picture. The syntax element ph_multiple_slice_types_in_pic_flag equal to 1 indicates that the codec slices in the picture may have different slice_type values. The syntax element ph_multiple_slice_types_in_pic_flag equal to 0 indicates that all codec slices in the picture have the same slice_type value.

[0131] When ph_multiple_slice_types_in_pic_flag is equal to 1, a syntax element ph_allowed_slice_types_idc is further signaled to derive the slice types allowed in a slice of a picture. When ph_multiple_slice_types_in_pic_flag is equal to 0, a syntax element ph_slice_type is further signaled to specify the slice_type value to be used for all slices in the picture. The values ​​of the variables BSliceAllowed, PSliceAllowed, and ISliceAllowed indicate whether slice types B, P, and I are allowed in the picture, respectively. The variables BSliceAllowed, PSliceAllowed, and ISliceAllowed can be derived from the values ​​of ph_multiple_slice_types_in_pic_flag, ph_allowed_slice_types_idc, and ph_slice_type. The value of the PH syntax element for codecs that apply only to unused slice types is not encoded. When no_pic_partition_flag is equal to 1 or rect_slice_flag is equal to 1 and num_slices_in_pic_minus1 is equal to 1, ph_multiple_slice_types_in_pic_flag is inferred to be equal to 0 and is not explicitly encoded in the bitstream.

[0132] Alternatively, a syntax element pps_multiple_slice_types_in_pic_flag may be further signaled in the PPS to indicate whether more than one slice type is present in each picture referencing the PPS. When the value of pps_multiple_slice_types_in_pic_flag is equal to 0, the syntax element ph_multiple_slice_types_in_pic_flag is inferred to be equal to 0 and is not encoded into the bitstream. Tables 18 and 19 below show a picture header and a slice header, where the syntax includes signaling to indicate whether each slice type may be present.

[0133] Table 18:

[0134]

[0135]

[0136]

[0137] Table 19:

[0138]

[0139]

[0140] The syntax element pred_weight_Table is provided by the syntax table shown in Table 20 below.

[0141] Table 20:

[0142]

[0143]

[0144] When both pps_weighted_bipred_flag and wp_info_in_ph_flag are equal to 1, the syntax element num_l1_weights specifies the number of weights signaling the number of weights used for entries in reference picture list 1. The value of num_l1_weights is constrained to be in the range of 0 to Min(15, num_ref_entries[1][RplsIdx[1]]), inclusive. The variable NumWeightsL1 is derived as follows:

[0145] if (pps_weighted_bipred_flag)

[0146] NumWeightsL1=0

[0147] else if (wp_info_in_ph_flag)

[0148] NumWeightsL1=BSliceAllowed? num_l1_weights:0

[0149] Otherwise NumWeightsL1 = NumRefIdxActive[1]

[0150] In the picture header shown in Table 18 above, the syntax element ph_multiple_slice_types_in_pic_flag equal to 1 indicates that codec slices in the picture can have different slice_type values. The value of ph_multiple_slice_types_in_pic_flag equal to 0 indicates that all codec slices in the picture have the same slice_type value. When the syntax element ph_multiple_slice_types_in_pic_flag is not present, its value is inferred to be equal to 0. When ph_multiple_slice_types_in_pic_flag is equal to 1, the syntax element ph_allowed_slice_types_idc indicates the slice_type values ​​allowed for codec slices in the picture. Table 21 below shows the allowed slice_types indicated by ph_allowed_slice_types_idc. (The codec type of the slice is indicated by slice_type according to Table 23 below).

[0151] Table 21:

[0152]

[0153] When ph_multiple_slice_types_in_pic_flag is equal to 0, the syntax element ph_slice_type specifies the slice_type value used for all slice headers in the picture. The variables BSliceAllowed, PSliceAllowed, and ISliceAllowed are derived as follows:

[0154] If ph_multiple_slice_types_in_pic_flag is equal to 0, BSLiceAllowed is set equal to (ph_slice_type == B), PSliceAllowed is set equal to (ph_slice_type == P) and ISliceAllowed is set equal to (ph_slice_type == I)

[0155] – Otherwise, BSliceAllowed is set equal to (ph_allowed_slice_types_idc=0), PSliceAllowed is set equal to (ph_allowed_slice_types_idc=1) and ISliceAllowed is set equal to (ph_allowed_slice_types_idc=2).

[0156] In some embodiments, for a bitstream that supports subpicture-based bitstream merging without changing the PH NAL unit, the encoder sets the values ​​of both ph_inter_slice_allowed_flag and ph_intra_slice_allowed_flag to 1.

[0157] (g) Signaling the slice type based on the conditions of the PH information

[0158] In some embodiments, when ph_inter_slice_allowed_flag is equal to 1 and ph_intra_slice_allowed is equal to 0, the value of slice_type is only allowed to be equal to 0 or 1 and can be signaled by a syntax flag signaled by one bin. In some embodiments, based on the condition that the value of ph_inter_slice_allowed_flag is equal to 1 and ph_intra_slice_allowed in PH, the video codec signals information to derive the value of slice_type in the slice header. In some embodiments, when ph_inter_slice_allowed_flag is equal to 1 and ph_intra_slice_allowed is equal to 0, the syntax element P_slice_flag indicates whether the value of slice_type is equal to 0 or 1. In this way, a bitstream with ph_intra_slice_allowed equal to 0 and slice_type equal to 2 can be avoided. Table 22 below shows all slice headers that signal slice type based on the condition of PH information:

[0159] Table 22:

[0160]

[0161] (h) Slice Header Semantics

[0162] Table 23:

[0163] slice_type slice_type name 0 B(B slice) 1 P(P slice) 2 I(I slice)

[0164] In some embodiments, when ph_multiple_slice_types_in_pic_flag is equal to 0, the value of slice_type (when not present) is inferred to be equal to ph_slice_type, otherwise it is inferred to be equal to the value of P_slice_flag. When ph_multiple_slice_types_in_pic_flag is equal to 1 and IntraSliceAllowed is equal to 0, P_slice_flag equal to 1 specifies that the value of slice_type is equal to P. When ph_multiple_slice_types_in_pic_flag is equal to 1 and IntraSliceAllowed is equal to 0, P_slice_flag equal to 0 specifies that the value of slice_type is equal to B.

[0165] In some embodiments, when ph_inter_slice_allowed_flag is equal to 0, the value of slice_type (when not present) is inferred to be equal to 2, otherwise it is inferred to be equal to P_slice_flag. P_slice_flag equal to 1 specifies that when ph_inter_slice_allowed_flag is equal to 1 and ph_intra_slice_allowed is equal to 0, the value of slice_type is equal to P. P_slice_flag equal to 0 specifies that when ph_inter_slice_allowed_flag is equal to 1 and ph_intra_slice_allowed is equal to 0, the value of slice_type is equal to B.

[0166] In some embodiments, the value of slice_type (when not present) is determined as follows:

[0167] – If ph_multiple_slice_types_in_pic_flag is equal to 1, the value of slice_type is set equal to (slice_type_flag>=ph_allowed_slice_types_idc?slice_type_flag+1:slice_type_flag).

[0168] – Otherwise, the value of slice_type is set equal to ph_slice_type.

[0169] When ph_multiple_slice_types_in_pic_flag is equal to 1 and the value of ph_allowed_slice_types_idc is not equal to 3, the syntax element slice_type_flag indicates the value of slice_type.

[0170] In some embodiments, when nal_unit_type is in the range of IDR_W_RADL to CRA_NUT (inclusive), and vps_independent_layer_flag[GeneralLayerIdx[nuh_layer_id]] is equal to 1, slice_type is constrained to be equal to 2.

[0171] Figure 1 A portion of an encoded video is conceptually illustrated, wherein syntax elements in a picture header and a PPS are used to indicate whether a particular slice type or types are present in a corresponding picture. As illustrated, encoded video 100 includes pictures 111-114. These pictures reference a picture parameter set (PPS) 105. Encoded video 100 includes picture headers 121-124 associated with video pictures 111-114, respectively. Each video picture 111-114 includes one or more slices, and each slice has a corresponding slice header. For example, slices of video picture 114 are associated with slice headers 151-154.

[0172] As shown, the picture header 124 for the picture 114 signals one or more slice information syntax elements to indicate the presence of one or more slice types present in the current picture. In some embodiments, these slice information syntax elements may include ph_mixed_slice_types_in_pic_flag, and ph_intra_slice_present_flag (e.g., Figure 1 As shown in Table 12 above). In some embodiments, these slice information syntax elements may include ph_slice_B_present_flag, ph_slice_P_present_flag, ph_slice_I_present_flag (as shown in Table 7 above). This slice information syntax element may indicate whether a specific slice type (inter, intra, B, P, or I) is present in the current picture.

[0173] In addition, the slice information syntax element includes several syntax elements to indicate whether (i) a first slice type is present in the current picture, and (ii) a second and different slice type is present in the current picture. A slice of the first slice type (e.g., intra slice, I slice type) does not reference information of other pictures other than the current picture. A slice of the second slice type (e.g., inter slice, B or P slice type) has information of other pictures other than the current picture. For example, the slice information syntax elements ph_mixed_slice_type_in_pic_flag and ph_intra_slice_present_flag can be used to determine whether intra slices and / or inter slices are allowed in the video picture 114 (by determining the variables IntraSliceAllowed and InterSliceAllowed).

[0174] A video codec may bypass and not encode or decode picture header syntax elements that are not associated with one or more slice types present in the current picture identified by one or more slice information syntax elements. For example, if IntraSliceAllowed is false, syntax element 160 in picture header 124 is skipped or bypassed; and if InterSliceAllowed is false, syntax element 162 in picture header 140 is skipped or bypassed.

[0175] In some embodiments, the slice information syntax element of a picture header includes a multi-slice-type syntax element to indicate whether the current picture includes slices of more than one slice type. When the multi-slice-type syntax element indicates that the current picture includes slices of more than one slice type, all slice headers of all slices of the current picture signal or indicate a slice type. In the picture header 124, the syntax element ph_mixed_slice_types_in_pic_flag is a multi-slice-type syntax element to indicate that there may be a mixture of different slices of different slice types in the picture (e.g., one slice in the picture 114 may be an I slice, another slice may be a B slice, etc.). In the slice header 153, the multi-slice-type syntax element ph_mixed_slice_types_in_pic_flag is used to determine whether the syntax element "slice_type" is signaled.

[0176] The PPS 105 includes a PPS syntax element to indicate whether slices of different slice types are allowed (eg, pps_mixed_slice_types_in_pic_flag). When the PPS syntax element indicates that slices of more than one slice type are allowed, the picture header 140 signals the multi-slice-type syntax element ph_mixed_slice_types_in_pic_flag.

[0177] Any of the above-mentioned proposed methods may be implemented in an encoder and / or a decoder. For example, any of the above-mentioned methods may be implemented in a high-level syntax encoding module of an encoder and / or a high-level syntax decoding module of a decoder. Alternatively, any of the above-mentioned methods may be implemented as a circuit integrated in a high-level syntax encoding module of an encoder and / or a high-level syntax decoding module of a decoder. Any of the above-mentioned methods may also be implemented in an image encoder and / or a decoder, wherein the generated bitstream corresponds to a coded frame using only intra-frame prediction.

[0178] II. Exemplary Video Encoders

[0179] Figure 2 An exemplary video encoder 200 is shown. As shown, the video encoder 200 receives an input video signal from a video source 205 and encodes the signal into a bitstream 295. The video encoder 200 has several components or modules to encode the signal from the video source 205, including at least some components selected from the transform module 210, the quantization module 211, the inverse quantization module 214, the inverse transform module 215, the intra-frame estimation module 320, the intra-frame prediction module 225, the motion compensation module 230, the motion estimation module 235, the loop filter 245, the reconstructed frame buffer 250, the MV buffer 265, the MV prediction module 275, and the entropy encoder 290. The motion compensation module 230 and the motion estimation module 235 are part of the inter-frame prediction module 240.

[0180] In some embodiments, modules 210-290 are modules of software instructions executed by one or more processing units (e.g., processors) of a computing device or electronic device. In some embodiments, modules 210-290 are modules of hardware circuits implemented by one or more integrated circuits (ICs) of an electronic device. Although modules 210-290 are shown as separate modules, some modules may be combined into a single module.

[0181] The video source 205 provides a raw video signal that represents pixel data for each video frame without compression. The subtractor 208 calculates the difference between the raw video pixel data of the video source 205 and the predicted pixel data 213 from the motion compensation module 230 or the intra-prediction module 225. The transform module 210 converts this difference (or residual pixel data or residual signal 209) into transform coefficients (e.g., by performing a discrete cosine transform, or DCT). The quantization module 211 quantizes the transform coefficients into quantized data (or quantized coefficients) 212, which are encoded into the bitstream 295 by the entropy encoder 290.

[0182] The inverse quantization module 214 inversely quantizes the quantized data (or quantized coefficients) 212 to obtain transform coefficients, and the inverse transform module 215 performs inverse transform on the transform coefficients to generate a reconstructed residual 219. The reconstructed residual 219 is added to the predicted pixel data 213 to generate reconstructed pixel data 217. In some embodiments, the reconstructed pixel data 217 is temporarily stored in a line buffer (not shown) for intra-frame prediction and spatial MV prediction. The reconstructed pixels are filtered by the loop filter 245 and stored in the reconstructed picture buffer 250. In some embodiments, the reconstructed picture buffer 250 is a memory outside the video encoder 200. In some embodiments, the reconstructed picture buffer 250 is a memory within the video encoder 200.

[0183] The intra-frame estimation module 220 performs intra-prediction based on the reconstructed pixel data 217 to generate intra-prediction data. The intra-prediction data is provided to the entropy encoder 290 to be encoded into the bitstream 295. The intra-prediction data is also used by the intra-prediction module 225 to generate the predicted pixel data 213.

[0184] The motion estimation module 235 performs inter-prediction by providing MVs to reference pixel data of previously decoded video frames stored in the reconstructed picture buffer 250. These MVs are provided to the motion compensation module 230 to generate predicted pixel data.

[0185] Instead of encoding the complete actual MV into the bitstream, the video encoder 200 uses MV prediction to generate a predicted MV, and the difference between the MV used for motion compensation and the predicted MV is encoded as residual motion data and stored in the bitstream 295.

[0186] The MV prediction module 275 generates a predicted MV based on a reference MV generated when encoding a previous video frame, that is, a motion compensated MV for performing motion compensation. The MV prediction module 275 retrieves the reference MV from the previous video frame from the MV buffer 265. The video encoder 200 stores the MV generated for the current video frame in the MV buffer 265 as a reference MV for generating the predicted MV.

[0187] The MV prediction module 275 uses the reference MV to create a predicted MV. The predicted MV can be calculated by spatial MV prediction or temporal MV prediction. The difference between the predicted MV and the motion compensated MV (MC MV) of the current video frame (residual motion data) is encoded into the bitstream 295 by the entropy encoder 290.

[0188] The entropy encoder 290 encodes various parameters and data into the bitstream 295 by using entropy coding techniques such as Context-based Adaptive Binary Arithmetic Coding (CABAC) or Huffman encoding. The entropy encoder 290 encodes various header elements, flags, and quantized transform coefficients 212 along with residual motion data as syntax elements into the bitstream 295. The bitstream 295 is then stored in a storage device or transmitted to a decoder via a communication medium such as a network.

[0189] The loop filter 245 performs a filtering operation or a smoothing operation on the reconstructed pixel data 217 to reduce coding artifacts, especially at the boundaries of pixel blocks. In some embodiments, the filtering operation performed includes a sample adaptive offset (SAO). In some embodiments, the filtering operation includes an adaptive loop filter (ALF).

[0190] In some embodiments, the entropy encoder 290 signals or encodes slice information syntax elements into the bitstream 295. Slice information syntax elements, such as ph_slice_B_present_flag, ph_slice_P_present_flag, ph_slice_I_present_flag, and ph_mixed_slice_types_in_pic_flag, are encoded into a picture header associated with a video picture. Depending on the slice information syntax elements signaled, the entropy encoder 290 may bypass or omit encoding certain syntax elements based on the presence or absence of slice types therein.

[0191] Figure 3 A process 300 is conceptually depicted to signal one or more syntax elements in a picture header to indicate the slice types that may be present in the current picture. In some embodiments, one or more processing units (e.g., processors) on a computing device implementing the encoder 200 execute the process 300 by executing instructions stored on a computer-readable medium. In some embodiments, an electronic device implementing the encoder 200 executes the process 300.

[0192] The encoder receives (at block 310) raw pixel data to be encoded as a current picture of a video into a bitstream. The encoder signals (at block 320) a picture header for the current picture, the picture header including a set of one or more slice information syntax elements in the picture header to indicate the presence of one or more slice types present in the current picture.

[0193] The set of slice information syntax elements may indicate whether a slice of a particular type is present in the current picture (e.g., ph_slice_B_present_flag indicates that a B-type slice is present in the current picture, etc.). The set of slice information syntax elements may include one or more syntax elements to indicate whether (i) a first slice type is present in the current picture, and (ii) a second and different slice type is present in the current picture. A slice of the first slice type does not reference information of a picture other than the current picture (e.g., an intra slice or an I slice), and a slice of the second slice type has information that references a picture other than the current picture (e.g., an inter slice, a B or P slice).

[0194] When signaling a picture header, the video decoder may bypass parsing picture header syntax elements associated with a codec that is not associated with one or more slice types present in the current picture identified by the set of slice information syntax elements.

[0195] The encoder encodes (at block 330) slices of the current picture using the set of slice information syntax elements. In some embodiments, the set of slice information syntax elements includes a multi-slice-type syntax element to indicate whether the current picture includes slices of more than one slice type (e.g., ph_mixed_slice_types_in_pic_flag indicates that a mixture of multiple slice types may be present in the current picture). In some embodiments, when the multi-slice-type syntax element indicates that the current picture includes slices of more than one slice type, all slice headers of all slices of the current picture indicate a slice type. In some embodiments, the current picture references a picture parameter set (PPS), the PPS including a PPS syntax element to indicate whether slices of different slice types are allowed in the current picture (e.g., pps_mixed_slice_types_in_pic_flag). When the PPS syntax element indicates that slices of more than one slice type are allowed, the picture header of the current picture includes the multi-slice-type syntax element.

[0196] III. Exemplary Video Decoder

[0197] Figure 4 An exemplary video decoder 400 is shown. As shown, the video decoder 400 is an image-decoding or video-decoding circuit that receives a bitstream 495 and decodes the contents of the bitstream into pixel data of a video frame for display. The video decoder 400 has several components or modules for decoding the bitstream 495, including some components selected from an inverse quantization module 411, an inverse transform module 410, an intra-prediction module 425, a motion compensation module 430, a loop filter 445, a decoded picture buffer 450, an MV buffer 465, an MV prediction module 475, and a parser 490. The motion compensation module 430 is a part of the inter-prediction module 440.

[0198] In some embodiments, modules 410-490 are modules of software instructions executed by one or more processing units (e.g., processors) of a computing device. In some embodiments, modules 410-490 are modules of hardware circuits implemented by one or more integrated circuits of an electronic device. Although modules 410-490 are shown as separate modules, some modules may be combined into a single module.

[0199] The parser 490 (or entropy decoder) receives the bitstream 495 and performs preliminary parsing according to the syntax defined by the video-codec or image-codec standard. The parsed syntax elements include various header elements, flags, and quantized data (or quantized coefficients) 412. The parser 490 parses out the various syntax elements by using entropy coding techniques such as context-adaptive binary arithmetic coding (CABAC) or Huffman coding.

[0200] The inverse quantization module 411 de-quantizes the quantized data (or quantized coefficients) 412 to obtain transform coefficients, and the inverse transform module 410 performs an inverse transform operation on the transform coefficients 416 to generate a reconstructed residual signal 419. The reconstructed residual signal 419 is added to the predicted pixel data 413 from the intra-prediction module 425 or the motion compensation module 430 to generate decoded pixel data 417. The decoded pixel data is filtered by the loop filter 445 and stored in the decoded picture buffer 450. In some embodiments, the decoded picture buffer 450 is a memory outside the video decoder 400. In some embodiments, the decoded picture buffer 450 is a memory inside the video decoder 400.

[0201] The intra-prediction module 425 receives intra-prediction data from the bitstream 495 and generates predicted pixel data 413 from decoded pixel data 417 stored in the decoded picture buffer 450. In some embodiments, the decoded pixel data 417 is also stored in a line buffer (not shown) for intra-picture prediction and spatial MV prediction.

[0202] In some embodiments, the contents of the decoded frame buffer 450 are used for display. The display device 455 directly retrieves the contents of the decoded frame buffer 450 for display, or retrieves the contents of the decoded frame buffer back to a display buffer. In some embodiments, the display device receives pixel values ​​from the decoded frame buffer 450 via a pixel transfer.

[0203] The motion compensation module 430 generates predicted pixel data 413 from the decoded pixel data 417 stored in the decoded picture buffer 450 according to motion compensated MVs (MC MVs). These motion compensated MVs are decoded by adding the residual motion data received from the bitstream 495 to the predicted MVs received from the MV prediction module 475.

[0204] The MV prediction module 475 generates a predicted MV based on a reference MV generated when decoding a previous video frame, that is, a motion compensated MV for performing motion compensation. The MV prediction module 475 retrieves the reference MV of the previous video frame from the MV buffer 465. The video decoder 400 stores the motion compensated MV generated for decoding the current video frame in the MV buffer 465 as a reference MV for generating the predicted MV.

[0205] The loop filter 445 performs a filtering operation or smoothing operation on the decoded pixel data 417 to reduce coding artifacts, particularly at the boundaries of pixel blocks. In some embodiments, the filtering operation performed includes a sample adaptive offset (SAO). In some embodiments, the filtering operation includes an adaptive loop filter (ALF).

[0206] In some embodiments, the entropy decoder 490 parses or decodes slice information syntax elements into the bitstream 495. The slice information syntax elements, such as ph_slice_B_present_flag, ph_slice_P_present_flag, ph_slice_I_present_flag, and ph_mixed_slice_types_in_pic_flag, are parsed from a picture header associated with a video picture. Depending on the slice information syntax elements signaled in the picture header, the entropy decoder 490 may bypass or omit decoding or parsing certain syntax elements based on the presence or absence of slice types therein.

[0207] Figure 5 A process 500 is conceptually illustrated for using one or more syntax elements in a picture header to indicate the slice types that may be present in the current picture. In some embodiments, one or more processing units (e.g., processors) on a computing device implementing the decoder 400 execute the process 500 by executing instructions stored on a computer-readable medium. In some embodiments, an electronic device implementing the decoder 400 executes the process 500.

[0208] The decoder receives (at block 510) data from a bitstream to be decoded as a current picture of a video. The decoder parses (at block 520) a picture header of the current picture, the picture header including a set of one or more slice information syntax elements in the picture header to indicate the presence of one or more slice types present in the current picture. The set of slice information syntax elements may indicate whether a slice of a particular type is present in the current picture (e.g., ph_slice_B_present_flag indicates that a B-type slice is present in the current picture, etc.). The set of slice information syntax elements may include one or more syntax elements to indicate (i) whether a first slice type is present in the current picture, and (ii) whether a second and different slice type is present in the current picture; wherein a slice of the first slice type does not reference information of pictures other than the current picture (e.g., slice type I), and a slice of the second slice type has information that references pictures other than the current picture (e.g., slice types B and P).

[0209] When parsing a picture header, the video decoder may bypass parsing picture header syntax elements related to codecs that are not associated with one or more slice types present in the current picture identified by the set of slice information syntax elements.

[0210] The video decoder reconstructs (at block 530) slices of the current picture by using the set of slice information syntax elements. In some embodiments, the set of slice information syntax elements includes a multi-slice-type syntax element to indicate whether the current picture includes slices of more than one slice type (e.g., ph_mixed_slice_types_in_pic_flag indicates that a mixture of multiple slice types may be present in the current picture). In some embodiments, when the multi-slice-type syntax element indicates that the current picture includes slices of more than one slice type, all slice headers of all slices of the current picture indicate a slice type. In some embodiments, the current picture references a picture parameter set (PPS), the PPS including a PPS syntax element to indicate whether slices of different slice types are allowed in the current picture (e.g., pps_mixed_slice_types_in_pic_flag). When the PPS syntax element indicates that slices of more than one slice type are allowed, the picture header of the current picture includes the multi-slice-type syntax element.

[0211] IV. Example Electronic Systems

[0212] Many of the above features and applications can be implemented as software processing, which is specified as a set of instructions recorded on a computer readable storage medium (computer readable storage medium) (also referred to as a computer readable medium). When these instructions are executed by one or more computing units or processing units (for example, one or more processors, processor cores or other processing units), these instructions cause the processing unit to perform the actions represented by these instructions. Examples of computer readable media include, but are not limited to, CD-ROMs, flash memory drives, random access memory (RAM) chips, hard disks, erasable programmable read only memory (EPROM), electrically erasable programmable read only memory (EEPROM), etc. The computer readable medium does not include carrier waves and electrical signals connected wirelessly or wired.

[0213] In this specification, the term "software" means firmware in a read-only memory or an application stored in a magnetic storage device, which can be read into the memory for processing by the processor. At the same time, in some embodiments, multiple software inventions can be implemented as sub-parts of a larger program, while retaining different software inventions. In some embodiments, multiple software inventions can be implemented as separate programs. Finally, any combination of separate programs that implement the software inventions described herein together is within the scope of the present invention. In some embodiments, when installed to operate on one or more electronic systems, the software program defines one or more specific machine implementations that execute and implement the operations of the software program.

[0214] Figure 6 An electronic system 600 is conceptually illustrated in which some embodiments of the present disclosure may be implemented. The electronic system 600 may be a computer (e.g., a desktop computer, a personal computer, a tablet computer, etc.), a phone, a PDA, or other types of electronic devices. The electronic system includes various types of computer-readable media and interfaces for various other types of computer-readable media. The electronic system 600 includes a bus 605, a processing unit 610, a graphics-processing unit (GPU) 615, a system memory 620, a network 625, a read-only memory (ROM) 630, a permanent storage device 635, an input device 640, and an output device 645.

[0215] Buses 605 collectively represent all system buses, peripheral buses, and chipset buses that are communicatively coupled to a multitude of internal devices of electronic system 600. For example, bus 605 is communicatively coupled to processing unit 610 via image processing unit 615, read-only memory 630, system memory 620, and permanent storage device 635.

[0216] To these various memory units, the processing unit 610 retrieves instructions to execute and data to process in order to perform the processes of the present invention. In various embodiments, the processing unit can be a single processor or a multi-core processor. Certain instructions are transmitted to and executed by the image processing unit 615. The image processing unit 615 can offload various calculations or supplement the image processing provided by the processing unit 610.

[0217] The read-only memory 630 stores static data and instructions required by the processing unit 610 or other modules of the electronic system. On the other hand, the permanent storage device 635 is a read-and-write memory device. This device is a non-volatile memory unit that stores instructions and data even when the electronic system 600 is turned off. Some embodiments of the present invention use a large-capacity storage device (such as a disk or optical disk and its corresponding disk drive) as the permanent storage device 635.

[0218] Other embodiments use an unloadable storage device (such as a floppy disk, a flash memory device, etc., and its corresponding disk drive) as the permanent storage device. Like the permanent storage device 635, the system memory 620 is a read-write memory device. However, unlike the storage device 635, the system memory 620 is a volatile read-write memory, such as a random access memory. The system memory 620 stores some instructions and data that the processor needs when running. In some embodiments, the processing according to the present invention is stored in the system memory 620, the permanent storage device 635 and / or the read-only memory 630. For example, various memory units include instructions for processing multimedia clips according to some embodiments. For these various memory units, the processing unit 610 retrieves the executed instructions and processed data in order to perform the processing of certain embodiments.

[0219] The bus 605 is also connected to an input device 640 and an output device 645. The input device 640 enables a user to communicate information and select commands to the electronic system. The input device 640 includes an alphanumeric keyboard and pointing device (also known as a "cursor control device"), a camera (such as a webcam), a microphone or similar device for receiving voice commands, etc. The output device 645 displays images generated by the electronic system or data output in other ways. The output device 645 includes a printer and a display device, such as a cathode ray tube (CRT) or a liquid crystal display (LCD), and a speaker or similar audio output device. Some embodiments include devices such as a touch screen that serves as both an input device and an output device.

[0220] Finally, as shown in FIG. 6 , bus 605 also couples electronic system 600 to network 625 via a network interface card (not shown). In this manner, the computer may be part of a network of computers (e.g., a local area network (LAN), a wide area network (WAN), or an intranet) or a network of networks (e.g., the Internet). Any or all of the components of electronic system 600 may be used in conjunction with the present invention.

[0221] Some embodiments include electronic components, such as microprocessors, storage devices, and memories, which store computer program instructions in machine-readable media or computer-readable media (optionally referred to as computer-readable storage media, machine-readable media, or machine-readable storage media). Some examples of computer-readable media include RAM, ROM, read-only compact discs (CD-ROM), recordable compact discs (CD-R), rewritable compact discs (CD-RW), read-only digital versatile discs (e.g., DVD-ROM, dual-layer DVD-ROM), various recordable / rewritable DVDs (e.g., DVD RAM, DVD-RW, DVD+RW, etc.), flash memories (e.g., SD cards, mini SD cards, micro SD cards, etc.), magnetic and / or solid-state hard drives, read-only and Computer readable media may store computer programs executed by at least one processing unit and include sets of instructions for performing various operations. Examples of computer programs or computer code include machine code, such as that produced by a compiler, and files containing high-level code that is executed by a computer, electronic component, or microprocessor using an interpreter.

[0222] While the above discussion refers primarily to microprocessors or multi-core processors that execute software, many of the above functions and applications are performed by one or more integrated circuits, such as application specific integrated circuits (ASICs) or field programmable gate arrays (FPGAs). In some embodiments, such integrated circuits execute instructions stored on the circuits themselves. In addition, some embodiments execute software stored in programmable logic devices (PLDs), ROM, or RAM devices.

[0223] As used in the specification and any claims of the present invention, the terms "computer", "server", "processor" and "memory" refer to electronic devices or other technical equipment. These terms do not include people or groups. For the purpose of this specification, the term display or display device refers to displaying on an electronic device. As used in the specification and any claims of the present invention, the terms "computer-readable medium", "computer-readable media" and "machine-readable medium" are entirely limited to tangible, physical objects that store information in a computer-readable form. These terms do not include any wireless signals, wired download signals and any other transient signals.

[0224] While the present invention has been described in conjunction with many specific details, a person skilled in the art will recognize that the present invention may be implemented in other specific forms without departing from the spirit of the present invention. In addition, a large number of figures (including Figures 3 and 5) conceptually illustrate the processing. The specific operations of these processes may not be performed in the exact order shown and described. These specific operations may not be performed in a continuous series of operations, and different specific operations may be performed in different embodiments. In addition, the processing is implemented using several sub-processes, or as part of a larger macro process. Therefore, a person skilled in the art will understand that the present invention is not limited by the foregoing illustrative details, but is defined by the claims.

[0225] Additional Notes

[0226] The subject matter described herein sometimes represents different elements, which are included in or connected to other different elements.It is understandable that the described structure is only an example, and in fact, it can be implemented by many other structures to achieve the same function.Conceptually, the arrangement of any components that achieve the same function is actually "associated", so as to achieve the required function.Therefore, regardless of structure or intermediate components, any two elements combined to achieve a specific function are considered to be "interrelated", to achieve the required function.Similarly, any two associated elements are considered to be "operably connected" or "operably coupled" to achieve a specific function.Any two components that can be associated with each other are also considered to be "operably coupled" to achieve a specific function.The specific example of operable connection includes but is not limited to physically pairable and / or physically interacting elements, and / or wirelessly interactive and / or wirelessly interactive elements, and / or logically interactive and / or logically interactive elements.

[0227] Furthermore, with respect to the use of substantially any plural and / or singular terms, those skilled in the art can translate from the plural to the singular and / or from the singular to the plural as appropriate to the context and / or application. For clarity, the different singular / plural permutations are expressly provided herein.

[0228] In addition, it will be understood by those of ordinary skill in the art that, in general, the terms used in the present invention, especially in the claims, are generally used as "open" terms according to the subject matter of the claims, for example, "including" should be interpreted as "including but not limited to", "having" should be interpreted as "at least having", "including" should be interpreted as "including but not limited to", etc. It will be further understood by those of ordinary skill in the art that if a specific number of claim contents are planned to be introduced, it will be explicitly indicated in the claims, and will not be displayed in the absence of such content. For example, to aid understanding, the following claims may include the phrases "at least one" and "one or more" to introduce the claim contents. However, the use of these phrases should not be understood to imply the use of the indefinite article "a" or "an" to introduce the claim contents and limit any specific claim. Even when the same claim includes the introductory phrases "one or more" or "at least one", the indefinite article, such as "a" or "an", should be interpreted to mean at least one or more, and the same is true for the use of the explicit description used to introduce the claim. In addition, even if a specific number is explicitly cited, the specific number of claim contents will be clearly indicated in the claims. Introductory content with a certain number of references, a person of ordinary skill in the art will recognize that such content should be interpreted as indicating the number of references, for example, "two references" without other modifications means at least two references, or two or more references. In addition, when a statement similar to "at least one of A, B, and C" is used, it is usually stated so that a person of ordinary skill in the art can understand the statement, for example, "a system includes at least one of A, B, and C" will include but is not limited to a system with A alone, a system with B alone, a system with C alone, a system with A and B, a system with A and C, a system with B and C, and / or a system with A, B, and C, etc. A person of ordinary skill in the art will further understand that any separated words and / or phrases represented by two or more alternative terms, whether in the specification, in the claims, or in the drawings, should be understood to include the possibility of one of these terms, one of them, or both of these terms. For example, "A or B" should be understood as the possibility of "A", or "B", or "A and B".

[0229] As can be seen from the foregoing, various embodiments have been described herein for illustrative purposes, and various modifications may be made without departing from the scope and spirit of the invention. Therefore, the various embodiments disclosed herein are not intended to be limiting, and the scope of the patent application represents the true scope and spirit.

Claims

1. A video decoding method, comprising: Receiving data from a bitstream to be decoded as a current frame of a video; parsing a picture header of the current picture, the picture header comprising a set of one or more slice information syntax elements in the picture header to indicate the presence of one or more slice types present in the current picture, wherein the set of one or more slice information syntax elements comprises: a multiple slice types syntax element to indicate whether the current picture includes slices of more than one slice type; as well as A slice of the current picture is reconstructed by using the set of one or more slice information syntax elements.

2. The video decoding method according to claim 1, characterized in that: Further including: Picture header syntax elements that are not associated with one or more slice types present in the current picture identified by the set of slice information syntax elements are bypassed without being parsed.

3. The video decoding method according to claim 1, characterized in that: The set of slice information syntax elements indicates whether a slice of a specific type exists in the current picture.

4. The video decoding method according to claim 1, characterized in that: The set of slice information syntax elements includes one or more syntax elements: used to indicate (i) whether a first slice type exists in the current picture, and (ii) whether a second slice type exists in the current picture, wherein the second slice type is different from the first slice type.

5. The video decoding method according to claim 4, characterized in that: wherein a slice of the first slice type does not refer to information of other pictures except the current picture, wherein the first slice type is an I slice type, and the slice of the first slice type is an intra slice; And a slice of the second slice type has information referring to other pictures except the current picture, wherein the second slice type is a B or P slice type, and the slice of the second slice type is an inter-frame slice.

6. The video decoding method according to claim 1, characterized in that: When the multiple slice type syntax element indicates that the current picture includes slices of more than one slice type, all slice headers of all slices of the current picture indicate all slice types.

7. The video decoding method according to claim 1, characterized in that: The current picture refers to a picture parameter group, the picture parameter group comprising: a picture parameter group syntax element to indicate whether slices of different slice types are allowed in the current picture, wherein when the picture parameter group syntax element indicates that slices of more than one slice type are allowed, the picture header of the current picture comprises the multiple slice type syntax element.

8. A video encoding method, comprising: Receiving raw pixel data for encoding as a current frame of a video into a bitstream; signaling a picture header of the current picture, the picture header comprising a set of one or more slice information syntax elements in the picture header to indicate the presence of one or more slice types present in the current picture, wherein the set of one or more slice information syntax elements comprises: a multiple slice types syntax element to indicate whether the current picture includes slices of more than one slice type; as well as A slice of the current picture is encoded by using the set of one or more slice information syntax elements.

9. An electronic device comprising: A video decoder circuit is configured to operate, comprising: Receiving data from a bitstream to be decoded as a current frame of a video; parsing a picture header of the current picture, the picture header comprising a set of one or more slice information syntax elements in the picture header to indicate the presence of one or more slice types present in the current picture, wherein the set of one or more slice information syntax elements comprises: a multiple slice types syntax element to indicate whether the current picture includes slices of more than one slice type; and A slice of the current picture is reconstructed by using the set of one or more slice information syntax elements.

Images