Coding of picture order count in video bitstreams - Patents.com

Abstract

A method, system, and device for coding a picture order count in video bitstream processing is disclosed. An example video processing method includes converting a video including one or more pictures to a bitstream of the video, the bitstream conforming to a format rule, the format rule specifying that a picture derivation in a decoding process for the picture order count is based on a second flag associated with a first flag, the picture associated with the first flag being a previous picture in decoding order having (i) a first identifier equal to a slice or picture header that references a reference picture list syntax structure, (ii) a second identifier and second flag equal to 0, and (iii) a picture type different from a random access skipped leading picture and a random access decodable leading picture.

Description

[Technical Field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application is based on International Patent Application No. PCT / US2021 / 033780, filed May 21, 2021, which claims priority to and benefit of U.S. Provisional Patent Application No. 63 / 029,321, filed May 22, 2020. All of the foregoing patent applications are incorporated herein by reference in their entireties.

[0002] This patent specification relates to image and video coding and decoding. [Background technology]

[0003] Digital video accounts for the largest bandwidth usage on the Internet and other digital communications networks, and the bandwidth demands for digital video use are expected to continue to grow as the number of connected user devices capable of receiving and displaying video increases. Summary of the Invention

[0004] This specification discloses techniques that may be used by video encoders and decoders to perform video encoding or decoding.

[0005] In one exemplary aspect, a video processing method is disclosed that includes converting between video including one or more pictures and a bitstream of the video, the bitstream conforming to a format rule that specifies that a picture timing (PT) supplemental enhancement information (SEI) message, if included in the bitstream, is access unit (AU) specific, and each picture of the one or more pictures that is a random access skipped reading (RASL) picture includes only a RASL network abstraction layer unit type (NUT).

[0006] In another exemplary aspect, another video processing method is disclosed that includes converting video including one or more pictures to a bitstream of the video, the bitstream conforming to format rules that permit use of a random access skipped leading (RADL) sub-picture in a RADL picture as a reference sub-picture for predicting a co-located RADL picture in a RADL sub-picture associated with the same clean random access (CRA) picture as the RASL picture.

[0007] In yet another exemplary aspect, another video processing method is disclosed. The method includes converting a video including one or more pictures to a bitstream of the video, the bitstream conforming to a format rule, the format rule being associated with a first flag and specifying that a picture derivation in a decoding process for a picture order count is based on a second flag, the picture associated with the first flag being a previous picture in decoding order having (i) a first identifier that is the same as a slice or picture header that references a reference picture list syntax structure, (ii) a second identifier and second flag that are equal to 0, and (iii) a picture type that is different from a random access skipped leading (RASL) picture and a random access decodable leading (RADL) picture, the first flag indicating whether a third flag is present in the bitstream, the second flag indicating whether the current picture is used as a reference picture, and the third flag being used to determine values ​​of one or more most significant bits of a picture order count value of a long-term reference picture.

[0008] In yet another exemplary aspect, another video processing method is disclosed. The method includes converting video including one or more pictures to a bitstream of the video, where the bitstream conforms to a format rule that specifies that a variable used to determine timing of removing or decoding a decoding unit (DU) is access unit (AU) specific and is derived based on a flag indicating whether the current picture is allowed to be used as a reference picture.

[0009] In yet another exemplary aspect, another video processing method is disclosed. The method includes converting video including one or more pictures to a bitstream of the video, the bitstream conforming to a format rule, the format rule specifying that a buffering period supplemental enhancement information (SEI) message and a picture timing SEI message, if included in the bitstream, are access unit (AU) specific, and a first variable associated with the buffering period SEI message and a second variable associated with the buffering period SEI message and the picture timing SEI message are allowed to use a current picture as a reference picture. The first variable indicates an access unit that contains (i) an identifier equal to 0 and (ii) a picture that is not a random access skipped leading (RASL) picture or a random access decodable leading (RADL) picture and whose flag is equal to 0, and the second variable indicates that the access unit is not the first AU in decoding order and that the previous AU in decoding order contains (i) an identifier equal to 0 and (ii) a picture that is not a random access skipped leading (RASL) picture or a random access decodable leading (RADL) picture and whose flag is equal to 0.

[0010] In yet another exemplary aspect, another video processing method is disclosed. The method includes converting a video including one or more pictures to a bitstream of the video, the bitstream conforming to a format rule, the format rule specifying that derivation of a first variable and a second variable associated with a first picture and a second picture is based on a flag, the first picture being a current picture, and the second picture (i) including a first identifier equal to 0, (ii) including a flag equal to 0, and (iii) not being a random access skipped leading (RASL) picture or a random access decodable leading (RADL) picture. The first variable and the second variable are the maximum and minimum values, respectively, of the picture order counts of (i) the first picture, (ii) the second picture, (iii) one or more short-term reference pictures referenced by all entries in the reference picture list of the first picture, and (iv) each picture output with a coded picture buffer (CPB) removal time shorter than the CPB removal time of the first picture and a decoded picture buffer (DPB) output time greater than or equal to the CPB removal time of the first picture, which are the previous picture in decoding order, and have a second identifier equal to that of the first picture.

[0011] In yet another exemplary aspect, another video processing method is disclosed. The method includes converting between video including one or more pictures and a bitstream of the video, the bitstream conforming to a format rule, the format rule specifying a flag and a syntax element, if included in the bitstream, that are access unit (AU) specific, the flag indicating whether a nominal Coded Picture Buffer (CPB) removal time of the current AU is determined relative to (a) a nominal CPB removal time of a previous AU associated with a buffering period supplemental enhancement information (SEI) message or (b) a nominal CPB removal time of the current AU, responsive to the current AU not being the first AU of the bitstream in decoding order, and the syntax element specifying a CPB removal delay increment value relative to the nominal CPB removal time of the current AU, responsive to the current AU not being the first AU of the bitstream in decoding order.

[0012] In yet another exemplary aspect, another video processing method is disclosed, the method including converting between video including one or more pictures and a bitstream of the video, the bitstream conforming to a format rule, the format rule specifying that a plurality of variables and a picture timing supplemental enhancement information (SEI) message, if included in the bitstream, are access unit (AU) specific, the picture timing SEI message including a plurality of syntax elements, a first variable of the plurality of variables indicating whether a current AU is associated with a buffering period SEI message, and second and third variables of the plurality of variables associated with an indication of whether the current AU is an AU that initializes a hypothetical reference decoder (HRD); A first syntax element of the plurality of syntax elements specifies the number of clock ticks to wait after the AU is removed from the coded picture buffer (CPB) until one or more decoded pictures of the AU are output from the decoded picture buffer (DPB), a second syntax element of the plurality of syntax elements specifies the number of sub-clock ticks to wait after the last decoding unit (DU) of the AU is removed from the CPB until one or more decoded pictures of the AU are output from the DPB, and a third syntax element of the plurality of syntax elements specifies the number of picture period intervals of the element that the one or more decoded pictures of the current AU occupy relative to the display model.

[0013] In yet another exemplary aspect, another video processing method is disclosed. The method includes converting video including one or more pictures to a bitstream of the video, the bitstream conforming to a format rule, the format rule specifying that a syntax element associated with a decoding picture buffer (DPB), when included in the bitstream, is access unit (AU) specific, and the syntax element specifies a number of sub-clock ticks to wait after removing a last decoding unit (DU) of the AU from a coded picture buffer (CPB) until one or more decoded pictures of the AU are output from the DPB.

[0014] In yet another exemplary aspect, another video processing method is disclosed. The method includes converting video including one or more pictures to a bitstream of the video, the bitstream conforming to a format rule, where the format rule specifies that a flag, if included in the bitstream, is access unit (AU) specific, the value of the flag being based on whether the associated AU is an intra random access point (IRAP) AU or a gradual decoding update (GDR) AU, the value of the flag specifying (i) whether a syntax element is present in a buffering period supplemental enhancement information (SEI) message and (ii) whether alternative timing information is present in a picture timing SEI message for a current buffering period.

[0015] In yet another exemplary aspect, another video processing method is disclosed. The method includes converting video including one or more pictures to a bitstream of the video, the bitstream conforming to a format rule, the format rule specifying that a value of a first syntax element is based on a flag indicating whether a temporal distance between output times of consecutive pictures of a hypothetical reference decoder (HRD) is constrained and a variable specifying a highest temporal sub-layer to be decoded, the first syntax element specifying a number of element picture period intervals that one or more decoded pictures of a current AU occupy with respect to a display model.

[0016] In yet another exemplary aspect, a video encoder apparatus is disclosed, the video encoder comprising a processor configured to implement the above-described method.

[0017] In yet another exemplary aspect, a video decoder apparatus is disclosed, the video decoder comprising a processor configured to implement the above-described method.

[0018] In yet another exemplary aspect, a computer readable medium having code stored thereon is disclosed, the code performing one of the methods described herein in the form of processor executable code.

[0019] These and other features are described throughout this document. [Brief explanation of the drawings]

[0020] [Figure 1] FIG. 1 is a block diagram illustrating an example video processing system in which various techniques disclosed herein may be implemented. [Figure 2] FIG. 1 is a block diagram illustrating an example of a hardware platform used for video processing. [Figure 3] 1 is a block diagram illustrating an example video coding system in which some embodiments of the present disclosure may be implemented. [Figure 4] FIG. 1 is a block diagram illustrating an example of an encoder capable of implementing some embodiments of the present disclosure. [Figure 5] FIG. 2 is a block diagram illustrating an example of a decoder capable of implementing some embodiments of the present disclosure. [Figure 6] 1 is a flowchart illustrating an example of a method for video processing. [Figure 7] 1 is a flowchart illustrating an example of a method for video processing. [Figure 8] 1 is a flowchart illustrating an example of a method for video processing. [Figure 9] 1 is a flowchart illustrating an example of a method for video processing. [Figure 10] 1 is a flowchart illustrating an example of a method for video processing. [Figure 11] 1 is a flowchart illustrating an example of a method for video processing. [Figure 12] 1 is a flowchart illustrating an example of a method for video processing. [Figure 13] 1 is a flowchart illustrating an example of a method for video processing. [Figure 14]1 is a flowchart illustrating an example of a method for video processing. [Figure 15] 1 is a flowchart illustrating an example of a method for video processing. [Figure 16] 1 is a flowchart illustrating an example of a method for video processing. DETAILED DESCRIPTION OF THE INVENTION

[0021] Section headings are used herein for ease of understanding and are not intended to limit the applicability of the technology and embodiments described in each section to that section alone. Furthermore, the term H.266 is used in certain descriptions for ease of understanding only and is not intended to limit the scope of the disclosed technology. Thus, the technology described herein is applicable to other video codec protocols and designs.

[0022] 1. Introduction This specification relates to video coding technology. Specifically, it relates to the handling of discardable pictures / AUs in video coding and the semantics of HRD-related SEI messages. Examples of discardable pictures that can be discarded in certain scenarios include RASL pictures, RADL pictures, and pictures with ph_non_ref_pic_flag equal to 1. HRD-related SEI messages include BP, PT, and DUI SEI messages. This concept may be applied, individually or in various combinations, to any video coding standard or non-standard video codec that supports multi-layer video coding, such as the currently developed Versatile Video Coding (VVC).

[0023] 2. Abbreviations APS Adaptation Parameter Set AU Access Unit AUD Access Unit Delimiter AVC Advanced Video Coding CLVS Coded Layer Video Sequence CPB Coded Picture Buffer CRA Clean Random Access CTU Coding Tree Unit CVS Coded Video Sequence DCI Decoding Capability Information DPB Decoded Picture Buffer EOB End Of Bitstream EOS End Of Sequence GDR Gradual Decoding Refresh HEVC High Efficiency Video Coding HRD Hypothetical Reference Decoder IDR Instantaneous Decoding Refresh ILP Inter-Layer Prediction ILRP Inter-Layer Reference Picture JEM Joint Exploration Model LTRP Long-Term Reference Picture MCTS Motion-Constrained Tile Sets NAL Network Abstraction Layer OLS Output Layer Set PH Picture Header PPS Picture Parameter Set PTL Profile, Tier and Level PU Picture Unit RAP Random Access Point RADL Random Access Decodable Leading Picture RASL Random Access Skipped Leading Picture RBSP Raw Byte Sequence Payload SEI Supplemental Enhancement Information SPS Sequence Parameter Set STRP Short-Term Reference Picture SVC Scalable Video Coding VCL Video Coding Layer VPS Video Parameter Set VTM VVC Test Model VUI Video Usability Information VVC Versatile Video Coding

[0024] 3. Initial consultations Video coding standards have evolved primarily through the development of well-known ITU-T and ISO / IEC standards. ITU-T developed H.261 and H.263, while ISO / IEC developed MPEG-1 and MPEG-4 Visual. The two organizations jointly developed H.262 / MPEG-2 Video, H.264 / MPEG-4 AVC (Advanced Video Coding), and H.265 / HEVC. Since H.262, video coding standards have been based on hybrid video coding architectures that utilize temporal prediction and transform coding. To explore future video coding technologies beyond HEVC, VCEG and MPEG jointly established the Joint Video Exploration Team (JVET) in 2015. Since then, many new methods have been adopted by the JVET and incorporated into reference software called the Joint Exploration Model (JEM). JVET meets quarterly, and the new coding standard aims to achieve a 50% bitrate reduction compared to HEVC. At the April 2018 JVET meeting, the new video coding standard was officially named "VVC (Versatile Video Coding)" and the first version of the VVC Test Model (VTM) was released at that time. As efforts to contribute to the standardization of VVC continue, new coding techniques are adopted for the VVC standard at every JVET meeting. After each meeting, the VVC Working Draft and Test Model VTM are updated. The VVC project is currently aiming for Technical Finalization (FDIS) at the July 2020 meeting.

[0025] 3.1 Reference Picture Management and Reference Picture List (RPL) Reference picture management is a core function required for any video coding scheme that uses inter prediction: it manages the storage and removal of reference pictures into and from the decoded picture buffer (DPB), and places reference pictures in the proper order within the RPL.

[0026] HEVC's reference picture management, which includes reference picture marking and removal from the decoded picture buffer (DPB) and reference picture list construction (RPLC), differs from that of AVC. Instead of the reference picture marking mechanism based on a sliding window plus adaptive memory management control operations (MMCO) in AVC, HEVC specifies a reference picture management and marking mechanism based on a so-called reference picture set (RPS). As a result, RPLC is based on the RPS mechanism. An RPS consists of a set of reference pictures associated with a picture, consisting of all reference pictures preceding the associated picture in decoding order, and may be used for inter-prediction of the associated picture or any picture following the associated picture in decoding order. A reference picture set consists of five lists of reference pictures. The first three lists contain all reference pictures that may be used in inter-prediction of the current picture and in inter-prediction of one or more pictures following the current picture in decoding order. The other two lists consist of all reference pictures that are not used in the inter prediction of the current picture but may be used in the inter prediction of one or more pictures that follow the current picture in decoding order. RPS provides "intra-coded" signaling of the DPB status instead of "inter-coded" signaling as in AVC, mainly to improve error resilience. RPLC processing in HEVC is based on RPS by signaling an index to an RPS subset of each reference index, and this processing is simpler than the RPLC processing in AVC.

[0027] Reference picture management in VVC is more similar to HEVC than AVC, but somewhat simpler and more robust. As in these standards, two RPLs, list0 and list1, are derived, but these are signaled more directly rather than based on the reference picture set concept used in HEVC or the automatic sliding window processing used in AVC. Reference pictures are listed as either active or inactive entries for the RPL, and only active entries may be used as reference indexes in inter-prediction of CTUs of the current picture. Inactive entries indicate other pictures that should be kept in the DPB for reference by other pictures arriving later in the bitstream.

[0028] 3.2. Random Access and its Support in HEVC and VVC Random access refers to starting access and decoding of a bitstream from a picture that is not the first picture of the bitstream in decoding order. To support tuning and channel switching in broadcast / multicast and multi-party video conferencing, local playback and seeking in streaming, and stream adaptation in streaming, the bitstream needs to contain frequent random access points, which are typically intra-coded pictures but may also be inter-coded pictures (e.g., in the case of gradual decoding update).

[0029] HEVC includes signaling of Intra Random Access Point (IRAP) pictures in the header of a NAL unit via the NAL unit type. Three types of IRAP pictures are supported: Instant Decoder Refresh (IDR), Clean Random Access (CRA), and Broken Link Access (BLA) pictures. IDR pictures constrain the inter-picture prediction structure to not reference any pictures before the current group of pictures (GOP) and are traditionally called closed GOP random access points. CRA pictures relax the restrictions by allowing a picture to reference pictures before the current GOP, all of which are discarded in the case of random access. CRA pictures are traditionally called open GOP random access points. BLA pictures are typically generated by splicing two bitstreams or parts of them at a CRA picture, for example, during a stream switch. To enable better system usage of IRAP pictures, a total of six different NAL units are defined to signal properties of IRAP pictures that can be used to better suit the types of Stream Access Points, such as those defined in the ISO Base Media File Format (ISOBMFF), used for random access support in Dynamic Adaptive Streaming over HTTP (DASH).

[0030] VVC supports three types of IRAP pictures, two types of IDR pictures (one with an associated RADL picture and the other without), and one type of CRA picture, which are essentially the same as HEVC. The BLA picture type in HEVC is not included in VVC for two main reasons: i) the basic functionality of a BLA picture can be achieved by adding an end of sequence NAL unit to a CRA picture, and the presence of this end of sequence NAL unit indicates that the following picture starts a new CVS in a single-layer bitstream; and ii) in the development of VVC, it was desirable to specify fewer NAL unit types than HEVC, as indicated by using 5 bits instead of 6 bits for the NAL unit type field in the NAL unit header.

[0031] Another important difference in random access support between VVC and HEVC is that VVC supports GDR in a more prescriptive way. In GDR, bitstream decoding can start with an inter-coded picture, and initially, the entire picture region cannot be correctly decoded, but after several pictures, the entire picture region can be correctly decoded. AVC and HEVC also support GDR using a recovery point SEI message for signaling GDR random access points and recovery points. In VVC, a new NAL unit type is specified to indicate GDR pictures, and recovery points are signaled in the picture header syntax structure. CVS and bitstreams can start with a GDR picture. This means that an entire bitstream can contain only inter-coded pictures without a single intra-coded picture. The main advantage of specifying GDR support in this way is that it provides GDR-compatible operation. GDR allows the encoder to smooth the bitstream bitrate by distributing intra-coded slices or blocks across multiple pictures rather than intra-coding the entire picture, thereby enabling a significant reduction in end-to-end delay, which is more important today than ever before as ultra-low latency applications such as wireless displays, online gaming, and drone-based applications become more common.

[0032] Another GDR-related feature in VVC is virtual boundary signaling. The boundary between an updated region (i.e., a correctly decoded region) and an unupdated region in a picture between a GDR picture and its recovery point may be signaled as a virtual boundary, and when signaled, in-loop filtering across the boundary will not be applied, thus preventing inconsistencies in decoding of some samples at and near the boundary. This can be useful if an application decides to display correctly decoded regions during GDR processing.

[0033] IRAP pictures and GDR pictures can be collectively referred to as Random Access Point (RAP) pictures.

[0034] 3.3 Changing picture resolution within a sequence In AVC and HEVC, the spatial resolution of a picture cannot be changed unless a new sequence using the new SPS begins with an IRAP picture. VVC allows changing the resolution of a picture within a sequence at a certain position without encoding an IRAP picture, which is always intra-coded. This feature is called reference picture resampling (RPR) because it is necessary to resample the reference picture used for inter prediction when the reference picture has a different resolution than the current picture being decoded.

[0035] The scaling ratio is limited to between 1 / 2 (2x downsampling from the reference picture to the current picture) and 8 (8x upsampling). To accommodate various scaling ratios between the reference picture and the current picture, three sets of resampling filters with different frequency cutoffs are defined. The three sets of resampling filters are applicable to scaling ratios ranging from 1 / 2 to 1 / 1.75, 1 / 1.75 to 1 / 1.25, and 1 / 1.25 to 8, respectively. Each set of resampling filters has 16 phases for luma and 32 phases for chroma, similar to the motion-compensated interpolation filters. In practice, the typical MC interpolation process is a special case of the resampling process, with scaling ratios ranging from 1 / 1.25 to 8. The horizontal and vertical scaling ratios are derived based on the picture width and height, as well as the left, right, top, and bottom scaling offsets specified for the reference picture and the current picture.

[0036] Other aspects of the VVC design to support this feature that differ from HEVC include: i) signaling picture resolution and corresponding compatibility window in the PPS instead of the SPS, signaling the maximum picture resolution in the SPS; ii) in the case of a single-layer bitstream, each picture store (one slot in the DPB for storing one decoded picture) occupies the buffer size required to store a decoded picture with the maximum picture resolution.

[0037] 3.4 Scalable Video Coding (SVC) in General and VVC Scalable Video Coding (SVC), sometimes referred to as scalability in video coding, refers to video coding in which a Base Layer (BL) (sometimes called a Reference Layer (RL)) and one or more Enhancement Layers (ELs) are used. In SVC, the base layer can carry video data at a basic quality level. One or more enhancement layers can carry additional video data, e.g., to support higher spatial, temporal, and / or signal-to-noise (SNR) levels. Enhancement layers may be defined relative to a previous, coded layer. For example, a lower layer can function as a BL and an upper layer can function as an EL. An intermediate layer can function as either an EL or an RL, or both. For example, an intermediate layer (e.g., a layer that is neither the lowest nor the highest layer) may be an EL for a layer below it, e.g., a base layer or any intervening enhancement layer, and simultaneously serve as an RL for one or more enhancement layers above it. Similarly, in multiview or 3D extensions of the HEVC standard, there may be multiple views, and information from one view may be utilized to code (e.g., encode or decode) information from another view (e.g., motion estimation, motion vector prediction, and / or other redundancy).

[0038] In SVC, parameters used by an encoder or decoder are grouped into parameter sets based on the coding level at which they can be utilized (e.g., the video level, the sequence level, the picture level, the slice level, etc.). For example, parameters that can be utilized by a coded video sequence in different layers in a bitstream may be included in a video parameter set (VPS), and parameters utilized by one or more pictures in a coded video sequence may be included in a sequence parameter set (SPS). Similarly, parameters utilized by one or more slices of a picture may be included in a picture parameter set (PPS), and other parameters specific to a slice may be included in a slice header. Similarly, an indication of which parameter set a particular layer is using at a given time may be provided at various coding levels.

[0039] Thanks to VVC's support for Reference Picture Resampling (RPR), the upsampling required for spatial scalability support can be performed simply using an RPR upsampling filter. This allows a bitstream containing multiple layers, such as two layers for SD and HD resolutions in VVC, to be designed without requiring additional signal processing-level coding tools. Nevertheless, scalability support requires high-level syntax changes (compared to cases without scalability support). Scalability support is specified in VVC Version 1. Unlike scalability support in any previous video coding standard, including extensions to AVC and HEVC, the design of VVC's scalability has been optimized for single-layer decoders. Decoding capabilities for multi-layer bitstreams are specified as if the bitstream had only one layer. For example, decoding capabilities, such as DPB size, are specified in a manner independent of the number of layers in the bitstream being decoded. Essentially, a decoder designed for a single-layer bitstream does not require significant modifications to be able to decode a multi-layer bitstream. Compared to the multi-layer extension designs of AVC and HEVC, aspects of HLS have been significantly simplified at the expense of some flexibility: for example, an IRAP AU is required to contain a picture for each layer present in the CVS.

[0040] 3.5 Parameter Set AVC, HEVC, and VVC specify parameter sets. Parameter set types include SPS, PPS, APS, and VPS. SPS and PPS are supported by all of AVC, HEVC, and VVC. VPS was introduced in HEVC and is included in both HEVC and VVC. APS was not included in AVC or HEVC, but is included in the text of the recent VVC draft.

[0041] The SPS is designed to carry sequence-level header information, while the PPS is designed to carry picture-level header information that does not change frequently. Using the SPS and PPS avoids redundant signaling of frequently changing information because it does not need to be repeated for each sequence or picture. Furthermore, using the SPS and PPS enables out-of-band transmission of important header information, thereby not only avoiding the need for redundant transmission but also improving error resilience.

[0042] VPS was introduced to carry sequence-level header information that is common to all layers of a multi-layer bitstream.

[0043] APS was introduced to carry such picture-level or slice-level information, which requires a significant number of bits to code, is shared by multiple pictures, and can exist in a large number of different variations in the sequence.

[0044] 4. Technical problem solved by the disclosed technical solution The existing design for handling discardable pictures in the recent VVC text (JVET-R2001-vA / v10) has the following problems: 1) In Clause 3 (Definitions), the sentence "A RASL picture is used as a reference picture for the decoding process of a non-RASL picture" as part of the definition of a RASL picture is problematic because it is no longer true in some scenarios and may cause confusion and interoperability problems. 2) In Section D.4.2 (Semantics of Picture Timing SEI Messages), the constraint on pt_cpb_alt_timing_info_present_flag equal to 0 is specified in a picture-specific manner. However, since HRD operation is based on OLS, the semantics is technically inaccurate and may cause interoperability issues. 3) The semantics of delta_poc_msb_cycle_present_flag[i][j] and the derivation of prevTid0Pic in the decoding process of the Picture Order Count (POC) do not take into account the value of ph_non_ref_pic_flag. This can cause problems when discarding pictures with ph_non_ref_pic_flag equal to 1, as the derived POC values ​​for pictures and / or signaled reference pictures may be incorrect, resulting in unexpected and incorrect decoding behavior, including decoder crashes. 4) In Section C.2.3 (Timing of DU Removal and DU Decoding), the variable prevNonDiscardablePic is specified in a picture-specific manner, and the value of ph_non_ref_pic_flag is not taken into account. As a result, the same problem as above may occur. 5) The same issues as in issue 4 also apply to notDiscardablePic and prevNonDiscardablePic in section D.3.2 (Semantics of Buffering Period SEI Messages) and prevNonDiscardablePic in section D.4.2 (Semantics of Picture Timing SEI Messages). 6) The semantics of bp_concatenation_flag and bp_cpb_removal_delay_delta_minus1 are specified in a picture-specific manner. However, since HRD operation is based on OLS, the semantics may be technically imprecise and may cause interoperability issues. 7) Constraints on the value of bp_alt_cpb_params_present_flag have two problems: first, the semantics of bp_alt_cpb_params_present_flag should be specified in a picture-specific manner and be AU-specific; and second, it should only consider IRAP pictures and also GDR pictures. 8) The variables BpResetFlag, CpbRemovalDelayMsb[i], CpbRemovalDelayVal[i], and the syntax elements pt_dpb_output_delay, pt_dpb_output_du_delay, and pt_display_elemental_periods_minus1 in Section D.4.2 (Semantics of Picture Timing SEI Messages) are specified in a picture-specific manner. However, because HRD operation is based on OLS, the semantics are technically inaccurate and may cause interoperability issues. 9) In Section C.4 (Bitstream Conformance), the derivation of maxPicOrderCnt and minPicOrderCnt does not consider the value of ph_non_ref_pic_flag. This causes problems when discarding pictures with ph_non_ref_pic_flag equal to 1, as the derived POC values ​​for pictures and / or signaled reference pictures may be incorrect, resulting in unexpected and incorrect decoding behavior, including decoder crashes. 10) In the semantics of pt_display_elemental_periods_minus1, the syntax element fixed_pic_rate_within_cvs_flag[TemporalId] is used. However, the semantics should be described in the context of the maximum TemporalId value of interest, as is the case in other semantics of BP, PT, and DUI SEI messages, and therefore fixed_pic_rate_within_cvs_flag[Htid] should be used instead. 11) The semantics of dui_dpb_output_du_delay are specified in a picture-specific manner. However, since the HRD operation is based on OLS, the semantics may be technically imprecise and may cause interoperability issues.

[0045] 5. List of technical solutions and embodiments In order to solve the above problems, the following methods have been disclosed. These items are examples for explaining general concepts and should not be interpreted in a narrow sense. Furthermore, these items may be applied individually or may be combined in any manner. 1) To solve problem 1, in Clause 3 (Definitions), the sentence in the note "RASL pictures are used as reference pictures for the decoding process of non-RASL pictures" is changed to "RASL pictures are not used as reference pictures for the decoding process of non-RASL pictures. However, if a RASL picture has a RADL sub-picture, that RADL picture may be used when performing RADL inter-prediction that is co-located in a RADL picture associated with the same CRA picture as the RASL picture." as part of the RASL picture definition. 2) To solve issue 2, in Section D.4.2 (Semantics of Picture Timing SEI Message), change the description of the constraint on pt_cpb_alt_timing_info_present_flag equal to 0 from picture-specific to AU-specific, and add that RASL pictures now contain only RASL NUTs. In one example, this constraint is specified as follows: If all pictures in the associated AU are RASL pictures with pps_mixed_nalu_types_in_pic_flag equal to 0, then the value of pt_cpb_alt_timing_info_present_flag is equal to 0. b. In another example, the constraint is specified as follows: If all pictures in the associated AU are RASL pictures with pps_mixed_nalu_types_in_pic_flag equal to 0, then the value of pt_cpb_alt_timing_info_present_flag is equal to 0. c. In yet another example, this constraint is specified as follows: If all pictures in the associated AU are RASL pictures that contain all of the VCL NAL units with nal_unit_type equal to RASL_NUT, then the value of pt_cpb_alt_timing_info_present_flag is equal to 0. 3) To solve problem 3, add ph_non_ref_pic_flag to the derivation of prevTid0Pic in the decoding process for the semantics of delta_poc_msb_cycle_present_flag[i][j] and picture order count. 4) To resolve issue 4, in Section C.2.3 (DU Removal and DU Decoding Timing), change the specification of prevNonDiscardablePic from picture-specific to AU-specific, including renaming it to prevNonDiscardableAu, and add ph_non_ref_pic_flag to the derivation of the same variable. 5) To resolve issue 5, for notDiscardablePic, prevNonDiscardablePic in Section D.3.2 (Semantics of Buffering Period SEI Messages), and prevNonDiscardablePic in Section D.4.2 (Semantics of Picture Timing SEI Messages), change the specification of these variables from picture-specific to AU-specific, including renaming them to notDiscardableAu and prevNonDiscardableAu, respectively, and add ph_non_ref_pic_flag to the derivation of these two variables. 6) To solve issue 6, change the semantic description of bp_concatenation_flag, bp_cpb_removal_delay_delta_minus1 from picture-specific to AU-specific. 7) To solve problem 7, we specify constraints on the value of bp_alt_cpb_params_present_flag in an AU-specific manner, and specify that the value of bp_alt_cpb_params_present_flag further depends on whether the connected AU is a GDR AU or not. 8) To resolve issue 8, change the specifications of the variables BpResetFlag, CpbRemovalDelayMsb[i], CpbRemovalDelayVal[i], and the syntax elements pt_dpb_output_delay, pt_dpb_output_du_delay, and pt_display_elemental_periods_minus1 in Section D.4.2 (Semantics of Picture Timing SEI Messages) from picture-specific to AU-specific. 9) To resolve issue 9, in Section C.4 (Bitstream Conformance), in the derivation of maxPicOrderCnt and minPicOrderCnt, add ph_non_ref_pic_flag to "the previous picture in decoding order whose TemporalId is equal to 0 and which is not a RASL or RADL picture." 10) To solve problem 10, the semantics of pt_display_elemental_periods_minus1 is specified using fixed_pic_rate_within_cvs_flag[TemporalId] instead of fixed_pic_rate_within_cvs_flag[Htid]. 11) To solve problem 11, the semantics of dui_dpb_output_du_delay are specified in an AU-specific manner.

[0046] 6. Implementation [ka]

[0047] 6.1. First embodiment This embodiment corresponds to items 1 to 9.

[0048] 3 Definition ... [ka] ...

[0049] 7.4.9 Reference Picture List Semantics ... [ka] -PicOrderCntVal of prevTid0Pic, - the PicOrderCntVal of each picture referenced by an entry in RefPicList[0] or RefPicList[1] of prevTid0Pic and having the same nuh_layer_id as the current picture, The PicOrderCntVal of each picture following in decoding order of -prevTid0Pic has the same nuh_layer_id as the current picture and precedes the current picture in decoding order. If setOfPrevPocVals contains multiple values ​​whose value modulo MaxPicOrderCntLsb is equal to PocLsbLt[i][j], the value of delta_poc_msb_cycle_present_flag[i][j] is equal to 1. ...

[0050] 8.3.1 Decoding process for picture order counting ... [ka] - The variable prevPicOrderCntLsb is set equal to the ph_pic_order_cnt_lsb of prevTid0Pic. - The variable prevPicOrderCntMsb is set equal to the PicOrderCntMsb of prevTid0Pic. ...

[0051] C.2.3 Timing of DU Removal and Decoding ... The nominal removal time of AU n from the CPB is defined as follows: If AU n is an AU with n equal to 0 (the AU that initializes the HRD), the nominal removal time from the CPB of the AU is defined as follows: AuNominalRemovalTime[0]=InitCpbRemovalDelay[Htid][ScIdx]÷90000 (C.9) - Otherwise, the following applies: If AU n is the first AU of a BP that does not initialize the HRD, the following applies: The nominal removal time of AU n from the CPB is specified below. [ka] tmpCpbRemovalDelay1=(auCpbRemovalDelayDeltaMinus1+1) baseTime2=AuNominalRemovalTime[n-1] tmpCpbRemovalDelay2= (C.10) Ceil((InitCpbRemovalDelay[Htid][ScIdx]÷90000+ AuFinalArrivalTime[n-1]-AuNominalRemovalTime[n-1])÷ClockTick) if(baseTime1+ClockTick * tmpCpbRemovalDelay1< baseTime2+ClockTick * tmpCpbRemovalDelay2){ baseTime=baseTime2 tmpCpbRemovalDelay=tmpCpbRemovalDelay2 }else{ baseTime=baseTime1 tmpCpbRemovalDelay=tmpCpbRemovalDelay1 } } AuNominalRemovalTime[n]= baseTime+(ClockTick * tmpCpbRemovalDelay-CpbDelayOffset) [ka] After deriving the nominal CPB removal time and before deriving the DPB output time for access unit n, the variables DpbDelayOffset and CpbDelayOffset are derived as follows: If one or more of the following conditions are true, DpbDelayOffset is set equal to the value of the PT SEI message syntax element dpb_delay_offset[Htid] of AU n+1, CpbDelayOffset is set equal to the value of the PT SEI message syntax element cpb_delay_offset[Htid] of AU n+1, and the PT SEI message containing the syntax elements is selected as specified in clause C.1: -AU n's UseAltCpbParamsFlag is equal to 1. -DefaultInitCpbParamsFlag equals 0. Otherwise, DpbDelayOffset and CpbDelayOffset are both set equal to 0. If AU n is not the first AU of a BP, the nominal removal time of AU n from the CPB is defined as follows: [ka] ...

[0052] C.4 Bitstream Conformance ... Let currPicLayerId equal the nuh_layer_id of the current picture. For each current picture, set the variables maxPicOrderCnt and minPicOrderCnt equal to the maximum and minimum PicOrderCntVal values, respectively, of the following pictures whose nuh_layer_id is equal to currPicLayerId. - Current Picture [ka] -STRP is referenced by all entries in RefPicList[0] and all entries in RefPicList[1] of the current picture. All pictures n that have PictureOutputFlag equal to -1, and where currPic is the current picture, AuCpbRemovalTime[n] is less than AuCpbRemovalTime[currPic] and DpbOutputTime[n] is greater than or equal to AuCpbRemovalTime[currPic]. ...

[0053] D.3.2 Buffering Period SEI Message Semantics ... [ka] The presence of a BP SEI message is defined as follows: If NalHrdBpPresentFlag is equal to 1 or VclHrdBpPresentFlag is equal to 1, then for each AU in the CVS the following applies: - If the AU is an IRAP or GDR AU, the BP SEI message applicable to the operating point shall be associated with the AU. [ka] Otherwise (NalHrdBpPresentFlag and VclHrdBpPresentFlag are both equal to 0), the AU in the CVS is not associated with the BP SEI message. [ka] ...

[0054] D.4.2 Picture Timing SEI Message Semantics ... [ka] If the current AU is the AU that initializes the HRD, then CpbRemovalDelayMsb[i] and CpbRemovalDelayVal[i] are both set equal to 0, and the value of cpbRemovalDelayValTmp[i] is set equal to pt_cpb_removal_delay_minus1[i]+1. [ka] [ka] ...

[0055] D.5.2 DU Information SEI Message Semantics ... [ka] ...

[0056] FIG. 1 is a block diagram illustrating an example video processing system 1000 in which various techniques disclosed herein may be implemented. Various implementations may include some or all of the modules of system 1000. System 1000 may include an input 1002 for receiving video content. The video content may be received in a raw or uncompressed format, e.g., 8- or 10-bit multi-module pixel values, or in a compressed or encoded format. Input 1002 may represent a network interface, a peripheral bus interface, or a storage interface. Examples of network interfaces include wired interfaces such as Ethernet, passive optical networks (PONs), and wireless interfaces such as Wi-Fi or cellular interfaces.

[0057] System 1000 may include a coding module 1004 capable of implementing various coding or encoding methods described herein. Coding module 1004 may reduce the average bit rate of video from input 1002 to its output, generating a coded representation of the video. Accordingly, this coding technique may be referred to as a video compression or video transcoding technique. The output of coding module 1004 may be stored or transmitted via a connected communication, as represented by component 1006. The bitstream (or coded) representation of the video received, stored, or communicated at input 1002 may be used by component 1008 to generate pixel values ​​or displayable video that are transmitted to display interface 1010. The process of generating user-viewable video from the bitstream representation may be referred to as video decompression (video unpacking). Furthermore, while certain video processing operations are referred to as "coding" operations or tools, it will be understood that the coding tools or operations are performed by an encoder and corresponding decoding tools or operations that reverse the results of the coding are performed by a decoder.

[0058] Examples of peripheral bus interface units or display interface units may include Universal Serial Bus (USB) or High-Definition Multimedia Interface (HDMI), or DisplayPort, etc. Examples of storage interfaces include Serial Advanced Technology Attachment (SATA), PCI, IDE interfaces, etc. The techniques described herein may be implemented in various electronic devices such as mobile phones, laptops, smartphones, or other devices capable of digital data processing and / or video display.

[0059] FIG. 2 is a block diagram of a video processing device 2000. The device 2000 may be used to implement one or more of the methods described herein. The device 2000 may be implemented in a smartphone, a tablet, a computer, an Internet of Things (IoT) receiver, etc. The device 2000 may include one or more processors 2002, one or more memories 2004, and video processing hardware 2006. The one or more processors 2002 may be configured to implement one or more methods described herein (e.g., those described in FIGS. 6 through 9). The memory(s) 2004 may be used to store data and code used to implement the methods and techniques described herein. The video processing hardware 2006 may be used to implement the techniques described herein in hardware circuitry. In some embodiments, the hardware 2006 may be partially or entirely included within the one or more processors 2002, e.g., a graphics processor.

[0060] FIG. 3 is a block diagram illustrating an example video coding system 100 that can utilize techniques of this disclosure. As shown in FIG. 3, video coding system 100 may include a source device 110 and a destination device 120. Source device 110 generates encoded video data and may also be referred to as a video encoding device. Destination device 120 decodes the encoded video data generated by source device 110 and may also be referred to as a video decoding device. Source device 110 may include a video source 112, a video encoder 114, and an input / output (I / O) interface 116.

[0061] The video source 112 may include a source such as a video capture device, an interface for receiving video data from a video content provider, and / or a computer graphics system for generating video data, or a combination of these sources. The video data may include one or more pictures. The video encoder 114 encodes the video data from the video source 112 and generates a bitstream. The bitstream may include a sequence of bits forming a coded representation of the video data. The bitstream may include coded pictures and associated data. A coded picture is a coded representation of a picture. The associated data may include sequence parameter sets, picture parameter sets, and other syntax structures. The I / O interface 116 may include a modulator / demodulator (modem) and / or transmitter. The coded video data may be transmitted directly to the destination device 120 via the I / O interface 116 over the network 130a. The coded video data may be stored on a recording medium / server 130b for access by the destination device 120.

[0062] Destination device 120 may include an I / O interface 126 , a video decoder 124 , and a display device 122 .

[0063] I / O interface 126 may include a receiver and / or a modem. I / O interface 126 may obtain encoded video data from source device 110 or storage medium / server 130b. Video decoder 124 may decode the encoded video data. Display device 122 may display the decoded video data to a user. Display device 122 may be integrated with destination device 120 or may be external to destination device 120 configured to interface with an external display device.

[0064] Video encoder 114 and video decoder 124 may operate according to video compression standards such as the High Efficiency Video Coding (HEVC) standard, the Versatile Video Coding (VVVM) standard, and other current and / or future standards.

[0065] FIG. 4 is a block diagram illustrating an example of a video encoder 200, which may be video encoder 114 in system 100 shown in FIG.

[0066] Video encoder 200 may be configured to perform any or all of the techniques of this disclosure. In the example of FIG. 4, video encoder 200 includes multiple functional modules. The techniques described in this disclosure may be shared among various components of video encoder 200. In some examples, a processor may be configured to perform any or all of the techniques described in this disclosure.

[0067] The functional components of the video encoder 200 may include a partitioning unit 201, a prediction unit 202, a residual generation unit 207, a transform unit 208, a quantization unit 209, an inverse quantization unit 210, an inverse transform unit 211, a reconstruction unit 212, a buffer 213, and an entropy coding unit 214, and the prediction unit 202 includes a mode selection unit 203, a motion estimation unit 204, a motion compensation unit 205, and an intra prediction unit 206.

[0068] In other examples, video encoder 200 may include more, fewer, or different functional components. In one example, prediction unit 202 may include an intra block copy (IBC) unit. The IBC unit may perform prediction in an IBC mode in which at least one reference picture is the picture in which the current video block is located.

[0069] Furthermore, some components, such as the motion estimation unit 204 and the motion compensation unit 205, may be highly integrated, but are represented separately in the example of FIG. 4 for purposes of explanation.

[0070] Division unit 201 may divide a picture into one or more video blocks. Video encoder 200 and video decoder 300 may support a variety of video block sizes.

[0071] The mode select unit 203 may, for example, select one of intra or inter coding modes based on the error result, and provide the resulting intra- or inter-coded block to the residual generation unit 207 to generate and provide residual block data to the reconstruction unit 212 to reconstruct the coded block for use as a reference picture. In some examples, the mode select unit 203 may select a Combination of Intra and Inter Prediction (CIIP) mode that performs prediction based on an inter prediction signal and an intra prediction signal. In the case of inter prediction, the mode select unit 203 may select a resolution of motion vectors (e.g., sub-pixel or integer pixel precision) for the block.

[0072] To perform inter prediction on the current video block, motion estimation unit 204 may generate motion information for the current video block by comparing the current video block with one or more reference frames from buffer 213. Motion compensation unit 205 may determine a prediction video block for the current video block based on the motion information and decoded samples of pictures from buffer 213 other than the picture associated with the current video block.

[0073] Motion estimation unit 204 and motion compensation unit 205 may, for example, perform different operations on the current video block based on whether the current video block is an I-slice, a P-slice, or a B-slice.

[0074] In some examples, motion estimation unit 204 may perform unidirectional prediction on the current video block, and motion estimation unit 204 may search reference pictures in list 0 or list 1 for a reference video block for the current video block. Motion estimation unit 204 may then generate a reference index indicating the reference picture in list 0 or list 1, including the reference video block and a motion vector indicating a spatial displacement between the current video block and the reference video block. Motion estimation unit 204 may output the reference index, prediction direction indicator, and motion vector as motion information for the current video block. Motion compensation unit 205 may generate a prediction video block for the current block based on the reference video block indicated by the motion information of the current video block.

[0075] In another example, motion estimation unit 204 may perform bidirectional prediction on the current video block, and motion estimation unit 204 may search for reference pictures from list 0 for a reference video block for the current video block, and may also search for reference pictures in list 1 for another reference video block for the current video block. Motion estimation unit 204 may then generate reference indexes that indicate the reference pictures in lists 0 and 1 that contain the reference video blocks, and motion vectors that indicate spatial displacements between the reference video blocks and the current video block. Motion estimation unit 204 may output the reference index and motion vector for the current video block as motion information for the current video block. Motion compensation unit 205 may generate a prediction video block for the current video block based on the reference video block indicated by the motion information of the current video block.

[0076] In some examples, the motion estimation unit 204 may output a full set of motion information for the decoding process of the decoder.

[0077] In some examples, motion estimation unit 204 may not output a full set of motion information for the current picture. Rather, motion estimation unit 204 may signal motion information for the current video block by reference to motion information of another video block. For example, motion estimation unit 204 may determine that the motion information of the current video block is sufficiently similar to the motion information of a neighboring video block.

[0078] In one example, motion estimation unit 204 may indicate, in a syntax structure associated with the current video block, a value that indicates to video decoder 300 that the current video block has the same motion information as another video block.

[0079] In another example, motion estimation unit 204 may identify another video block and a motion vector difference (MVD) in a syntax structure associated with the current video block. The motion vector difference indicates the difference between the motion vector of the current video block and the motion vector of the indicated video block. Video decoder 300 may use the motion vector of the indicated video block and the motion vector difference to determine the motion vector of the current video block.

[0080] As mentioned above, video encoder 200 may predictively signal motion vectors. Two examples of predictive signaling techniques that may be implemented by video encoder 200 include Advanced Motion Vector Prediction (AMVP) and merge mode signaling.

[0081] Intra prediction unit 206 may perform intra prediction on the current video block. If intra prediction unit 206 performs intra prediction on the current video block, intra prediction unit 206 may generate predictive data for the current video block based on decoded samples of other video blocks in the same picture. The predictive data for the current video block may include a predicted video block and various syntax elements.

[0082] Residual generation unit 207 may generate residual data for the current video block by subtracting (e.g., as indicated by a minus sign) the predicted video block of the current video block from the current video block. The residual data for the current video block may include residual video blocks that correspond to different sample components of the samples in the current video block.

[0083] In other examples, for example, in skip mode, there may be no residual data for the current video block, and residual generation unit 207 may not perform the subtraction operation.

[0084] Transform processing unit 208 may generate one or more transform coefficient video blocks for the current video block by applying one or more transforms to the residual video block associated with the current video block.

[0085] After transform processing unit 208 generates a transform coefficient video block associated with the current video block, quantization unit 209 may quantize the transform coefficient video block associated with the current video block based on one or more quantization parameter (QP) values ​​associated with the current video block.

[0086] Inverse quantization unit 210 and inverse transform unit 211 may apply inverse quantization and inverse transform, respectively, to the transform coefficient video block to reconstruct a residual video block from the transform coefficient video block. Reconstruction unit 212 may add the reconstructed residual video block to corresponding samples from one or more prediction video blocks generated by prediction unit 202 to generate a reconstructed video block associated with the current block for storing in buffer 213.

[0087] After reconstruction unit 212 reconstructs the video block, a loop filtering operation may be performed to reduce video blocking artifacts in the video block.

[0088] Entropy encoding unit 214 may receive data from other functional components of video encoder 200. If entropy encoding unit 214 receives data, entropy encoding unit 214 may perform one or more entropy encoding operations to generate entropy-coded data and output a bitstream that includes the entropy-coded data.

[0089] FIG. 5 is a block diagram illustrating an example of a video decoder 300, which may be video decoder 114 in system 100 shown in FIG.

[0090] Video decoder 300 may be configured to perform any or all of the techniques of this disclosure. In the example of FIG. 5, video decoder 300 includes multiple functional components. Techniques described in this disclosure may be shared among various components of video decoder 300. In some examples, a processor may be configured to perform any or all of the techniques described in this disclosure.

[0091] 5, video decoder 300 includes entropy decoding unit 301, motion compensation unit 302, intra prediction unit 303, inverse quantization unit 304, inverse transform unit 305, and reconstruction unit 306, as well as a buffer 307. Video decoder 300 may, in some examples, perform a decoding path that is approximately the reverse of the encoding path described with respect to video encoder 200 (FIG. 4).

[0092] The entropy decoding unit 301 retrieves an encoded bitstream. The encoded bitstream may include entropy-coded video data (e.g., encoded blocks of video data). The entropy decoding unit 301 decodes the entropy-coded video data, and from the entropy-decoded video data, the motion compensation unit 302 may determine motion information including motion vectors, motion vector precision, reference picture list indexes, and other motion information. The motion compensation unit 302 may determine such information by performing, for example, AMVP and merge mode.

[0093] The motion compensation unit 302 may generate motion-compensated blocks, possibly performing interpolation based on an interpolation filter, and an identifier for the interpolation filter used with sub-pixel precision may be included in the syntax element.

[0094] Motion compensation unit 302 may calculate interpolated values ​​for sub-integer pixels of the reference block using interpolation filters such as those used by video encoder 200 during encoding of the video block. Motion compensation unit 302 may determine the interpolation filters used by video encoder 200 based on received syntax information and use the interpolation filters to generate the prediction block.

[0095] The motion compensation unit 302 may use syntax information to determine the size of the blocks used to code the frames and / or slices of the coded video sequence, partitioning information describing how each macroblock of a picture of the coded video sequence is divided, a mode indicating how each division is coded, one or more reference frames (and reference frame lists) for each inter-coded block, and some other information to decode the coded video sequence.

[0096] The intra prediction unit 303 may form a prediction block from spatially adjacent blocks, for example, using an intra prediction mode received in the bitstream. The inverse quantization unit 303 inverse quantizes (i.e., dequantizes) the quantized video block coefficients provided in the bitstream and decoded by the entropy decoding unit 301. The inverse transform unit 303 applies an inverse transform.

[0097] Reconstruction unit 306 may sum the residual block with the corresponding prediction block generated by motion compensation unit 202 or intra prediction unit 303 to form a decoded block. If desired, a deblocking filter may be applied to filter the decoded block to remove block artifacts. The decoded video block is stored in buffer 307, which provides reference blocks for subsequent motion compensation / intra prediction and also generates decoded video for display on a display device.

[0098] 6 to 10 illustrate an exemplary way in which the above-mentioned technical solution can be implemented, for example, this embodiment is shown in FIGS.

[0099] 6 shows a flowchart illustrating an example method 600 of video processing. The method 600 includes, at operation 610, converting video including one or more pictures to a bitstream of the video, the bitstream conforming to a format rule that specifies that picture timing (PT) supplemental enhancement information (SEI) messages, if included in the bitstream, are access unit (AU) specific, and each picture of the one or more pictures that are random access skipped reading (RASL) pictures includes only RASL network abstraction layer unit types (NUTs).

[0100] 7 shows a flowchart illustrating an example of a method 700 for video processing. The method 700 includes, at operation 710, converting video including one or more pictures to a bitstream of the video, where the bitstream conforms to format rules that allow a random access decodable (RADL) sub-picture in a random access skip-leading (RASL) picture to be used as a reference sub-picture for predicting a co-located clean random access (CRA) picture in an RADL picture that is associated with the same CRA picture as the RASL picture.

[0101] 8 shows a flowchart illustrating an example of a video processing method 800. The method 800 includes, at operation 810, converting a video including one or more pictures to a bitstream of the video, the bitstream conforming to a format rule, the format rule being associated with a first flag and specifying that derivation of a picture in a decoding process for a picture order count is based on a second flag, the picture associated with the first flag being a previous picture in decoding order having (i) a first identifier equal to a slice or picture header referencing a reference picture list syntax structure, (ii) a second identifier and second flag equal to 0, and (iii) a picture type different from a random access skipped leading (RASL) picture and a random access decodable leading (RADL) picture, the first flag indicating whether a second flag is present in the bitstream, the second flag indicating whether the current picture is used as a reference picture, and the third flag being used to determine values ​​of one or more most significant bits of a picture order count of a long-term reference picture.

[0102] 9 shows a flowchart illustrating an example of a method 900 for video processing. The method 900 includes, at operation 910, converting video including one or more pictures to a bitstream for the video, where the bitstream conforms to format rules that specify that variables used to determine the timing of removing or decoding a decoding unit (DU) are access unit (AU) specific and are derived based on a flag indicating whether the current picture is allowed to be used as a reference picture.

[0103] 10 shows a flowchart illustrating an example of a method 1000 of video processing. The method 1000 includes, at operation 1010, converting a video including one or more pictures to a bitstream of the video, the bitstream conforming to format rules specifying that a buffering period supplemental enhancement information (SEI) message and a picture timing SEI message, if included in the bitstream, are access unit (AU) specific, and a first variable associated with the buffering period SEI message and a second variable associated with the buffering period SEI message and the picture timing SEI message are allowed to use the current picture as a reference picture. The first variable indicates an access unit that includes (i) an identifier equal to 0 and (ii) a picture that is not a random access skipped leading (RASL) picture or a random access decodable leading (RADL) picture and has this flag equal to 0, and the second variable indicates that the current AU is not the first AU in decoding order and that the previous AU in decoding order includes (i) an identifier equal to 0 and (ii) a picture that is not a random access skipped leading (RASL) picture or a random access decodable leading (RADL) picture and has its flag equal to 0.

[0104] 11 shows a flowchart illustrating an example of a method 1100 for video processing. The method 1100 includes, at act 1110, converting a video including one or more pictures to a bitstream of the video, the bitstream conforming to a format rule, the format rule specifying that derivation of a first variable and a second variable associated with a first picture and a second picture is based on a flag, the first picture being a current picture, and the second picture (i) including a first identifier equal to 0, (ii) including a flag equal to 0, and (iii) being a random access skipped reading (RASL) picture or a random access decodable reading (RADL) picture. the first variable and the second variable are the maximum and minimum values, respectively, of the picture order counts of (i) the first picture, (ii) the second picture, (iii) one or more short-term reference pictures referenced by all entries in the reference picture list of the first picture, and (iv) each picture output with a coded picture buffer (CPB) removal time shorter than the CPB removal time of the first picture and a decoded picture buffer (DPB) output time greater than or equal to the CPB removal time of the first picture, which have a second identifier equal to that of the first picture.

[0105] 12 shows a flowchart illustrating an example of a method 1200 for video processing. The method 1200 includes, at act 1210, converting between video including one or more pictures and a bitstream of the video, the bitstream conforming to a format rule, the format rule specifying a flag and a syntax element that, if included in the bitstream, are access unit (AU) specific, the flag indicating whether a nominal Coded Picture Buffer (CPB) removal time of the current AU is determined relative to (a) a nominal CPB removal time of a previous AU associated with a buffering period supplemental enhancement information (SEI) message or (b) a nominal CPB removal time of the current AU, responsive to the current AU not being the first AU of the bitstream in decoding order, and the syntax element specifying a CPB removal delay increment value relative to the nominal CPB removal time of the current AU, responsive to the current AU not being the first AU of the bitstream in decoding order.

[0106] 13 shows a flowchart illustrating an example of a method 1300 for video processing. The method 1300 includes, at operation 1310, converting between video including one or more pictures and a bitstream of the video, the bitstream conforming to a format rule, the format rule specifying that a plurality of variables and a picture timing supplemental enhancement information (SEI) message and a picture timing SEI message, if included in the bitstream, are access unit (AU) specific, the picture timing SEI message including a plurality of syntax elements, a first variable of the plurality of variables indicating whether a current AU is associated with a buffering period SEI message, and second and third variables of the plurality of variables indicating whether the current AU is an AU for initializing a hypothetical reference decoder (HRD). a first syntax element of the plurality of syntax elements specifies a number of clock ticks to wait after the AU is removed from the coded picture buffer (CPB) until one or more decoded pictures of the AU are output from the decoded picture buffer (DPB); a second syntax element of the plurality of syntax elements specifies a number of sub-clock ticks to wait after the last decoding unit (DU) of the AU is removed from the CPB until one or more decoded pictures of the AU are output from the DPB; and a third syntax element of the plurality of syntax elements specifies a number of picture period intervals of the element that the one or more decoded pictures of the current AU occupy with respect to the display model.

[0107] 14 shows a flowchart illustrating an example of a method 1400 for video processing. The method 1400 includes, at operation 1410, converting video including one or more pictures to a bitstream of the video, the bitstream conforming to a format rule, the format rule specifying that a syntax element associated with a decoded picture buffer (DPB), when included in the bitstream, is access unit (AU) specific, the syntax element specifying a number of sub-clock ticks to wait after removing a last decoding unit (DU) of the AU from a coded picture buffer (CPB) before outputting one or more decoded pictures of the AU from the DPB.

[0108] 15 shows a flowchart illustrating an example of a method 1500 for video processing. The method 1500 includes, at operation 1510, converting video including one or more pictures to a bitstream for the video, where the bitstream conforms to a format rule that specifies that a flag, if included in the bitstream, is access unit (AU) specific, the value of the flag being based on whether the associated AU is an intra random access point (IRAP) AU or a gradual decoding update (GDR) AU, the value of the flag specifying (i) whether a syntax element is present in a buffering period supplemental enhancement information (SEI) message and (ii) whether alternative timing information is present in a picture timing SEI message for the current buffering period.

[0109] 16 shows a flowchart illustrating an example of a method 1600 for video processing. The method 1600 includes, at operation 1610, converting a video including one or more pictures to a bitstream of the video, the bitstream conforming to a format rule specifying that a value of a first syntax element is based on a flag indicating whether a temporal distance between output times of consecutive pictures of a hypothetical reference decoder (HRD) is constrained and a variable identifying the highest temporal sub-layer to be decoded, the first syntax element specifying a number of element picture period intervals that one or more decoded pictures of a current AU occupy relative to a display model.

[0110] Next, preferred solutions are listed in some embodiments.

[0111] A1. The video processing method includes converting video including one or more pictures to a bitstream of the video, the bitstream conforming to a format rule, the format rule specifying that a picture timing (PT) supplemental enhancement information (SEI) message, if included in the bitstream, is access unit (AU) specific, and each picture of the one or more pictures that is a random access skipped reading (RASL) picture includes only a RASL network abstraction layer unit type (NUT).

[0112] A2. In the method described in Solution A1, in response to a RASL picture having a first flag equal to 0 and a second flag equal to 0 in the associated AU, the first flag indicates whether each picture referencing a picture parameter set (PPS) is two or more video coding layer (VCL) network abstraction layer (NAL) units and whether at least two of the two or more VCL NAL units are of different types, and the second flag indicates whether one or more syntax elements related to temporal information are allowed to be present in the picture timing SEI message.

[0113] A3. The method according to solution A2, wherein the first flag is pps_mixed_nalu_types_in_pic_flag and the second flag is pt_cpb_alt_timing_info_present_flag.

[0114] A4. The one or more syntax elements include: a first syntax element indicating an alternative initial CPB removal delay delta for the ith sublayer for the jth coded picture buffer (CPB) for the NAL hypothetical reference decoder (HRD) in units of a 90 kHz clock; a second syntax element indicating an alternative initial CPB removal offset delta for the ith sublayer for the jth CPB for the NAL HRD in units of a 90 kHz clock; a third syntax element indicating an offset to use in deriving the nominal CPB removal time of the AU associated with the PT SEI message and one or more subsequent AUs in decoding order, if the AU associated with the PT SEI message directly follows in decoding order the AU associated with the BP SEI message; and a fourth syntax element indicating an offset to use in deriving the decoded picture buffer (DPB) output time of the Intra Random Access Point (IRAP) AU associated with the BP SEI message, if the AU associated with the PT SEI message directly follows in decoding order the IRAP AU associated with the BP SEI message; and a VCL a fifth syntax element indicating an alternative initial CPB removal delay delta for the i-th sublayer for the j-th CPB for the HRD in units of 90 kHz clock; a sixth syntax element indicating an alternative initial CPB removal offset delta for the i-th sublayer for the j-th CPB for the VCL HRD in units of 90 kHz clock; and a seventh syntax element indicating, for the i-th sublayer for the VCL HRD, an offset to use in deriving the nominal CPB removal time of the AU associated with the PT SEI message and one or more subsequent AUs in decoding order, if the AU associated with the PT SEI message directly follows in decoding order the AU associated with the BP SEI message; and an eighth syntax element indicating an offset used to derive the DPB output time of the IRAP AU associated with the BP SEI message if the AU associated with the PT SEI message directly follows the IRAP AU associated with the BP SEI message in decoding order, for the i-th sublayer for the VCL HRD.

[0115] A5. The method according to Solution A1, wherein a flag is equal to 0 in response to each picture in the associated AU being a RASL picture containing a Video Coding Layer (VCL) Network Abstraction Layer (NAL) unit, where each VCL NAL unit is a RASL NUT, and the flag indicates whether one or more syntax elements related to timing information are present in the picture timing SEI message.

[0116] A6. The method according to solution A5, wherein the flag is pt_cpb_alt_timing_info_present_flag.

[0117] A7. A video processing method comprising converting video including one or more pictures to a bitstream of the video, the bitstream conforming to format rules that permit the use of a random access skipped leading (RADL) sub-picture in a RASL picture as a reference sub-picture for predicting a co-located RADL picture in a random access decodable leading (RADL) picture associated with the same clean random access (CRA) picture as the RASL picture.

[0118] Next, we list alternative solutions that are preferred in some embodiments.

[0119] B1. A video processing method, comprising: converting a video including one or more pictures to a bitstream of the video, the bitstream conforming to a format rule, the format rule being associated with a first flag and specifying that derivation of a picture in a decoding process for a picture order count is based on a second flag; the picture associated with the first flag is a previous picture in decoding order having (i) a first identifier that is the same as a slice or picture header that references a reference picture list syntax structure, (ii) a second identifier and second flag that are equal to 0, and (iii) a picture type that is different from a random access skipped leading (RASL) picture and a random access decodable leading (RADL) picture; the first flag indicating whether a third flag is present in the bitstream, the second flag indicating whether the current picture is used as a reference picture, and the third flag being used to determine values ​​of one or more most significant bits of a picture order count value of a long-term reference picture.

[0120] B2. The method according to solution B1, wherein the first identifier is a layer identifier and the second identifier is a temporal identifier.

[0121] B3. The method according to solution B1, wherein the first identifier is a syntax element and the second identifier is a variable.

[0122] B4. The method according to any of Solutions B1 to B3, wherein the first flag is delta_poc_msb_cycle_present_flag, the second flag is ph_non_ref_pic_flag, the third flag is delta_poc_msb_cycle_present_flag, the first identifier is nuh_layer_id, and the second identifier is TemporalId.

[0123] B5. A video processing method, comprising: converting a video including one or more pictures to a bitstream of the video, the bitstream conforming to a format rule, the format rule specifying that a variable used to determine the timing of removing a decoding unit (DU) or decoding a DU is access unit (AU) specific and derived based on a flag indicating whether the current picture is allowed to be used as a reference picture.

[0124] B6. The method described in solution B5, where the variable is prevNonDiscardableAu and the flag is ph_non_ref_pic_flag.

[0125] B7. The method according to solution B6, wherein ph_non_ref_pic_flag equal to 1 specifies that the current picture is never used as a reference picture.

[0126] B8. The method according to solution B6, wherein ph_non_ref_pic_flag equal to 0 specifies that the current picture may or may not be used as a reference picture.

[0127] B9. A method for converting a video including one or more pictures to a bitstream of the video, the bitstream conforming to a format rule, the format rule specifying that a buffering period supplemental enhancement information (SEI) message and a picture timing SEI message, if included in the bitstream, are access unit (AU) specific, and a first variable associated with the buffering period SEI message and a second variable associated with the buffering period SEI message and the picture timing SEI message specify whether the current picture is allowed to be used as a reference picture. a first variable indicating an access unit that includes an access unit having (i) an identifier equal to 0 and (ii) a picture that is not a random access skipped leading (RASL) picture or a random access decodable leading (RADL) picture and whose flag is equal to 0; and a second variable indicating a previous AU in decoding order that is not the first AU in decoding order and that includes (i) an identifier equal to 0 and (ii) a picture that is not a random access skipped leading (RASL) picture or a random access decodable leading (RADL) picture and whose flag is equal to 0.

[0128] B10. The method of solution B9, wherein the identifier is a temporal identifier.

[0129] B11. The method according to Solution B9, wherein the first variable is notDiscardableAu, the second variable is prevNonDiscardableAu, the flag is ph_non_ref_pic_flag, and the identifier is TemporalId.

[0130] B12. A method for converting between a video including one or more pictures and a bitstream of the video, the bitstream conforming to a format rule, the format rule specifying a first variable associated with a first picture and a second picture, and the derivation being based on a flag, the first picture being a current picture, and the second picture being a previous picture in decoding order that (i) includes a first identifier equal to 0, (ii) includes a flag equal to 0, and (iii) is not a random access skipped leading (RASL) picture or a random access decodable leading (RADL) picture. wherein the first variable and the second variable are maximum and minimum values ​​of picture order counts, respectively, of (i) the first picture, (ii) the second picture, (iii) one or more short-term reference pictures referenced by all entries in the reference picture list of the first picture, which have a second identifier equal to that of the first picture, and (iv) each picture output with a Coded Picture Buffer (CPB) removal time shorter than the CPB removal time of the first picture and a Decoded Picture Buffer (DPB) output time greater than or equal to the CPB removal time of the first picture.

[0131] B13. The method of Solution B12, wherein the first variable indicates a maximum value of the picture order count and the second variable indicates a minimum value of the picture order count.

[0132] B14. The method according to solution B12, wherein the flag indicates whether the current picture is allowed to be used as a reference picture.

[0133] B15. The method of solution B12, wherein the first identifier is a temporal identifier and the second identifier is a layer identifier.

[0134] B16. The method according to any of Solutions B12 to B15, wherein the first variable is maxPicOrderCnt, the second variable is minPicOrderCnt, the first identifier is TemporalId, the second identifier is nuh_layer_id, and the flag is ph_non_ref_pic_flag.

[0135] Further solutions that may be suitable in some embodiments are listed below.

[0136] C1. A video processing method, comprising: converting a video including one or more pictures to a bitstream of the video, the bitstream conforming to a format rule; the format rule specifying a flag and a syntax element that are access unit (AU) specific if the bitstream includes a buffering period supplemental enhancement information (SEI) message; the flag indicating whether a nominal Coded Picture Buffer (CPB) removal time of the current AU is determined relative to (a) a nominal CPB removal time of a previous AU associated with the buffering period supplemental enhancement information (SEI) message, or (b) a nominal CPB removal time of the current AU, in response to the current AU not being the first AU of the bitstream in decoding order; and the syntax element specifying a CPB removal delay increment value relative to the nominal CPB removal time of the current AU, in response to the current AU not being the first AU of the bitstream in decoding order.

[0137] C2. The method according to solution C1, wherein the length of the syntax element is indicated in the syntax structure of the buffering period SEI message.

[0138] C3. The method according to solution C1, wherein the length of the syntax element is (bp_cpb_removal_delay_length_minus1+1) bits.

[0139] C4. The method according to any one of solutions C1 to C3, wherein the flag is bp_concatenation_flag and the syntax element is bp_cpb_removal_delay_delta_minus1.

[0140] C5. A method for converting a video including one or more pictures to a bitstream of the video, the bitstream conforming to a format rule, the format rule specifying that a plurality of variables and a picture timing supplemental enhancement information (SEI) message and a picture timing SEI message, when included in the bitstream, are access unit (AU) specific, the picture timing SEI message including a plurality of syntax elements, a first variable of the plurality of variables indicating whether a current AU is associated with a buffering period SEI message, and a second variable and a third variable of the plurality of variables associated with an indication of whether the current AU is an AU that initializes a hypothetical reference decoder (HRD). a first syntax element among the plurality of syntax elements specifies a number of clock ticks to wait after removing the AU from a coded picture buffer (CPB) until one or more decoded pictures of the AU are output from a decoded picture buffer (DPB); a second syntax element among the plurality of syntax elements specifies a number of sub-clock ticks to wait after removing a last decoding unit (DU) of the AU from the CPB until one or more decoded pictures of the AU are output from the DPB; and a third syntax element among the plurality of syntax elements specifies a number of picture period intervals of an element that the one or more decoded pictures of the current AU occupy with respect to a display model.

[0141] C6. The method according to solution C5, wherein the first variable is BpResetFlag, the second variable is CpbRemovalDelayMsb, and the third variable is CpbRemovalDelayVal.

[0142] C7. The method according to solution C5, wherein the first syntax element is pt_dpb_output_delay, the second syntax element is pt_dpb_output_du_delay, and the third syntax element is pt_display_elemental_periods_minus1.

[0143] C8. A video processing method, comprising: converting video including one or more pictures to a bitstream of the video, the bitstream conforming to a format rule, the format rule specifying that a syntax element associated with a decoding picture buffer (DPB), when included in the bitstream, is access unit (AU) specific, the syntax element specifying the number of sub-clock ticks to wait after removing the last decoding unit (DU) of the AU from a coded picture buffer (CPB) until one or more decoded pictures of the AU are output from the DPB.

[0144] C9. The method according to solution C8, in which the syntax element is used to calculate the DPB output time.

[0145] C10. The method according to solution C8, wherein the syntax element is dui_dpb_output_du_delay.

[0146] Further solutions that may be suitable in some embodiments are listed below.

[0147] D1. A video processing method comprising: converting a video including one or more pictures to a bitstream of the video, the bitstream conforming to a format rule that specifies that a flag, if included in the bitstream, is access unit (AU) specific; the value of the flag is based on whether the associated AU is an Intra Random Access Point (IRAP) AU or a Gradual Decoding Update (GDR) AU; and the value of the flag specifies (i) whether a syntax element is present in a buffering period supplemental enhancement information (SEI) message and (ii) whether alternative timing information is present in a picture timing SEI message for the current buffering period.

[0148] D2. The method according to solution D1, wherein the value of the flag is equal to 0, corresponding to the associated AU not being an IRAP AU or a GDR AU.

[0149] D3. The method of solution D1, wherein the value of the flag is inferred to be equal to 0, corresponding to the flag not being included in the bitstream.

[0150] D4. The method according to solution D1, wherein a value of the flag equal to 1 specifies that the syntax element is present in the buffering period SEI message.

[0151] D5. The method according to any of solutions D1 to D4, wherein the flag is bp_alt_cpb_params_present_flag and the syntax element is bp_use_alt_cpb_params_flag.

[0152] Further solutions that may be suitable in some embodiments are listed below.

[0153] E1. A video processing method, comprising: converting a video including one or more pictures to a bitstream of the video, the bitstream conforming to a format rule, the format rule specifying that the value of a first syntax element is based on a flag indicating whether the temporal distance between output times of consecutive pictures of a hypothetical reference decoder (HRD) is constrained and a variable identifying the highest temporal sub-layer to be decoded, the first syntax element specifying the number of element picture period intervals that one or more decoded pictures of a current AU occupy relative to a display model.

[0154] E2. The method according to solution E1, wherein the variable identifies the highest temporal sub-layer to be decoded.

[0155] E3. The method according to solution E1 or E2, where the variable is Htid.

[0156] E4. The method according to Solution E1, wherein the flag is a second syntax element included in the syntax structure of the timing and HRD parameters of the output layer set (OLS).

[0157] E5. The method according to Solution E1, wherein the first syntax element is included in a Picture Timing Supplemental Enhancement Information message.

[0158] E6. The method according to any of Solutions E1 to E5, wherein the flag is fixed_pic_rate_within_cvs_flag, the first syntax element is pt_display_elemental_periods_minus1, and the variable is Htid.

[0159] The following applies to one or more of the above solutions:

[0160] O1. A method according to any of the preceding solutions, wherein the conversion includes decoding the video from a bitstream.

[0161] O2. A method according to any of the preceding solutions, wherein the conversion includes encoding the video into a bitstream.

[0162] O3. A method for storing a bitstream representing video on a computer-readable recording medium, the method comprising generating a bitstream from video according to a method described in any one or more of the preceding solutions, and storing the bitstream on a computer-readable recording medium.

[0163] O4. A video processing device comprising a processor configured to implement a method according to any one or more of the preceding solutions.

[0164] O5. A computer-readable medium having instructions stored thereon that, when executed, cause a processor to implement one or more of the methods set forth in the preceding solutions.

[0165] O6. A computer-readable medium storing a bitstream generated by any one or more of the preceding solutions.

[0166] O7. A video processing device for storing a bitstream, configured to implement a method according to any one or more of the preceding solutions.

[0167] Further solutions that may be suitable in some embodiments are listed below.

[0168] P1. A video processing method comprising: converting between video including one or more video pictures and a coded representation of the video, the coded representation conforming to a format rule that permits use of a random access skipped leading (RADL) sub-picture in a RASL picture as a reference sub-picture for predicting a co-located RADL picture in a random access decodable leading (RADL) picture associated with the same clean random access picture as the RASL picture.

[0169] P2. A video processing method comprising: converting between video including one or more video pictures and a coded representation of the video, the bitstream conforming to a format rule, the format rule specifying that a picture timing supplemental enhancement information message, if included in the coded representation, is access unit specific, and the corresponding random access skipped reading (RASL) picture comprises a RASL network abstraction layer unit type (NUT).

[0170] P3. The method of solution P1 or P2, wherein performing the conversion includes parsing and decoding the coded representation to generate the video.

[0171] P4. The method according to solution P1 or P2, wherein performing the conversion comprises encoding the video into a coded representation.

[0172] P5. A video decoding device comprising a processor configured to implement the methods described in one or more of solutions P1 to P4.

[0173] P6. A video encoding device comprising a processor configured to implement the methods described in one or more of solutions P1 to P4.

[0174] P7. A computer program product having computer code stored thereon, the code causing the processor to implement a method according to any one of solutions P1 to P4 when executed by the processor.

[0175] As used herein, the term "video processing" can refer to video encoding, video decoding, video compression, or video decompression. For example, a video compression algorithm may be applied during the conversion of a pixel representation of video to a corresponding bitstream representation, or vice versa. The bitstream representation (or simply bitstream) of a current video block may correspond to bits spread across the same or different locations within the bitstream, e.g., as specified by a syntax. For example, one macroblock may be coded in terms of transformed and coded error residual values ​​and using bits in a header and other fields in the bitstream.

[0176] Implementations of the disclosed and other solutions, embodiments, implementations, modules, and functional operations described herein, including the structures disclosed herein and their structural equivalents, may be implemented in digital electronic circuitry, or in computer software, firmware, or hardware, or in one or more combinations thereof. The disclosed and other embodiments may be implemented as one or more computer program products, i.e., one or more modules of computer program instructions encoded on a computer-readable medium for implementation by or controlling the operation of a data processing apparatus. The computer-readable medium may be a machine-readable storage device, a machine-readable storage substrate, a memory device, a composition of matter providing a machine-readable propagated signal, or one or more combinations thereof. The term "data processing apparatus" includes all apparatuses, devices, and machines for processing data, including, for example, a programmable processing apparatus, a computer, or multiple processing apparatuses, or computers. In addition to hardware, the apparatus may include code that creates an execution environment for the computer program, such as code constituting processor firmware, a protocol stack, a database management system, an operating system, or one or more combinations thereof. A propagated signal is an artificially generated signal, for example, a machine-generated electrical, optical, or electromagnetic signal, that is generated to encode information for transmission to an appropriate receiving device.

[0177] A computer program (also called a program, software, software application, script, or code) can be written in any form of programming language, including compiled or interpreted languages, and can be deployed in any form, including as a stand-alone program or as modules, components, subroutines, or other units suitable for use in a computing environment. A computer program does not necessarily correspond to a file in a file system. A program may be recorded as part of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), may be stored in a single file dedicated to the program, or may be stored in multiple coordinated files (e.g., files containing one or more modules, subprograms, or portions of code). A computer program can be deployed to run on one computer located at a single site or on multiple computers distributed across multiple sites and interconnected by a communications network.

[0178] The processes and logic flows described herein may be performed by one or more programmable processing devices executing one or more computer programs to perform functions by operating on input data and generating output. The processes and logic flows may also be performed by, and devices may be implemented as, special purpose logic circuitry, such as an FPGA (Field Programmable Gate Array) or an ASIC (Application Specific Integrated Circuit).

[0179] Processors suitable for executing a computer program include, for example, both general-purpose and special-purpose microprocessors, and any one or more processors of any kind of digital computer. Typically, a processor receives instructions and data from a read-only memory or a random-access memory, or both. The essential elements of a computer are a processor for executing instructions and one or more memory devices for storing instructions and data. Typically, a computer will include one or more mass storage devices, e.g., magnetic, magneto-optical, or optical disks, for storing data, or be operatively coupled to receive data from or transfer data to these mass storage devices. However, a computer need not have such devices. Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, including, for example, EPROMs, EEPROMs, flash storage devices, magnetic disks, e.g., internal or removable hard disks, magneto-optical disks, and semiconductor storage devices such as CD-ROM and DVD-ROM disks. The processor and memory may be supplemented by, or incorporated in, special-purpose logic circuitry.

[0180] While this patent specification contains many details, these should not be construed as limiting the scope of any subject matter or the scope of the claims, but rather as descriptions of features that may be specific to particular embodiments of a particular technology. Certain features described in this patent document in the context of separate embodiments may also be implemented in combination in a single example. Conversely, various features described in the context of a single example may also be implemented in multiple embodiments separately or in any suitable subcombination. Furthermore, while features may be described above as acting in a particular combination and initially claimed as such, one or more features from a claimed combination may, in some cases, be extracted from the combination, and the claimed combination may be directed to subcombinations or variations of the subcombination.

[0181] Similarly, although operations are shown in a particular order in the figures, this should not be understood as requiring such operations to be performed in the particular order or sequential order shown, or that all of the operations shown be performed, to achieve desired results. Also, the separation of various system components in the examples described in this patent specification should not be understood as requiring such separation in all embodiments.

[0182] Only a few implementations and examples are described; other embodiments, extensions and variations are possible based on the content described and illustrated in this patent document.

Claims

1. 1. A method for processing a video, comprising: converting a video including one or more pictures to a bitstream of the video, the method comprising: the bitstream conforms to format rules; the format rule specifies that in a semantics associated with a first flag and in a decoding process for a picture order count, derivation of a previous picture is based on a second flag; the previous picture is a previous picture in decoding order having (i) a first identifier that is the same as a slice or picture header that references a reference picture list syntax structure, (ii) a second identifier and the second flag that are both equal to 0, and (iii) a picture type that is different from a random access skipped leading (RASL) picture and a random access decodable leading (RADL) picture; the first flag indicates whether a third flag is present in the bitstream; the second flag indicates whether the current picture is allowed to be used as a reference picture; the third flag is used to determine a first variable associated with a value of a picture order count of a long-term reference picture; Image processing method.

2. The first identifier is a layer identifier and a syntax element, and the second identifier is a temporal identifier. The method of claim 1.

3. the first flag is delta_poc_msb_cycle_present_flag, the second flag is ph_non_ref_pic_flag, the third flag is delta_poc_msb_cycle_lt, the first identifier is nuh_layer_id, and the second identifier is TemporalId; The method of claim 1.

4. The format rule specifies that a second variable prevNonDiscardableAu used to determine the timing of removal of a decoding unit (DU) or decoding of the DU is access unit (AU) specific and is derived based on the second flag. The method of claim 1.

5. When the second flag is equal to 1, it specifies that the current picture is not used as a reference picture, and when the second flag is equal to 0, it specifies that the current picture is allowed or not allowed to be used as the reference picture. The method of claim 1.

6. The format rules specify that a buffering period (BF) supplemental enhancement information (SEI) message and a picture timing SEI message, when included in the bitstream, are access unit (AU) specific; a third variable associated with the buffering period SEI message and a fourth variable associated with the buffering period SEI message and the picture timing SEI message are derived based on a second flag; the third variable indicates an AU that includes: (i) the second identifier of the AU equal to 0; and (ii) at least one picture that is not a RASL picture or a RADL picture and whose second flag is equal to 0; If the current AU is not the first AU in decoding order, the fourth variable indicates a previous AU in the decoding order that includes (i) the second identifier equal to 0 and (ii) at least one picture that is not a RASL picture or a RADL picture and whose second flag is equal to 0. The method of claim 1.

7. The third variable is notDiscardableAu, and the fourth variable is prevNonDiscardableAu. The method according to claim 6.

8. the formatting rules specify that derivation of a fifth variable and a sixth variable associated with the first picture and the second picture is based on the second flag; the first picture is a current picture, and the second picture (i) includes the second identifier equal to 0; (ii) includes the second flag equal to 0; and (iii) is a previous picture in decoding order that is not a RASL picture or a RADL picture; the fifth and sixth variables are maximum and minimum picture order counts of (i) the first picture, (ii) the second picture, (iii) one or more short-term reference pictures referenced by all entries of a reference picture list of the first picture, and (iv) each picture output with a coded picture buffer (CPB) removal time less than the picture buffer (CPB) removal time of the first picture and a decoded picture buffer (DPB) output time greater than or equal to the CPB removal time of the first picture, each having a first identifier equal to an identifier of the first picture; The method of claim 1.

9. the fifth variable is maxPicOrderCnt, which indicates the maximum value of the picture order count; and the sixth variable is minPicOrderCnt, which indicates the minimum value of the picture order count. The method according to claim 8.

10. the converting includes decoding the video from the bitstream.

10. The method according to any one of claims 1 to 9.

11. the converting includes encoding the video into the bitstream; 10. The method according to any one of claims 1 to 9.

12. 1. A video data processing apparatus comprising a processor and a non-transitory memory comprising instructions which, when executed by the processor, cause the processor to convert between a video comprising one or more pictures and a bitstream of the video; the bitstream conforms to format rules; the format rule specifies that in a semantics associated with a first flag and in a decoding process for picture order counting, derivation of a previous picture is based on a second flag; the previous picture is a previous picture in decoding order having (i) a first identifier that references a reference picture list syntax structure and is the same as a slice or picture header, (ii) a second identifier and the second flag that are both equal to 0, and (iii) a picture type that is different from a random access skipped leading (RASL) picture and a random access decodable leading (RADL) picture; the first flag indicates whether a third flag is present in the bitstream; the second flag indicates whether the current picture is allowed to be used as a reference picture; the third flag is used to determine a first variable associated with a picture order count value of a long-term reference picture; Video data processing device.

13. The first identifier is a layer identifier and is a syntax element nuh_layer_id, and the second identifier is a temporal identifier and is a variable TemporalId; The first flag is delta_poc_msb_cycle_present_flag and the second flag is ph_non_ref_pic_flag, the second flag being equal to 1 specifies that the current picture is never used as the reference picture, and the second flag being equal to 0 specifies that the current picture is allowed or not allowed to be used as the reference picture.

13. The apparatus of claim 12.

14. The format rule specifies that a second variable prevNonDiscardableAu used to determine the timing of deleting a decoding unit (DU) or decoding the DU is access unit (AU) specific and is derived based on the second flag.

13. The apparatus of claim 12.

15. The format rules specify that a buffering period (BF) supplemental enhancement information (SEI) message and a picture timing SEI message, when included in the bitstream, are access unit (AU) specific; a third variable associated with the buffering period SEI message and a fourth variable associated with the buffering period SEI message and the picture timing SEI message are derived based on the second flag; the third variable indicates an AU that includes (i) the second identifier of the AU equal to 0; and (ii) at least one picture that is not a RASL picture or a RADL picture and whose second flag is equal to 0; if the current AU is not the first AU in decoding order, a fourth variable indicates a previous AU in the decoding order that includes: (i) the second identifier equal to 0; and (ii) at least one picture that is not a RASL picture or a RADL picture and has the second flag equal to 0; The third variable is notDiscardableAu, and the fourth variable is prevNonDiscardableAu.

13. The apparatus of claim 12.

16. the formatting rules specify that derivation of a fifth variable and a sixth variable associated with the first picture and the second picture is based on the second flag; the first picture is a current picture, and the second picture (i) includes the second identifier equal to 0; (ii) includes the second flag equal to 0; and (iii) is a previous picture in decoding order that is not a RASL picture or a RADL picture; the fifth and sixth variables are respectively maximum and minimum values ​​of picture order counts of each of the following pictures having the first identifier equal to that of the first picture, the following pictures being: (i) the first picture, (ii) the second picture, (iii) one or more short-term reference pictures referenced by all entries in a reference picture list of the first picture, and (iv) each picture that has a coded picture buffer (CPB) removal time less than the CPB removal time of the first picture and a decoded picture buffer (DPB) output time equal to or greater than the CPB removal time of the first picture; the fifth variable is maxPicOrderCnt, which indicates the maximum value of the picture order count; and the sixth variable is minPicOrderCnt, which indicates the minimum value of the picture order count.

13. The apparatus of claim 12.

17. A non-transitory computer-readable carrier medium storing instructions for causing a processor to convert between a video including one or more pictures and a bitstream of the video; the bitstream conforms to format rules; the format rule specifies that in a semantics associated with a first flag and in a decoding process for picture order counting, derivation of a previous picture is based on a second flag; the previous picture is a previous picture in decoding order having (i) a first identifier that references a reference picture list syntax structure and is the same as a slice or picture header, (ii) a second identifier and the second flag that are both equal to 0, and (iii) a picture type that is different from a random access skipped leading (RASL) picture and a random access decodable leading (RADL) picture; the first flag indicates whether a third flag is present in the bitstream; the second flag indicates whether the current picture is allowed to be used as a reference picture; the third flag is used to determine a first variable associated with a picture order count value of a long-term reference picture; A non-transitory computer-readable storage medium.

18. The format rule specifies that a second variable prevNonDiscardableAu used to determine the timing of deleting a decoding unit (DU) or decoding the DU is access unit (AU) specific and is derived based on the second flag; The format rules specify that a buffering period (BF) supplemental enhancement information (SEI) message and a picture timing SEI message, when included in the bitstream, are access unit (AU) specific; a third variable associated with the buffering period SEI message and a fourth variable associated with the buffering period SEI message and the picture timing SEI message are derived based on the second flag; the third variable indicates an AU that includes (i) the second identifier of the AU equal to 0; and (ii) at least one picture that is not a RASL picture or a RADL picture and whose second flag is equal to 0; if the current AU is not the first AU in decoding order, the fourth variable indicates a previous AU in the decoding order that includes: (i) the second identifier equal to 0; and (ii) at least one picture that is not a RASL picture or a RADL picture and has the second flag equal to 0; the third variable is notDiscardableAu, the fourth variable is prevNonDiscardableAu, the formatting rules specify that derivation of a fifth variable and a sixth variable associated with the first picture and the second picture is based on the second flag; the first picture is a current picture, and the second picture (i) includes the second identifier equal to 0; (ii) includes the second flag equal to 0; and (iii) is a previous picture in decoding order that is not a RASL picture or a RADL picture; the fifth and sixth variables are respectively maximum and minimum values ​​of picture order counts of each of the following pictures having the first identifier equal to that of the first picture, the following pictures being: (i) the first picture, (ii) the second picture, (iii) one or more short-term reference pictures referenced by all entries in a reference picture list of the first picture, and (iv) each picture that has a coded picture buffer (CPB) removal time less than the CPB removal time of the first picture and a decoded picture buffer (DPB) output time equal to or greater than the CPB removal time of the first picture; the fifth variable is maxPicOrderCnt, which indicates a maximum value of the picture order count; and the sixth variable is minPicOrderCnt, which indicates a minimum value of the picture order count.

20. The non-transitory computer-readable storage medium of claim 17.

19. 1. A method for storing a video bitstream, comprising: generating a bitstream of the video comprising one or more pictures; storing the bitstream on a non-transitory computer readable recording medium; the bitstream conforming to a format rule; the formatting rules specify that in a semantics associated with a first flag and in a decoding process for picture order counting, derivation of a previous picture is based on a second flag; the previous picture is a previous picture in decoding order having (i) a first identifier that references a reference picture list syntax structure, the first identifier being the same as a slice or picture header, (ii) a second identifier and the second flag both being equal to 0, and (iii) a picture type different from a random access skipped leading (RASL) picture and a random access decodable leading (RADL) picture; the first flag indicates whether a third flag is present in the bitstream; the second flag indicates whether the current picture is allowed to be used as a reference picture; the third flag is used to determine a first variable associated with a picture order count value of a long-term reference picture; method.

20. The format rule specifies that a second variable prevNonDiscardableAu used to determine the timing of deleting a decoding unit (DU) or decoding the DU is access unit (AU) specific and is derived based on the second flag; The format rules specify that a buffering period (BF) supplemental enhancement information (SEI) message and a picture timing SEI message, when included in the bitstream, are access unit (AU) specific; a third variable associated with the buffering period SEI message and a fourth variable associated with the buffering period SEI message and the picture timing SEI message are derived based on the second flag; the third variable indicates an AU that includes (i) the second identifier of the AU equal to 0, and (ii) at least one picture that is not a RASL picture or a RADL picture and whose second flag is equal to 0; if the current AU is not the first AU in decoding order, the fourth variable indicates a previous AU in decoding order that includes: (i) the second identifier equal to 0; and (ii) at least one picture that is not a RASL picture or a RADL picture and has the second flag equal to 0; the third variable is notDiscardableAu, the fourth variable is prevNonDiscardableAu, the formatting rules specify that derivation of a fifth variable and a sixth variable associated with the first picture and the second picture is based on the second flag; the first picture is a current picture, and the second picture (i) includes the second identifier equal to 0; (ii) includes the second flag equal to 0; and (iii) is a previous picture in decoding order that is not a RASL picture or a RADL picture; the fifth and sixth variables are respectively maximum and minimum values ​​of picture order counts of each of the following pictures having the first identifier equal to that of the first picture, the following pictures being: (i) the first picture, (ii) the second picture, (iii) one or more short-term reference pictures referenced by all entries in a reference picture list of the first picture, and (iv) each picture that has a coded picture buffer (CPB) removal time less than the CPB removal time of the first picture and a decoded picture buffer (DPB) output time equal to or greater than the CPB removal time of the first picture; the fifth variable is maxPicOrderCnt, which indicates the maximum value of the picture order count; and the sixth variable is minPicOrderCnt, which indicates the minimum value of the picture order count.

20. The method of claim 19.

Images