Constraints on the order of reference pictures

Abstract

Methods and apparatuses for video processing such as video encoding or decoding are described. An example video processing method includes performing a conversion between a video having one or more video layers including a current picture containing a current slice and a bitstream of the video according to a rule, and wherein the rule specifies a condition that a reference picture list of the current slice is not allowed to have an active entry that references a picture before an intra random access point picture associated with the current picture in decoding order or output order.

Description

[0001] Cross - Reference to Related Applications

[0002] This application claims priority and benefit of U.S. Provisional Patent Application No. 62 / 992,046, filed on Mar. 19, 2020, and is based on International Patent Application No. PCT / US2021 / 022584, filed on Mar. 16, 2021. All of the above - mentioned patent applications are hereby incorporated by reference in their entirety. Technical Field

[0003] This patent document relates to image and video encoding and decoding. Background Art

[0004] Digital video accounts for the largest bandwidth usage on the Internet and other digital communication networks. As the number of connected user devices capable of receiving and displaying video increases, the bandwidth demand for digital video usage is expected to continue to grow. Summary of the Invention

[0005] This document discloses techniques that can be used by video encoders and decoders to process coded representations of video using a bit - stream syntax that provides improved performance. The disclosed methods can be used by apparatuses performing video processing such as video encoding, video decoding, or video transcoding.

[0006] In one example aspect, a video processing method is disclosed. The method includes performing a conversion between a video having one or more video layers including one or more video pictures and a coded representation of the video; wherein the coded representation is organized according to rules that specify that a first video picture and a second picture that is an intra - random - access - point picture as a second picture are constrained to belong to the same video layer.

[0007] In another example aspect, another video processing method is disclosed. The method includes performing a conversion between a video having one or more video layers including one or more video pictures and a coded representation of the video, wherein the coded representation conforms to format rules that specify that a post - picture following a first - type picture that is an intra - random - access - point in the coded representation is also permitted to be associated with a second - type picture including a progressive - decoded - refresh picture.

[0008] In another example aspect, another video processing method is disclosed. The method includes performing a conversion between a video having one or more video layers including one or more video pictures and a coded representation of the video, wherein the coded representation conforms to format rules that specify a constraint on the output order of pictures that are before an intra - random - access - point in decoding order, such that the output order only applies to pictures in the same video layer.

[0009] In another example aspect, another video processing method is disclosed. The method includes performing a conversion between a video having one or more video layers including one or more video pictures and a coded representation of the video, wherein the coded representation complies with formatting rules that specify the following constraints: (1) a post picture must be after an associated instantaneous random access point (IRAP) picture or a gradual decoder refresh (GDR) picture in output order, or (2) a picture having the same layer id as a GDR picture must be before the GDR picture and all associated pictures of the GDR picture in output order.

[0010] In another example aspect, another video processing method is disclosed. The method includes performing a conversion between a video having one or more video layers including one or more video pictures and a coded representation of the video, wherein the conversion complies with the rule that sequential constraints apply to pictures, IRAP pictures, and non-preceding pictures if and only if the pictures, IRAP pictures, and non-preceding pictures are in the same layer, and wherein the rule is one of the following: (a) a first rule specifying values of a field sequence and a decoding order, or (b) an order of preceding and / or non-preceding pictures of a layer.

[0011] In another example aspect, another video processing method is disclosed. The method includes performing a conversion between a video having one or more video layers including one or more video pictures and a coded representation of the video, wherein the conversion complies with a rule specifying an order of a preceding picture, a random access decodable preceding (RADL) picture, and a random access skipped preceding (RASL) picture associated with a gradual decoder refresh (GDR) picture.

[0012] In another example aspect, another video processing method is disclosed. The method includes performing a conversion between a video having one or more video layers including one or more video pictures and a coded representation of the video, wherein the conversion complies with a rule specifying that constraints on a reference picture list for a fully random access picture are restricted to a layer.

[0013] In another example aspect, another video processing method is disclosed. The method includes performing a conversion between a video having one or more video layers including one or more video pictures and a coded representation of the video, wherein the conversion complies with a rule specifying conditions under which a current picture is allowed to reference an entry in a reference picture list generated by a decoding process for generating an unavailable reference picture.

[0014] In another example aspect, another video processing method is disclosed. The method includes performing a conversion between a video having one or more video layers including one or more video pictures and a coded representation of the video, wherein the conversion complies with a rule for an order between a current picture and a reference picture list corresponding to the current picture.

[0015] In another example aspect, another video processing method is disclosed. The method includes performing a conversion between a video having one or more video layers including one or more video pictures and a bitstream of the video according to format rules, and wherein the format rules specify that a first video picture and a second picture, which is an associated intra random access point picture as a second picture, are constrained to belong to the same video layer.

[0016] In another example aspect, another video processing method is disclosed. The method includes performing a conversion between a video having one or more video layers including one or more video pictures and a bitstream of the video according to format rules, and wherein the format rules specify that a post picture in the bitstream is permitted to be associated with a progressive decode refresh picture.

[0017] In another example aspect, another video processing method is disclosed. The method includes performing a conversion between a video having one or more video layers including one or more video pictures and a bitstream of the video according to format rules, and wherein the format rules specify that a constraint on the output order of pictures that are before an intra random access point in decoding order applies to pictures in the same video layer.

[0018] In another example aspect, another video processing method is disclosed. The method includes performing a conversion between a video having one or more video layers including one or more video pictures and a bitstream of the video according to format rules, and wherein the format rules specify the following constraints: (1) a post picture is after an associated intra random access point picture or a progressive decoder refresh picture in output order, or (2) a picture having the same NAL (Network Abstraction Layer) unit header layer identifier as a progressive decoder refresh picture is before the progressive decoder refresh picture and all associated pictures of the progressive decoder refresh picture in output order.

[0019] In another example aspect, another video processing method is disclosed. The method includes performing a conversion between a video having one or more video layers including one or more video pictures and a bitstream of the video according to rules, and wherein the rules specify that a constraint on the decoding order of a picture and a non-pre picture is applied if and only if a picture associated with an intra random access point picture, the intra random access point picture, and the non-pre picture are in the same layer.

[0020] In another example aspect, another video processing method is disclosed. The method includes performing a conversion between a video having one or more video layers including one or more video pictures and a bitstream of the video according to rules, and wherein the rules specify the order of a pre picture associated with a progressive decode refresh picture, a randomly accessible decodable pre picture, and a randomly accessible skipped pre picture.

[0021] In another example aspect, another video processing method is disclosed. The method includes performing a conversion between a video having one or more video layers each including one or more video pictures and a bitstream of the video according to a rule, where the rule specifies that a constraint on a reference picture list for a strip of a full random access picture is restricted to a layer.

[0022] In another example aspect, another video processing method is disclosed. The method includes performing a conversion between a video having one or more video layers each including one or more video pictures and a bitstream of the video according to a rule, and where the rule specifies a condition that no picture that has been generated by a decoding process for generating an unavailable reference picture is referenced by an active entry in a reference picture list of a current strip of a current picture.

[0023] In another example aspect, another video processing method is disclosed. The method includes performing a conversion between a video having one or more video layers each including one or more video pictures and a bitstream of the video according to a rule, and where the rule specifies a condition that no picture that has been generated by a decoding process for generating an unavailable reference picture is referenced by an entry in a reference picture list of a current strip of a current picture.

[0024] In another example aspect, another video processing method is disclosed. The method includes performing a conversion between a video having one or more video layers each including a current picture including a current strip and a bitstream of the video according to a rule, and where the rule specifies a condition that a reference picture list of the current strip is not allowed to have an active entry that references a picture before an intra random access point picture in a frame associated with the current picture in decoding order or output order.

[0025] In another example aspect, another video processing method is disclosed. The method includes performing a conversion between a video having one or more video layers each including a current picture including a current strip and a bitstream of the video according to a rule, and where the rule specifies a condition that a reference picture list of the current strip is not allowed to have an entry that references a picture before an intra random access point picture in a frame associated with the current picture in decoding order or output order.

[0026] In yet another example aspect, a video encoder device is disclosed. The video encoder includes a processor configured to implement the above method.

[0027] In yet another example aspect, a video decoder device is disclosed. The video decoder includes a processor configured to implement the above method.

[0028] In yet another example aspect, a computer-readable medium storing code is disclosed. The code embodies one of the methods described herein in the form of processor-executable code.

[0029] These and other features will be described in this document. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 is a block diagram of an example video processing system.

[0031] Figure 2 is a block diagram of a video processing apparatus.

[0032] Figure 3 is a flowchart of an example method of video processing.

[0033] Figure 4 is a block diagram showing a video codec system according to some embodiments of the present disclosure.

[0034] Figure 5 is a block diagram showing an encoder according to some embodiments of the present disclosure.

[0035] Figure 6 is a block diagram showing a decoder according to some embodiments of the present disclosure.

[0036] Figures 7A to 7G shows a flowchart of an example method of video processing based on some implementations of the disclosed technology.

[0037] Figure 8A and Figure 8B shows a flowchart of an example method of video processing based on some implementations of the disclosed technology.

[0038] Figure 9A and Figure 9B shows a flowchart of an example method of video processing based on some implementations of the disclosed technology. DETAILED DESCRIPTION

[0039] Section headings are used in this document for ease of understanding, and do not limit the applicability of the technologies and embodiments disclosed in each section only to that section. Additionally, the use of H.266 technical terms in some descriptions is merely for ease of understanding and does not limit the scope of the disclosed technologies. Thus, the technologies described herein are also applicable to other video codec protocols and designs.

[0040] 1. BRIEF SUMMARY

[0041] This document relates to video coding and decoding techniques. Specifically, it pertains to various aspects of supporting random access, sub-layer switching, and scalability, including the definition of different types of pictures and their relationships in terms of decoding order, output order, and prediction relationships. This idea can be applied alone or in various combinations to any video coding standard or non-standard video codec that supports multi-layer video coding and decoding, such as the Versatile Video Coding (VVC) currently under development.

[0042] 2. Abbreviations

[0043] APS Adaptive Parameter Set

[0044] AU Access Unit

[0045] AUD Access Unit Delimiter

[0046] AVC Advanced Video Coding

[0047] CLVS Coded Layer Video Sequence

[0048] CPB Coded Picture Buffer

[0049] CRA Complete Random Access

[0050] CTU Coding Tree Unit

[0051] CVS Coded Video Sequence

[0052] DCI Decoding Capability Information

[0053] DPB Decoded Picture Buffer

[0054] EOB End of Bitstream

[0055] EOS End of Sequence

[0056] GDR Gradual Decoding Refresh

[0057] HEVC High Efficiency Video Coding

[0058] HRD Hypothetical Reference Decoder

[0059] IDR Instantaneous Decoding Refresh

[0060] JEM Joint Exploration Model

[0061] MCTS Motion Constrained Tile Set

[0062] NAL Network Abstraction Layer

[0063] OLS Output Layer Set

[0064] PH Picture Header

[0065] PPS Picture Parameter Set

[0066] PTL Profile, Tier and Level

[0067] PU Picture Unit

[0068] RADL Random Access Decodable Leading (picture)

[0069] RAP Random Access Point

[0070] RASL Random Access Skipped Leading (picture)

[0071] RBSP Raw Byte Sequence Payload

[0072] RPL Reference Picture List

[0073] SEI Supplemental Enhancement Information

[0074] SPS Sequence Parameter Set

[0075] STSA Stepwise Temporal Sub-layer Access

[0076] SVC Scalable Video Coding

[0077] VCL Video Coding Layer

[0078] VPS Video Parameter Set

[0079] VTM VVC Test Model

[0080] VUI Video Usability Information

[0081] VVC Versatile Video Coding

[0082] 3. Preliminary Discussion

[0083] Video coding standards have evolved mainly through the development of well-known ITU-T and ISO / IEC standards. The ITU-T developed H.261 and H.263, the ISO / IEC developed MPEG-1 and MPEG-4 Visual, and the two organizations jointly developed the H.262 / MPEG-2 video, H.264 / MPEG-4 Advanced Video Coding (AVC), and H.265 / HEVC standards. Since H.262, video coding standards have been based on a hybrid video coding structure, in which temporal prediction plus transform coding is employed. To explore future video coding technologies beyond HEVC, VCEG and MPEG jointly established the Joint Video Exploration Team (JVET) in 2015. Since then, the JVET has adopted many new methods and incorporated them into a reference software called the Joint Exploration Model (JEM). JVET meetings are held quarterly simultaneously. The goal of the new coding standard is to reduce the bit rate by 50% compared to HEVC. The new video coding standard was officially named Versatile Video Coding (VVC) at the JVET meeting in April 2018, and the first version of the VVC Test Model (VTM) was also released at that time. Due to the continuous efforts on VVC standardization, new coding technologies have been adopted into the VVC standard at each JVET meeting. The working draft and test model VTM of VVC are updated after each meeting. The VVC project now aims to be technically completed (FDIS) at the meeting in July 2020.

[0084] 3.1. Overview and Scalable Video Coding (SVC) in VVC

[0085] Scalable Video Coding (SVC, sometimes also referred to as scalability in video coding) refers to video coding that uses a Base Layer (BL) (sometimes referred to as a Reference Layer (RL)) and one or more Scalable Enhancement Layers (ELs). In SVC, the base layer can carry video data with a basic quality level. One or more enhancement layers can carry additional video data to support, for example, higher spatial, temporal, and / or signal-to-noise ratio (SNR) levels. Enhancement layers can be defined relative to previously encoded layers. For example, the bottom layer can act as the BL, while the top layer can act as the EL. Intermediate layers can act as ELs or RLs, or both. For example, an intermediate layer (e.g., a layer that is neither the lowest nor the highest layer) can be an EL of a layer below it (such as the base layer or any intervening enhancement layer) and at the same time act as an RL for one or more enhancement layers above it. Similarly, in the multi-view or 3D extension of the HEVC standard, there can be multiple views, and the information of one view can be used for coding or decoding (e.g., motion estimation, motion vector prediction, and / or other redundancies) the information of another view.

[0086] In SVC, the parameters used by the encoder or decoder are grouped into parameter sets based on the coding levels at which they can be utilized (e.g., video level, sequence level, picture level, slice level, etc.). For example, the parameters that can be utilized by one or more coded video sequences of different layers in a bitstream can be included in a Video Parameter Set (VPS), and the parameters that can be utilized by one or more pictures in a coded video sequence can be included in a Sequence Parameter Set (SPS). Similarly, the parameters utilized by one or more slices in a picture can be included in a Picture Parameter Set (PPS), and other parameters specific to a single slice can be included in the slice header. Similarly, an indication of which (which) parameter set a given layer uses at a given time can be provided at various coding levels.

[0087] Due to the support for reference picture resampling (RPR) in VVC, it is possible to design support for bitstreams containing multiple layers (e.g., two layers with SD and HD resolutions in VVC) without any additional signal processing level codec tools, since the upsampling required for spatial scalability support can be achieved using only the RPR upsampling filter. However, for scalability support, high-level syntax changes are required (compared to non-scalability support). Scalability support was specified in VVC version 1. Different from the scalability support in any earlier video codec standards (including extensions of AVC and HEVC), the design of VVC scalability has been made as friendly as possible to single-layer decoder design. The decoding capabilities of multi-layer bitstreams are specified in a way that is as if there were only a single layer in the bitstream. For example, decoding capabilities such as DPB size are specified independently of the number of layers in the bitstream to be decoded. Basically, a decoder designed for single-layer bitstreams does not need much modification to be able to decode multi-layer bitstreams. Compared to the design of multi-layer extensions of AVC and HEVC, the HLS aspect is significantly simplified at the cost of some flexibility. For example, it is required that an IRAP AU contains pictures of each layer present in the CVS.

[0088] 3.2. Random Access and Its Support in HEVC and VVC

[0089] Random access refers to accessing and decoding a bitstream starting from a picture that is not the first picture of the bitstream in decoding order. To support tuning and channel switching in broadcast / multicast and multiparty video conferencing, search in local playback and streaming, and stream adaptation in streaming, the bitstream needs to include randomly accessible points that are close together, which are usually intra-coded pictures but can also be inter-coded pictures (e.g., in the case of progressive decoding refresh).

[0090] HEVC includes signaling of Intra Random Access Point (IRAP) pictures in the NAL unit header by NAL unit type. Three types of IRAP pictures are supported, namely Instantaneous Decoder Refresh (IDR), Complete Random Access (CRA), and Broken Link Access (BLA) pictures. IDR pictures constrain the inter-picture prediction structure to not reference any pictures before the current Group of Pictures (GOP), which is traditionally referred to as a closed GOP random access point. CRA pictures are less restrictive by allowing a particular picture to reference pictures before the current GOP, where in the case of random access, all pictures are discarded. CRA pictures are traditionally referred to as open GOP random access points. BLA pictures typically result from the concatenation of two bitstreams or parts thereof at a CRA picture, such as during stream switching. To better enable the system to use IRAP pictures, a total of six different NAL units are defined to signal the attributes of IRAP pictures, which can be used to better match the stream access point types defined in the ISO Base Media File Format (ISOBMFF) for random access support in HTTP-based Dynamic Adaptive Streaming over HTTP (DASH).

[0091] VVC supports three types of IRAP pictures, two types of IDR pictures (one type with or the other type without an associated RADL picture), and one type of CRA picture. These are basically the same as in HEVC. The BLA picture type in HEVC is not included in VVC, mainly for two reasons: i) The basic function of BLA pictures can be achieved by a CRA picture plus the end of the sequence NAL unit, the presence of which indicates that the subsequent pictures start a new CVS in a single-layer bitstream. ii) During the development of VVC, it was desired to specify fewer NAL unit types than in HEVC, as indicated by using 5 bits instead of 6 bits for the NAL unit type field in the NAL unit header.

[0092] Another key difference between VVC and HEVC in random access support is that GDR is supported in a more standardized way in VVC. In GDR, the decoding of the bitstream can start from an inter-coded picture. Although not the entire picture area can be correctly decoded at the beginning, after multiple pictures, the entire picture area will be correct. AVC and HEVC also support GDR by using recovery point SEI messages to signal GDR random access points and recovery points. In VVC, a new NAL unit type is specified for the indication of GDR pictures, and the recovery point is signaled in the picture header syntax structure. It is allowed for the CVS and the bitstream to start with GDR pictures. This means that it is allowed for the entire bitstream to contain only inter-coded pictures without a single intra-coded picture. The main benefit of specifying GDR support in this way is to provide consistent behavior of GDR. GDR enables the encoder to smooth the bitrate of the bitstream by distributing intra-coded strips or blocks over multiple pictures, as opposed to intra-coding the entire picture, thus allowing a significant reduction in end-to-end latency, which is considered more important today than before as ultra-low latency applications such as wireless display, online gaming, and drone-based applications become more popular.

[0093] Another GDR-related feature in VVC is virtual boundary signaling. The boundary between the refreshed area (i.e., the correctly decoded area) and the non-refreshed area at the pictures between a GDR picture and its recovery point can be signaled as a virtual boundary, and when signaled, loop filtering across the boundary will not be applied, so there will be no decoding mismatch of some samples at or near the boundary. This can be useful when the application determines to display the correctly decoded area during the GDR process.

[0094] IRAP pictures and GDR pictures can be collectively referred to as random access point (RAP) pictures.

[0095] 3.3. Reference Picture Management and Reference Picture List (RPL)

[0096] Reference picture management is a core function essential for any video coding scheme that uses inter prediction. It manages the storage of reference pictures in the decoded picture buffer (DPB) and their removal from the DPB, and places the reference pictures in the RPL in their correct order.

[0097] Reference picture management in HEVC, including reference picture marking and removal from the decoded picture buffer (DPB) as well as reference picture list construction (RPLC), differs from that in AVC. Instead of the reference picture marking mechanism based on a sliding window plus adaptive memory management control operation (MMCO) in AVC, HEVC specifies a reference picture management and marking mechanism based on the so-called reference picture set (RPS), and RPLC is thus based on the RPS mechanism. An RPS consists of a set of reference pictures associated with a picture, which consists of all reference pictures in decoding order before the associated picture and can be used for inter prediction of the associated picture or any picture in decoding order after the associated picture. The reference picture set consists of five reference picture lists. The first three lists contain all reference pictures that can be used for inter prediction of the current picture and can be used for inter prediction of one or more pictures in decoding order after the current picture. The other two lists consist of all reference pictures that are not used for inter prediction of the current picture but can be used for inter prediction of one or more pictures in decoding order after the current picture. The RPS provides "intra-coded" signaling of the DPB state, rather than "inter-coded" signaling as in AVC, mainly to improve fault tolerance. The RPLC process in HEVC is based on the RPS, by signaling the index of an RPS subset for each reference index; this process is simpler than the RPLC process in AVC.

[0098] Reference picture management in VVC is more similar to HEVC than to AVC, but is slightly simpler and more robust. As in those standards, two RPLs, list 0 and list 1, are derived, but they are not based on the reference picture set concept used in HEVC or the automatic sliding window process used in AVC; instead, they are signaled more directly. Reference pictures are listed as active and non-active entries for the RPL, and only active entries can be used as reference indices in the inter prediction of the CTUs of the current picture. Non-active entries indicate other pictures to be saved in the DPB for reference by other pictures arriving later in the bitstream.

[0099] 3.4. Parameter Sets

[0100] AVC, HEVC, and VVC specify parameter sets. The types of parameter sets include SPS, PPS, APS, and VPS. SPS and PPS are supported in all of AVC, HEVC, and VVC. VPS was introduced starting from HEVC and is included in HEVC and VVC. APS is not included in AVC or HEVC but is included in the latest VVC draft text.

[0101] The SPS is designed to carry sequence-level header information, and the PPS is designed to carry picture-level header information that changes infrequently. By using the SPS and PPS, the information that changes infrequently does not need to be repeated for each sequence or picture, thus avoiding redundant signaling of this information. In addition, the use of the SPS and PPS enables out-of-band transmission of important header information, thus not only avoiding the need for redundant transmission but also improving fault tolerance.

[0102] The VPS is introduced to carry sequence-level header information common to all layers in a multi-layer bitstream.

[0103] The APS is introduced to carry picture-level or slice-level information that requires a significant number of bits to encode and decode, can be shared by multiple pictures, and can vary considerably in the sequence.

[0104] 3.5. Related Definitions in VVC

[0105] The related definitions in the latest VVC text (in JVET-Q2001-vE / v15) are as follows.

[0106] Associated IRAP picture (of a specific picture): The previous IRAP picture in decoding order (when present) has the same nuh_layer_id value as the specific picture.

[0107] Completely Random Access (CRA) PU: A PU for which the decoded picture is a CRA picture.

[0108] Completely Random Access (CRA) picture: An IRAP picture for which each VCL NAL unit has a nal_unit_type equal to CRA_NUT.

[0109] Coded Video Sequence (CVS): A sequence of AUs consisting of CVSS AUs in decoding order, followed by zero or more AUs that are not CVSS AUs, including all subsequent AUs up to but not including any subsequent AU that is a CVSS AU.

[0110] Coded Video Sequence Start (CVSS) AU: An AU for which there is a PU for each layer in the CVS and the decoded picture in each PU is a CLVSS picture.

[0111] Gradual Decoding Refresh (GDR) AU: An AU for which the decoded picture in each current PU is a GDR picture.

[0112] Gradual Decoding Refresh (GDR) PU: A PU for which the decoded picture is a GDR picture.

[0113] Gradual Decoding Refresh (GDR) Picture: A picture where each VCL NAL unit has a nal_unit_type equal to GDR_NUT.

[0114] Instantaneous Decoding Refresh (IDR) PU: A PU where the decoded picture is an IDR picture.

[0115] Instantaneous Decoding Refresh (IDR) Picture: An IRAP picture where each VCL NAL unit has a nal_unit_type equal to IDR_W_RADL or IDR_N_LP.

[0116] Intra Random Access Point (IRAP) AU: An AU where there is a PU for each layer in the CVS and the decoded picture in each PU is an IRAP picture.

[0117] Intra Random Access Point (IRAP) PU: A PU where the decoded picture is an IRAP picture.

[0118] Intra Random Access Point (IRAP) Picture: A decoded picture where all VCL NAL units have the same nal_unit_type value in the range from IDR_W_RADL to CRA_NUT (including IDR_W_RADL and CRA_NUT).

[0119] Preceding Picture: A picture in the same layer as the associated IRAP picture and preceding the associated IRAP picture in output order.

[0120] Random Access Decodable Leading (RADL) PU: A PU where the decoded picture is a RADL picture.

[0121] Random Access Decodable Leading (RADL) Picture: A picture where each VCL NAL unit has a nal_unit_type equal to RADL_NUT.

[0122] Random Access Skippable Leading (RASL) PU: A PU where the decoded picture is a RASL picture.

[0123] Random Access Skippable Leading (RASL) Picture: A picture where each VCL NAL unit has a nal_unit_type equal to RASL_NUT.

[0124] Stepwise Temporal Sub-layer Access (STSA) PU: A PU where the decoded picture is a STSA picture.

[0125] Stepwise Temporal Sub-layer Access (STSA) Picture: A picture where each VCL NAL unit has a nal_unit_type equal to STSA_NUT.

[0126] Note – The STSA picture does not use pictures with the same TemporalId as the STSA picture for inter-prediction reference. Pictures following the STSA picture in decoding order with the same TemporalId as the STSA picture do not use pictures preceding the STSA picture in decoding order with the same TemporalId as the STSA picture for inter-prediction reference. The STSA picture implementation switches from the immediately lower sublayer to the sublayer containing the STSA picture at the STSA picture. The STSA picture must have a TemporalId greater than 0.

[0127] Trailing picture: A non-IRAP picture that follows the associated IRAP picture in output order and is not an STSA picture.

[0128] Note – Trailing pictures associated with an IRAP picture are also after the IRAP picture in decoding order. A picture that follows the associated IRAP picture in output order and precedes the associated IRAP picture in decoding order is not allowed.

[0129] 3.6. NAL Unit Header Syntax and Semantics in VVC

[0130] In the latest VVC text (in JVET-Q2001-vE / v15), the NAL unit header syntax and semantics are as follows.

[0131] 7.3.1.2 NAL Unit Header Syntax

[0132]

[0133] 7.4.2.2 NAL Unit Header Semantics

[0134] The forbidden_zero_bit shall be equal to 0.

[0135] The nuh_reserved_zero_bit shall be equal to 0. A value of 1 for the nuh_reserved_zero_bit may be specified by ITU-T|ISO / IEC in the future. The decoder shall ignore (i.e., remove and discard from the bitstream) NAL units with nuh_reserved_zero_bit equal to 1.

[0136] The nuh_layer_id specifies the identifier of the layer to which the VCL NAL unit belongs or the identifier of the layer applicable to the non-VCL NAL unit. The value of the nuh_layer_id shall be in the range from 0 to 55 (including 0 and 55). Other values of the nuh_layer_id are reserved for future use by ITU-T|ISO / IEC.

[0137] The value of nuh_layer_id shall be the same for all VCL NAL units of a coded or decoded picture. The value of nuh_layer_id for a coded picture or PU is the value of nuh_layer_id for the VCL NAL units of the coded picture or PU.

[0138] The values of nuh_layer_id for AUD, PH, EOS, and FD NAL units are constrained as follows:

[0139] - If nal_unit_type is equal to AUD_NUT, then nuh_layer_id shall be equal to vps_layer_id[0].

[0140] - Otherwise, when nal_unit_type is equal to PH_NUT, EOS_NUT, or FD_NUT, nuh_layer_id shall be equal to the nuh_layer_id of the associated VCL NAL unit.

[0141] Note 1 – The values of nuh_layer_id for DCI, VPS, and EOB NAL units are not constrained.

[0142] The value of nal_unit_type shall be the same for all pictures of a CVSS AU.

[0143] nal_unit_type specifies the NAL unit type, i.e., the type of the RBSP data structure contained in the NAL unit as specified in Table 5.

[0144] NAL units with nal_unit_type in the range of UNSPEC_28..UNSPEC_31 (unspecified semantics), including UNSPEC_28 and UNSPEC_31, shall not affect the decoding process specified in this specification.

[0145] Note 2 – NAL unit types in the range UNSPEC_28..UNSPEC_31 may be used as determined by the application. The decoding processes for these values of nal_unit_type are not specified in this specification. Since different applications may use these NAL unit types for different purposes, special care must be taken in the design of encoders that generate NAL units with these nal_unit_type values, as well as in the design of decoders that interpret the content of NAL units with these nal_unit_type values. This specification does not define any management of these values. These nal_unit_type values may only be suitable for use in contexts where the "conflicts" in use (i.e., different definitions of the meaning of the content of NAL units with the same nal_unit_type value) are not important, or not possible, or are managed (e.g., defined or managed in a control application or transport specification, or in an environment distributed via a control bitstream).

[0146] For purposes other than determining the amount of data in the DU of the bitstream (as specified in Annex C), the decoder shall ignore (remove and discard from the bitstream) the content of all NAL units that use reserved values of nal_unit_type.

[0147] Note 3 – This requirement allows for future definition of compliant extensions to this specification.

[0148] Table 5 – NAL unit type codes and NAL unit type classifications

[0149]

[0150]

[0151] Note 4 – A completely random access (CRA) picture may have associated RASL or RADL pictures present in the bitstream.

[0152] Note 5 – An instant decoding refresh (IDR) picture with a nal_unit_type equal to IDR_N_LP does not have an associated previous picture present in the bitstream. An IDR picture with a nal_unit_type equal to IDR_W_RADL does not have an associated RASL picture present in the bitstream, but may have an associated RADL picture in the bitstream.

[0153] The value of nal_unit_type shall be the same for all VCL NAL units of a sub-picture. The sub-picture is said to have the same NAL unit type as the VCL NAL units of the sub-picture.

[0154] For the VCL NAL units of any particular picture, the following applies:

[0155] - If the mixed_nalu_types_in_pic_flag is equal to 0, the value of nal_unit_type shall be the same for all VCL NAL units of the picture, and the picture or PU is said to have the same NAL unit type as the coded slice NAL unit of the picture or PU.

[0156] - Otherwise (mixed_nalu_types_in_pic_flag equal to 1), the picture shall have at least two sub - pictures, and the VCL NAL units of the picture shall have exactly two different nal_unit_type values as follows: the VCL NAL units of at least one sub - picture of the picture shall all have a specific value of nal_unit_type equal to STSA_NUT, RADL_NUT, RASL_NUT, IDR_W_RADL, IDR_N_LP or CRA_NUT, while the VCL NAL units of the other sub - pictures in the picture shall all have a different specific value of nal_unit_type equal to TRAIL_NUT, RADL_NUT or RASL_NUT.

[0157] For a single - layer bitstream, the following constraints apply:

[0158] - Every picture other than the first picture in decoding order in the bitstream is considered to be associated with the previous IRAP picture in decoding order.

[0159] - When a picture is a pre - picture of an IRAP picture, it shall be a RADL or RASL picture.

[0160] - When a picture is a post - picture of an IRAP picture, it shall not be a RADL or RASL picture.

[0161] - There shall not be a RASL picture associated with an IDR picture in the bitstream.

[0162] - There shall not be a RADL picture associated with an IDR picture having a nal_unit_type equal to IDR_N_LP in the bitstream.

[0163] Note 6 – Random access can be performed at the location of an IRAP PU (and the IRAP picture and all subsequent non - RASL pictures in decoding order can be correctly decoded) by discarding all PUs before the IRAP PU, provided that each parameter set is available when it is referenced (in the bitstream or by external means not specified in this specification).

[0164] - Any picture before an IRAP picture in decoding order shall be before the IRAP picture in output order and shall be before any RADL picture associated with the IRAP picture in output order.

[0165] - Any RASL picture associated with a CRA picture shall be before any RADL picture associated with the CRA picture in output order.

[0166] - Any RASL picture associated with a CRA picture shall be after any IRAP picture before the CRA picture in decoding order in output order.

[0167] - If field_seq_flag is equal to 0 and the current picture is a pre-picture associated with an IRAP picture, it shall be before all non-pre-pictures associated with the same IRAP picture in decoding order. Otherwise, let picA and picB be the first and last pre-pictures associated with the IRAP picture in decoding order respectively. There shall be at most one non-pre-picture before picA in decoding order and there shall be no non-pre-picture between picA and picB in decoding order.

[0168] nuh_temporal_id_plus1 minus 1 specifies the temporal identifier of the NAL unit.

[0169] The value of nuh_temporal_id_plus1 shall not be equal to 0.

[0170] The variable TemporalId is derived as follows:

[0171] TemporalId = nuh_temporal_id_plus1 - 1 (36)

[0172] When nal_unit_type is in the range from IDR_W_RADL to RSV_IRAP_12 (including IDR_W_RADL and RSV_IRAP_12), TemporalId shall be equal to 0.

[0173] When nal_unit_type is equal to STSA_NUT and vps_independent_layer_flag[GeneralLayerIdx[nuh_layer_id]] is equal to 1, TemporalId shall not be equal to 0.

[0174] The value of TemporalId shall be the same for all VCL NAL units of an AU. The value of TemporalId for an encoded / decoded picture, PU, or AU is the value of TemporalId of the VCL NAL units of the encoded / decoded picture, PU, or AU. The value of TemporalId for a sub-layer representation is the maximum value of TemporalId of all VCL NAL units in the sub-layer representation.

[0175] The value of TemporalId for non-VCL NAL units is constrained as follows:

[0176] - If nal_unit_type is equal to DCI_NUT, VPS_NUT, or SPS_NUT, then TemporalId shall be equal to 0, and the TemporalId of the AU containing the NAL unit shall be equal to 0.

[0177] - Otherwise, if nal_unit_type is equal to PH_NUT, then TemporalId shall be equal to the TemporalId of the PU containing the NAL unit.

[0178] - Otherwise, if nal_unit_type is equal to EOS_NUT or EOB_NUT, then TemporalId shall be equal to 0.

[0179] - Otherwise, if nal_unit_type is equal to AUD_NUT, FD_NUT, PREFIX_SEI_NUT, or SUFFIX_SEI_NUT, then TemporalId shall be equal to the TemporalId of the AU containing the NAL unit.

[0180] - Otherwise, when nal_unit_type is equal to PPS_NUT, PREFIX_APS_NUT, or SUFFIX_APS_NUT, TemporalId shall be greater than or equal to the TemporalId of the PU containing the NAL unit.

[0181] Note 7 – When the NAL unit is a non-VCL NAL unit, the value of TemporalId is equal to the minimum of the TemporalId values of all AUs applicable to the non-VCL NAL unit. When nal_unit_type is equal to PPS_NUT, PREFIX_APS_NUT, or SUFFIX_APS_NUT, TemporalId can be greater than or equal to the TemporalId of the containing AU, since all PPSs and APSs can be included at the beginning of the bitstream (e.g., when they are transmitted out-of-band and the receiver places them at the beginning of the bitstream), where the first decoded picture has a TemporalId equal to 0.

[0182] 3.7. Picture Header Structure Syntax and Semantics in VVC

[0183] In the latest VVC text (in JVET-Q2001-vE / v15), the picture header structure syntax and semantics most relevant to the present invention in this document are as follows.

[0184] 7.3.2.7 Picture Header Structure Syntax

[0185]

[0186] 7.4.3.7 Picture Header Structure Semantics

[0187] The PH syntax structure contains information common to all slices of the decoded picture associated with the PH syntax structure.

[0188] gdr_or_irap_pic_flag being equal to 1 specifies that the current picture is a GDR or IRAP picture. gdr_or_irap_pic_flag being equal to 0 specifies that the current picture may or may not be a GDR or IRAP picture.

[0189] gdr_pic_flag being equal to 1 specifies that the picture associated with the PH is a GDR picture. gdr_pic_flag being equal to 0 specifies that the picture associated with the PH is not a GDR picture. When absent, the value of gdr_pic_flag is inferred to be equal to 0. When gdr_enabled_flag is equal to 0, the value of gdr_pic_flag shall be equal to 0.

[0190] Note 1 – When gdr_or_irap_pic_flag is equal to 1 and gdr_pic_flag is equal to 0, the picture associated with the PH is an IRAP picture. ...

[0192] ph_pic_order_cnt_lsb specifies the picture order count of the current picture modulo MaxPicOrderCntLsb. The ph_pic_order_cnt_lsb syntax element has a length of log2_max_pic_order_cnt_lsb_minus4 + 4 bits. The value of ph_pic_order_cnt_lsb shall be in the range of 0 to MaxPicOrderCntLsb - 1, inclusive (including 0 and MaxPicOrderCntLsb - 1).

[0193] no_output_of_prior_pics_flag affects the output of prior decoded pictures in the DPB after decoding a CLVSS picture that is not the first picture in the bitstream as specified in Annex C.

[0194] recovery_poc_cnt specifies the recovery point of decoded pictures in output order. If the current picture is a GDR picture associated with PH and there is a picture picA in the CLVS with a PicOrderCntVal equal to the value of PicOrderCntVal of the current GDR picture plus recovery_poc_cnt and picA is after the current GDR picture in decoding order, then picture picA is called a recovery point picture. Otherwise, the first picture in output order with a PicOrderCntVal greater than the value of PicOrderCntVal of the current picture plus recovery_poc_cnt is called a recovery point picture. The recovery point picture shall not be before the current GDR picture in decoding order. The value of recovery_poc_cnt shall be in the range of 0 to MaxPicOrderCntLsb - 1, inclusive (including 0 and MaxPicOrderCntLsb - 1).

[0195] When the current picture is a GDR picture, the variable RpPicOrderCntVal is derived as follows:

[0196] RpPicOrderCntVal = PicOrderCntVal + recovery_poc_cnt (81)

[0197] Note 2 – When gdr_enabled_flag is equal to 1 and the PicOrderCntVal of the current picture is greater than or equal to the RpPicOrderCntVal of the associated GDR picture, the current decoded picture and subsequent decoded pictures in output order exactly match the corresponding pictures generated by starting the decoding process from the prior IRAP picture (when present) before the associated GDR picture in decoding order. ...

[0199] 3.8 Constraints on RPL in VVC

[0200] In the latest VVC text (in JVET-Q2001-vE / v15), the constraints on RPL in VVC are as follows (as part of the decoding process for constructing the reference picture list in Clause 8.3.2 of VVC).

[0201] 8.3.2 Decoding process for constructing the reference picture list ...

[0203] For each i equal to 0 or 1, the first NumRefIdxActive[i] entries in RefPicList[i] are called the active entries in RefPicList[i], and the other entries in RefPicList[i] are called the inactive entries in RefPicList[i].

[0204] Note 2 – A particular picture may be referenced by an entry in RefPicList[0] and an entry in RefPicList[1]. A particular picture may also be referenced by more than one entry in RefPicList[0] or more than one entry in RefPicList[1].

[0205] Note 3 – The active entries in RefPicList[0] and the active entries in RefPicList[1] together reference all the reference pictures that can be used for inter prediction of the current picture and one or more pictures after the current picture in decoding order. The inactive entries in RefPicList[0] and the inactive entries in RefPicList[1] together reference all the reference pictures that are not used for inter prediction of the current picture but can be used for inter prediction of one or more pictures after the current picture in decoding order.

[0206] Note 4 – There may be one or more entries equal to "no reference picture" in RefPicList[0] or RefPicList[1] because the corresponding pictures do not exist in the DPB. Each inactive entry equal to "no reference picture" in RefPicList[0] or RefPicList[0] should be ignored. An unintentional picture loss should be inferred for each active entry equal to "no reference picture" in RefPicList[0] or RefPicList[1].

[0207] The requirements for bitstream conformance are that the following constraints apply:

[0208] - For each i equal to 0 or 1, num_ref_entries[i][RplsIdx[i]] shall not be less than NumRefIdxActive[i].

[0209] - Each picture referenced by an active entry in RefPicList[0] or RefPicList[1] shall be present in the DPB and shall have a TemporalId less than or equal to the TemporalId of the current picture.

[0210] - Each picture referenced by an entry in RefPicList[0] or RefPicList[1] shall not be the current picture and shall have a non_reference_picture_flag equal to 0.

[0211] - The STRP entry in RefPicList[0] or RefPicList[1] for a strip of a picture and the LTRP entry in RefPicList[0] or RefPicList[1] for the same strip or a different strip of the same picture shall not reference the same picture.

[0212] - There shall be no LTRP entry in RefPicList[0] or RefPicList[1] such that the difference between the PicOrderCntVal of the current picture and the PicOrderCntVal of the picture referenced by the entry is greater than or equal to 2 24 .

[0213] - Let setOfRefPics be the set of unique pictures referenced by all entries in RefPicList[0] having the same nuh_layer_id as the current picture and all entries in RefPicList[1] having the same nuh_layer_id as the current picture. The number of pictures in setOfRefPics shall be less than or equal to MaxDpbSize - 1 (including MaxDpbSize - 1), where MaxDpbSize is as specified in Clause A.4.2, and setOfRefPics shall be the same for all strips of the picture.

[0214] - When the current strip has a nal_unit_type equal to STSA_NUT, there shall be no active entry in RefPicList[0] or RefPicList[1] having a TemporalId equal to the TemporalId of the current picture and a nuh_layer_id equal to the nuh_layer_id of the current picture.

[0215] - When the current picture is a picture that has a TemporalId equal to the TemporalId of the current picture and a nuh_layer_id equal to the nuh_layer_id of the current picture, and that comes after the STSA picture in decoding order, pictures that come before the STSA picture in decoding order, have a TemporalId equal to the TemporalId of the current picture, and have a nuh_layer_id equal to the nuh_layer_id of the current picture shall not be included as active entries in RefPicList[0] or RefPicList[1].

[0216] - When the current picture is a CRA picture, there shall be no picture that is referenced by an entry in RefPicList[0] or RefPicList[1] and that comes before any previous IRAP picture (when present) in decoding order, either in output order or in decoding order.

[0217] - When the current picture is a trailing picture, there shall be no picture that is referenced by an active entry in RefPicList[0] or RefPicList[1] and that is generated by the decoding process of unavailable reference pictures used to generate the IRAP picture associated with the current picture.

[0218] - When the current picture is a trailing picture that comes after one or more leading pictures (if any) associated with the same IRAP picture in both decoding order and output order, there shall be no picture that is referenced by an entry in RefPicList[0] or RefPicList[1] and that is generated by the decoding process of unavailable reference pictures used to generate the IRAP picture associated with the current picture.

[0219] - When the current picture is a recovery point picture or a picture that comes after the recovery point picture in output order, there shall be no entry in RefPicList[0] or RefPicList[1] that contains a picture generated by the decoding process of unavailable reference pictures of the GDR picture used to generate the recovery point picture.

[0220] - When the current picture is a trailing picture, there shall be no picture that is referenced by an active entry in RefPicList[0] or RefPicList[1] and that comes before the associated IRAP picture, either in output order or in decoding order.

[0221] - When the current picture is a post picture that is after one or more pre pictures (if any) associated with the same IRAP picture in both decoding order and output order, there should be no picture in RefPicList[0] or RefPicList[1] that is referenced by an entry and is before the associated IRAP picture in output order or decoding order.

[0222] - When the current picture is a RADL picture, there should be no active entry in RefPicList[0] or RefPicList[1] that is any of the following:

[0223] ο RASL picture

[0224] ο A picture generated by a decoding process for generating unavailable reference pictures

[0225] ο A picture before the associated IRAP picture in decoding order

[0226] - The picture referenced by each ILRP entry in RefPicList[0] or RefPicList[1] of the current picture's slice should be in the same AU as the current picture.

[0227] - The picture referenced by each ILRP entry in RefPicList[0] or RefPicList[1] of the current picture's slice should exist in the DPB and should have a nuh_layer_id less than that of the current picture.

[0228] - Each ILRP entry in RefPicList[0] or RefPicList[1] of the slice should be an active entry. ...

[0230] 4. Technical problems solved by the disclosed technical solution

[0231] The existing designs in the latest VVC text (in JVET-Q2001-vE / v15) have the following problems:

[0232] 1) The definition of the associated IRAP picture should be updated so that the associated IRAP picture of a specific picture belongs to the same layer as the specific picture.

[0233] 2) The current definition of the post picture is as follows:

[0234] Post picture: A non-IRAP picture that is after the associated IRAP picture in output order and is not a STSA picture.

[0235] Therefore, an IRAP picture needs to exist in the bitstream for subsequent pictures to exist, and if the bitstream does not have an IRAP picture, the NAL unit type value TRAIL_NUT cannot be used. However, non-STSA pictures associated with GDR pictures need to use the NAL unit type value TRAIL_NUT.

[0236] 3) Existing constraints on the output order of pictures before the IRAP picture in decoding order need to be specified to apply only to pictures within a layer.

[0237] 4) Constraints on the relative decoding order and output order between GDR pictures and pictures before and after in decoding order are missing.

[0238] 5) Existing constraints on the decoding order of pictures associated with IRAP pictures and some non-preceding pictures need to be specified to apply only to pictures within a layer.

[0239] 6) Currently, preceding pictures, RADL pictures, and RASL pictures associated with GDR pictures are not supported.

[0240] 7) Existing constraints on the RPL of CRA pictures need to be specified to apply only to pictures within a layer.

[0241] 8) For STSA pictures, subsequent pictures associated with GDR pictures, and GDR pictures with NoOutputBeforeRecoveryFlag equal to 0, there is a lack of constraints on active entries in the RPL that are not generated by the decoding process for generating unavailable reference pictures.

[0242] 9) For STSA pictures, IDR pictures, CRA pictures with NoOutputBeforeRecoveryFlag equal to 0, etc., there is a lack of constraints on entries in the RPL that are not generated by the decoding process for generating unavailable reference pictures.

[0243] 10) For STSA pictures, there is a lack of constraints on active entries in the RPL that are not before the associated IRAP picture in output order or decoding order.

[0244] 11) For STSA pictures, there is a lack of constraints on entries in the RPL that are not before the associated IRAP picture in output order or decoding order.

[0245] 5. Examples of Embodiments and Technical Solutions

[0246] To solve the above problems and other problems, the following summarized methods are disclosed. The present invention should be considered as an example for explaining general concepts and should not be interpreted in a narrow way. In addition, these inventions can be applied separately or combined in any way.

[0247] 1) To solve Problem 1, update the definition of the associated IRAP pictures such that the associated IRAP pictures of a specific picture belong to the same layer as the specific picture.

[0248] 2) To solve Problem 2, update the definition of the post pictures such that the post pictures can also be associated with GDR pictures.

[0249] a. Additionally, add the definition of the associated GDR pictures and update the definition of the associated IRAP pictures such that, except for the first picture of the layer in the bitstream, each picture of the layer is designated to be associated with the previous IRAP or GDR picture in the same layer in decoding order, whichever is closer.

[0250] b. Additionally, add a constraint to require that the post pictures should be after the associated IRAP or GDR pictures in output order.

[0251] 3) To solve Problem 3, update the existing constraint on the output order of the pictures before the IRAP pictures in decoding order such that it only imposes restrictions on the pictures within the layer.

[0252] a. In one example, the constraint is specified as follows: Any picture with nuh_layer_id equal to a specific value layerId and before the IRAP picture with nuh_layer_id equal to layerId in decoding order should be before the IRAP picture and all its associated RADL pictures in output order.

[0253] 4) To solve Problem 4, add one or more of the following constraints:

[0254] a. The post pictures should be after the associated IRAP or GDR pictures in output order.

[0255] b. Any picture with nuh_layer_id equal to a specific value layerId and before the GDR picture with nuh_layer_id equal to layerId in decoding order should be before the GDR picture and all its associated pictures in output order.

[0256] 5) To solve Problem 5, update the existing constraint on the decoding order of the pictures associated with the IRAP pictures and some non-pre pictures such that it only imposes restrictions on the pictures within the layer.

[0257] a. In one example, the constraint is specified as follows: If field_seq_flag is equal to 0, and the current picture with nuh_layer_id equal to a specific value layerId is a previous picture associated with an IRAP picture, then it should be before all non-previous pictures associated with the same IRAP picture in decoding order. Otherwise, let picA and picB be the first and last previous pictures associated with the IRAP picture in decoding order respectively, then there should be at most one non-previous picture with nuh_layer_id equal to layerId that is before picA in decoding order, and there should be no non-previous pictures with nuh_layer_id equal to layerId between picA and picB in decoding order.

[0258] b. In another example, the constraint is specified as follows: If field_seq_flag is equal to 0, and the current picture is a previous picture associated with an IRAP picture, then it should be before all non-previous pictures associated with the same IRAP picture in decoding order. Otherwise, let picA and picB be the first and last previous pictures associated with the IRAP picture in decoding order respectively, then there should be at most one non-previous picture associated with the IRAP picture that is before picA in decoding order, and there should be no non-previous pictures associated with the IRAP picture between picA and picB in decoding order.

[0259] 6) To solve Problem 6, define and specify the previous pictures, RADL pictures, and RASL pictures associated with GDR pictures.

[0260] a. The previous pictures associated with a GDR picture are those pictures that are after the GDR picture in decoding order and before the GDR picture in output order.

[0261] b. The RADL pictures associated with a GDR picture are the previous pictures associated with the GDR picture and having a nal_unit_type equal to RADL_NUT.

[0262] c. The RASL pictures associated with a GDR picture are the previous pictures associated with the GDR picture and having a nal_unit_type equal to RASL_NUT.

[0263] 7) To solve Problem 7, update the existing constraint on the RPL of CRA pictures such that it only imposes restrictions on the pictures within a layer.

[0264] a. In one example, the constraint is specified as follows: When the current picture with nuh_layer_id equal to a specific value layerId is a CRA picture, there should be no picture referenced by an entry in RefPicList[0] or RefPicList[1] that is before any previous IRAP picture (when present) with nuh_layer_id equal to layerId in decoding order or output order and before the current picture in decoding order.

[0265] 8) To solve Problem 8, the following constraint is specified:

[0266] When the current picture with nuh_layer_id equal to a specific value layerId is not a RASL picture associated with a CRA picture with NoOutputBeforeRecoveryFlag equal to 1, a GDR picture with NoOutputBeforeRecoveryFlag equal to 1, or a recovery picture of a GDR picture with NoOutputBeforeRecoveryFlag equal to 1 and nuh_layer_id equal to layerId, there should be no picture referenced by an active entry in RefPicList[0] or RefPicList[1] that is generated by the decoding process for generating unavailable reference pictures.

[0267] 9) To solve Problem 9, the following constraint is specified:

[0268] When the current picture with nuh_layer_id equal to a specific value layerId is not a CRA picture with NoOutputBeforeRecoveryFlag equal to 1, a picture before a previous picture associated with the same CRA picture with NoOutputBeforeRecoveryFlag equal to 1 in decoding order, a previous picture associated with a CRA picture with NoOutputBeforeRecoveryFlag equal to 1, a GDR picture with NoOutputBeforeRecoveryFlag equal to 1, or a recovery picture of a GDR picture with NoOutputBeforeRecoveryFlag equal to 1 and nuh_layer_id equal to layerId, there should be no picture referenced by an entry in RefPicList[0] or RefPicList[1] that is generated by the decoding process for generating unavailable reference pictures.

[0269] 10) To solve Problem 10, the following constraint is specified:

[0270] When the current picture is associated with an IRAP picture and is after the IRAP picture in output order, there shall be no picture in RefPicList[0] or RefPicList[1] that is referenced by an active entry and that is before the associated IRAP picture in output order or decoding order.

[0271] 11) To solve problem 11, the following constraints are specified:

[0272] When the current picture is associated with an IRAP picture, is after the IRAP picture in output order, and is after the preceding picture (if any) associated with the same IRAP picture in both decoding order and output order, there shall be no picture in RefPicList[0] or RefPicList[1] that is referenced by an entry and that is before the associated IRAP picture in output order or decoding order.

[0273] 6. Embodiments

[0274] The following are some example embodiments of some aspects of the present invention summarized in Section 5 above, which can be applied to the VVC specification. The changed text is based on the latest VVC text in JVET-Q2001-vE / v15. The most relevant parts that have been added or modified are highlighted in bold italics, and some deleted parts are marked with double brackets (e.g., [[a]] means deleting the character 'a'). There are some other changes of an editorial nature and thus not highlighted.

[0275] 6.1. First Embodiment

[0276] This embodiment addresses items 1, 2, 3, 4, 5, and 5a.

[0277] 3 Definitions ...

[0279] Associated GDR picture (of a specific picture with a specific value layerId of nuh_layer_id): The previous GDR picture in decoding order (when it exists) with nuh_layer_id equal to layerId, where there is no IRAP picture with nuh_layer_id equal to layerId between this picture and the specific picture in decoding order.

[0280] Associated IRAP picture (of a specific picture with a specific value layerId of nuh_layer_id): The previous IRAP picture in decoding order (when it exists) with nuh_layer_id equal to layerId, where there is no GDR picture with nuh_layer_id equal to layerId between this picture and the specific picture in decoding order.

[0281] Output order: The order of pictures or sub - pictures within a CLVS as indicated by increasing POC values, and for decoded pictures output from the DPB, this is the order in which the decoded pictures are output from the DPB.

[0282] Trailing picture: A picture for which each VCL NAL unit has a nal_unit_type equal to TRAIL_NUT.

[0283] Note – Trailing pictures associated with an IRAP or GDR picture are also after the IRAP or GDR picture in decoding order. A picture that is after the associated IRAP or GDR picture in output order and before the associated IRAP or GDR picture in decoding order is not allowed. ...

[0285] 7.4.2.2 NAL unit header semantics ...

[0287] [[For a single - layer bitstream, the requirements for bitstream conformance are that the following constraints apply:

[0288] - Every picture except the first picture in decoding order in the bitstream is considered to be associated with the previous IRAP picture in decoding order.]]

[0289] - Trailing pictures shall be after the associated IRAP or GDR picture in output order.

[0290] - When a picture is a leading picture of an IRAP picture, it shall be a RADL or RASL picture.

[0291] - There shall be no RASL pictures associated with an IDR picture in the bitstream.

[0292] - There shall be no RADL pictures associated with an IDR picture having a nal_unit_type equal to IDR_N_LP in the bitstream.

[0293] Note 6 – Random access can be performed at the position of an IRAP PU (and the IRAP picture and all subsequent non - RASL pictures in decoding order can be correctly decoded) by discarding all PUs before the IRAP PU, provided that each parameter set is available when it is referenced (in the bitstream or by external means not specified in this specification).

[0294] - Any picture with nuh_layer_id equal to a specific value layerId that is before an IRAP picture with nuh_layer_id equal to layerId in decoding order shall be before the IRAP picture and all its associated RADL pictures in output order.

[0295] Any picture with -nuh_layer_id equal to a specific value layerId and preceding the GDR picture with nuh_layer_id equal to layerId in decoding order shall be before the GDR picture and all its associated pictures in output order.

[0296] Any RASL picture associated with a CRA picture shall be before any RADL picture associated with the CRA picture in output order.

[0297] Any RASL picture associated with a CRA picture shall be after any IRAP picture preceding the CRA picture in decoding order in output order.

[0298] - If field_seq_flag equals 0 and the current picture with nuh_layer_id equal to a specific value layerId is a preceding picture associated with an IRAP picture, it shall be before all non - preceding pictures associated with the same IRAP picture in decoding order. Otherwise, let picA and picB be the first and last preceding pictures associated with the IRAP picture in decoding order respectively. Then, there shall be at most one non - preceding picture with nuh_layer_id equal to layerId before picA in decoding order, and there shall be no non - preceding picture with nuh_layer_id equal to layerId between picA and picB in decoding order. ...

[0300] 7.4.3.7 Picture header structure semantics ...

[0302] recovery_poc_cnt specifies the recovery point of decoded pictures in output order.

[0303] When the current picture is a GDR picture, the variable recoveryPointPocVal is derived as follows:

[0304] recoveryPointPocVal = PicOrderCntVal+recovery_poc_cnt (81)

[0305] If the current picture is a GDR picture associated with PH, and there is a picture picA in the CLVS with a PicOrderCntVal equal to recoveryPointPocVal [[the value of PicOrderCntVal of the current GDR picture plus recovery_poc_cnt]] and following the current GDR picture in decoding order, then the picture picA is called a recovery point picture. Otherwise, the first picture in the CLVS with a PicOrderCntVal greater than recoveryPointPocVal [[the value of PicOrderCntVal of the current picture plus recovery_poc_cnt]] in output order is called a recovery point picture. The recovery point picture shall not be before the current GDR picture in decoding order. Pictures associated with the current GDR picture and having a PicOrderCntVal less than recoveryPointPocVal are called recovery pictures of the GDR picture. The value of recovery_poc_cnt shall be in the range from 0 to MaxPicOrderCntLsb - 1 (including 0 and MaxPicOrderCntLsb - 1).

[0306] [[When the current picture is a GDR picture, the variable RpPicOrderCntVal is derived as follows:

[0307] RpPicOrderCntVal = PicOrderCntVal + recovery_poc_cnt(81)]

[0308] Note 2 – When the gdr_enabled_flag is equal to 1 and the PicOrderCntVal of the current picture is greater than or equal to the recoveryPointPocVal [[RpPicOrderCntVal]] of the associated GDR picture, the current decoded picture and subsequent decoded pictures in output order exactly match the corresponding pictures generated by starting the decoding process from the previous IRAP picture (when present) before the associated GDR picture in decoding order. ...

[0310] 8.3.2 Decoding Process for Reference Picture List Construction ...

[0312] The requirements for bitstream conformance are that the following constraints apply:

[0313] -...

[0314] - When the current picture with nuh_layer_id equal to a specific value layerId is a CRA picture, there should be no picture referred to by an entry in RefPicList[0] or RefPicList[1] that is an earlier IRAP picture (when present) with nuh_layer_id equal to layerId in decoding order and that is before any previous IRAP picture in output order or decoding order.

[0315] or a picture referred to by an entry in RefPicList[1].

[0316] - When the current picture with nuh_layer_id equal to a specific value layerId [[is a post picture]] is not a RASL picture associated with a CRA picture with NoOutputBeforeRecoveryFlag equal to 1, a GDR picture with NoOutputBeforeRecoveryFlag equal to 1, or a recovery picture of a GDR picture with NoOutputBeforeRecoveryFlag equal to 1 and nuh_layer_id equal to layerId, there should be no picture referred to by an active entry in RefPicList[0] or RefPicList[1] that is generated by the decoding process of an unavailable reference picture used to generate [[the IRAP picture associated with the current picture]].

[0317] - When the current picture with nuh_layer_id equal to a specific value layerId [[is a post picture that is after one or more pre pictures (if any) associated with the same IRAP picture in both decoding order and output order]] is not a CRA picture with NoOutputBeforeRecoveryFlag equal to 1, a picture that is before a pre picture associated with the same CRA picture with NoOutputBeforeRecoveryFlag equal to 1 in decoding order, a pre picture associated with a CRA picture with NoOutputBeforeRecoveryFlag equal to 1, a GDR picture with NoOutputBeforeRecoveryFlag equal to 1, or a recovery picture of a GDR picture with NoOutputBeforeRecoveryFlag equal to 1 and nuh_layer_id equal to layerId, there should be no picture referred to by an entry in RefPicList[0] or RefPicList[1] that is generated by the decoding process of an unavailable reference picture used to generate [[the IRAP picture associated with the current picture]].

[0318] - When the current picture is a recovery point picture or a picture that comes after the recovery point picture in the output order, there should be no entry in RefPicList[0] or RefPicList[1] that contains a picture generated by the decoding process of an unavailable reference picture that includes the GDR picture used to generate the recovery point picture.

[0319] - When the current picture [[is a post-picture]] is associated with an IRAP picture and comes after the IRAP picture in the output order, there should be no picture that is referenced by an active entry in RefPicList[0] or RefPicList[1] and that comes before the associated IRAP picture in the output order or decoding order.

[0320] - When the current picture [[is]] is associated with an IRAP picture, comes after the IRAP picture in the output order, and comes after the pre-picture (if any) associated with the same IRAP picture in both the decoding order and the output order [[as a post-picture]], there should be no picture that is referenced by an entry in RefPicList[0]

[0321] or RefPicList[1].

[0322] - When the current picture is a RADL picture, there should be no active entry in RefPicList[0] or RefPicList[1] that is any of the following:

[0323] ο RASL picture

[0324] [[a picture generated by the decoding process used to generate an unavailable reference picture]]

[0325] ο a picture that comes before the associated IRAP picture in the decoding order

[0326] -...

[0327] Figure 1 is a block diagram showing an example video processing system 1900 in which various techniques disclosed herein can be implemented. Various embodiments may include some or all components of system 1900. System 1900 may include an input 1902 for receiving video content. The video content may be received in a raw or uncompressed format, such as 8 or 10-bit multi-component pixel values, or may be in a compressed or encoded format. Input 1902 may represent a network interface, a peripheral bus interface, or a storage interface. Examples of network interfaces include wired interfaces such as Ethernet, passive optical network (PON), etc., and wireless interfaces such as Wi-Fi or cellular interfaces.

[0328] System 1900 may include a codec component 1904 that may implement various codec or encoding methods described in this document. The codec component 1904 may reduce the average bit rate of the video from input 1902 to the output of the codec component 1904 to produce a coded representation of the video. Codec techniques are thus sometimes referred to as video compression or video transcoding techniques. The output of the codec component 1904 may be stored or transmitted via a communication connection represented by component 1906. The stored or communicatively transmitted bitstream (or coded) representation of the video received at input 1902 may be used by component 1908 to generate pixel values or a displayable video that is transmitted to the display interface 1910. The process of generating a user-viewable video from the bitstream representation is sometimes referred to as video decompression. Additionally, while certain video processing operations are referred to as “codec” operations or tools, it will be understood that codec tools or operations are used at the encoder and the corresponding decoding tools or operations that reverse the codec results will be performed by the decoder.

[0329] Examples of a peripheral bus interface or a display interface may include Universal Serial Bus (USB), or High-Definition Multimedia Interface (HDMI), or Displayport, etc. Examples of a storage interface include SATA (Serial Advanced Technology Attachment), PCI, IDE interface, etc. The techniques described in this document may be embodied in various electronic devices, such as a mobile phone, a laptop computer, a smartphone, or other devices capable of performing digital data processing and / or video display.

[0330] Figure 2 is a block diagram of a video processing apparatus 3600. The apparatus 3600 may be used to implement one or more methods described herein. The apparatus 3600 may be embodied in a smartphone, a tablet computer, a computer, an Internet of Things (IoT) receiver, etc. The apparatus 3600 may include one or more processors 3602, one or more memories 3604, and video processing hardware 3606. The (multiple) processors 3602 may be configured to implement one or more methods described in this document. The memory (multiple memories) 3604 may be used to store data and code for implementing the methods and techniques described herein. The video processing hardware 3606 may be used to implement some of the techniques described in this document in hardware circuitry.

[0331] Figure 4 is a block diagram showing an example video codec system 100 that may utilize the techniques of the present disclosure.

[0332] As Figure 4As shown, the video encoding and decoding system 100 may include a source device 110 and a destination device 120. The source device 110 generates encoded video data, where the source device 110 may be referred to as a video encoding device. The destination device 120 may decode the encoded video data generated by the source device 110, where the destination device 120 may be referred to as a video decoding device.

[0333] The source device 110 may include a video source 112, a video encoder 114, and an input / output (I / O) interface 116.

[0334] The video source 112 may include sources such as a video capture device, an interface for receiving video data from a video content provider, and / or a computer graphics system for generating video data, or a combination of these sources. The video data may include one or more pictures. The video encoder 114 encodes the video data from the video source 112 to generate a bitstream. The bitstream may include a sequence of bits forming a coded representation of the video data. The bitstream may include coded pictures and associated data. A coded picture is a coded representation of a picture. The associated data may include a sequence parameter set, a picture parameter set, and other syntax structures. The I / O interface 116 may include a modulator / demodulator (modem) and / or a transmitter. The encoded video data may be sent directly to the destination device 120 via the I / O interface 116 over a network 130a. The encoded video data may also be stored on a storage medium / server 130b for access by the destination device 120.

[0335] The destination device 120 may include an I / O interface 126, a video decoder 124, and a display device 122.

[0336] The I / O interface 126 may include a receiver and / or a modem. The I / O interface 126 may obtain the encoded video data from the source device 110 or the storage medium / server 130b. The video decoder 124 may decode the encoded video data. The display device 122 may display the decoded video data to a user. The display device 122 may be integrated with the destination device 120 or may be external to the destination device 120 configured to interface with an external display device.

[0337] The video encoder 114 and the video decoder 124 may operate according to a video compression standard, such as the High Efficiency Video Coding (HEVC) standard, the Versatile Video Coding (VVC) standard, and other current and / or additional standards.

[0338] Figure 5 is a block diagram showing an example of a video encoder 200, which may be the video encoder 114 in the system 100 shown Figure 4 in the system 100 as shown.

[0339] Video encoder 200 may be configured to perform any or all of the techniques of the present disclosure. In Figure 5 the example, video encoder 200 includes multiple functional components. The techniques described in the present disclosure may be shared among various components of video encoder 200. In some examples, a processor may be configured to perform any or all of the techniques described in the present disclosure.

[0340] The functional components of video encoder 200 may include a splitting unit 201, a prediction unit 202 (which may include a mode selection unit 203, a motion estimation unit 204, a motion compensation unit 205, and an intra prediction unit 206), a residual generation unit 207, a transformation unit 208, a quantization unit 209, an inverse quantization unit 210, an inverse transformation unit 211, a reconstruction unit 212, a buffer 213, and an entropy coding unit 214.

[0341] In other examples, video encoder 200 may include more, fewer, or different functional components. In an example, prediction unit 202 may include an intra block copy (IBC) unit. The IBC unit may perform prediction in the IBC mode, where at least one reference picture is the picture in which the current video block is located.

[0342] In addition, some components such as motion estimation unit 204 and motion compensation unit 205 may be highly integrated, but are shown separately in the Figure 5 example for purposes of explanation.

[0343] Splitting unit 201 may split a picture into one or more video blocks. Video encoder 200 and video decoder 300 may support various video block sizes.

[0344] Mode selection unit 203 may select one of the coding / decoding modes (e.g., intra or inter) based on error results, and provide the resulting intra-coded / decoded block or inter-coded / decoded block to residual generation unit 207 to generate residual block data, and to reconstruction unit 212 to reconstruct the coded block to be used as a reference picture. In some examples, mode selection unit 203 may select a combination of intra and inter prediction modes (CIIP), where the prediction is based on an inter prediction signal and an intra prediction signal. In the case of inter prediction, mode selection unit 203 may also select the resolution of the motion vector of the block (e.g., sub-pixel or integer-pixel accuracy).

[0345] To perform inter prediction on the current video block, the motion estimation unit 204 may generate motion information for the current video block by comparing one or more reference frames from buffer 213 with the current video block. The motion compensation unit 205 may determine a predicted video block for the current video block based on the motion information and decoded samples of pictures from buffer 213 other than the picture associated with the current video block.

[0346] The motion estimation unit 204 and the motion compensation unit 205 may perform different operations on the current video block, e.g., depending on whether the current video block is in an I-slice, a P-slice, or a B-slice.

[0347] In some examples, the motion estimation unit 204 may perform uni-directional prediction on the current video block, and the motion estimation unit 204 may search for a reference picture in list 0 or list 1 of reference pictures for the current video block. The motion estimation unit 204 may then generate a reference index indicating the reference picture in list 0 or list 1 that includes the reference video block and a motion vector indicating a spatial displacement between the current video block and the reference video block. The motion estimation unit 204 may output the reference index, a prediction direction indicator, and the motion vector as the motion information for the current video block. The motion compensation unit 205 may generate a predicted video block for the current block based on the reference video block indicated by the motion information of the current video block.

[0348] In other examples, the motion estimation unit 204 may perform bi-directional prediction on the current video block. The motion estimation unit 204 may search for a reference video block of the current video block among the reference pictures in list 0 and may also search for another reference video block of the current video block in list 1. The motion estimation unit 204 may then generate a reference index that indicates the reference pictures in list 0 and list 1 that include the reference video block and a motion vector indicating a spatial displacement between the reference video block and the current video block. The motion estimation unit 204 may output the reference index and the motion vector of the current video block as the motion information for the current video block. The motion compensation unit 205 may generate a predicted video block for the current video block based on the reference video block indicated by the motion information of the current video block.

[0349] In some examples, the motion estimation unit 204 may output a complete set of motion information for use in the decoding process of the decoder.

[0350] In some examples, the motion estimation unit 204 may not output a complete set of motion information for the current video. Instead, the motion estimation unit 204 may signal the motion information of the current video block by referring to the motion information of another video block. For example, the motion estimation unit 204 may determine that the motion information of the current video block is similar enough to the motion information of a neighboring video block.

[0351] In one example, the motion estimation unit 204 may indicate a value in a syntax structure associated with the current video block, and the value indicates to the video decoder 300 that the current video block has the same motion information as another video block.

[0352] In another example, the motion estimation unit 204 may identify another video block and a motion vector difference (MVD) in a syntax structure associated with the current video block. The motion vector difference indicates the difference between the motion vector of the current video block and the motion vector of the indicated video block. The video decoder 300 may use the motion vector of the indicated video block and the motion vector difference to determine the motion vector of the current video block.

[0353] As discussed above, the video encoder 200 may predictively signal motion vectors. Two examples of predictive signaling techniques that may be implemented by the video encoder 200 include advanced motion vector prediction (AMVP) and Merge mode signaling.

[0354] The intra prediction unit 206 may perform intra prediction on the current video block. When the intra prediction unit 206 performs intra prediction on the current video block, the intra prediction unit 206 may generate prediction data for the current video block based on decoded samples of other video blocks in the same picture. The prediction data for the current video block may include a predicted video block and various syntax elements.

[0355] The residual generation unit 207 may generate residual data for the current video block by subtracting (e.g., indicated by a minus sign) the (multiple) predicted video blocks of the current video block from the current video block. The residual data for the current video block may include residual video blocks corresponding to different sample components of the samples in the current video block.

[0356] In other examples, such as in the skip mode, there may be no residual data for the current video block, and the residual generation unit 207 may not perform the subtraction operation.

[0357] The transform processing unit 208 may generate one or more transform coefficient video blocks for the current video block by applying one or more transforms to the residual video block associated with the current video block.

[0358] After the transform processing unit 208 generates the transform coefficient video block associated with the current video block, the quantization unit 209 may quantize the transform coefficient video block associated with the current video block based on one or more quantization parameter (QP) values associated with the current video block.

[0359] The inverse quantization unit 210 and the inverse transform unit 211 can respectively apply inverse quantization and inverse transform to the transformed coefficient video block to reconstruct the residual video block from the transformed coefficient video block. The reconstruction unit 212 can add the reconstructed residual video block to the corresponding samples of one or more predicted video blocks generated by the prediction unit 202 to generate a reconstructed video block associated with the current block for storage in the buffer 213.

[0360] After the reconstruction unit 212 reconstructs the video block, a loop filtering operation can be performed to reduce the video block effect in the video block.

[0361] The entropy encoding unit 214 can receive data from other functional components of the video encoder 200. When the entropy encoding unit 214 receives the data, the entropy encoding unit 214 can perform one or more entropy encoding operations to generate entropy encoded data and output a bitstream including the entropy encoded data.

[0362] Figure 6 is a block diagram showing an example of a video decoder 300, and the video decoder 300 can be Figure 4 the video decoder 114 in the system 100 shown.

[0363] The video decoder 300 can be configured to perform any or all of the techniques of the present disclosure. In Figure 6 the example, the video decoder 300 includes a plurality of functional components. The techniques described in the present disclosure can be shared among various components of the video decoder 300. In some examples, the processor can be configured to perform any or all of the techniques described in the present disclosure.

[0364] In Figure 6 the example, the video decoder 300 includes an entropy decoding unit 301, a motion compensation unit 302, an intra prediction unit 303, an inverse quantization unit 304, an inverse transform unit 305, a reconstruction unit 306, and a buffer 307. In some examples, the video decoder 300 can perform a decoding process that is generally opposite to the encoding process described for the video encoder 200 ( Figure 5 ).

[0365] The entropy decoding unit 301 can retrieve the encoded bitstream. The encoded bitstream can include entropy encoded video data (e.g., encoded blocks of video data). The entropy decoding unit 301 can decode the entropy encoded video data, and from the entropy decoded video data, the motion compensation unit 302 can determine motion information including motion vectors, motion vector precision, reference picture list indices, and other motion information. The motion compensation unit 302 can determine such information, for example, by performing AMVP and Merge modes.

[0366] The motion compensation unit 302 may generate motion-compensated blocks and may perform interpolation based on an interpolation filter. Identifiers of interpolation filters to be used with sub-pixel accuracy may be included in syntax elements.

[0367] The motion compensation unit 302 may use an interpolation filter such as that used by the video encoder 200 during the encoding of video blocks to compute the interpolation of sub-integer pixels of a reference block. The motion compensation unit 302 may determine the interpolation filter used by the video encoder 200 according to received syntax information and use that interpolation filter to generate a prediction block.

[0368] The motion compensation unit 302 may use some syntax information to determine the size of blocks used to encode (multiple) frames and / or (multiple) slices of an encoded video sequence, partitioning information that describes how each macroblock of a picture describing the encoded video sequence is partitioned, the modes indicating how each partition is encoded, one or more reference frames (and reference frame lists) for each inter-coded block, and other information for decoding the encoded video sequence.

[0369] The intra prediction unit 303 may form a prediction block from spatially adjacent blocks using, for example, an intra prediction mode received in a bitstream. The inverse quantization unit 303 inverse quantizes the quantized video block coefficients provided in the bitstream and decoded by the entropy decoding unit 301, i.e., de-quantizes. The inverse transform unit 303 applies an inverse transform.

[0370] The reconstruction unit 306 may add a residual block to the corresponding prediction block generated by the motion compensation unit 202 or the intra prediction unit 303 to form a decoded block. If necessary, a de-blocking filter may also be applied to filter the decoded block to remove block artifacts. The decoded video blocks are then stored in a buffer 307 to provide reference blocks for subsequent motion compensation / intra prediction and also to produce decoded video for presentation on a display device.

[0371] Next, a list of preferred examples of some embodiments is provided.

[0372] A first set of clauses illustrates example embodiments of the techniques discussed in the previous section. The following clauses illustrate example embodiments of the techniques discussed in the previous section (e.g., item 1).

[0373] 1. A video processing method (e.g., Figure 3 method 3000 as shown), comprising: performing a conversion (3002) between a video and a coded representation of the video having one or more video layers including one or more video pictures; wherein the coded representation is organized according to rules that specify that a first video picture and a second picture that is an intra-random access point picture are constrained to belong to the same video layer.

[0374] The following clauses illustrate example embodiments of the techniques discussed in the previous section (e.g., item 2).

[0375] 2. A video processing method, comprising: performing a conversion between a video having one or more video layers including one or more video pictures and a coded representation of the video, wherein the coded representation conforms to a format rule that specifies that a post picture following a first type of picture that is an intra random access point in the coded representation is also permitted to be associated with a second type of picture including a progressive decode refresh picture.

[0376] 3. The method according to clause 2, wherein the format rule further specifies that for each layer, except for the first picture of the layer in the bitstream, each picture of the layer is designated to be associated with the one that is closer in decoding order among the previous intra random access point or progressive decoder refresh picture of the same layer.

[0377] The following clauses illustrate example embodiments of the techniques discussed in the previous section (e.g., item 3).

[0378] 4. A video processing method, comprising: performing a conversion between a video having one or more video layers including one or more video pictures and a coded representation of the video, wherein the coded representation conforms to a format rule that specifies a constraint on the output order of pictures that are before an intra random access point in decoding order, such that the output order applies only to pictures in the same video layer.

[0379] 5. The method according to clause 1, wherein the constraint specifies that any picture that has a nuh_layer_id equal to a specific value layerId and is before an intra random access point picture with a nuh_layer_id equal to layerId in decoding order is required to be in the output order before the intra random access point picture and all associated randomly accessible decodable pre-pictures.

[0380] The following clauses illustrate example embodiments of the techniques discussed in the previous section (e.g., item 4).

[0381] 6. A video processing method, comprising: performing a conversion between a video having one or more video layers including one or more video pictures and a coded representation of the video, wherein the coded representation conforms to a format rule that specifies the following constraints: (1) a post picture must be after an associated intra random access point picture (IRAP) or progressive decoder refresh (GDR) picture in output order, or (2) a picture having the same layer id as a GDR picture must be before the GDR picture and all associated pictures of the GDR picture in output order.

[0382] The following clauses illustrate example embodiments of the techniques discussed in the previous section (e.g., item 5).

[0383] 7. A video processing method, comprising: performing a conversion between a video having one or more video layers each including one or more video pictures and a coded representation of the video, wherein the conversion complies with a rule that order constraints apply to pictures, instant random access point (IRAP) pictures, and non-preceding pictures if and only if the pictures, IRAP pictures, and non-preceding pictures are in the same layer, and wherein the rule is one of the following:

[0384] (a) A first rule specifying values of field sequences and decoding order, or

[0385] (b) The order of preceding and / or non-preceding pictures of a layer.

[0386] The following clauses illustrate example embodiments of the techniques discussed in the previous section (e.g., item 6).

[0387] 8. A video processing method, comprising: performing a conversion between a video having one or more video layers each including one or more video pictures and a coded representation of the video, wherein the conversion complies with a rule specifying the order of preceding pictures, randomly accessible decodable preceding (RADL) pictures, and randomly accessible skipped preceding (RASL) pictures associated with gradual decoding refresh (GDR) pictures.

[0388] The following clauses illustrate example embodiments of the techniques discussed in the previous section (e.g., item 7).

[0389] 9. A video processing method, comprising: performing a conversion between a video having one or more video layers each including one or more video pictures and a coded representation of the video, wherein the conversion complies with a rule specifying that constraints on reference picture lists for fully randomly accessible pictures are restricted to a layer.

[0390] 10. The method according to clause 9, wherein the constraint specifies that, for a layer having a fully randomly accessible picture, the preceding instant random access point picture in decoding or output order is not referenced by an entry in the reference picture list.

[0391] The following clauses illustrate example embodiments of the techniques discussed in the previous section (e.g., item 8).

[0392] 11. A video processing method, comprising: performing a conversion between a video having one or more video layers each including one or more video pictures and a coded representation of the video, wherein the conversion complies with a rule specifying conditions under which a current picture is allowed to reference an entry in a reference picture list generated by a decoding process for generating unavailable reference pictures.

[0393] 12. The method according to clause 11, wherein the condition is that the current picture is a random access skipped pre-RASL picture associated with a full random access CRA picture with NoOutputBeforeRecoveryFlag equal to 1, a progressive decoder refresh GDR picture with NoOutputBeforeRecoveryFlag equal to 1, or a recovery picture of a GDR picture with NoOutputBeforeRecoveryFlag equal to 1.

[0394] The following clauses illustrate example embodiments of the techniques discussed in the previous section (e.g., items 9, 10, 11).

[0395] 13. A video processing method, comprising: performing a conversion between a video having one or more video layers including one or more video pictures and a coded representation of the video, wherein the conversion complies with rules regarding the order between the current picture and a reference picture list corresponding to the current picture.

[0396] 14. The method according to clause 13, wherein the rules specify that when the current picture with nuh_layer_id equal to a specific value layerId is not a full random access (CRA) picture with NoOutputBeforeRecoveryFlag equal to 1, a picture before a pre-picture associated with the same CRA picture with NoOutputBeforeRecoveryFlag equal to 1 in decoding order, a pre-picture associated with a CRA picture with NoOutputBeforeRecoveryFlag equal to 1, a progressive decoder refresh GDR picture with NoOutputBeforeRecoveryFlag equal to 1, or a recovery picture of a GDR picture with NoOutputBeforeRecoveryFlag equal to 1 and nuh_layer_id equal to layerId, there should be no picture referenced by an entry in RefPicList[0] or RefPicList[1] generated by a decoding process for generating unavailable reference pictures.

[0397] 15. The method according to clause 13, wherein the rules specify that when the current picture is associated with an intra random access point IRAP picture and is after the IRAP picture in output order, there should be no picture referenced by an active entry in RefPicList[0] or RefPicList[1] before the associated IRAP picture in output order or decoding order.

[0398] 16. The method according to clause 13, wherein the rule specifies that when the current picture is associated with an Intra Random Access Point (IRAP) picture, is after the IRAP picture in the output order, and is after the preceding picture (if any) associated with the same IRAP picture both in the decoding order and the output order, there shall be no picture in RefPicList[0] or RefPicList[1] that is referenced by an entry and is before the associated IRAP picture either in the output order or the decoding order.

[0399] 17. The method according to any one of clauses 1 to 16, wherein the conversion includes encoding the video into a codec representation.

[0400] 18. The method according to any one of clauses 1 to 16, wherein the conversion includes decoding the codec representation to generate pixel values of the video.

[0401] 19. A video decoding apparatus, comprising a processor configured to implement the method according to one or more of clauses 1 to 18.

[0402] 20. A video encoding apparatus, comprising a processor configured to implement the method according to one or more of clauses 1 to 18.

[0403] 21. A computer program product storing computer code, which when executed by a processor causes the processor to implement the method according to any one of clauses 1 to 18.

[0404] 22. A method, apparatus, or system described in this document.

[0405] The second set of clauses shows example embodiments of the techniques discussed in the previous section (e.g., items 1 - 7).

[0406] 1. A method for video processing (e.g., method 710 as Figure 7A shown), comprising: performing a conversion 712 between a video having one or more video layers including one or more video pictures and a bitstream of the video according to format rules, and wherein the format rules specify that a first video picture and a second picture, which is an associated Intra Random Access Point picture as the second picture, are constrained to belong to the same video layer.

[0407] 2. The method according to clause 1, wherein, between the first video picture and the second picture in the decoding order, there is no gradual decoding refresh picture in the same video layer.

[0408] 3. The method according to clause 1, wherein the first video picture and the second picture have the same identifier of the layer to which the video codec layer network abstraction layer unit belongs or the same identifier of the layer applicable to the non-video codec layer network abstraction layer unit.

[0409] 4. The method according to any one of clauses 1-3, wherein, between the first video picture and the second picture in the decoding order, there is no progressive decoding refresh picture having the same identifier.

[0410] 5. The method according to clause 1, wherein the format rule further specifies that the post picture is after the associated intra random access point picture or progressive decoding refresh picture in the output order.

[0411] 6. A method for video processing (e.g., the method 720 as shown in Figure 7B ), including: performing a conversion 722 between a video and a bitstream of the video having one or more video layers including one or more video pictures according to a format rule, and wherein the format rule specifies that a post picture in the bitstream is permitted to be associated with a progressive decoding refresh picture.

[0412] 7. The method according to clause 6, wherein the post picture is a picture in which each video codec layer network abstraction layer unit has a post network abstraction layer unit type.

[0413] 8. The method according to clause 6 or 7, wherein the post picture is permitted to be associated with an intra random access point picture.

[0414] 9. The method according to any one of clauses 6-8, wherein the post picture associated with the intra random access point picture or the progressive decoding refresh picture is after the intra random access point picture or the progressive decoding refresh picture in the decoding order.

[0415] 10. The method according to any one of clauses 6-8, wherein a picture that is after the associated intra random access point picture in the output order and before the associated intra random access point picture in the decoding order is not allowed.

[0416] 11. The method according to clause 6, wherein the format rule further specifies that, for each layer, except for the first picture of the layer in the bitstream, each picture of the layer is designated to be associated with the one of the previous intra random access point or progressive decoder refresh picture of the same layer that is closer in the decoding order.

[0417] 12. The method according to clause 6 or 11, wherein the post picture is required to be after the associated intra random access point or progressive decoder refresh picture in the output order.

[0418] 13. A method for video processing (e.g., method 730 as shown in Figure 7C ), comprising: performing a conversion between a video having one or more video layers including one or more video pictures and a bitstream of the video according to format rules 732, and wherein the format rules specify that a constraint on the output order of pictures before an intra random access point in decoding order applies to pictures in the same video layer.

[0419] 14. The method according to clause 13, wherein the constraint specifies that any picture having a NAL (Network Abstraction Layer) unit header layer identifier equal to a specific value and before an intra random access point picture with a NAL unit header layer identifier equal to the specific value in decoding order is required to be in the output order before the intra random access point picture and all associated random access decodable pre-pictures.

[0420] 15. The method according to clause 14, wherein the NAL (Network Abstraction Layer) unit header layer identifier is nuh_layer_id.

[0421] 16. The method according to clause 14, wherein the NAL (Network Abstraction Layer) unit header layer identifier specifies an identifier of a layer to which a video codec layer network abstraction layer unit belongs or an identifier of a layer applicable to a non-video codec layer network abstraction layer unit.

[0422] 17. A method for video processing (e.g., method 740 as shown in Figure 7D ), comprising: performing a conversion between a video having one or more video layers including one or more video pictures and a bitstream of the video according to format rules 742, and wherein the format rules specify the following constraints: (1) a post-picture is in the output order after an associated intra random access point picture or a gradual decoder refresh picture, or (2) a picture having the same NAL (Network Abstraction Layer) unit header layer identifier as a gradual decoder refresh picture is in the output order before the gradual decoder refresh picture and all associated pictures of the gradual decoder refresh picture.

[0423] 18. A method for video processing (e.g., method 750 as shown in Figure 7E ), comprising: performing a conversion between a video having one or more video layers including one or more video pictures and a bitstream of the video according to rules 752, and wherein the rules specify that a constraint on the decoding order of pictures and non-pre-pictures is applied if and only if pictures associated with an intra random access point picture, the intra random access point picture, and non-pre-pictures are in the same layer.

[0424] 19. The method according to clause 18, wherein the constraint specifies that, in the case where the value of the field sequence flag is equal to 0 and the picture for which the NAL (Network Abstraction Layer) unit header layer identifier is equal to a specific value is a pre-picture associated with an intra random access point picture, the picture is, in decoding order, before all non-pre-pictures associated with the intra random access point picture.

[0425] 20. The method according to clause 19, wherein the value of the field sequence flag equal to 0 indicates that the picture in the coded layer video sequence conveys a picture of a frame.

[0426] 21. The method according to clause 18, wherein the constraint specifies that, in the case where the value of the field sequence flag is equal to 0 and the picture is a pre-picture associated with an intra random access point picture, the picture is, in decoding order, before all non-pre-pictures associated with the intra random access point picture.

[0427] 22. A method for video processing (e.g., the method 760 as Figure 7F shown), comprising: performing a conversion 762 between a video having one or more video layers including one or more video pictures and a bitstream of the video according to rules, and wherein the rules specify the order of pre-pictures, randomly accessible decodable pre-pictures, and randomly accessible skipped pre-pictures associated with a progressive decoded refresh picture.

[0428] 23. The method according to clause 22, wherein the pre-picture associated with a progressive decoded refresh picture is, in decoding order, after the progressive decoded refresh picture and, in output order, before the progressive decoded refresh picture.

[0429] 24. The method according to clause 22, wherein the randomly accessible decodable pre-picture associated with a progressive decoded refresh picture is a pre-picture associated with the progressive decoded refresh picture and has a NAL (Network Abstraction Layer) unit type corresponding to the coded slice of the randomly accessible decodable pre-picture.

[0430] 25. The method according to clause 22, wherein the randomly accessible decodable pre-picture associated with a progressive decoded refresh picture is a pre-picture associated with the progressive decoded refresh picture and has a NAL (Network Abstraction Layer) unit type corresponding to the coded slice of the randomly accessible decodable pre-picture.

[0431] 26. A method for video processing (e.g., the method 770 as Figure 7G shown), comprising: performing a conversion 772 between a video having one or more video layers including one or more video pictures and a bitstream of the video according to rules, wherein the rules specify that the constraint on the reference picture list for the slices of a fully randomly accessible picture is restricted to the layer.

[0432] 27. The method according to clause 26, wherein the constraint specifies that, for a layer with a fully randomly accessible picture, the in-picture random access point picture in the previous frames in the decoding or output order is not referenced by an entry in the reference picture list.

[0433] 28. The method according to any one of clauses 1 to 27, wherein the conversion includes encoding the video into a bitstream.

[0434] 29. The method according to any one of clauses 1 to 27, wherein the conversion includes decoding the video from the bitstream.

[0435] 30. The method according to clauses 1 to 27, wherein the conversion includes generating a bitstream from the video, and the method further includes: storing the bitstream in a non-transitory computer-readable recording medium.

[0436] 31. A video processing apparatus, including a processor configured to implement the method according to any one or more of clauses 1 to 30.

[0437] 32. A method of storing a bitstream of a video, including the method according to any one of clauses 1 to 30, and further including storing the bitstream into a non-transitory computer-readable recording medium.

[0438] 33. A computer-readable medium storing program code that, when executed, causes a processor to implement the method according to any one or more of clauses 1 to 30.

[0439] 34. A computer-readable medium storing a bitstream generated according to any one of the above methods.

[0440] 35. A video processing apparatus for storing a bitstream representation, wherein the video processing apparatus is configured to implement the method according to any one or more of clauses 1 to 30.

[0441] The third set of clauses shows example embodiments of the techniques discussed in the previous section (e.g., items 8 and 9).

[0442] 1. A video processing method (e.g., the method 810 as shown Figure 8A in method 810), including: performing a conversion 810 between a video having one or more video layers including one or more video pictures and a bitstream of the video according to a rule, and wherein the rule specifies a condition that no picture that has been generated by a decoding process for generating an unavailable reference picture is referenced by an active entry in the reference picture list of the current slice of the current picture.

[0443] 2. The method according to Clause 1, wherein the active entry corresponds to an entry that can be used as a reference index in the inter-prediction of the current picture.

[0444] 3. The method according to Clause 1, wherein the condition is that the current picture whose NAL (Network Abstraction Layer) unit header layer identifier is equal to a specific value is not a random access skip pre-picture associated with a full random access picture where a variable indicating no output before recovery is equal to 1, a progressive decoder refresh picture where the variable is equal to 1, or a recovery picture of a progressive decoder refresh picture where the variable is equal to 1 and the NAL unit header layer identifier is equal to a specific value.

[0445] 4. A video processing method (e.g., the method 820 as shown in Figure 8B ), comprising: performing a conversion 822 between a video having one or more video layers including one or more video pictures and a bitstream of the video according to a rule, and wherein the rule specifies a condition that a picture that has not been generated by a decoding process for generating unavailable reference pictures is not referenced by an entry in the reference picture list of the current slice of the current picture.

[0446] 5. The method according to Clause 4, wherein the condition is that the current picture whose NAL (Network Abstraction Layer) unit header layer identifier is equal to a specific value is not a full random access picture where a variable indicating no output before recovery is equal to 1, a picture before the pre-picture associated with the full random access picture where the variable is equal to 1 in decoding order, a progressive decoder refresh picture where the variable is equal to 1, or a recovery picture of a progressive decoder refresh picture where the variable is equal to 1 and the NAL unit header layer identifier is equal to a specific value.

[0447] 6. The method according to any one of Clauses 1 to 5, wherein the conversion includes encoding the video into a bitstream.

[0448] 7. The method according to any one of Clauses 1 to 5, wherein the conversion includes decoding the video from the bitstream.

[0449] 8. The method according to any one of Clauses 1 to 5, wherein the conversion includes generating a bitstream from the video, and the method further includes: storing the bitstream in a non-transitory computer-readable recording medium.

[0450] 9. A video processing apparatus, comprising a processor configured to implement the method according to any one or more of Clauses 1 to 8.

[0451] 10. A method of storing a bitstream of a video, comprising the method according to any one of Clauses 1 to 8, and further comprising storing the bitstream into a non-transitory computer-readable recording medium.

[0452] 11. A computer-readable medium storing program code which, when executed, causes a processor to perform the method according to any one or more of clauses 1 to 8.

[0453] 12. A computer-readable medium storing a bitstream generated according to any one of the above methods.

[0454] 13. A video processing apparatus for storing a bitstream representation, wherein the video processing apparatus is configured to perform the method according to any one or more of clauses 1 to 12.

[0455] The fourth set of clauses shows example embodiments of the techniques discussed in the previous section (e.g., items 10 and 11).

[0456] 1. A method for video processing (e.g., the method 910 as Figure 9A shown), including: performing a conversion 912 between a video having one or more video layers including a current picture containing a current slice and a bitstream of the video according to a rule, and wherein the rule specifies a condition that the reference picture list of the current slice is not allowed to have an active entry that references a picture before an intra random access point picture associated with the current picture in decoding order or output order.

[0457] 2. The method according to clause 1, wherein the active entry corresponds to an entry that can be used as a reference index in inter prediction of the current picture.

[0458] 3. The method according to clause 1 or 2, wherein the rule specifies a condition that the reference picture list of the current picture is not allowed to have an active entry that references a picture before an intra random access point picture associated with the current picture in decoding order.

[0459] 4. The method according to any one of clauses 1 to 3, wherein the rule specifies a condition that the reference picture list of the current picture is not allowed to have an active entry that references a picture before an intra random access point picture associated with the current picture in output order.

[0460] 5. The method according to any one of clauses 1 to 4, wherein the condition is that the current picture is associated with an intra random access point picture and is after the intra random access point picture in decoding order and / or output order.

[0461] 6. The method according to any one of clauses 1 to 4, wherein the condition is that the current picture is after an intra random access point picture having the same value of the identifier of the layer to which the video codec layer network abstraction layer unit belongs or the identifier of the layer applicable to the non-video codec layer network abstraction layer unit in decoding order and / or output order.

[0462] 7. A method for video processing (e.g., method 920 as shown in Figure 9B ), comprising: performing a conversion between a video having one or more video layers including a current picture containing a current slice and a bitstream of the video according to a rule 922, and wherein the rule specifies a condition that a reference picture list of the current slice is not allowed to have an entry that references a picture before an intra random access point picture associated with the current picture in decoding order or output order.

[0463] 8. The method according to clause 7, wherein the rule specifies a condition that a reference picture list of the current picture is not allowed to have an entry that references a picture before an intra random access point picture associated with the current picture in decoding order.

[0464] 9. The method according to clause 7 or 8, wherein the rule specifies a condition that a reference picture list of the current picture is not allowed to have an entry that references a picture before an intra random access point picture associated with the current picture in output order.

[0465] 10. The method according to any one of clauses 7 to 9, wherein the intra random access point picture is associated with zero or more preceding pictures, and wherein the condition is that the current picture is associated with the intra random access point picture, after the intra random access point picture in decoding order and / or output order, and after zero or more preceding pictures associated with the intra random access point picture in both decoding order and output order.

[0466] 11. The method according to any one of clauses 7 to 9, wherein the intra random access point picture is associated with zero or more preceding pictures, and wherein the condition is that the current picture is after an intra random access point picture having the same value of the identifier of the layer to which the video codec layer network abstraction layer unit belongs or the identifier of the layer applicable to the non - video codec layer network abstraction layer unit and zero or more preceding pictures in decoding order and / or output order.

[0467] 12. The method according to any one of clauses 1 to 11, wherein the conversion includes encoding the video into a bitstream.

[0468] 13. The method according to any one of clauses 1 to 11, wherein the conversion includes decoding the video from the bitstream.

[0469] 14. The method according to any one of clauses 1 to 11, wherein the conversion includes generating a bitstream from the video, and the method further includes: storing the bitstream in a non - transitory computer - readable recording medium.

[0470] 15. A video processing apparatus includes a processor configured to implement the method according to any one or more of clauses 1 to 14.

[0471] 16. A method of storing a bitstream of a video includes the method according to any one of clauses 1 to 14, and further includes storing the bitstream into a non-transitory computer-readable recording medium.

[0472] 17. A computer-readable medium storing program code which, when executed, causes a processor to implement the method according to any one or more of clauses 1 to 14.

[0473] 18. A computer-readable medium storing a bitstream generated according to any one of the above methods.

[0474] 19. A video processing apparatus for storing a bitstream representation, wherein the video processing apparatus is configured to implement the method according to any one or more of clauses 1 to 14.

[0475] In this document, the term "video processing" may refer to video encoding, video decoding, video compression, or video decompression. For example, during the conversion from the pixel representation of a video to the corresponding bitstream representation, a video compression algorithm may be applied, and vice versa. As defined by the syntax, the bitstream representation of the current video block may, for example, correspond to bits co-located or scattered in different places within the bitstream. For example, macroblocks may be encoded according to the transform and coding / decoding error residual values and also using bits in the headers and other fields in the bitstream. Further, during the conversion, the decoder may parse the bitstream based on this determination, knowing that some fields may or may not be present, as described in the above solution. Similarly, the encoder may determine whether to include or exclude a particular syntax field and generate the coded / decoded representation accordingly by including the syntax field or excluding the syntax field from the coded / decoded representation.

[0476] The disclosed and other solutions, examples, embodiments, modules, and functional operations described in this document can be implemented in digital electronic circuitry, or in computer software, firmware, or hardware (including the structures disclosed in this document and their structural equivalents), or in combinations of one or more of them. The disclosed and other embodiments can be implemented as one or more computer program products, i.e., one or more modules of computer program instructions encoded on a computer-readable medium for execution by, or to control the operation of, a data processing apparatus. The computer-readable medium can be a machine-readable storage device, a machine-readable storage substrate, a memory device, a combination of substances affecting a machine-readable propagated signal, or a combination of one or more of them. The term "data processing apparatus" encompasses all apparatus, devices, and machines for processing data, including, for example, programmable processors, computers, or multiple processors or computers. In addition to hardware, the apparatus can also include code that creates an execution environment for the computer programs being discussed, e.g., code constituting processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of them. A propagated signal is an artificially generated signal that is generated to encode information for transmission to a suitable receiver apparatus, e.g., a machine-generated electrical, optical, or electromagnetic signal.

[0477] A computer program (also known as a program, software, software application, script, or code) can be written in any form of programming language (including compiled or interpreted languages), and it can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A computer program does not necessarily correspond to a file in a file system. The program can be stored as part of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program being discussed, or in multiple coordinated files (e.g., files that store one or more modules, subroutines, or portions of code). A computer program can be deployed to execute on one computer or on multiple computers distributed across one site or multiple sites and interconnected by a communication network.

[0478] The processes and logical flows described in this document can be performed by one or more programmable processors executing one or more computer programs to perform functions by operating on input data and generating output. The processes and logical flows can also be performed by special-purpose logic circuitry, and the apparatus can also be implemented as special-purpose logic circuitry, e.g., an FPGA (Field Programmable Gate Array) or an ASIC (Application Specific Integrated Circuit).

[0479] Processors suitable for executing computer programs include, for example, any one or more processors of general and special microprocessors, as well as any type of digital computer. Generally, a processor will receive instructions and data from a read-only memory or a random access memory or both. The basic elements of a computer are a processor for executing instructions and one or more memory devices for storing the instructions and data. Generally, a computer will also include one or more mass storage devices (e.g., magnetic disks, magneto-optical disks, or optical disks) for storing data, or be operatively coupled to receive data from or transfer data to or both from and to the one or more mass storage devices. However, a computer does not necessarily require such devices. Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, including, for example, semiconductor memory devices such as EPROM, EEPROM, and flash memory devices; magnetic disks, such as internal hard disks or removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks. The processor and the memory may be supplemented by, or incorporated in, special purpose logic circuitry.

[0480] Although this patent document contains many details, these details should not be construed as limitations on any subject or the scope that may be claimed, but rather as descriptions of features of particular embodiments specific to a particular technology. Certain features described in the context of separate embodiments in this patent document may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented separately in multiple embodiments or in any suitable sub-combination. Additionally, although features may be described as acting in certain combinations and even initially claimed as such, in some cases, one or more features from a claimed combination may be excluded from the combination, and the claimed combination may be directed to a sub-combination or variations of a sub-combination.

[0481] Similarly, although operations are depicted in the drawings in a particular order, this should not be construed as requiring that the operations be performed in the particular order shown or in sequential order, or that all of the illustrated operations be performed to achieve a desired result. Additionally, the separation of various system components in the embodiments described in this patent document should not be construed as requiring such separation in all embodiments.

[0482] Only some embodiments and examples have been described, and other embodiments, enhancements, and variations may be made based on what is described and illustrated in this patent document.

Claims

1. A method for video processing, comprising: Performing a conversion between a video having one or more video layers including a current picture of a current strip containing the current picture and a bitstream of the video according to a rule, and wherein the rule specifies a condition under which the reference picture list of the current strip is not allowed to have an active entry that references a picture before an intra random access point picture associated with the current picture in decoding order or output order, wherein the condition is that the current picture is after the intra random access point picture having a value identical to the identifier of the layer to which the video codec layer network abstraction layer unit belongs or a value identical to the identifier of the layer applicable to the non - video codec layer network abstraction layer unit in both decoding order and output order.

2. The method according to claim 1, wherein, The active entry corresponds to the entry indicated by the reference index in the inter - prediction of the current picture.

3. The method according to claim 1, wherein The condition is that the current picture is associated with the intra random access point picture and is after the intra random access point picture in decoding order and / or output order.

4. The method according to claim 1, wherein, When there are preceding pictures, the intra random access point picture is associated with one or more preceding pictures, and wherein the condition is that the current picture is associated with the intra random access point picture, is after the intra random access point picture in decoding order and / or output order, and is after the one or more preceding pictures associated with the intra random access point picture in both decoding order and output order.

5. The method according to claim 1, wherein When there are preceding pictures, the intra random access point picture is associated with one or more preceding pictures, and wherein the condition is that the current picture is after the intra random access point picture having a value identical to the identifier of the layer to which the video codec layer network abstraction layer unit belongs or a value identical to the identifier of the layer applicable to the non - video codec layer network abstraction layer unit and the one or more preceding pictures in both decoding order and output order.

6. The method according to claim 1, wherein The conversion includes encoding the video into the bitstream.

7. The method according to claim 1, wherein, The conversion includes decoding the video from the bitstream.

8. An apparatus for processing video data, comprising a processor and a non - transitory memory having instructions thereon, wherein the instructions, when executed by the processor, cause the processor to: Perform a conversion between a video having one or more video layers including a current picture of a current strip containing the current picture and a bitstream of the video according to a rule, and Among them, The rule specifies a condition under which the reference picture list of the current strip is not allowed to have an active entry that references a picture before an intra random access point picture associated with the current picture in decoding order or output order, wherein the condition is that the current picture is after the intra random access point picture having a value identical to the identifier of the layer to which the video codec layer network abstraction layer unit belongs or a value identical to the identifier of the layer applicable to the non - video codec layer network abstraction layer unit in both decoding order and output order.

9. The device according to claim 8, wherein, The condition is that the current picture is associated with the intra random access point picture and is after the intra random access point picture in the decoding order and / or the output order.

10. The apparatus according to claim 8, wherein, When there are preceding pictures, the intra random access point picture is associated with one or more preceding pictures, and the condition is that the current picture is associated with the intra random access point picture, is after the intra random access point picture in the decoding order and / or the output order, and is after the one or more preceding pictures associated with the intra random access point picture in both the decoding order and the output order.

11. The apparatus according to claim 8, wherein, When there are preceding pictures, the intra random access point picture is associated with one or more preceding pictures, and the condition is that the current picture is after the intra random access point picture having the same value as the identifier of the layer to which the video coding layer network abstraction layer unit belongs or the same value as the identifier of the layer applicable to the non-video coding layer network abstraction layer unit and the one or more preceding pictures in both the decoding order and the output order.

12. A non-transitory computer-readable storage medium storing instructions that cause a processor to: perform a conversion between a video having one or more video layers including a current picture including a current slice of the current picture and a bitstream of the video according to a rule, and Among them, the rule specifies a condition under which the reference picture list of the current slice is not allowed to have an active entry that references a picture before the intra random access point picture associated with the current picture in the decoding order or the output order, wherein the condition is that the current picture is after the intra random access point picture having the same value as the identifier of the layer to which the video coding layer network abstraction layer unit belongs or the same value as the identifier of the layer applicable to the non-video coding layer network abstraction layer unit in both the decoding order and the output order.

13. The medium according to claim 12, wherein When there are preceding pictures, the intra random access point picture is associated with one or more preceding pictures, and the condition is that the current picture is after the intra random access point picture having the same value as the identifier of the layer to which the video coding layer network abstraction layer unit belongs or the same value as the identifier of the layer applicable to the non-video coding layer network abstraction layer unit and the one or more preceding pictures in both the decoding order and the output order.

14. A non-transitory computer-readable recording medium storing a bitstream of a video generated by a method executed by a video processing device, wherein the method includes: generating the bitstream of the video having one or more video layers including a current picture including a current slice of the current picture according to a rule, and wherein the rule specifies a condition under which the reference picture list of the current slice is not allowed to have an active entry that references a picture before the intra random access point picture associated with the current picture in the decoding order or the output order. Wherein, the condition is that the current picture is after the intra random access point picture having a value same as the identifier of the layer to which the video coding layer network abstraction layer unit belongs or a value same as the identifier of the layer applicable to the non-video coding layer network abstraction layer unit, in both the decoding order and the output order.

15. The medium according to claim 14, wherein, When there is a previous picture, the intra random access point picture is associated with one or more previous pictures, and wherein the condition is that the current picture is after the intra random access point picture having a value same as the identifier of the layer to which the video coding layer network abstraction layer unit belongs or a value same as the identifier of the layer applicable to the non-video coding layer network abstraction layer unit, and the one or more previous pictures, in both the decoding order and the output order.

16. A method for storing a bitstream of video, comprising: generating the bitstream of the video having one or more video layers including a current slice containing a current picture according to a rule; and storing the bitstream in a non-transitory computer-readable recording medium, wherein the rule specifies a condition under which the reference picture list of the current slice is not allowed to have an entry that references an active picture that is before the intra random access point picture associated with the current picture in decoding order or output order, wherein the condition is that the current picture is after the intra random access point picture having a value same as the identifier of the layer to which the video coding layer network abstraction layer unit belongs or a value same as the identifier of the layer applicable to the non-video coding layer network abstraction layer unit, in both the decoding order and the output order.

Images