DECODER, ENCODER AND METHOD FOR MIXING NAL UNITS OF DIFFERENT NAL UNIT TYPES IN A VIDEO STREAM - Patent application

Abstract

The present invention relates to a decoder, an encoder, and corresponding methods for processing a video data stream (11) comprising a first sub-bitstream (11-1) and a second sub-bitstream (11-2). The concepts described herein provide a solution for mixing different NAL units (301, 302, 3030, 304) of different NAL unit types within an access unit (30, 31, 32). For example, IRAP NAL unit types can be mixed with different IRAP NAL unit types or non-IRAP NAL unit types, and non-IRAP NAL unit types can be mixed with different non-IRAP NAL unit types.

Description

[Technical field]

[0001] The embodiments of the present disclosure relate to a decoder and a corresponding encoder for decoding / encoding a video stream, in particular a video stream comprising at least two video sub-streams. The video stream may comprise a number of access units, each of which may comprise at least one picture included in each of the two or more video sub-streams. Each picture may be associated with a Network Abstraction Layer (NAL) unit of a particular NAL unit type. The innovative concept described herein introduces a solution for mixing NAL units of different NAL unit types. [Background technology]

[0002] NAL units may be classified as VCL (Video Coding Layer) and non-VCL NAL units according to whether they contain coded pictures or other associated data, respectively. The HEVC standard includes several VCL NAL unit types that identify categories of pictures for decoder initialization and random access purposes. The contents of the two-byte NAL unit header can be used to easily identify the purpose of the associated payload data.

[0003] Video coding techniques allow the combination of random access and bitstreams. For random access, a bitstream can start with an Instantaneous Decoding Refresh (IDR) access unit. An IDR access unit contains independently coded pictures, i.e. coded pictures that can be decoded without decoding previous pictures in the NAL unit stream. The presence of an IDR access unit indicates that a subsequent picture in the bitstream does not require reference to pictures earlier than the one it contains in order to be decoded. IDR pictures are used within a coding structure known as a closed Group Of Pictures (GOP). An alternative to the above mentioned IDR syntax is the use of independently coded pictures at RAP or IRAP ((Intra) Random Access Point) positions, i.e. the Clean Random Access (CRA) picture syntax that specifies the position in the bitstream. This syntax specifies the position in the bitstream from which a decoder can successfully begin decoding a picture without having to decode pictures that appeared earlier in the bitstream, supporting an efficient temporal coding order known as open GOP operation.

[0004] Good support of random access may be important to enable channel switching, seek operations, and dynamic streaming services. Some pictures following a CRA picture in decoding order and preceding it in display order may contain interpicture prediction references to pictures that are not available at the decoder. These non-decodable pictures may therefore be discarded by a decoder that starts the decoding process at a CRA point. For this purpose, such non-decoded pictures may be identified as RASL pictures (Random Access Skipped Leading). An IRAP picture may be an IDR or CRA picture, and a CRA picture may be followed by a RASL picture in the bitstream. A RASL picture may be discarded by the decoder, for example, because of a combining operation, since it may contain references to pictures that are not actually present in the bitstream.

[0005] A further type of picture that may follow the IRAP picture in decoding order and precede the IRAP picture in output order is a Random Access Decodable Leading (RADL) picture, which may not contain a reference to any picture preceding the IRAP picture in decoding order. RASL and RADL pictures may be collectively referred to as Leading Pictures (LP). Pictures that follow the IRAP picture in both decoding order and output order are known as Trailing Pictures (TRAIL). They may not contain any reference to LP for inter-picture prediction.

[0006] In today's video coding, Temporal Sublayering Support may also be provided. Thus, a temporal identifier indicating a level in a hierarchical temporal prediction structure may be specified in the NAL unit header. This helps to achieve temporal scalability without the need to parse parts of the bitstream other than the NAL unit header. Under certain circumstances, the number of decoded temporal sublayers may be adjusted during the decoding process of one coded video sequence. The location of the point in the bitstream where a sublayer switch is possible to start decoding some higher temporal layers may be indicated by the presence of STSA pictures (Stepwise Temporal Sublayer Access). At the location of the STSA picture, it may be possible to switch from decoding lower temporal sublayers to decoding one particular higher temporal sublayer (but not further layers above it unless they also contain an STSA picture).

[0007] In the case of management of multiple reference pictures, a certain set of previously decoded pictures may be present in the Decorded Picture Buffer (DPB) for decoding the remaining part of the pictures in the bitstream. To identify these pictures, a list of Picture Order Count (POC) identifiers may be transmitted in each slice header. The set of reference pictures that is retained is called Reference Picture Set (RPS). The POC may include a least significant bit portion (LSB) and a most significant bit portion (MSB) for each picture in the bitstream to indicate the respective current picture order count (POC value), and the picture order may be different from the decoding order of the respective pictures. Therefore, decoding a picture after an IRAP picture, which refers to a reference picture before the IRAP picture, may be difficult.

[0008] It is therefore desirable to improve upon existing encoders and decoders that can provide proper handling of pictures and sequences of pictures in a bitstream, especially when they appear after an IRAP picture.

[0009] It is therefore proposed to provide a decoder having at least the features of the independent claims and a corresponding device for encoding and / or merging having at least the features of the independent claims. It is further proposed to provide corresponding methods of encoding and decoding, respectively, as well as corresponding computer program products and bitstreams. Advantageous embodiments are presented in the dependent claims.

[0010] As mentioned above, a bitstream may comprise pictures that are associated with an IRAP NAL unit type, which may also be referred to as IRAP pictures, and pictures that are not associated with an IRAP NAL unit type, i.e., pictures associated with non-IRAP NAL unit types, may also be referred to as non-IRAP pictures. Summary of the Invention [Means for solving the problem]

[0011] According to one aspect, a decoder for decoding video content is provided, the decoder being configured to decode a bitstream including a first sub-bitstream associated with a first spatial segment (131) of a picture of the video content and a second sub-bitstream associated with a second spatial segment (132) of a picture of the video content. In this case, the bitstream comprises at least one access unit, in which at least one first NAL unit of the first sub-bitstream is of a non-IRAP NAL unit type, e.g., RASL, RADL, TRAIL, STSA. The at least one access unit further comprises at least one different second NAL unit of the second sub-bitstream, which is of a non-IRAP NAL unit type, e.g., RASL, RADL, TRAIL, STSA, and the first NAL unit of the first sub-bitstream and the second NAL unit of the second sub-bitstream are mixed. In other words, the non-IRAP NAL unit is mixed with a different non-IRAP NAL unit. For example, a NAL unit of a TRAIL picture NAL unit type can be mixed with a NAL unit of a RASL picture NAL unit type. According to this non-limiting example, within one access unit, a TRAIL picture of the first sub-bitstream may be mixed with a RASL picture of the second sub-bitstream. Other non-limiting examples may allow for mixing a TRAIL picture of the first sub-bitstream with a RADL picture of the second sub-bitstream within one access unit. Some further non-limiting examples may allow for mixing a TRAIL picture of the first sub-bitstream with a STSA picture of the second sub-bitstream within one access unit. Some further non-limiting examples may allow for mixing a STSA picture of the first sub-bitstream with a RADL picture of the second sub-bitstream within one access unit. Some further non-limiting examples may allow for mixing a STSA picture of the first sub-bitstream with a RASL picture of the second sub-bitstream within one access unit.Some further non-limiting examples may allow mixing RASL pictures of a first sub-bitstream and RADL pictures of a second sub-bitstream within one access unit.

[0012] According to a further aspect, a decoder for decoding video content is provided, the decoder configured to decode a bitstream comprising a first sub-bitstream associated with a first spatial segment of a picture of the video content and a second sub-bitstream associated with a second spatial segment of a picture of the video content. In this case, the bitstream comprises at least one access unit of an IRAP NAL unit type, in which at least one first NAL unit of the first sub-bitstream is mixed with at least one second NAL unit of the second sub-bitstream of a further IRAP NAL unit type. In other words, the first IRAP NAL unit is mixed with a further second IRAP NAL unit. The further second IRAP NAL unit may be of the same NAL unit type as the first IRAP NAL unit or may be of a different NAL unit type than the first IRAP NAL unit. For example, a NAL unit of a CRA picture NAL unit type may be mixed with further pictures of the same NAL unit type, e.g., further CRA picture NALs, or with further pictures of a different NAL unit type, e.g., a NAL unit of an IDR picture NAL unit type with a leading picture, i.e., an IDR-with-RADL picture unit type (IDR_W_RADL). According to this non-limiting example, within one access unit, a CRA picture of a first sub-bitstream may be mixed with a further CRA picture of a second sub-bitstream or an IDR_W_RADL picture of the second sub-bitstream.

[0013] According to a further aspect, a decoder for decoding video content is provided, the decoder being configured to decode a bitstream comprising a first sub-bitstream associated with a first spatial segment of a picture of the video content and a second sub-bitstream associated with a second spatial segment of a picture of the video content. In this case, the bitstream comprises at least one access unit in which at least one first NAL unit of the first sub-bitstream is an IRAP NAL unit type mixed with at least one second NAL unit of the second sub-bitstream that is either a non-IRAP leading picture NAL unit type (e.g., RASL or RADL) or an STSA NAL unit type. In other words, within one access unit, an IRAP NAL unit (IDR_W_RADL or CRA) is mixed with a non-IRAP NAL unit, which is either a RASL picture, a RADL picture, or an STSA picture.

[0014] According to a further aspect, a decoder for decoding video content is provided, the decoder being configured to decode a bitstream including a first sub-bitstream and a second sub-bitstream, the decoder being configured to derive from slice headers of NAL units of the first and second sub-bitstreams, the NAL units being of an IRAP NAL unit type (e.g., CRA, IDR_W_RADL), information for setting a top POC portion of a POC of a picture to which the NAL units refer. In this example, the top POC portion needs to be set to a predefined value for all NAL units of the IRAP NAL unit type in an access unit of the bitstream if at least one NAL unit in the access unit is of a non-Instantaneous Decoding Refresh (non-IDR) unit type. In other words, if at least one NAL unit in the access unit refers to a non-IDR picture, e.g., a CRA picture, the top POC portion needs to be set to a predefined value. As a non-limiting example, the predetermined value may be equal to or greater than the most significant POC portion of the POC associated with a picture preceding the picture referenced by the NAL unit, In other words, if a non-IDR picture is present in the access unit, a POC reset may not occur.

[0015] According to a further aspect, a decoder for decoding video content is provided, the decoder being configured to decode a bitstream including a first sub-bitstream and a second sub-bitstream, the decoder being configured to derive from slice headers of NAL units of the first and second sub-bitstreams, the NAL units being an IRAP NAL unit type (e.g., CRA, IDR_W_RADL) and information for setting a top POC portion of a POC of a picture to which the NAL units refer. In this example, no picture reference is allowed to cross any access unit of a bitstream in decoding order, in which all NAL units are an IRAP NAL unit type and have a slice header with a top POC portion set to a predetermined value. In other words, if each picture in an access unit may refer to an IRAP NAL unit type (e.g., CRA or IDR_W_RADL) and may have a top POC portion set to a predetermined value, no picture referencing from a reference picture preceding said access unit may be allowed. As a non-limiting example, the predetermined value may be smaller than the most-top POC part of the POC associated with a picture preceding the picture to which the NAL unit refers. The predetermined value may be smaller, for example, if a reset of the most-top POC part occurs. Thus, according to the above non-limiting example, an access unit may only comprise an IRAP picture, and if a reset of the most-top POC part occurs, picture reference by referring to a preceding reference picture may not be allowed.

[0016] According to a further aspect, a decoder for decoding video content is provided, the decoder configured to decode a bitstream including a first sub-bitstream and a second sub-bitstream, the decoder configured to derive from slice headers of NAL units of the first and second sub-bitstreams, the NAL units being of an IRAP NAL unit type (e.g., CRA, IDR_W_RADL) and information for setting a top POC part of a POC of a picture to which the NAL units refer. In this example, a reference picture of a first access unit having a slice header whose top POC part is set to a first predetermined value and preceding, in bitstream order, a second access unit of the bitstream having a slice header in which all NAL units are of the IRAP NAL unit type and whose top POC part is set to a second predetermined value is not allowed to be referenced by a picture following, in bitstream order, the second access unit of the bitstream. According to a non-limiting example, the first predetermined value may be greater than the second predetermined value. In other words, if the first predetermined value of the topmost POC part of the (preceding) first access unit is greater than the second predetermined value of the topmost POC part of the second access unit, then pictures following the second access unit may not be allowed to refer to reference pictures preceding the second access unit.

[0017] According to further aspects, respective corresponding encoders, methods for encoding and methods for decoding are proposed.

[0018] According to further aspects, computer programs are provided, where each of the computer programs is configured to implement one of the methods described herein when executed on a computer or signal processor, whereby a method described herein is implemented by one of the computer programs. [Brief description of the drawings]

[0019] In the following, embodiments of the present disclosure are explained in more detail with reference to the drawings. [Figure 1] FIG. 1 is a schematic diagram of a bitstream that can be processed by the inventive decoder and apparatus for encoding / merging according to an embodiment. [Diagram 2] FIG. 2 is a schematic diagram of a bitstream having an access unit with an IDR_N_LP NAL unit type and all other access units with the same NAL unit type. [Diagram 3] FIG. 3 is a schematic diagram of a bitstream with access units having an IDR_W_LP NAL unit type and corresponding RADL unit types mixed with TRAIL unit types. [Figure 4] FIG. 4 is a schematic diagram of a bitstream having access units with CRA NAL unit types and corresponding RASL unit types mixed with TRAIL unit types. [Diagram 5] FIG. 5 illustrates a full 360 degree low resolution thumbnail with selective high resolution tiles. [Figure 6] FIG. 6 illustrates an example coding structure in which in one tile (top example), a picture uses an STSA NAL unit type to indicate step-wise temporal sublayer access, and in another tile (bottom example), the same picture uses an RASL NAIL unit type. [Figure 7] FIG. 7 shows an example coding structure in the high resolution domain with a closed GOP structure (top) and in the high resolution domain with an open GOP structure (bottom). [Figure 8] FIG. 8 is a diagram illustrating an example coding structure for a multi-layer bitstream with a reset POC at a layer-aligned RAP picture. [Figure 9] FIG. 9 is a diagram illustrating an example coding structure for a multi-layer bitstream with POC MSB resetting when all NAL unit IDRs are disallowed. [Figure 10] FIG. 10 is a diagram illustrating an example coding structure for a multi-layer bitstream with POC MSB resetting when all NAL unit IDRs are allowed. [Figure 11] FIG. 11 illustrates an example coding structure for a multi-layer bitstream when POC MSB reset and LT referencing are not allowed. [Figure 12] FIG. 12 illustrates an example coding structure for a multi-layer bitstream when all POC MSBs are equal to 0 and LT referencing is not allowed. [Figure 13] FIG. 13 illustrates an example coding structure for a multi-layer bitstream when all POC MSBs are equal to 0 and LT referencing is allowed. [Figure 14] FIG. 14 illustrates an example coding structure for a multi-layer bitstream when the POC MSB is not reset and LT references are allowed. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0020] Equivalent or equivalent elements or elements having equivalent or equivalent functions are indicated in the following description by equivalent or equivalent reference numbers.

[0021] Method steps depicted by and described with reference to block diagrams may also be performed in an order different from that depicted and / or described. Furthermore, method steps relating to particular features of an apparatus may be interchangeable with said features of the apparatus or in other ways.

[0022] Mixing NAL Unit Types in Pictures and Access Units 1. Mixed unit types within a coded picture 1 illustrates, for introductory purposes, the structure of an exemplary video bitstream 11 that may be processed by a decoder 100 and an apparatus 10 for encoding and / or merging a video bitstream 11 according to an embodiment of the present invention. The video bitstream 11 comprises a sequence of consecutive pictures 12a, 12b, 12c, two or more of which may be grouped together in a Group of Pictures (GOP) 12 that may represent at least a portion of an intended video content.

[0023] For coding purposes, each picture 12a, 12b, 12c may be subdivided into spatial segments, e.g., slices, tiles, or blocks. In this non-limiting example, pictures 12a, 12b, 12c are each subdivided into two spatial segments, a first (e.g., left) spatial segment 131 and a second (e.g., right) spatial segment 132.

[0024] The bitstream 11 can be subdivided into a first sub-bitstream 11-1 associated with a first spatial segment 131 of the pictures 12a, 12b, 12c and a second sub-bitstream 11-2 associated with a second spatial segment 132 of the pictures 12a, 12b, 12c.

[0025] Some pictures 12a, 12b may be dependently coded, for example by predictive coding. Some of these pictures 12b may be predicted by uniprediction, for example by referring to a single preceding picture, while some other pictures 12a may be predicted by biprediction, for example by referring to a preceding and a following picture. Some other pictures 12c may be independently coded, i.e., these pictures 12c may not refer to preceding or following pictures. The independently coded pictures 12c may provide (intra) random access points ((I)RAPs) that represent positions in the bitstream 11 at which a decoder can start decoding a following picture without needing to decode a picture that appeared earlier in the bitstream 11. For example, picture 12c is an independently coded picture that may also be referred to as an (I)RAP picture. In this disclosure, the terms RAP and IRAP may be used synonymously.

[0026] The bitstream 11 may comprise one or more access units 30 for accessing the bitstream 11. In the non-limiting example of Fig. 1, the access unit 30 may align (in terms of presentation order) the picture segments 131, 132 contained in the first and second sub-bitstreams 11-1, 11-2. The pictures 12a, 12b, 12c containing these aligned picture segments 131, 132 may be grouped into different categories, e.g., IRAP pictures and non-IRAP pictures (as described above).

[0027] The Network Abstraction Layer (NAL) may provide a High Level Syntax for defining those picture segments 131, 132 and pictures 12a, 12b, 12c. In particular, a NAL unit may be associated with said picture segment 131, 132 for each of the pictures 12a, 12b, 12c. Thus, in this disclosure, the above-mentioned picture segments 131, 132 and the first and second sub-bitstreams 11-1, 11-2 may also be referred to as NAL units. Thus, the first and second sub-bitstreams 11-1, 11-2 may also be referred to as NAL unit streams. Each NAL unit 131, 132 may include a specific NAL unit type. The NAL unit type may then identify the aforementioned category of the respective picture contained in the NAL unit, i.e., whether the respective NAL unit is of IRAP type or non-IRAP type.

[0028] There are several applications that rely on mixing IRAP types within a picture, where the IRAP type is a random access property of an Access Unit (AU) 30 or part of it, indicated through a NAL unit type. Use cases are in either single-layer or multi-layer codec domains, i.e., single-layer codec bitstreams where a picture consists of slices that are IRAP while other slices of the same picture are not, or multi-layer codec bitstreams where an access unit contains an IRAP in one layer and non-IRAP pictures in other layers.

[0029] Currently, support for mixed NAL unit types is provided in the prior art where a single type of IRAP NAL unit type can be mixed within a picture together with a TRAIL picture, however this does not cover the use case that adequately motivates the need for the feature as described below and for which the present invention provides a solution.

[0030] The main use case that motivates the need for mixed NAL unit types is 360-degree video, where tile streaming is used and only a subset of the entire 360 ​​degrees is transmitted in high resolution and the rest in low resolution. When a change in viewing direction occurs, some tiles in high resolution are no longer needed, but new tiles shown in low resolution are needed in high resolution from this time onwards. These tiles are the only tiles that encounter a change (from high resolution to low resolution and vice versa). These tiles are downloaded using a representation that starts with IRAP, while other tiles do not need this property. When merging bitstreams corresponding to different tiles into a single bitstream, some of the tiles contain a NAL unit type corresponding to an IRAP picture and others contain a NAL unit type for a non-IRAP picture.

[0031] Furthermore, there are some other use cases where it is desirable to allow AUs to have NAL units of different types. For example, in a scenario where there is a 1080p picture with a RoI inside a 720p, it can be assumed that different parts of the video (RoI or otherwise) have different RAP durations. For example, when the same bitstream is used to feed two receivers, one interested in the whole video and the other only interested in the RoI.

[0032] In either case of 360-degree video streaming, where merging of different bitstreams occurs, or RoI streaming, where the video is originally coded into different parts with different RAP durations for different regions, the decoder will encounter bitstreams where the NAL units in the coded pictures are not necessarily the same. Therefore, the present invention provides a solution for mixing NAL units of different NAL unit types.

[0033] 1.1 Trailing Picture Blending (Prior Art) The solution currently proposed in the prior art is that a PPS flag indicates whether a RAP picture is mixed with a non-RAP picture, and then the NAL unit types that can be encountered in the NAL units of a mixed AU are TRAIL NAL unit types (up to VCL_RSV_6 or GDR) for non-IRAP types, and a single one type IDR_W_RADL (IDR with Leading Pictures), IDR_N_LP (IDR with No Leading Pictures), and CRA (Clean Random Access). Thus, the prior art suggests mixing IRAP pictures with TRAIL pictures.

[0034] 2 shows a prior art example where a NAL unit 201 of IRAP NAL unit type is mixed with a NAL unit 202 of trailing picture type (TRAIL). The IRAP NAL unit type in this example is IDR without leading pictures (IDR_N_LP). All other access units have NAL units of the same NAL unit type (i.e., TRAIL).

[0035] 1.2 Mixed Leading Pictures However, these measures are not sufficient for the envisaged applications where an open GOP structure may be applied with the introduction of leading pictures (LPs). Figure 3 shows an exemplary case where a NAL unit 301 of an IRAP NAL unit type (e.g., IDR_W_RADL) is shown with its associated leading picture 302, which in this case may be a NAL unit 302 of NAL unit type RADL (Random Access Decodable Leading Picture). The IRAP type NAL unit 301 (e.g., IDR_W_RADL) may be mixed with a NAL unit 303 of a non-IRAP NAL unit type (e.g., TRAIL). The associated NAL unit 302 of a leading picture type (RADL), which is a non-IRAP type, may be mixed with a NAL unit 304 of a different non-IRAP type (e.g., TRAIL).

[0036] 4 illustrates a further exemplary case in which a NAL unit 301 of an IRAP NAL unit type (e.g., CRA) is shown with its associated leading picture 302, which in this case may be a NAL unit 302 of NAL unit type RASL (Random Access Skipped Leading Picture). Alternatively, not shown, the associated leading picture 302 may be a NAL unit 302 of NAL unit type RADL (Random Access Decodable Leading Picture). A NAL unit 301 of an IRAP type (e.g., CRA) may be mixed with a NAL unit 303 of a non-IRAP NAL unit type (e.g., TRAIL). An associated NAL unit 302 of a leading picture type (e.g., RASL) that is a non-IRAP type may be mixed with a NAL unit 304 of a different non-IRAP type (e.g., TRAIL).

[0037] Therefore, it is necessary to treat coded pictures with mixed NAL unit types not only when IRAP NAL units are included, but also only for non-IRAP NAL units included in the mixed picture.

[0038] To briefly summarize, the prior art can suggest mixing IRAP NAL units with non-IRAP NAL units of TRAIL type, or mixing non-IRAP NAL units of the same type, i.e., TRAIL type NAL units with TRAIL type NAL units.

[0039] The present invention instead proposes mixing a first non-IRAP NAL unit type (eg, RASL, RADL, TRAIL, STSA) with at least one different second non-IRAP NAL unit type (eg, RASL, RADL, TRAIL, STSA).

[0040] Thus, according to an embodiment, a decoder 100 is provided for decoding a video content 12, the decoder 100 being configured to decode a bitstream 11 comprising a first sub-bitstream 11-1 relating to a first spatial segment 131 of a picture 12a, 12b, 12c of the video content 12 and a second sub-bitstream 11-2 relating to a second spatial segment 132 of a picture 12a, 12b, 12c of the video content 12. According to this embodiment, the bitstream 11 comprises at least one access unit 30 of a non-IRAP NAL unit type (e.g. RASL, RADL, TRAIL, STSA) in which at least one first NAL unit 302 of the first sub-bitstream 11-1 is mixed with at least one different second NAL unit 304 of the second sub-bitstream 11-2 of a different non-IRAP NAL unit type (e.g. RASL, RADL, TRAIL, STSA).

[0041] For example, a TRAIL picture may be mixed with a RADL or RASL picture. TRAIL + RADL TRAIL + RASL

[0042] Thus, according to one embodiment, the decoder 100 may be configured to process a bitstream 11 in which the non-IRAP NAL unit type of the NAL units 302 of the first sub-bitstream 11-1 is a random access decodable leading picture unit type (RADL) and the non-IRAP NAL unit type of the NAL units 304 of the second sub-bitstream 11-2 is a trailing picture unit type (TRAIL).

[0043] According to a further embodiment, the decoder 100 may be configured to process a bitstream 11 in which the non-IRAP NAL unit type of the NAL unit 302 of the first sub-bitstream 11-1 is a random access skip leading picture unit type (RASL) and the non-IRAP NAL unit type of the NAL unit 304 of the second sub-bitstream 11-2 is a trailing picture unit type (TRAIL).

[0044] In one embodiment of the present invention, there is an indication in the bitstream 11, for example as a flag in the parameter set (PPS, SPS, VPS) that NAL unit types within a coded picture can be mixed as follows: TRAIL + RADL TRAIL + RASL

[0045] Such signaling can be carried in the PPS to indicate that TRAIL+(CRA xor IDR_W_LD xor IDR_N_P) is mixable.

[0046] For example, if the flag is not set (i.e., flag = 0), all NAL units of a picture may have the same NAL unit type, and the picture or PU is said to have the same NAL unit type as the coded slice NAL units of the picture or PU.

[0047] Otherwise, if the flag is set (i.e., flag=1), a picture may contain at least two sub-pictures. Additionally or alternatively, the NAL units of the picture may comprise at least two different NAL unit types. Further, additionally or alternatively, the NAL units of the picture may not comprise a GDR NAL unit type. Further additionally or alternatively, if a NAL unit of the picture is of at least one of the following NAL unit types: Instant decoder refresh with random access decoded leading picture (IDR_W_RADL) Instant Decoder Refresh without Leading Picture (IDR_N_LP) Clean Random Access (CRA)

[0048] In that case, all other available NAL units of the picture may comprise one of the three unit types mentioned above (IDR_W_RADL, IDR_N_LP, CRA) or a trailing picture unit type (TRAIL).

[0049] Alternatively, this indication may be made at the SPS level, since RADL and RASL pictures do not have a different normative decoding process from TRAIL pictures, as, for example, IRAP pictures have. In one embodiment, for example, the indication may have the form of a constraint flag, e.g., a general constraint flag. The general constraint flag may indicate whether a particular mix of NAL unit types is allowed. If such a general constraint flag is set (i.e., general constraint flag = 1), it may indicate that there is no mix of NAL unit types in the picture, and that the above flags in the parameter sets (PPS, SPS, VPS) are not set (i.e., flags = 0). If the general constraint flag is not set (i.e., general constraint flag = 0), the above constraints may not be imposed.

[0050] Thus, according to one embodiment, the decoder 100 may be configured to derive an indication from the bitstream 11 that explicitly informs the decoder 100 that a non-IRAP NAL unit type NAL unit 304 of the second sub-bitstream 11-2 (which NAL unit 304 is a trailing picture unit type (TRAIL)) is mixed with a non-IRAP NAL unit type NAL unit 302 of the first sub-bitstream 11-1, which NAL unit 302 is either a random access skip leading picture unit type (RASL) or a random access decodable leading picture unit type (RADL).

[0051] According to one embodiment, the indication may comprise a syntax element in a parameter set, for example, the syntax element may be included in at least one of a Picture Parameter Set (PPS) or a Sequence Parameter Set (SPS).

[0052] However, assuming a 360-degree video tile-based streaming scenario, when tiles with RASL properties and tiles of TRAIL pictures are mixed in a picture, it may happen that the tiles change position at partial RAP points and therefore the RASL tiles cannot be correctly reconstructed because the reference pictures used are incorrect (i.e., still depicting the content of other tiles). It is crucial for the application to recognize these defective reconstructed picture regions, while dropping the entire mixed leading picture (including TRAIL+RASL tiles) is also not an option for playout continuity reasons. Instead, the client recognizes the defective regions in the decoded and outputted picture and does not use them for subsequent processing.

[0053] Instead, given a configuration as shown in, for example, Fig. 5, clients can use the lower resolution version 51 included in the picture for content that is incorrectly reconstructed in their mixed leading picture. In one embodiment, regions corresponding to potentially defective reconstructed tiles are indicated to subsequent processing, for example through an SEI message or by external means, for example an application. The described indication means that regions corresponding to NAL units with NAL unit type RASL are ignored / not output.

[0054] Thus, according to one embodiment, if the non-IRAP NAL unit type of the NAL unit 302 of the first sub-bitstream 11-1 is a random access skip leading picture unit type (RASL) and the non-IRAP NAL unit type of the NAL unit 304 of the second sub-bitstream 11-2 is a trailing picture unit type (TRAIL), the decoder 100 may be configured to derive an indication from the bitstream 11.

[0055] In one example, the instruction may indicate to the decoder 100 to ignore or not output a first spatial segment 131 in pictures 12a, 12b, 12c of the video content 12 of the first sub-bitstream 11-1 that corresponds to a NAL unit 302 of a random access skip leading picture unit type in the first sub-bitstream 11-1.

[0056] In another example, the instruction may instruct the decoder 100 to output a first spatial segment 131 of a picture 12a, 12b, 12c of the video 12 content of the first sub-bitstream 11-1 that corresponds to said NAL unit 302 of a Random Access Skip Read Picture Unit Type (RASL) of the first sub-bitstream 11-1, and further to mark the first spatial segment 131 as a corrupted spatial segment. For example, the decoder 100 may be configured to mark said corrupted spatial segment 13 for further processing chains by a Supplemental Enhancement information (SEI) message.

[0057] Furthermore, another option is to avoid RASL pictures being mixed with trailing pictures (TRAIL) at all, and thus only allow RADL pictures to be mixed with TRAIL pictures. Thus, in one embodiment, there is a constraint shown in the SPS, where when a CRA is mixed with a TRAIL picture, the associated leading pictures must only be of RADL type.

[0058] Thus, according to one embodiment, the decoder 100 may be configured to decode a constraint indication from the parameter set, the said constraint indication indicating to the decoder 100 that if the non-IRAP NAL unit type of the NAL unit 304 of the second sub-bitstream 11-2 is a non-leading picture NAL unit type (e.g., TRAIL or STSA), then the non-IRAP NAL unit type of the NAL unit 302 of the first sub-bitstream 11-1 is a random access decodable leading (RADL) picture unit type only if its associated IRAP NAL unit 301 of the first sub-bitstream 11-1 is a clean random access unit type (CRA).

[0059] This may be especially true for embodiments where the non-IRAP NAL unit type of the NAL unit 304 of the second bitstream 11-2 is a trailing picture unit type (TRAIL), ie, mixing TRAIL+CRA with associated RADL leading pictures.

[0060] In the previous aspects, we mainly focused on the TRAIL NAL unit type. However, in many cases, there may also be an STSA NAL unit type, allowing step-wise temporal sublayer access. That is, although the above examples were mainly related to non-IRAP NAL units of the TRAIL NAL unit type, the present invention can also provide a solution for mixing non-IRAP NAL units of the STSA (Stepwise Temporal Sublayer Access) unit type with at least one of the RASL, RADL, and TRAIL unit types. · STSA + RASL ·STSA + RADL ·STSA + TRAIL

[0061] Thus, according to one embodiment, the decoder 100 may be configured to decode a bitstream 11 in which the non-IRAP NAL unit type of the NAL unit 302 of the first sub-bitstream 11-1 is a step-wise temporal sublayer access unit type (STSA) and the non-IRAP NAL unit type of the NAL unit 304 of the second sub-bitstream 11-2 is a trailing picture unit type (TRAIL).

[0062] In a further embodiment, the decoder 100 may be configured to decode a bitstream 11, where the non-IRAP NAL unit type of the NAL unit 302 of the first sub-bitstream 11-1 is a random access decodable reading picture unit type (RADL) and the non-IRAP NAL unit type of the NAL unit 304 of the second sub-bitstream 11-2 is a stepwise temporal sublayer access unit type (STSA).

[0063] In a further embodiment, the decoder 100 may be configured to decode a bitstream 11 in which the non-IRAP NAL unit type of the NAL unit 302 of the first sub-bitstream 11-1 is a random access skip leading picture unit type (RASL) and the non-IRAP NAL unit type of the NAL unit 304 of the second sub-bitstream 11-2 is a step-wise temporal sublayer access unit type (STSA).

[0064] FIG. 6 shows an example coding structure in which in one picture segment 132 (e.g., a tile), a picture 12a indicates step-wise temporal sublayer access using a NAL unit 304 of the STSA NAL unit type, while the same picture 12a in another picture segment 131 (e.g., a tile) uses a NAL unit 302 of the RASL NAL unit type.

[0065] STSA NAL units cannot be mixed with IRAP NAL units because the STSA NAL unit type cannot be placed at the lowest temporal level 0. However, they can be mixed with RADL or RASL NAL unit types, similar to the TRAIL NAL units described above.

[0066] In one embodiment, a flag is included in the parameter set, for example in the PPS, that indicates that STSA NAL units are mixed with RASL or RADL NAL unit types.

[0067] Thus, in one embodiment of the present invention, there is an indication in the bitstream 11, for example as a flag in a parameter set (PPS, SPS, VPS), that NAL unit types within a coded picture can be mixed as follows: ·STSA + RADL · STSA + RASL ·STSA + TRAIL

[0068] In further embodiments, whether such mixing is permitted may be indicated using a constraint flag, for example, on the sequence level or picture level.

[0069] Thus, according to one embodiment, the decoder 100 may be configured to derive an indication from the bitstream 11, where said indication explicitly informs the decoder 100 that a NAL unit 304 of a non-IRAP NAL unit type of the second sub-bitstream 11-2, in which the NAL unit 304 is of a stepwise temporal sublayer access unit type (STSA), is mixed with a NAL unit 302 of a non-IRAP NAL unit type of the first sub-bitstream 11-1, and that said NAL unit 302 is either of a random access skip leading picture unit type (RASL) or of a random access decodable leading picture unit type (RADL).

[0070] According to one embodiment, the indication comprises a syntax element in a parameter set. According to a further embodiment, the syntax element is comprised in at least one of a picture parameter set or a sequence parameter set.

[0071] Furthermore, an indication may be added to the bitstream 11 that effectively indicates that all NAL units 302, 304 of the picture 12a, including the STSA NAL unit 304, satisfy the constraint related to the STSA NAL unit type, even if they have another NAL unit type, for example, RASL (see NAL unit 302, for example). This may be indicated by a flag in the SPS indicating that the (spatial) subset bitstream 11-1 (see NAL unit 302, for example) that includes the RASL picture also satisfies the constraint that each RASL picture is an STSA picture. Such signaling is even more important for the merging use case, where the original bitstream containing the RASL or RADL NAL units can indicate whether the RADL and RASL NAL units satisfy the properties of STSA pictures, so that when mixed with STSA, it can be easily derived whether the mixed coded picture has STSA properties.

[0072] Thus, according to one embodiment, the decoder 100 may be configured to decode a constraint flag from a parameter set of the bitstream 11, which constraint flag indicates to the decoder 100 that any access unit 30 of the bitstream 11 that comprises a mix of one or more NAL units 302 of a non-IRAP leading picture NAL unit type (e.g., RASL or RADL) and one or more NAL units 304 of a stepwise temporal sublayer access unit type (STSA) satisfies the constraint associated with the stepwise temporal sublayer access unit type (STSA), i.e., each RASL picture 302 may be treated as an STSA picture.

[0073] The above constraints may indicate to the decoder 100 that any picture associated with a next higher temporal sublayer than the stepwise temporal sublayer access (STSA) unit type, according to the decoding order, is not allowed to reference a picture in the next higher temporal sublayer if the next higher temporal sublayer precedes the stepwise temporal sublayer access unit type (STSA).

[0074] Alternatively, a bitstream 11 with a coded picture 12a with mixed NAL unit types (e.g., STSA+RASL or RADL) may include a flag indicating that the picture 12a for which the flag is set may contain NAL units 304 of STSA NAL unit type, but does not (in total) obey the constraints associated with the STSA NAL unit type. This indicates that the coded picture 12a may have a NAL unit type equal to STSA, but the properties implied by STSA (i.e., further temporal levels may be decoded from that AU onwards) do not apply. Thus, parsing of STSA is ignored and stepwise temporal sublayer access is inferred.

[0075] Thus, according to one embodiment, the decoder 100 may be configured to decode a flag from the parameter set, which indicates to the decoder 100 that any access unit 30 comprising a mix of one or more NAL units 302 of a non-IRAP leading picture NAL unit type (RASL or RADL) and one or more NAL units 304 of a stepwise temporal sub-layer access unit type (STSA) does not comply with the constraints associated with the stepwise temporal sub-layer access (STSA) unit type. Furthermore, the decoder 100 may be configured in response to the aforementioned flag to refrain from parsing one or more NAL units 304 of the stepwise temporal sub-layer access (STSA) unit type and to implicitly infer the decoding of the stepwise temporal sub-layer access, i.e., the STSA constraints are not applied to the respective access unit.

[0076] This concept may also be applied independently of mixing NAL unit types, as explained above. Thus, according to one embodiment, a decoder 100 for decoding a video content 12 is proposed, the decoder 100 being configured to decode a bitstream 11 and to decode a constraint flag from a parameter set of the bitstream 11, the aforementioned constraint flag indicating to the decoder 100 that any access unit 30 comprising a NAL unit 302 of a non-IRAP leading picture NAL unit type (e.g., RASL or RADL) in the bitstream 11 satisfies a constraint related to a stepwise temporal sublayer access (STSA) unit type.

[0077] 1.3 Mixing different RAP types A further important case that is not allowed by the prior art is to allow mixing pictures of an open Group of Pictures (GOP) structure with pictures of a closed GOP structure. For example, mixing of two different types of IRAP NAL unit types (e.g., IDR, CRA), e.g., IDR_W_RADL and CRA NAL unit types, may be allowed, or mixing of NAL units of the same NAL unit type, e.g., CRA and CRA, may be allowed. ·IDR_W_RADL + CRA · CRA + CRA

[0078] As mentioned above, this embodiment shall allow mixing IRAP NAL units of NAL unit types associated with open GOP structures and IRAP NAL units of NAL unit types associated with closed GOP structures, for example, a CRA with a RADL may be associated with a closed GOP structure, while a CRA with a RASL may be associated with an open GOP structure.

[0079] Thus, according to an embodiment, a decoder 100 for decoding a video content 12 may be provided, the decoder 100 being configured to decode a bitstream 11 comprising a first sub-bitstream 11-1 related to a first spatial segment 131 of a picture 12a, 12b, 12c of the video content 12 and a second sub-bitstream 11-2 related to a second spatial segment 132 of a picture 12a, 12b, 12c of the video content 12. According to this example, the bitstream 11 comprises at least one access unit 30 of an IRAP NAL unit type, in which at least one first NAL unit 302 of the first sub-bitstream 11-1 is mixed with at least one second NAL unit 304 of a further (same or different) second sub-bitstream 11-2 of an IRAP NAL unit type.

[0080] There is at least one use case where this is extremely useful. For example, one approach to 360-degree video transmission with tiled streaming is where the entire low-resolution video content is transmitted (whether or not parts of it are also transmitted as high-resolution tiles). In such a case, the change in viewing direction and the corresponding change in high-resolution tile selection from one viewport to another are illustrated in Figure 5.

[0081] Since the entire low-resolution video content is always available at the client side and no stream switching per tile is required for this region, the low-resolution content can be coded with CRA, thus taking advantage of higher coding efficiency compared to closed GOP structures using IDR. Also, the longer random access point period required for stream switching is sufficient for this region, again benefiting the coding efficiency of the low-resolution video. On the other hand, high-resolution tiles may change frequently in fast progression from one viewport to another, as shown in Figure 5, and therefore, using CRA and associated RASL pictures does not allow smooth switching of viewports, as in the case of low-resolution videos. This is because RASL regions in pictures representing repositioned or newly added tile streams cannot be properly decoded (due to lack of references) and are discarded when a change of viewport is encountered.

[0082] 7 shows an example in which a NAL unit 304 associated with a closed GOP structure is mixed with a NAL unit 302 associated with an open GOP structure. In this particular, but non-limiting, example, a NAL unit 304 of CRA type with a leading RADL 314 (closed GOP) is mixed with a NAL unit 302 of CRA type with a leading RASL 312 (open GOP). This non-limiting example shows the coding structure of a high resolution region in the second sub-bitstream 11-2 using a closed GOP structure with CRA in combination with a leading picture of RADL type and a low resolution region in the first sub-bitstream 11-1 using an open GOP structure with CRA in combination with a leading picture of RASL type.

[0083] According to one embodiment, the decoder 100 is configured to process a bitstream 11, where the IRAP NAL unit type of the NAL unit 304 of the second sub-bitstream 11-2 is an IRAP NAL unit type associated with a closed GOP structure and the IRAP NAL unit type of the NAL unit 302 of the first sub-bitstream 11-1 is an IRAP NAL unit type associated with an open GOP structure.

[0084] In this particular non-limiting embodiment as shown in FIG. 7, the IRAP NAL unit type of the NAL unit 304 of the second sub-bitstream 11-2 is a CRA unit type with a random access decodable picture unit type (CRA with RADL) and the IRAP NAL unit type of the NAL unit 302 of the first sub-bitstream 11-1 is a CRA unit type with a random access skip leading picture unit type (CRA with RASL).

[0085] It should be noted that Fig. 7 illustrates a sequence of access units and the pictures contained therein in their display order, although the display order may differ from the coding order, also referred to as the bitstream order. As can be seen, the access unit 30 includes a mixed picture 12a that includes a first VCL NAL unit 302 representing a first spatial segment 131a of the picture 12a in the first sub-bitstream 11-1 and a second VCL NAL unit 304 representing a second spatial segment 132a of the picture 12a in the second sub-bitstream 11-2. In this case, the first and second NAL units are IRAP NAL units. In this particular but non-limiting example, both the first and second NAL units 302, 304 are of IRAP NAL unit type CRA. Fig. 7 also illustrates the display order of the successive pictures.

[0086] Thus, in the display order, a further access unit 31 may precede the above mentioned access unit 30. The above mentioned leading access unit 31 includes a mixed picture 12b including a first VCL NAL unit 312 representing a first spatial segment 131b of a picture 12b in the first sub-bitstream 11-1 and a second VCL NAL unit 314 representing a second spatial segment 132b of a picture 12b in the second sub-bitstream 11-2. The first NAL unit 312 of this AU 31 is of non-IRAP NAL unit type RASL and the second NAL unit 314 of this AU 31 is of non-IRAP NAL unit type RADL. Figure 7 also shows the display order of successive pictures.

[0087] Optionally, further in display order, one or more further access units, such as the exemplarily depicted access unit 32, may precede the access unit 31 described above. More generally, the access unit 31 comprising non-IRAP NAL units 312, 314 (e.g., RASL or RADL) may precede in display order at least one further access unit 32, which may also preferably comprise non-IRAP NAL units 322, 324 (e.g., RASL or RADL). To maintain an open GOP structure in the first sub-bitstream 11-1, the RASL NAL unit 312 of the access unit 31 may be preceded (in display order) by a NAL unit 322 contained in the preceding access unit 32 (in display order) and which may be of RASL type. Then, to maintain a closed GOP structure in the second sub-bitstream 11-2, the RADL NAL unit 314 of the access unit 31 may be preceded (in display order) by a NAL unit 324 contained in the preceding access unit 32 (in display order) and which may also be of RADL type.

[0088] The coding order, also called bitstream order, may differ from the display order shown in Fig. 7. For example, RADL and RASL pictures 131b, 132b, 131c, 132c are dependently coded, which means that CRA pictures 131a, 132a (in display order) derive differentially coded information from their respective CRA pictures 13. For example, RADL and RASL pictures 131b, 132b included in access unit 31 refer to CRA pictures 131a, 132a (in display order) included in access unit 30, even though access unit 30 follows access unit 31 (in display order).

[0089] RASL picture 131b may optionally further reference pictures (in display and coding order), such as the preceding RASL picture 131c illustrated in Fig. 7, and thus the preceding RASL picture 131c of the preceding access unit 32 must be coded before RASL picture 13b of access unit 31. However, the further preceding RASL picture 131c of the preceding access unit 32 may itself reference a picture that follows in display order but precedes in coding order, such as IRAP picture 131a of access unit 30. That is, IRAP (CRA) picture 131a of access unit 30 is the last picture in the aforementioned order (i.e., CRA) picture 13, but IRAP (CRA) picture 131a of access unit 30 is the first picture in the coding order, i.e., must be coded first, since both RASL pictures 131b, 131c refer to IRAP (CRA) picture 131a during coding.

[0090] The same is true for RADL pictures: RADL picture 132b may further reference any preceding picture (in display and coding order), such as preceding RADL picture 132c illustrated in Figure 7, and thus further previous and subsequent RADL pictures 132c of the preceding access unit 32 must be coded before RADL picture 132b of access unit 31. However, further preceding RADL picture 132c of the preceding access unit 32 may itself reference a picture that follows in display order but precedes in coding order, such as IRAP picture 132a of access unit 30. That is, although the IRAP (CRA) picture 132a of access unit 30 is the last picture in the aforementioned order (i.e., CRA) picture 13, the IRAP (CRA) picture 132a of access unit 30 is the first picture in the coding order, i.e., must be coded first, because both RADL pictures 132b, 132c reference the IRAP (CRA) picture 132a during coding.

[0091] More generally, non-IRAP pictures of leading picture (LP) type (e.g., RASL and RADL pictures 131b, 131c, 132b, 132c) can reference following IRAP pictures (e.g., CRA pictures 131a, 132a) (e.g., in display order), and the IRAP pictures (e.g., CRA pictures 131a, 132a) must be coded first, i.e., before leading non-IRAP pictures of leading picture (LP) type (e.g., RASL and RADL pictures 131b, 131c, 132b, 132c) (e.g., RASL and RADL pictures 131b, 131c, 132b, 132c). That is, the IRAP pictures (CRA pictures 131a, 132a, etc.) follow the display order of the non-IRAP pictures of leading picture (LP) type (RASL and RADL pictures 131b, 131c, 132b, 132c, etc.), but the IRAP pictures (CRA pictures 131a, 132a, etc.) are coded first. In other words, non-IRAP pictures of leading picture (LP) type (e.g., RASL and RADL pictures 131b, 131c, 132b, 132c) lead IRAP pictures (e.g., CRA pictures 131a, 132a) in display order, and IRAP pictures (e.g., CRA pictures 131a, 132a) lead non-IRAP pictures of leading picture (LP) type (e.g., RASL and RADL pictures 131b, 131c, 132b, 132c) in bitstream order (coding order). In other words, non-IRAP pictures of leading picture (LP) type (e.g., RASL and RADL pictures 131b, 131c, 132b, 132c) lead the IRAP pictures (e.g., CRA pictures 131a, 132a) in display order, but non-IRAP pictures of loading picture (LP) type (e.g., RASL and RADL pictures 131b, 131c, 132b, 132c) follow the IRAP pictures (e.g., CRA pictures 131a, 132a) in coding / bitstream order.

[0092] Considering the above cases and coding structure, it is an advantage of the present invention that a mixture of CRA with leading RASL (open GOP) and CRA with leading RADL (closed GOP) in a picture of an access unit 30 is allowed. When such a mixed NALU access unit 30 is encountered in a Coded Video Sequence (CVS) during normal playout (decoding the entire bitstream 11 from the beginning), all the necessary reference pictures will be available. Therefore, the pictures following the mixed picture with RASL NAL unit will be decoded and output normally.

[0093] However, when such a mixed NALU is encountered during a seek operation, or when the decoding process starts from such an AU during random access, a process for generating unavailable reference pictures needs to be invoked and is either: · The subsequent processing chain needs to be informed that the RASL region was not decoded correctly and that the corresponding low resolution region is to be used, e.g. through an SEI instruction. Affected pictures will be completely removed from the output.

[0094] This means that when starting to decode a mixed picture, one option is to treat it as a GDR picture, where some parts are decodable and can be shown, and other parts are not and are refreshed over time (until all RASL NAL units are finished). Such behavior is relevant, for example, to a 360 scenario where the RAP type is selected opposite to the way shown in Figure 7, i.e., the low resolution uses a closed GOP structure (e.g., CRA with RADL) and the high resolution content uses an open GOP structure (e.g., CRA with RASL). Then it can show the low resolution, since it uses a closed GOP structure and the player waits until the high resolution using the open GOP structure is cleanly decoded to show it.

[0095] Another option to handle such a situation is to not display any of the pictures that are affected (not even partially) by the missing references and to drop them, i.e., drop any mixed pictures with a NAL unit type equal to RASL.

[0096] In other words, any mixed picture 12b, 12c with a NAL unit type 312, 322 equal to RASL may be marked to be dropped / discarded to ensure continuous decoding behavior.

[0097] Thus, according to one embodiment, the decoder 100 may be configured to decode and present, in bitstream order, the co-located spatial segments 132b, 132c, ... of one or more pictures 12b, 12c, ... of access units 31, 32, ... that follow, in bitstream order, at least one access unit 30 to which the NAL unit 304 of the second sub-bitstream 11-2 is associated, in which the co-located spatial segments 132b, 132c, ... are arranged together with the second spatial segment 132a of picture 12a of the at least one access unit 30 to which the NAL unit 304 of the second sub-bitstream 11-2 is associated. Furthermore, the decoder 100 may be configured to mark for discard co-located spatial segments 131b, 131c, ... of one or more pictures 12b, 12c, ... of access units 31, 32, ... that follow at least one access unit 30 to which the NAL unit 302 of the first sub-bitstream 11-1 is associated, where the co-located segments 131b, 131c, ... are aligned together with the first spatial segment 131a of picture 12a of the at least one access unit 30 to which the NAL unit 302 of the first sub-bitstream 11-1 is associated.

[0098] According to further embodiments, the decoder 100 may be configured to operate depending on whether the bitstream 11 is to be decoded in the case of continuous decoding during normal playout (decoding the entire bitstream 11 from the beginning) or in the case of the start of decoding (e.g., during a seek operation or during random access, the decoding process starts from such an AU).

[0099] During normal playout (decoding the entire bitstream 11 from the beginning), the decoder 100 can be configured to decode and present the co-located spatial segments 132b, 132c, ... of one or more pictures 12b, 12c, ... of access units 31, 32, ... that follow in bitstream order at least one access unit 30 (e.g., CRA with RADL) to which the NAL unit 304 of the second sub-bitstream 11-2 is associated, where the co-located spatial segments 132b, 132c, ... are aligned together with the second spatial segment 132a of the picture 12a of the at least one access unit 30 to which the NAL unit 304 of the second sub-bitstream 11-2 is associated. Furthermore, in case of decoding start (e.g., during a seek operation or when the decoding process starts from such an AU during random access), the decoder 100 may be configured to discard one or more pictures 12b, 12c, ... of access units 31, 32, ... that follow in bitstream order the at least one access unit 30, and to resume picture output after the one or more discarded pictures 12b, 12c, .... In other words, any mixed picture 12b, 12c having at least one NAL unit 312, 322 equal to RASL is dropped / discarded.

[0100] According to a further embodiment, the decoder 100 may be configured to mark said corrupted spatial segments 131b, 131c, ... for the subsequent processing chain by means of an SEI message.

[0101] In a further embodiment, an indication is present in the bitstream that the AU is treated as a CRA, i.e., an IRAP with NoIncorrectPicOutputFlag equal to 1, regardless of whether the IDR_W_RADL NAL unit type is present or not, indicating that the AU is a mixed NALU, but has the mentioned characteristics of an IRAP and an open GOP structure. In other words, the decoder 100 may be configured to derive an indication from the bitstream 11 indicating towards the decoder 100 to decode at least one access unit 30 associated with the first sub-bitstream 11-1 and the second sub-bitstream 11-2 in a manner corresponding to the clean random access NAL unit type.

[0102] In the above description, an embodiment has been described in which the decoder 100 is configured to process a bitstream 11. A) a non-IRAP NAL unit of a non-IRAP NAL unit type (e.g., RASL, RADL, TRAIL, STSA) is mixed with at least one different second NAL unit of a different non-IRAP NAL unit type (e.g., RASL, RADL, TRAIL, STSA), or B) A NAL unit of an IRAP NAL unit type (eg, IDR_W_RADL, CRA) is mixed with at least one NAL unit of a further IRAP NAL unit type (eg, IDR_W_RADL, CRA).

[0103] However, there may be further embodiments in which the decoder 100 may be configured to process the bitstream 11, where C) NAL units of IRAP NAL unit types (IDR_W_RADL, CRA) are mixed with at least one NAL unit of a non-IRAP leading picture NAL unit type (eg, RASL, RADL) or one of the STSA NAL unit types.

[0104] Further embodiments also propose to provide a corresponding device for processing a video bitstream 11 during encoding of the bitstream 11 and / or merging of the first and second sub-bitstreams 11-1, 11-2. The above mentioned device 10 may be at least one of an encoder, a merger or a network node for processing the bitstream 11 according to the innovative principles described herein.

[0105] Thus, according to an embodiment, an apparatus 10 for processing a video content 12 is proposed, the apparatus 10 being configured to provide a first sub-bitstream 11-1 related to a first spatial segment 131 of a picture 12a, 12b, 12c of the video content 12 and to provide a second sub-bitstream 11-2 related to a second spatial segment 132 of a picture 12a, 12b, 12c of the video content 12, the first sub-bitstream 11-1 and the second sub-bitstream 11-2 being part of a bitstream 11, A) the bitstream 11 includes at least one access unit 30 in which at least one first NAL unit 302 of a first sub-bitstream 11-1 is a non-IRAP NAL unit type that is mixed with at least one different second NAL unit 304 of a second sub-bitstream 11-2 that is a different non-IRAP NAL unit type, or B) the bitstream 11 includes at least one access unit 30 of an IRAP NAL unit type in which at least one first NAL unit 302 of a first sub-bitstream 11-1 is mixed with at least one different second NAL unit 304 of a second sub-bitstream 11-2 of a further IRAP NAL unit type, or C) The bitstream 11 includes at least one access unit (30) in which at least one first NAL unit 302 of the first sub-bitstream 11-1 is an IRAP NAL unit type mixed with at least one different second NAL unit 304 of the second sub-bitstream 11-2, which is either a non-IRAP leading picture NAL unit type (RASL or RADL) or an STSA NAL unit type.

[0106] 1.4 Summary and Syntax Examples Below is an overview of the types of mixtures permitted within the present invention. [Table 1]

[0107] In the new mixture type described in this invention, there are two types of mixtures, one with IRAP type and the other without IRAP type.

[0108] Option 1 As mentioned above in the previous section, one option is to only signal mixing with IRAP types with flags in the PPS, for example reusing the existing mixing of IDR or CRA and TRAIL up to VCL_RSV_6 (mixed_nalu_types_in_pic_flag). Types other than IRAP can be signaled using constraint flags in the SPS / VPS / DPS.

[0109] mixed_nalu_types_in_pic_flag equal to 1 specifies that each picture that references the PPS has one or more VCL NAL units, that the VAL NAL units do not have the same value of nal_unit_type and have values ​​of nal_unit_type in the range from IDR_W_RADL to CRA_NUT, and that the picture is not an IRAP IDR picture. mixed_nalu_types_in_pic_flag equal to 0 specifies that each picture that references the PPS has one or more VCL NAL units, and that if the VCL NAL units have values ​​of nal_unit_type in the range from IDR_W_RADL to CRA_NUT, then the VCL NAL units of each picture that references the PPS have the same value of nal_unit_type.

[0110] Then, the NAL unit types of a coded picture MUST be CRA_NUT and IDR_W_RADL if the AU is the first AU in the bitstream or the first AU after a sequence (EOS) NAL unit that references a PPS with mixed_nalu_types_in_pic_flag equal to 1. Mixed pictures with IDR / CRA having a non-IRAP type are only allowed if the AU is not the first picture in the bitstream or in a sequence.

[0111] no_non_irap_mixed_nalu_types_in_pic_constraint_flag equal to 1 specifies that it is a bitstream conformance requirement that the VCL NAL units of a picture whose sequence contains TRAIL, STSA, RASL, RADL have the same value of nal_unit_type. A value equal to 0 does not impose such a constraint, i.e. a picture may have two different values ​​of nal_unit_type among TRAIL, STSA, RASL, RADL.

[0112] Option 2 Another option would be to specify an indication (idc) to indicate the different combinations instead of the PPS flag (mixed_nalu_types_in_pic_flag).

[0113] no_mixed_nalu_types_in_pic_constraint_flag equal to 1 specifies that it is a bitstream conformance requirement that mixed_nalu_types_in_pic_idc equal to 0. no_mixed_nalu_types_in_pic_constraint_flag equal to 0 imposes no such constraint.

[0114] mixed_nalu_types_in_pic_idc equal to 0 specifies that there is one or more VCL NAL units in each picture that references the PPS, and the VCL NAL units in each picture that references the PPS have the same value of nal_unit_type.

[0115] mixed_nalu_types_in_pic_idc equal to 1 specifies that each picture that references the PPS has multiple VCL NAL units, and all VCL NAL units have two specific values ​​of nal_unit_type ranging from TRAIL_NUT to RSV_VCL_6. [Table 2]

[0116] mixed_nalu_types_in_pic_idc equal to 2 specifies that each picture that references the PPS has multiple VCL NAL units, and one or more of the VCL NAL units all have a specific value of nal_unit_type ranging from IDR_W_RADL to CRA_NUT, and the other VCL NAL units all have a specific value of nal_unit_type ranging from TRAIL_NUT to RSV_VCL_6 or equal to GRA_NUT.

[0117] This value corresponds to the allowed mix of NAL units with existing flags. [Table 3]

[0118] mixed_nalu_types_in_pic_idc equal to 3 specifies that each picture that references the PPS has multiple VCL NAL units with a nal_unit_type value of IDR_W_RADL and one or more VCL NAL units with a nal_unit_type value of CRA_NUT. [Table 4]

[0119] For any particular picture's VCL NAL units, the following applies: If mixed_nalu_types_in_pic_idc is equal to 0, the value of nal_unit_type is the same for all coded slice NAL units of a picture. A picture or PU is said to have the same NAL unit type as the coded slice NAL units of the picture or PU. When mixed_nalu_types_in_pic_idc is 1 or 2, the following applies: If one of the VCL NAL units in a picture has a value of nal_unit_type equal to TRAIL_NUT, the picture or PU is called a trailing picture or trailing PU. Otherwise (there are no TRAIL_NUT VCL NAL units within the picture), the picture or PU is said to be a leading picture or leading PU. If one of the VCL NAL units in a picture has a value of nal_unit_type equal to RADL_NUT, the picture or PU is called a RADL picture or RADL PU. Otherwise (if no VCL NAL unit in the picture has a value of nal_unit_type equal to RADL_NUT), the picture or PU is called a RASL picture or RASL PU. Otherwise (mixed_nalu_types_in_pic_idc equals 3), the picture is called a CRA picture or a CRA PU.

[0120] Aspects related to treating the primary NAL unit type as an STSA picture may be implemented using constraint flags as follows. [Table 5]

[0121] leading_stsa_pictures_flag equal to 1 constrains the inter prediction references of RADL and RASL pictures as follows: no_mixed_nalu_types_in_pic_constraint_flag equal to 0 does not impose such constraints.

[0122] When leading_stsa_pictures_flag is equal to 1, the following applies: When the current picture is a RASL or RADL picture, there is no active entry in RefPicList[0] or RefPicList[1] with TemporalId equal to the current picture's TemporalId. When the current picture is a picture that, in decoding order, follows a RASL or RADL and precedes an associated IRAP picture with a TemporalId equal to that of the current picture, there shall be no picture with a TemporalId equal to that of the current picture included as an active entry in RefPicList[0] or RefPicList[1] preceding the RASL or RADL picture in decoding order.

[0123] In other words, the restriction related to the STSA picture prohibits a picture related to the next higher temporal sub-layer than STSA according to the decoding order from using any reference in the next higher temporal sub-layer preceding STSA, i.e. the decoder can start decoding the next temporal sub-layer from STSA onwards. Such a restriction can of course also be implemented for RASL / RADL pictures as described above, and thus the above signaling can serve as an indication of these properties.

[0124] 2. Mixing picture types within a multi-layer access unit This subsection may concern, as a non-limiting example, a multi-layer access unit in a multi-layer bitstream. However, the features described herein may already be applied to one single layer. Furthermore, this subsection describes the principles of the present invention by referring to long-term reference pictures, as a non-limiting example. However, the concepts of the present invention are equally applicable to short-term reference pictures, i.e., the features described herein may be applied to pictures that refer generally. Furthermore, this subsection may describe, as a non-limiting example, a POC MSB reset by setting the POC value to zero, i.e., setting MSB=0. However, it is also possible that the value zero only acts as a placeholder symbolizing a decreasing value of the MSB, e.g., the MSB reset may occur to any predefined value that is not equal to 0 but is less than the current MSB in the respective bitstream position.

[0125] Prior art may provide support for mixing pictures of different RAP types in a layered access unit by signaling the most significant POC (Picture Order Count) bit (poc_msb_val), i.e. for time instances where the access unit contains simultaneously RAP and non-RAP pictures in its layers. However, problems may remain for reference pictures of the Leading Picture NAL unit type, as illustrated below by the example of Long Term Reference Pictures, which involve references to pictures preceding their associated IRAP pictures in coding order, but are also applicable to Short Term Reference Pictures, for which the present invention provides a solution.

[0126] The current POC signaling related syntax is as follows: [Table 6] [Table 7]

[0127] An example usage of the above POC MSB signaling is to set the POC MSB to 0 when all layers have the same IRAP NAL unit type, e.g., all layers are IDR_W_RADL or CRA, as shown in FIG.

[0128] Figure 8 shows a multi-layer bitstream 11 consisting of a first sub-bitstream 11-1 of a first layer L0 and a second sub-bitstream 11-2 of a second layer L1. Each layer, L0, may comprise one or more temporal sub-layers T0, T1, T2. Thus, spatial segments of a picture may be distributed across the different layers L0, L1 and temporal sub-layers T0, T1, T2 mentioned above.

[0129] FIG. 8 further illustrates some exemplary multi-layer access units 30, 31, 32. In this example, the multi-layer access units 30, 31, 32 may comprise a first NAL unit 302 of a first sub-bitstream 11-1 in a first layer L0 mixed with a second NAL unit 304 of a second sub-bitstream 11-2 in a second layer L1. The NAL units may comprise different NAL unit types, in particular IRAP and non-IRAP NAL unit types, as explained above. Furthermore, the NAL units may comprise slice headers in which a picture order count (POC) is signaled, as mentioned above. The concepts described herein relate to the most significant POC parts, e.g., most significant POC bits, or POC MSBs. Thus, in the figure, the POC MSBs are indicated in each IRAP NAL unit.

[0130] For example, multi-layer access units 30, 32 represent access units in which the POC MSB of a first NAL unit 302 in a first sub-bitstream 11-1 is aligned with the POC MSB of a second NAL unit 304 in a second sub-bitstream 11-2, both NAL units being of the IRAP NAL unit type.

[0131] The multi-layer access unit 31 represents a non-aligned access unit, i.e., the POC MSB of the first NAL unit 302 in the first sub-bitstream 11-1 is non-aligned with the POC MSB (e.g., MSB=0) of the second NAL unit 304 of the second sub-bitstream 11-2 of non-IRAP NAL unit type. Thus, the POC MSB of the first NAL unit 302 of the first sub-bitstream 11-1 may be increased by a predetermined value, for example, by 1.

[0132] Figure 8 shows how the aforementioned MSB signaling in the slice header can be used in such a multi-layer case. The purpose of the signaling is to keep the POC value correct in unaligned RAPs, e.g. unaligned access unit 31 with CRA w MSB = 1, i.e. to avoid the POC (LSB and MSB) being reset to zero when all layers do not have an IRAP in the same access unit.

[0133] However, Figure 8 also shows that there is a problem with reference pictures 12r (e.g., long-term and / or short-term) when the POC is aligned. For ease of understanding, the following figures may refer only to long-term reference pictures. However, the concept is also applicable to short-term reference pictures.

[0134] It should be noted that the pictures 12a, 12b, 12c, ..., before the last CRA access unit 32, i.e., the second aligned RAP, have POC MSB>0 and POC LSB>0. Therefore, when resetting the MSB in the last CRA access unit 32 to 0 (as exemplarily shown in FIG. 8), it is not possible to refer to any of the previous pictures 12a, 12b, 12c, ..., using the POC MSB or its delta (used for long-term reference pictures). One such example is shown in FIG. 8 by the green arrow marked with "LT ref". Thus, the example shown in FIG. 8 is meant to show that picture reference may not be possible in this case.

[0135] Therefore, the present invention provides a solution to this problem so that picture referencing becomes possible.

[0136] In one embodiment, the constraint is expressed or indicated in the specification through bitstream flags (SPS, VPS, DPS) whose MSBs can be set to 0 only when all pictures in the access unit are of type IDR, i.e., IDR_W_RADL or IDR_N_LP, so that when CRA is used, no POC reset occurs and LT references are allowed (see Figures 9 and 10 for examples).

[0137] As can be seen in Figure 9, the access unit 32 may be an aligned access unit 32 in which both NAL units 302, 304 are IRAP NAL unit type. However, since the first NAL unit 302 of the first sub-bitstream 11-1 is of CRA type, no POC reset occurs. That is, Figure 9 illustrates a scenario in which POC reset is not allowed.

[0138] Instead, FIG. 10 illustrates a scenario in which a POC reset is permitted because the first NAL unit 302 of the first sub-bitstream 11-1 and the second NAL unit 304 of the second sub-bitstream 11-2 are both IDR unit types.

[0139] Thus, according to an embodiment, a decoder 100 for decoding a video content 12 is provided, the decoder 100 is configured to decode a bitstream 11 comprising a first sub-bitstream 11-1 and a second sub-bitstream 11-2, the decoder 100 is configured to derive from slice headers of NAL units 302, 304 of the first and second sub-bitstreams 11-1, 11-2, where the NAL units 302, 304 have an IRAP NAL unit type (e.g., CRA, IDR_W_LP, IDR_N_LP), information for setting the Most Significant POC Part (POC MSB) of the POC of the picture 12r to which the NAL units 302, 304 refer. According to this example, the Most Significant POC Part (POC MSB) needs to be set to a predefined value for all NAL units 302, 304 of IRAP NAL unit type within an access unit 32 of the bitstream 11 if at least one NAL unit 302, 304 within said access unit 32 is of a non-Instantaneous Decoding Refresh unit type, e.g., a CRA unit type.

[0140] As mentioned above, this subsection may describe, as a non-limiting example, the POC MSB reset by setting the POC value to zero, i.e., by setting MSB=0. However, it is also possible that the value zero only serves as a placeholder symbolizing a decreasing value of the MSB, and that, for example, the MSB reset may occur to any predefined value that is not equal to 0 but is less than the current MSB in the respective bitstream position. Thus, the predefined value mentioned above may be equal to or greater than the most significant POC portion of the POC associated with the pictures 12a, 12b, ... preceding the picture 12r to which the NAL unit 302 refers. In other words, if all NAL units 302, 304 in the access unit 32 are not of IDR unit type, no POC reset is performed.

[0141] In some examples, the above-mentioned predetermined value may be equal to zero, i.e., MSB = 0. In some other examples, the above-mentioned predetermined value may be equal to an implicitly derived most significant POC portion, for example, an implicit derivation of a normal SOTA POC MSB.

[0142] In another embodiment, there may be a constraint expressed in the specification or indicated via bitstream flags (SPS, VPS, DPS) that whenever an aligned RAP access unit 32 occurs and the POC MSB of the access unit 32 is set to 0, the (long-term or short-term) reference pictures 12a, 12b, 12c, ... preceding the aligned RAP access unit 32 are allowed to be referenced by the picture 12s following the aligned RAP access unit 32 in bitstream order.

[0143] For example, FIG. 11 shows that picture 12s may precede picture 12r in presentation order, but picture 12s may follow picture 12r in decoding order.

[0144] 11 and 12 show that any picture 12r with MSB equal to 0 does not have a leading picture 12a, 12b, 12c, ... with a LT reference picture (whereas FIG. 11 represents a reset from MSB=1 to MSB=0 and FIG. 12 represents an MSB equal to 0, for example, because the POC_LSB is long enough that no increment occurs). Thus, FIG. 11 shows a scenario where POC MSB reset and picture references are not allowed, and FIG. 12 shows a scenario where all POC MSBs are equal to zero, but picture references are not allowed. FIG. 13 shows that LT reference pictures are allowed for leading pictures if the MSB is not equal to 0.

[0145] Thus, according to an embodiment, a decoder 100 for decoding a video content 12 is provided, the decoder 100 is configured to decode a bitstream 11 including a first sub-bitstream 11-1 and a second sub-bitstream 11-2, the decoder 100 is configured to derive from slice headers of NAL units 302, 304 of the first and second sub-bitstreams 11-1, 11-2, where the NAL units 302, 304 are IRAP NAL unit types (e.g., CRA, IDR_W_LP, IDR_N_LP), information for setting the most significant POC part (POCMSB) of the POC of the picture 12r to which the NAL units 302, 304 refer. According to this example, no picture reference is allowed to intersect, in decoding order, with any access unit 32 of the bitstream 11 having a slice header in which all NAL units 302, 304 are IRAP NAL unit types and the most significant POC part (POC MSB) is set to a predefined value.

[0146] Again, as mentioned above, this subsection may, as a non-limiting example, describe the POC MSB reset by setting the POC value to zero, i.e., by setting MSB=0. However, it is also possible that the value zero acts only as a placeholder symbolizing a decreasing value of the MSB, e.g., the MSB reset may occur to any predefined value that is not equal to 0 but is smaller than the current MSB in the respective bitstream position. Thus, the predefined value in the example described with reference to Figures 11-13 may be smaller than the most significant POC portion (POC MSB) of the POC associated with the picture (12a, 12b, 12c, ...) preceding the picture (12r) referenced by the NAL unit (302, 304), i.e., the one at which the POC reset occurred.

[0147] In some examples, the above-mentioned predetermined value may be equal to zero, i.e., MSB = 0. In some other examples, the above-mentioned predetermined value may be equal to an implicitly derived most significant POC portion, for example, an implicit derivation of a normal SOTA POC MSB.

[0148] In another embodiment, there may be a constraint, expressed in the specification or indicated through bitstream flags (SPS, VPS, DPS), that whenever an aligned RAP access unit 32 occurs and the POC MSB of the access unit 32 is set to 0 (POC reset), reference pictures 12a, 12b, 12c, ... (short-term or long-term) having an MSB different from 0 before the aligned RAP access unit 32 are allowed to be referenced in bitstream order by pictures 12s following the aligned RAP access unit 32.

[0149] Note that if there was no MSB increase in the above example shown in FIG. 8, there would be no problem using the LT reference picture, since the reference of the LT reference picture is done only by the POC_LSB value.

[0150] 14 shows that in this example, no POC MSB reset has occurred. A POC MSB reset can be indicated by either increasing the POC MSB value of access unit 32 (in the example shown in FIG. 14, the POC MSB value is increased, i.e., MSB=3) or leaving the POC MSB value of access unit 32 at the same value as the previous access unit 31.

[0151] Thus, according to one embodiment, a decoder 100 for decoding a video content 12 is provided, the decoder 100 being configured to decode a bitstream 11 comprising a first sub-bitstream 11-1 and a second sub-bitstream 11-2, the decoder 100 being configured to derive from slice headers of NAL units 302, 304 of the first and second sub-bitstreams 11-1, 11-2, where the NAL units 302, 304 are IRAP NAL unit types (e.g., CRA, IDR_W_LP, IDR_N_LP), information for setting the most significant POC part (POC MSB) of the POC of the picture 12r to which the NAL units 302, 304 refer. According to this example, any picture reference of reference pictures 12a, 12b, 12c, ... of a first access unit 31 consisting of NAL units 301, 303 with their most significant POC parts set to a first predetermined value and with slice headers preceding in bitstream order is not allowed by a picture 12s following in bitstream order the second access unit 32 of bitstream 11 whose NAL units 302, 304 are of IRAP NAL unit type (e.g., CRA, IDR_W_LP, IDR_N_LP) and whose slice headers have their most significant POC parts (POC MSB) set to a second predetermined value.

[0152] Again, as mentioned above, this subsection may describe, as a non-limiting example, a POC MSB reset by setting the POC value to zero, i.e. by setting MSB=0. However, it is also possible that the value zero only serves as a placeholder symbolizing a decreasing value of the MSB, e.g. the MSB reset may occur to any predefined value not equal to 0 but less than the current MSB in the respective bitstream position. Thus, with respect to the example described with reference to FIG. 14, the first predefined value of the above-mentioned POC MSB of the preceding first access unit 31 may be greater than the second predefined value of the above-mentioned POC MSB of the following second access unit 32. In this case, picture reference is not allowed. Conversely, it may be stated that picture reference is allowed if the second predefined value of the MSB of the following second access unit 32 is greater than the first predefined value of the above-mentioned POC MSB of the previous first access unit 31. That is, if a POC reset did not occur, picture reference is allowed.

[0153] For example, the second predetermined value may be equal to zero to indicate a POC reset. Thus, no picture reference is allowed when a POC reset occurs. In some other examples, the second predetermined value may be equal to an implicitly derived most significant POC portion, such as an implicit derivation of a normal SOTA POC MSB.

[0154] Although some aspects have been described in the context of an apparatus, it will be apparent that these aspects also represent a description of a corresponding method, with the blocks or devices corresponding to method steps or features of a method step. Similarly, aspects described in the context of a method step also represent a description of a corresponding block or item or feature of a corresponding apparatus.

[0155] Some or all of the method steps may be performed by (or using) a hardware apparatus, such as, for example, a microprocessor, a programmable computer, or an electronic circuit. In some embodiments, one or more of the most important method steps may be performed by such an apparatus.

[0156] Depending on the particular implementation requirements, embodiments of the present invention can be implemented in hardware, or in software, or at least partially in hardware, or at least partially in software. The implementation can be performed using a digital storage medium, such as a floppy disk, DVD, Blu-ray, CD, ROM, PROM, EPROM, EEPROM, or FLASH memory, on which electronically readable control signals are stored that cooperate (or can cooperate) with a programmable computer system so that the respective method is performed. The digital storage medium may therefore be computer readable.

[0157] Some embodiments according to the invention comprise a data carrier having electronically readable control signals capable of cooperating with a programmable computer system to cause one of the methods described herein to be performed.

[0158] Generally, embodiments of the present invention can be implemented as a computer program product having program code which operates to perform one of the methods when the computer program product runs on a computer. The program code may for example be stored on a machine readable carrier.

[0159] Other embodiments comprise the computer program for performing one of the methods described herein, stored on a machine readable carrier.

[0160] In other words, an embodiment of the inventive method is, therefore, a computer program having a program code for performing one of the methods described herein, when the computer program runs on a computer.

[0161] A further embodiment of the inventive method is therefore a data carrier (or digital storage medium, or computer readable medium) having recorded thereon a computer program for performing one of the methods described herein. The data carrier, digital storage medium, or recording medium is typically tangible and / or non-transitory.

[0162] A further embodiment of the inventive method is therefore a data stream or a sequence of signals representing the computer program for performing one of the methods described herein. The data stream or the sequence of signals may for example be arranged to be transferred via a data communication connection, for example via the Internet.

[0163] A further embodiment comprises a processing means, for example a computer, or a programmable logic device, configured to or adapted to perform one of the methods described herein.

[0164] A further embodiment comprises a computer having installed thereon the computer program for performing one of the methods described herein.

[0165] Further embodiments according to the invention comprise an apparatus or system configured to transfer (e.g. electronically or optically) a computer program for performing one of the methods described herein to a receiver. The receiver may for example be a computer, a mobile device, a memory device, etc. The apparatus or system may for example comprise a file server for transferring the computer program to the receiver.

[0166] In some embodiments, a programmable logic device (e.g., a field programmable gate array) may be used to perform some or all of the functions of the methods described herein. In some embodiments, a field programmable gate array may cooperate with a microprocessor to perform one of the methods described herein. In general, the methods are preferably performed by any hardware apparatus.

[0167] The apparatus described herein may be implemented using a hardware apparatus, or using a computer, or using a combination of a hardware apparatus and a computer.

[0168] The methods described herein may be performed using a hardware apparatus, or using a computer, or using a combination of a hardware apparatus and a computer.

[0169] While the present disclosure has been described with reference to exemplary embodiments, this description is not intended to be construed in a limiting sense. Various modifications and combinations of the exemplary embodiments, as well as other embodiments of the present disclosure, will become apparent to those skilled in the art upon reference to the description. It is therefore intended that the appended claims cover any such modifications or embodiments.

Claims

1. A decoder (100) for decoding video content (12), comprising: The decoder (100) A first spatial segment (13) of a picture (12a, 12b, 12c) of said video content (12). １ ) a first sub-bitstream (11-1) related to A second spatial segment (13) of said pictures (12a, 12b, 12c) of said video content (12). ２ ) a second sub-bitstream (11-2) related to At least one coded picture, in which at least one first NAL unit (302) of the first sub-bitstream (11-1) is a non-IRAP NAL unit type; and a bitstream including: the coded picture is mixed with at least one different second NAL unit (304) of the second sub-bitstream (11-2) that is a different non-IRAP NAL unit type; the non-IRAP NAL unit type of the first NAL unit (302) of the first sub-bitstream (11-1) is a random access decodable reading unit type (RADL); the non-IRAP NAL unit type of the second NAL unit (304) of the second sub-bitstream (11-2) is a trailing picture unit type (TRAIL); A decoder (100).

2. A method for decoding video content (12), comprising the steps of: A first spatial segment (13) of a picture (12a, 12b, 12c) of said video content (12). １ ) a first sub-bitstream (11-1) related to A second spatial segment (13) of said pictures (12a, 12b, 12c) of said video content (12). ２ ) a second sub-bitstream (11-2) related to At least one coded picture, in which at least one first NAL unit (302) of the first sub-bitstream (11-1) is a non-IRAP NAL unit type; and decoding the bitstream including: the coded picture is mixed with at least one different second NAL unit (304) of the second sub-bitstream (11-2) that is a different non-IRAP NAL unit type; the non-IRAP NAL unit type of the first NAL unit (302) of the first sub-bitstream (11-1) is a random access decodable reading unit type (RADL); the non-IRAP NAL unit type of the second NAL unit (304) of the second sub-bitstream (11-2) is a trailing picture unit type (TRAIL); method.

Images