Video encoder, video decoder, encoding and decoding methods and video data streams for implementing advanced video encoding concepts

Abstract

An apparatus (200) for receiving an input video data stream according to an embodiment is provided. Video is encoded in the input video data stream. An apparatus (200) is configured to generate an output video data stream from an input video data stream.

Description

Division

[0001] This application is a divisional application of the application patent application with the application date of 21 May 2021, the application number 202180036764.4, and the invention name “Video encoder, video decoder, encoding and decoding method, and video data stream for implementing advanced video coding concepts”. TECHNICAL FIELD

[0002] The present invention relates to video encoding and video decoding, and in particular to a video encoder, a video decoder, an encoding and decoding method, and a video data stream for implementing advanced video coding concepts. BACKGROUND

[0003] H.265 / HEVC (HEVC = High Efficiency Video Coding) is a video codec that already provides tools for boosting or even enabling parallel processing at the encoder and / or decoder. For example, HEVC supports a subdivision of a picture into a set of tiles that are coded independently from each other. Another concept supported by HEVC relates to WPP, according to which CTU rows or CTU lines of a picture can be processed in parallel from left to right, e.g. in stripes, under the premise that some minimum CTU offset is respected when processing consecutive CTU rows. However, it would be advantageous to have a video codec that even more efficiently supports the parallel processing capabilities of a video encoder and / or a video decoder.

[0004] In the following, an introduction to VCL partitioning (VCL = Video Coding Layer) according to the prior art is described.

[0005] Generally, in video coding, the coding process of picture samples requires a small partitioning, in which samples are divided into rectangular regions for joint processing, e.g. prediction or transform coding. Therefore, pictures are partitioned into blocks of a certain size, which is constant during the encoding of a video sequence. In the H.264 / AVC (AVC = Advanced Video Coding) standard, a fixed size block of 16x16 samples, so-called macroblock, is used.

[0006] In the prior art HEVC standard (cf. [1]), there are coding tree blocks (CTB) or coding tree units (CTU) of a maximum size of 64x64 samples. In further descriptions of HEVC, the more common term CTU is used for such blocks.

[0007] CTUs are processed in a raster scan order (starting with the CTU in the top left, processing the CTUs in a picture row by row until the CTU in the bottom right).

[0008] Encoded CTU data is organized into a container called a slice. Initially, in previous video coding standards, a slice referred to a segment containing one or more consecutive CTUs of a picture. A slice was used for the segmentation of the encoded data. From another perspective, a complete picture can also be defined as one large segment, and therefore, historically, the term slice still applies. In addition to the encoded picture samples, a slice also includes additional information related to the encoding process of the slice itself, which is placed into a so-called slice header.

[0009] According to the state of the art, VCL (Video Coding Layer) also comprises techniques for segmentation and spatial partitioning. Such partitioning can be applied to video coding for various reasons, including among others parallelization of processing load balancing, CTU size matching in network transmission, error mitigation, etc.

[0010] Other examples are related to RoI (RoI = Region of Interest) coding, where for example in the middle of a picture there is a region that an observer can select, for example by a zoom operation (decoding of the RoI only) or gradual decoder refresh (GDR), where intra data (typically put into one frame of a video sequence) is distributed in time over several consecutive frames, for example as a column of intra blocks, which slide over the picture plane and locally reset the temporal prediction chain in the same way as an intra picture resets the temporal prediction chain for the whole picture plane. For the latter, there are two regions in each picture, one that is recently reset, and one that can be affected by errors and error propagation.

[0011] Reference Picture Resampling (RPR) is a technique for video coding that adapts the quality / rate of a video not only by using coarser quantization parameters but also by adapting the resolution of possibly each transmitted picture. Thus, the reference for inter prediction can have a different size than the picture that is currently being predicted for encoding. Basically, RPR requires a resampling process in the prediction loop, for example to define up- and down-sampling filters.

[0012] Depending on the style, RPR can lead to a change in the encoded picture size at any picture or be limited to occur only at certain specific pictures, for example only at specific locations that are delimited by for example segment boundary adaptive HTTP streaming. SUMMARY

[0013] It is an object of the present invention to provide an improved concept for video coding and video decoding.

[0014] It is an object of the present invention to provide an improved concept for video coding and video decoding.

[0015] According to a first aspect of the application, a device for receiving an input video data stream is provided. The input video data stream has a video encoded therein. The device is configured to generate an output video data stream from the input video data stream. Further, the device is to determine whether pictures of the video that precede a dependent random access picture should be output.

[0016] Further, a video data stream is provided. The video data stream has a video encoded therein. The video data stream comprises an indication whether pictures of the video that precede a dependent random access picture should be output.

[0017] Further, a video encoder is provided. The video encoder is configured to encode a video into a video data stream. The video encoder is configured to generate the video data stream such that the video data stream comprises an indication whether pictures of the video that precede a dependent random access picture should be output.

[0018] Further, a video decoder for receiving a video data stream having a video stored therein is provided. The video decoder is configured to decode the video from the video data stream. The video decoder is configured to decode the video depending on an indication whether pictures of the video that precede a dependent random access picture should be output.

[0019] Further, a method for receiving an input video data stream is provided. The input video data stream has a video encoded therein. The method comprises generating an output video data stream from the input video data stream. Further, the method comprises determining whether pictures of the video that precede a dependent random access picture should be output.

[0020] Further, a method of encoding a video into a video data stream is provided. The method comprises generating the video data stream such that the video data stream comprises an indication whether pictures of the video that precede a dependent random access picture should be output.

[0021] Further, a method for receiving a video data stream having a video stored therein is provided. The method comprises decoding the video from the video data stream. The video is decoded depending on an indication whether pictures of the video that precede a dependent random access picture should be output.

[0022] Further, a computer program for implementing one of the above methods when the computer program is executed by a computer or signal processor is provided.

[0023] According to a second aspect of the application, an apparatus for receiving one or more input video data streams is provided. Each of the one or more input video data streams has an input video encoded therein. The apparatus is configured to generate an output video data stream from the one or more input video data streams, the output video data stream encoding an output video, wherein the apparatus is configured to generate the output video data stream such that the output video is the input video encoded in one of the one or more input video data streams or such that the output video depends on the input video of at least one of the one or more input video data streams. Further, the apparatus is configured to determine an access unit removal time in a current picture of a plurality of pictures of the output video from a coded picture buffer. The apparatus is configured to determine whether the access unit removal time of the current picture from the coded picture buffer is determined using coded picture buffer delay offset information.

[0024] Further, a video data stream is provided. The video data stream has a video encoded therein. The video data stream comprises coded picture buffer delay offset information.

[0025] Further, a video decoder for receiving a video data stream having a video stored therein is provided. The video decoder is configured to decode the video from the video data stream. Further, the video decoder is configured to decode the video depending on an access unit removal time of a current picture of a plurality of pictures of the video from a coded picture buffer. The video decoder is configured to decode the video depending on an indication indicating whether the access unit removal time of the current picture from the coded picture buffer is determined using coded picture buffer delay offset information.

[0026] Further, a method for receiving one or more input video data streams is provided. Each of the one or more input video data streams has an input video encoded therein. The method comprises generating an output video data stream from the one or more input video data streams, the output video data stream encoding an output video, wherein generating the output video data stream is performed such that the output video is the input video encoded in one of the one or more input video data streams or such that the output video depends on the input video of at least one of the one or more input video data streams. Further, the method comprises determining an access unit removal time in a current picture of a plurality of pictures of the output video from a coded picture buffer. Further, the method comprises determining whether the access unit removal time of the current picture from the coded picture buffer is determined using coded picture buffer delay offset information.

[0027] Furthermore, a method for encoding a video into a video data stream according to an embodiment is provided. The method comprises generating the video data stream such that the video data stream comprises coded picture buffer delay offset information.

[0028] Furthermore, a method for receiving a video data stream having a video stored is provided. The method comprises decoding the video from the video data stream. Decoding the video depends on an access unit removal time of a current picture from a plurality of pictures of the video from a coded picture buffer. Furthermore, decoding the video is performed depending on an indication indicating whether to use coded picture buffer delay offset information for determining the access unit removal time of the current picture from the coded picture buffer.

[0029] Furthermore, a computer program for implementing one of the above-mentioned methods when the computer program is executed by a computer or signal processor is provided.

[0030] According to a third aspect of the present application, a video data stream is provided. The video data stream has a video encoded therein. Furthermore, the video data stream comprises an initial coded picture buffer removal delay. Furthermore, the video data stream comprises an initial coded picture buffer removal offset. Furthermore, the video data stream comprises information indicating whether a sum of the initial coded picture buffer removal delay and the initial coded picture buffer removal offset is defined to be constant over two or more buffering periods.

[0031] Furthermore, a video encoder is provided. The video encoder is configured to encode a video into a video data stream. Furthermore, the video encoder is configured to generate the video data stream such that the video data stream comprises an initial coded picture buffer removal delay. Furthermore, the video encoder is configured to generate the video data stream such that the video data stream comprises an initial coded picture buffer removal offset. Furthermore, the video encoder (100) is configured to generate the video data stream such that the video data stream comprises information indicating whether a sum of the initial coded picture buffer removal delay and the initial coded picture buffer removal offset is defined to be constant over two or more buffering periods.

[0032] Further, an apparatus for receiving two input video data streams is provided, the two input video data streams being a first input video data stream and a second input video data stream. An input video is encoded in each of the two input video data streams. The apparatus is configured to generate an output video data stream from the two input video data streams, the output video data stream encoding an output video, wherein the apparatus is configured to generate the output video data stream by concatenating the first input video data stream and the second input video data stream. Further, the apparatus is configured to generate the output video data stream such that the output video data stream comprises an initial coded picture buffer removal delay. Further, the apparatus is configured to generate the output video data stream such that the output video data stream comprises an initial coded picture buffer removal offset. Further, the apparatus is configured to generate the output video data stream such that the output video data stream comprises information indicating whether a sum of the initial coded picture buffer removal delay and the initial coded picture buffer removal offset is defined to be constant over two or more buffering periods.

[0033] Further, a video decoder for receiving a video data stream having a video stored therein is provided. The video decoder is configured to decode the video from the video data stream. Further, the video data stream comprises an initial coded picture buffer removal delay. Further, the video data stream comprises an initial coded picture buffer removal offset. Further, the video data stream comprises information indicating whether a sum of the initial coded picture buffer removal delay and the initial coded picture buffer removal offset is defined to be constant over two or more buffering periods. Further, the video decoder is configured to decode the video depending on the information indicating whether the sum of the initial coded picture buffer removal delay and the initial coded picture buffer removal offset is defined to be constant over two or more buffering periods.

[0034] Further, a method of encoding a video into a video data stream is provided. The method comprises generating the video data stream such that the video data stream comprises an initial coded picture buffer removal delay. The method comprises generating the video data stream such that the video data stream comprises an initial coded picture buffer removal offset. Further, the method comprises generating the video data stream such that the video data stream comprises information indicating whether a sum of the initial coded picture buffer removal delay and the initial coded picture buffer removal offset is defined to be constant over two or more buffering periods.

[0035] Further, a method for receiving two input video data streams is provided, the two input video data streams being a first input video data stream and a second input video data stream. An input video is encoded in each of the two input video data streams. The method comprises generating an output video data stream from the two input video data streams, the output video data stream encoding an output video, wherein the apparatus is configured to generate the output video data stream by concatenating the first input video data stream and the second input video data stream. Further, the method comprises generating the output video data stream such that the output video data stream comprises an initial coded picture buffer removal delay. Further, the method comprises generating the output video data stream such that the output video data stream comprises an initial coded picture buffer removal offset. Further, the method comprises generating the output video data stream such that the output video data stream comprises information indicating whether a sum of the initial coded picture buffer removal delay and the initial coded picture buffer removal offset is defined to be constant over two or more buffering periods.

[0036] Further, a method for receiving a video data stream storing a video is provided. The method comprises decoding the video from the video data stream. The video data stream comprises an initial coded picture buffer removal delay. Further, the video data stream comprises an initial coded picture buffer removal offset. Further, the video data stream comprises information indicating whether a sum of the initial coded picture buffer removal delay and the initial coded picture buffer removal offset is defined to be constant over two or more buffering periods. The method comprises decoding the video depending on the information indicating whether a sum of the initial coded picture buffer removal delay and the initial coded picture buffer removal offset is defined to be constant over two or more buffering periods.

[0037] Further, a computer program is provided for implementing one of the above methods when the computer program is executed by a computer or signal processor.

[0038] According to a fourth aspect of the present application, a video data stream is provided. The video data stream has encoded video. Furthermore, the video data stream comprises an indication indicating whether a non-scalable nested picture timing supplemental enhancement information message of a network abstraction layer unit of one access unit of a plurality of access units of one coded video sequence of one or more coded video sequences of the video data stream is defined to apply to all output layer sets of the plurality of output layer sets of the access unit. If the indication has a first value, the non-scalable nested picture timing supplemental enhancement information message of the network abstraction layer unit of the access unit is defined to apply to all output layer sets of the plurality of output layer sets of the access unit. If the indication has a value different from the first value, the indication does not define whether the non-scalable nested picture timing supplemental enhancement information message of the network abstraction layer unit of the access unit applies to all output layer sets of the plurality of output layer sets of the access unit.

[0039] Furthermore, a video encoder is provided. The video encoder is configured to encode video into a video data stream. Furthermore, the video encoder is configured to generate the video data stream such that the video data stream comprises an indication indicating whether a non-scalable nested picture timing supplemental enhancement information message of a network abstraction layer unit of one access unit of a plurality of access units of one coded video sequence of one or more coded video sequences of the video data stream is defined to apply to all output layer sets of the plurality of output layer sets of the access unit. If the indication has a first value, the non-scalable nested picture timing supplemental enhancement information message of the network abstraction layer unit of the access unit is defined to apply to all output layer sets of the plurality of output layer sets of the access unit. If the indication has a value different from the first value, the indication does not define whether the non-scalable nested picture timing supplemental enhancement information message of the network abstraction layer unit of the access unit applies to all output layer sets of the plurality of output layer sets of the access unit.

[0040] Furthermore, an apparatus for receiving an input video data stream is provided. The input video data stream has a video encoded therein. The apparatus is configured to generate a processed video data stream from the input video data stream. Furthermore, the apparatus is configured to generate the processed video data stream such that the processed video data stream comprises an indication indicating whether a non-scalable nested picture timing supplemental enhancement information message of a network abstraction layer unit of one access unit of one coded video sequence of one or more coded video sequences of the processed video data stream is defined to be applied to all output layer sets of a plurality of output layer sets of the access unit. If the indication has a first value, the non-scalable nested picture timing supplemental enhancement information message of the network abstraction layer unit of the access unit is defined to be applied to all output layer sets of the plurality of output layer sets of the access unit. If the indication has a value different from the first value, the indication does not define whether the non-scalable nested picture timing supplemental enhancement information message of the network abstraction layer unit of the access unit is applicable to all output layer sets of the plurality of output layer sets of the access unit.

[0041] Furthermore, a video decoder for receiving a video data stream having a video stored therein is provided. The video decoder is configured to decode the video from the video data stream. The video data stream comprises an indication indicating whether a non-scalable nested picture timing supplemental enhancement information message of a network abstraction layer unit of one access unit of one coded video sequence of one or more coded video sequences of the video data stream is defined to be applied to all output layer sets of a plurality of output layer sets of the access unit. If the indication has a first value, the non-scalable nested picture timing supplemental enhancement information message of the network abstraction layer unit of the access unit is defined to be applied to all output layer sets of the plurality of output layer sets of the access unit. If the indication has a value different from the first value, the indication does not define whether the non-scalable nested picture timing supplemental enhancement information message of the network abstraction layer unit of the access unit is applicable to all output layer sets of the plurality of output layer sets of the access unit.

[0042] Furthermore, a method of encoding a video into a video data stream is provided. The method comprises generating the video data stream such that the video data stream comprises an indication indicating whether a non-scalable nested picture timing supplemental enhancement information message of a network abstraction layer unit of one access unit of one coded video sequence of one or more coded video sequences of the video data stream is defined to apply to all output layer sets of a plurality of output layer sets of the access unit. If the indication has a first value, the non-scalable nested picture timing supplemental enhancement information message of the network abstraction layer unit of the access unit is defined to apply to all output layer sets of the plurality of output layer sets of the access unit. If the indication has a value different from the first value, the indication does not define whether the non-scalable nested picture timing supplemental enhancement information message of the network abstraction layer unit of the access unit applies to all output layer sets of the plurality of output layer sets of the access unit.

[0043] Furthermore, a method for receiving an input video data stream is provided. The input video data stream has a video encoded therein. The method comprises generating a processed video data stream from the input video data stream. Furthermore, the method comprises generating the processed video data stream such that the processed video data stream comprises an indication indicating whether a non-scalable nested picture timing supplemental enhancement information message of a network abstraction layer unit of one access unit of one coded video sequence of one or more coded video sequences of the processed video data stream is defined to apply to all output layer sets of a plurality of output layer sets of the access unit. If the indication has a first value, the non-scalable nested picture timing supplemental enhancement information message of the network abstraction layer unit of the access unit is defined to apply to all output layer sets of the plurality of output layer sets of the access unit. If the indication has a value different from the first value, the indication does not define whether the non-scalable nested picture timing supplemental enhancement information message of the network abstraction layer unit of the access unit applies to all output layer sets of the plurality of output layer sets of the access unit.

[0044] Furthermore, a method for receiving a video data stream having a video stored is provided. The method comprises decoding the video from the video data stream. The video data stream comprises an indication, the indication indicating whether a non-scalable nested picture timing supplemental enhancement information message of a network abstraction layer unit of one access unit of one coded video sequence of one or more coded video sequences of the video data stream is defined to apply to all output layer sets of a plurality of output layer sets of the access unit. If the indication has a first value, the non-scalable nested picture timing supplemental enhancement information message of the network abstraction layer unit of the access unit is defined to apply to all output layer sets of the plurality of output layer sets of the access unit. If the indication has a value different from the first value, the indication does not define whether the non-scalable nested picture timing supplemental enhancement information message of the network abstraction layer unit of the access unit applies to all output layer sets of the plurality of output layer sets of the access unit.

[0045] Furthermore, a computer program for implementing one of the above methods when the computer program is executed by a computer or signal processor is provided.

[0046] According to a fifth aspect of the present application, a video data stream is provided. The video data stream has a video encoded therein. Furthermore, the video data stream comprises one or more scalable-nested supplemental enhancement information messages. The one or more scalable-nested supplemental enhancement information messages comprise a plurality of syntax elements. Each syntax element of one or more syntax elements of the plurality of syntax elements is defined to have a same size in each scalable-nested supplemental enhancement information message of scalable-nested supplemental enhancement information messages of the video data stream or a portion of the video data stream.

[0047] Furthermore, a video encoder is provided. The video encoder is configured to encode a video into a video data stream. Furthermore, the video encoder is configured to generate the video data stream such that the video data stream comprises one or more scalable-nested supplemental enhancement information messages. Furthermore, the video encoder is configured to generate the video data stream such that the one or more scalable-nested supplemental enhancement information messages comprise a plurality of syntax elements. Furthermore, the video encoder is configured to generate the video data stream such that each syntax element of one or more syntax elements of the plurality of syntax elements is defined to have a same size in each scalable-nested supplemental enhancement information message of scalable-nested supplemental enhancement information messages of the video data stream or a portion of the video data stream.

[0048] Furthermore, an apparatus for receiving an input video data stream is provided. The input video data stream has video encoded therein. The apparatus is configured to generate an output video data stream from the input video data stream. The video data stream comprises one or more scalable-nested supplemental enhancement information messages. The one or more scalable-nested supplemental enhancement information messages comprise a plurality of syntax elements. Each syntax element of one or more syntax elements of the plurality of syntax elements is defined to have a same size in each scalable-nested supplemental enhancement information message of the video data stream or a portion of the video data stream. The apparatus is configured to process the one or more scalable-nested supplemental enhancement information messages.

[0049] Furthermore, a video decoder for receiving a video data stream having video stored therein is provided. The video decoder is configured to decode the video from the video data stream. The video data stream comprises one or more scalable-nested supplemental enhancement information messages. The one or more scalable-nested supplemental enhancement information messages comprise a plurality of syntax elements. Each syntax element of one or more syntax elements of the plurality of syntax elements is defined to have a same size in each scalable-nested supplemental enhancement information message of the video data stream or a portion of the video data stream. The video decoder is configured to decode the video depending on the one or more syntax elements of the plurality of syntax elements.

[0050] Furthermore, a method of encoding video into a video data stream is provided. The method comprises generating the video data stream such that the video data stream comprises one or more scalable-nested supplemental enhancement information messages. Furthermore, the method comprises generating the video data stream such that the one or more scalable-nested supplemental enhancement information messages comprise a plurality of syntax elements. Furthermore, the method comprises generating the video data stream such that each syntax element of one or more syntax elements of the plurality of syntax elements is defined to have a same size in each scalable-nested supplemental enhancement information message of the video data stream or a portion of the video data stream.

[0051] Furthermore, a method for receiving an input video data stream is provided. The input video data stream has video encoded therein. The method comprises generating an output video data stream from the input video data stream. The video data stream comprises one or more scalable-nested supplemental enhancement information messages. The one or more scalable-nested supplemental enhancement information messages comprise a plurality of syntax elements. Each syntax element of one or more syntax elements of the plurality of syntax elements is defined to have a same size in each scalable-nested supplemental enhancement information message of the video data stream or a portion of the video data stream. The method comprises processing the one or more scalable-nested supplemental enhancement information messages.

[0052] Furthermore, a method for receiving a video data stream having a video stored therein is provided. The method comprises decoding the video from the video data stream. The video data stream comprises one or more scalable-nested supplemental enhancement information messages. The one or more scalable-nested supplemental enhancement information messages comprise a plurality of syntax elements. Each syntax element of one or more syntax elements of the plurality of syntax elements is defined to have a same size in each scalable-nested supplemental enhancement information message of the video data stream or a portion of the video data stream. The video is decoded depending on the one or more syntax elements of the plurality of syntax elements.

[0053] Furthermore, a computer program for implementing one of the above methods when the computer program is executed by a computer or signal processor is provided.

[0054] Preferred embodiments are provided in dependent claims. BRIEF DESCRIPTION OF DRAWINGS

[0055] In the following, embodiments of the present application will be described in detail with reference to the attached drawings, in which:

[0056] Figure 1 A video encoder for encoding a video into a video data stream according to an embodiment is shown.

[0057] Figure 2 An apparatus for receiving an input video data stream according to an embodiment is shown.

[0058] Figure 3 A video decoder for receiving a video data stream having a video stored therein according to an embodiment is shown.

[0059] Figure 4 A raw bitstream (top) and a bitstream after discarding pictures (bottom) according to an embodiment are shown. Figure 4 Figure 4

[0060] Figure 5 A concatenation of two bitstreams after pictures have been discarded from one of the two bitstreams according to an embodiment is shown.

[0061] Figure 6 A concatenation of two bitstreams according to another embodiment is shown.

[0062] Figure 7 Two sets of HRD SEI (scalable-nested SEI and non-scalable-nested SEI) in a two-layer bitstream according to an embodiment are shown.

[0063] Figure 8 A video encoder is shown. ​​

[0064] Figure 9 A video decoder is shown.

[0065] Figure 10 The relationship between, on the one hand, a reconstructed signal (e.g. a reconstructed picture) and, on the other hand, a combination of a prediction residual signal and a prediction signal signaled in a data stream is shown. DETAILED DESCRIPTION

[0066] The following description of the drawings starts with a presentation of a description of an encoder and a decoder for block-based prediction coding of video pictures in order to form an example of an encoding framework into which embodiments of the present application can be built in. The respective encoder and decoder are described with respect to Figures 8 to 10 are described. In the following, the description of embodiments of the inventive concept is presented together with a description of how this concept can be built into Figure 8 and Figure 9 encoders and decoders, although the embodiments described using Figures 1 to 3 and in the following can also be used to form encoders and decoders operating under an encoding framework not according to Figure 8 and Figure 9 encoders and decoders.

[0067] Figure 8 A video encoder is shown, an apparatus for predictively encoding a picture 12 into a data stream 14 using, exemplarily, transform-based residual coding. The apparatus or encoder is indicated using reference sign 10. Figure 9 A corresponding video decoder 20 is shown, e.g. an apparatus 20 configured to predictively decode a picture 12' from the data stream 14 also using transform-based residual decoding, wherein the prime is used to indicate that the picture 12' reconstructed by the decoder 20 deviates from the picture 12 originally encoded by the apparatus 10 in terms of encoding losses introduced by quantization of the prediction residual signal. Figure 8 and Figure 9 uses transform-based prediction residual coding, although embodiments of the present application are not limited to such prediction residual coding. As will be outlined in the following, this is also true for other details described with respect to Figure 8 and Figure 9 .

[0068] The encoder 10 is configured to spatial-to-spectral transform the prediction residual signal and to encode the thus obtained prediction residual signal into the data stream 14. Likewise, the decoder 20 is configured to decode the prediction residual signal from the data stream 14 and to perform a spectral-to-spatial transformation on the thus obtained prediction residual signal.

[0069] Internally, the encoder 10 can comprise a prediction residual signal former 22 which generates a prediction residual 24 to measure a deviation of a prediction signal 26 from an original signal, e.g. from the picture 12. The prediction residual signal former 22 can for example be a subtractor which subtracts the prediction signal from the original signal, e.g. from the picture 12. The encoder 10 then further comprises a transformer 28 which spatially to spectrally transforms the prediction residual signal 24 to obtain a spectral domain prediction residual signal 24', which is then quantized by a quantizer 32, which is also comprised in the encoder 10. The thus quantized prediction residual signal 24" is encoded into the bitstream 14. To this end, the encoder 10 can optionally comprise an entropy encoder 34 which entropy encodes the prediction residual signal 24" which is transformed and quantized into the data stream 14. The prediction signal 26 is generated by a prediction stage 36 of the encoder 10 on the basis of the prediction residual signal 24" which is encoded into the data stream 14 and decodable from the data stream 14. To this end, as Figure 8 indicated, the prediction stage 36 can internally comprise a dequantizer 38 which dequantizes the prediction residual signal 24" in order to obtain a spectral domain prediction residual signal 24"' corresponding to the signal 24' except for a quantization loss, followed by an inverse transformer 40 which inverse transforms, e.g. spectrally to spatially, the latter prediction residual signal 24"' in order to obtain a prediction residual signal 24"" corresponding to the original prediction residual signal 24 except for a quantization loss. A combiner 42 of the prediction stage 36 then recombines the prediction signal 26 and the prediction residual signal 24"" by addition, e.g., to obtain a reconstructed signal 46, e.g. a reconstruction of the original signal 12. The reconstructed signal 46 can correspond to the signal 12'. A prediction module 44 of the prediction stage 36 then generates the prediction signal 26 on the basis of the signal 46 by using, e.g., spatial prediction, e.g. intra-picture prediction, and / or temporal prediction, e.g. inter-picture prediction.

[0070] Likewise, as Figure 9 indicated, the decoder 20 can internally comprise components corresponding to and interconnected in a manner corresponding to the prediction stage 36. In particular, an entropy decoder 50 of the decoder 20 can entropy decode the quantized spectral domain prediction residual signal 24" from the data stream, followed by a dequantizer 52, an inverse transformer 54, a combiner 56 and a prediction module 58 interconnected and cooperating in a manner described above with respect to the prediction stage 36 to recover a reconstructed signal on the basis of the prediction residual signal 24"', as Figure 9 indicated, the output of the combiner 56 yields the reconstructed signal, i.e. the picture 12'.

[0071] Although not specifically described above, it is clear that the encoder 10 can set coding parameters including, for example, prediction modes, motion parameters, etc., in accordance with some optimization scheme, e.g., in a way that optimizes some rate and distortion related criterion, e.g., the coding cost. For example, the encoder 10 and the decoder 20 and the respective corresponding modules 44, 58 can support different prediction modes such as intra coding modes and inter coding modes. The granularity at which the encoder and the decoder switch between these prediction mode types can correspond to a respective subdivision of the pictures 12 and 12' into coding segments or coding blocks. For example, in units of these coding segments, the pictures can be subdivided into blocks that are intra coded and blocks that are inter coded. As outlined in more detail below, intra coded blocks are predicted based on a spatial, already coded / decoded neighborhood of the respective block. Several intra coding modes can exist and be selected for respective intra coded segments including directional or angular intra coding modes according to which the respective segment is filled by extrapolating sample values of the neighborhood into the respective intra coded segment along a specific direction that is specific to the respective directional intra coding mode. For example, the intra coding modes can also include one or more other modes such as a DC coding mode according to which the prediction for the respective intra coded block assigns a DC value to all samples within the respective intra coded segment and / or a planar intra coding mode according to which the prediction for the respective block is approximated or determined as a spatial distribution of sample values described by a two-dimensional linear function over the sample positions of the respective intra coded block, wherein the driving tilt and offset of the plane are defined by the two-dimensional linear function based on neighboring samples. In contrast, inter coded blocks can be predicted, e.g., in time. For inter coded blocks, a motion vector can be signaled within the data stream that indicates a spatial displacement of a portion of a previously coded picture of the video to which the picture 12 belongs, from which portion of the previously coded / decoded picture the picture is sampled to obtain a prediction signal for the respective inter coded block. This means that, in addition to the residual signal coding included in the data stream 14, e.g., the entropy encoded transform coefficient levels representing the quantized spectral domain prediction residual signal 24'', the data stream 14 can have coded therein coding mode parameters for assigning coding modes to the various blocks, prediction parameters for some of the blocks, e.g., motion parameters for inter coded segments, and optionally other parameters, e.g., parameters for controlling and signaling the subdivision of the pictures 12 and 12', respectively, into segments. The decoder 20 uses these parameters to subdivide the pictures in the same way, to assign the same prediction modes to the segments, and to perform the same predictions to produce the same prediction signals.

[0072] Figure 10 The combination between the reconstructed signal, e.g., the reconstructed picture 12', of one aspect and the prediction residual signal 24'''' and the prediction signal 26 signaled in the data stream of the other aspect is shown. As already described above, this combination can be an addition. The prediction signal 26 is in the data stream 14, e.g., in the form of a prediction residual signal 24''', which is added to the quantized transform coefficients 24' of the prediction residual signal 24 to obtain the prediction signal 26.Figure 10 The picture area is shown to be subdivided into intra coded blocks, which are illustratively indicated using hatching, and inter coded blocks, which are illustratively indicated without hatching. The subdivision can be an arbitrary subdivision, e.g. a regular subdivision of the picture area into square blocks or rows and columns of non-square blocks, or a recursive multi-tree subdivision of the picture 12 from a tree root block into a plurality of leaf blocks of variable size, e.g. a quad-tree subdivision, etc., wherein, Figure 10 A hybrid thereof is shown, wherein the picture area is first subdivided into rows and columns of tree root blocks, and then further subdivided into one or more leaf blocks according to a recursive multi-tree subdivision.

[0073] Likewise, the data stream 14 can have intra coding modes encoded therein for the intra coded blocks 80, which assign one of a number of supported intra coding modes to a respective intra coded block 80. For the inter coded blocks 82, the data stream 14 can have one or more motion parameters encoded therein. Generally, the inter coded blocks 82 are not limited to being temporally coded. Alternatively, the inter coded blocks 82 can be any blocks predicted from a previously coded portion other than the current picture 12 itself, e.g. a previously coded picture of a video to which the picture 12 belongs, or a picture of another view or lower level in case the encoder and decoder are a scalable encoder and decoder, respectively.

[0074] Figure 10 The prediction residual signals 24'''' in the data stream 14 are also shown to subdivide the picture area into blocks 84. These blocks can be referred to as transform blocks in order to distinguish them from the coding blocks 80 and 82. In fact, Figure 10 It is shown that the encoder 10 and the decoder 20 can use two different subdivisions, which subdivide the picture 12 and the picture 12', respectively, into blocks, namely one into coding blocks 80 and 82, and the other into transform blocks 84. The two subdivisions can be identical, e.g. each coding block 80 and 82 can at the same time form a transform block 84, but Figure 10 It is shown that the case is possible that the subdivision into transform blocks 84 forms an extension of the subdivision into coding blocks 80 and 82, such that any boundary between two of the blocks 80 and 82 covers a boundary between two of the blocks 84, or in other words, each block 80, 82 coincides with one of the transform blocks 84, or with a cluster of transform blocks 84. However, it is also possible to determine or select the subdivisions independently from each other, such that the transform blocks 84 can alternatively span across block boundaries between the blocks 80, 82. Statements analogous to the ones made with respect to the subdivision into blocks 80, 82 are thus true with respect to the subdivision into transform blocks 84, e.g. the blocks 84 can be the result of a regular subdivision of the picture area into blocks (arranged or not arranged into rows and columns), a recursive multi-tree subdivision of the picture area, or a combination or any other type of blockation. Incidentally, it is noted that the blocks 80, 82 and 84 are not limited to being square, rectangular or any other shape.

[0075] Figure 10 It is also shown that the combination of the prediction signal 26 and the prediction residual signal 24'''' directly results in the reconstructed signal 12'. However, it should be noted that according to alternative embodiments more than one prediction signal 26 can be combined with the prediction residual signal 24'''' to result in the picture 12'.

[0076] In Figure 10 The transform blocks 84 shall have the following meaning. The transformer 28 and the inverse transformer 54 perform their transforms in units of these transform blocks 84. For example, many codecs use some kind of DST or DCT for all transform blocks 84. Some codecs allow to skip the transform so that for some transform blocks 84 the prediction residual signal is directly encoded in the spatial domain. However, according to the embodiments described below, the encoder 10 and the decoder 20 are configured in a way that they support several transforms. For example, the transforms that the encoder 10 and the decoder 20 support can include:

[0077] o DCT-II (or DCT-III), where DCT stands for Discrete Cosine Transform

[0078] o DST-IV, where DST stands for Discrete Sine Transform

[0079] o DCT-IV

[0080] o DST-VII

[0081] o Identity Transform (IT)

[0082] Of course, while the transformer 28 will support all forward transform versions of these transforms, the decoder 20 or the inverse transformer 54 will support their corresponding backward or inverse versions:

[0083] o Inverse DCT-II (or inverse DCT-III)

[0084] o Inverse DST-IV

[0085] o Inverse DCT-IV

[0086] o Inverse DST-VII

[0087] o Identity Transform (IT)

[0088] The following description provides more details on which transforms the encoder 10 and the decoder 20 can support. In any case, it should be noted that the set of supported transforms can include only one transform, e.g. one spectral-to-spatial or spatial-to-spectral transform.

[0089] As already outlined above, Figures 8 to 10An encoder and a decoder have been presented as examples in which the inventive concept described further below can be implemented in order to form specific examples of encoders and decoders according to the present application. In this regard, Figure 8 and Figure 9 may represent possible implementations of the encoder and the decoder described below. However, Figure 8 and Figure 9 are examples only. However, the encoder of embodiments according to the present application can use the encoding concept outlined in more detail below to perform block-based encoding of pictures 12 and can differ from the encoder of Figure 8 for example in that the encoder is not a video encoder but a still picture encoder, does not support inter prediction, or is performed in a different way than exemplified in Figure 10 Likewise, the decoder of embodiments according to the present application can use the encoding concept outlined further below to perform block-based decoding of pictures 12’ from data stream 14, but can differ from the decoder 20 of Figure 9 for example in that the decoder is not a video decoder but a still picture decoder, does not support intra prediction, or subdivides pictures 12’ into blocks in a different way than described with respect to Figure 10 and / or the decoder does not derive prediction residuals from data stream 14 in a transform domain but in a spatial domain.

[0090] Figure 1 A video encoder 100 for encoding a video into a video data stream according to an embodiment is shown. The video encoder 100 is configured to generate a video data stream such that the video data stream comprises an indication indicating whether pictures preceding a dependent random access picture should be outputted or not.

[0091] Figure 2 An apparatus 200 for receiving an input video data stream according to an embodiment is shown. A video is encoded in the input video data stream. The apparatus 200 is configured to generate an output video data stream from the input video data stream.

[0092] Figure 3 A video decoder 300 for receiving a video data stream having a video stored therein according to an embodiment is shown. The video decoder 300 is configured to decode the video from the video data stream. The video decoder 300 is configured to decode the video in dependence on an indication indicating whether pictures preceding a dependent random access picture should be outputted or not.

[0093] Further, a system according to an embodiment is provided. The system comprises Figure 2 the apparatus and Figure 3 the video decoder. The video decoder 300 of Figure 3 is configured to receive Figure 2an output video data stream of the apparatus. Figure 3 The video decoder 300 is configured to decode a video from Figure 2 an output video data stream of the apparatus 200.

[0094] In embodiments, the system may, for example, further comprise Figure 1 a video encoder 100. Figure 2 The apparatus 200 may, for example, be configured to receive a video data stream as an input video data stream from Figure 1 the video encoder 100.

[0095] The (optional) intermediate device 210 of the apparatus 200 may, for example, be configured to receive a video data stream as an input video data stream from the video encoder 100 and to generate an output video data stream from the input video data stream. For example, the intermediate device may, for example, be configured to modify (header / metadata) information of the input video data stream and / or may, for example, be configured to delete pictures from the input video data stream and / or may be configured to mix / splice the input video data stream with an additional second bitstream having a second video encoded into it.

[0096] The (optional) video decoder 221 may, for example, be configured to decode a video from the output video data stream.

[0097] The (optional) hypothetical reference decoder 222 may, for example, be configured to determine timing information of the video from the output video data stream or may, for example, be configured to determine buffer information of a buffer in which the video or a part of the video is to be stored.

[0098] The system comprises Figure 1 a video encoder 101 and Figure 2 a video decoder 151.

[0099] The video encoder 101 is configured to generate an encoded video signal. The video decoder 151 is configured to decode the encoded video signal to reconstruct pictures of the video.

[0100] In the following, the first aspect of the application will now be described in detail.

[0101] According to the first aspect of the application, an apparatus 200 for receiving an input video data stream is provided. The input video data stream has a video encoded therein. The apparatus 200 is configured to generate an output video data stream from the input video data stream. Furthermore, the apparatus 200 is configured to determine whether pictures of the video that precede dependent random access pictures should be outputted.

[0102] According to embodiments, the apparatus 200 may, for example, be configured to determine a first variable (e.g. NoOutputBeforeDrapFlag) indicating whether pictures of the video preceding a dependent random access picture should be outputted or not.

[0103] In embodiments, the apparatus 200 may, for example, be configured to generate the output video data stream such that the output video data stream may, for example, comprise an indication, which may, for example, indicate whether pictures of the video preceding a dependent random access picture should be outputted or not.

[0104] According to embodiments, the apparatus 200 may, for example, be configured to generate the output video data stream such that the output video data stream may, for example, comprise supplemental enhancement information comprising an indication, which may, for example, indicate whether pictures of the video preceding a dependent random access picture should be outputted or not.

[0105] In embodiments, the pictures of the video preceding a dependent random access picture may be independent random access pictures. The apparatus 200 may, for example, be configured to generate the output video data stream such that the output video data stream may, for example, comprise a flag (e.g. ph_pic_output_flag) with a predefined value (e.g. 0) in a picture header of an independent random access picture, such that the predefined value (e.g. 0) of the flag (e.g. ph_pic_output_flag) may, for example, indicate that the independent random access picture is directly preceded by said dependent random access picture within the video data stream and that said independent random access picture should not be outputted.

[0106] According to embodiments, the flag may, for example, be a first flag, wherein the apparatus 200 may, for example, be configured to generate the output video data stream such that the output video data stream may, for example, comprise a further flag in a picture parameter set of the video data stream, wherein the further flag may, for example, indicate whether the first flag (e.g. ph_pic_output_flag) is present in a picture header of an independent random access picture.

[0107] In embodiments, the apparatus 200 may, for example, be configured to generate the output video data stream such that the output video data stream may, for example, comprise as an indication, which may, for example, indicate whether pictures of the video preceding a dependent random access picture should be outputted or not, a supplemental enhancement information flag within supplemental enhancement information of the output video data stream, or a picture parameter set flag within a picture parameter set of the output video data stream, or a sequence parameter set flag within a sequence parameter set of the output video data stream, or an external device flag, wherein a value of the external device flag may, for example, be set by an external unit located outside the apparatus 200.

[0108] According to embodiments, the apparatus 200 may, for example, be configured to determine, depending on the first variable (e.g. NoOutputBeforeDrapFlag), a value of a second variable (e.g. PictureOutputFlag) of a picture of the video preceding a dependent random access picture, wherein the second variable (e.g. PictureOutputFlag) may, for example, indicate for the picture whether the picture should be output or not, and wherein the apparatus 200 may, for example, be configured to output or not output the picture depending on the second variable (e.g. PictureOutputFlag).

[0109] In embodiments, the picture of the video preceding a dependent random access picture can be an independent random access picture. The first variable (e.g. NoOutputBeforeDrapFlag) may, for example, indicate that the independent random access picture should not be output.

[0110] According to embodiments, the picture of the video preceding a dependent random access picture can be an independent random access picture. The apparatus 200 may, for example, be configured to set the first variable (e.g. NoOutputBeforeDrapFlag) such that the first variable (e.g. NoOutputBeforeDrapFlag) may, for example, indicate that the independent random access picture should be output.

[0111] In embodiments, the apparatus 200 may, for example, be configured to signal to the video decoder 300 whether the picture of the video preceding a dependent random access picture should be output or not.

[0112] Further, a video data stream is provided. The video is encoded in the video data stream. The video data stream comprises an indication indicating whether a picture of the video preceding a dependent random access picture should be output or not.

[0113] According to embodiments, the video data stream comprises supplemental enhancement information comprising the indication which may, for example, indicate whether the picture of the video preceding a dependent random access picture should be output or not.

[0114] In embodiments, the picture of the video preceding a dependent random access picture can be an independent random access picture. The video data stream may, for example, comprise a flag (e.g. ph_pic_output_flag) having a predefined value (e.g. 0) in a picture header of the independent random access picture, such that the predefined value (e.g. 0) of the flag (e.g. ph_pic_output_flag) may, for example, indicate that the independent random access picture is directly preceded by the dependent random access picture within the video data stream and that the independent random access picture should not be output.

[0115] According to embodiments, the flag may, for example, be a first flag, wherein the video data stream may, for example, comprise a further flag in a picture parameter set of the video data stream, wherein the further flag may, for example, indicate whether the first flag (e.g. ph_pic_output_flag) is present in a picture header of an independent random access picture.

[0116] In embodiments, the video data stream may, for example, comprise a supplemental enhancement information flag within a supplemental enhancement information of the output video data stream, or a picture parameter set flag within a picture parameter set of the output video data stream, or a sequence parameter set flag within a sequence parameter set of the output video data stream, or an external device flag as an indication which may, for example, indicate whether pictures of the video preceding a dependent random access picture should be outputted, wherein a value of the external device flag may, for example, be set by an external unit located outside of the device 200.

[0117] Further, a video encoder 100 is provided. The video encoder 100 may, for example, be configured to encode a video into a video data stream. Further, the video encoder 100 may, for example, be configured to generate a video data stream such that the video data stream comprises an indication which may, for example, indicate whether pictures of the video preceding a dependent random access picture should be outputted.

[0118] According to embodiments, the video encoder 100 may, for example, be configured to generate a video data stream such that the video data stream may, for example, comprise supplemental enhancement information comprising an indication which may, for example, indicate whether pictures of the video preceding a dependent random access picture should be outputted.

[0119] In embodiments, the pictures of the video preceding a dependent random access picture may be independent random access pictures. The video encoder 100 may, for example, be configured to generate a video data stream such that the video data stream may, for example, comprise a flag (e.g. ph_pic_output_flag) having a predefined value (e.g. 0) in a picture header of an independent random access picture, such that the predefined value (e.g. 0) of the flag (e.g. ph_pic_output_flag) may, for example, indicate that the independent random access picture is directly preceded by said dependent random access picture within the video data stream and that said independent random access picture should not be outputted.

[0120] According to embodiments, the flag may, for example, be a first flag, wherein the video encoder 100 may, for example, be configured to generate a video data stream such that the video data stream may, for example, comprise a further flag in a picture parameter set of the video data stream, wherein the further flag may, for example, indicate whether the first flag (e.g. ph_pic_output_flag) is present in a picture header of an independent random access picture.

[0121] In an embodiment, the video encoder 100 may be configured, for example, to generate a video data stream such that the video data stream may include, for example, the following flags as indicators of whether the video should be output before the random access image: a supplemental enhancement information flag in the supplemental enhancement information of the output video data stream, or a picture parameter set flag in the picture parameter set of the output video data stream, or a sequence parameter set flag in the sequence parameter set of the output video data stream, or an external device flag, wherein the value of the external device flag may be set, for example, by an external unit located outside the device 200.

[0122] In addition, a video decoder 300 is provided for receiving a video data stream in which video is stored. The video decoder 300 is configured to decode video from the video data stream. The video decoder 300 is configured to decode the video based on an indication of whether video should be output before a picture that depends on random access.

[0123] According to an embodiment, the video decoder 300 may be configured, for example, to decode the video based on a first variable (e.g., NoOutputBeforeDrapFlag), which indicates whether the video should output images prior to those relying on random access.

[0124] In one embodiment, the video data stream may include, for example, an indication of whether the video should be output before the random access images. The video decoder 300 may be configured, for example, to decode the video based on the indication within the video data stream.

[0125] According to an embodiment, the video data stream includes supplemental enhancement information, which includes, for example, an indication of whether the video should be output before the random access images. The video decoder 300 may be configured, for example, to decode the video based on the supplemental enhancement information.

[0126] In one embodiment, the image preceding the dependent random access image in the video can be an independent random access image. The video data stream may, for example, include a flag (e.g., ph_pic_output_flag) with a predefined value (e.g., 0) in the header of the independent random access image, such that the predefined value (e.g., ph_pic_output_flag) of the flag (e.g., ph_pic_output_flag) indicates, for example, that the independent random access image precedes the dependent random access image directly within the video data stream and should not be output. The video decoder 300 may, for example, be configured to decode the video based on this flag.

[0127] According to embodiments, the flag may, for example, be a first flag, wherein the video data stream may, for example, comprise a further flag in a picture parameter set of the video data stream, wherein the further flag may, for example, indicate whether the first flag (e.g. ph_pic_output_flag) is present in a picture header of an independent random access picture. Video decoder 300 may, for example, be configured to decode the video in dependence on the further flag.

[0128] In embodiments, the video data stream may, for example, comprise a flag as an indication whether pictures of the video preceding a dependent random access picture should be outputted or not, for example, a supplemental enhancement information flag within a supplemental enhancement information of the output video data stream, or a picture parameter set flag within a picture parameter set of the output video data stream, or a sequence parameter set flag within a sequence parameter set of the output video data stream, or an external device flag, wherein a value of the external device flag may, for example, be set by an external unit located outside of apparatus 200. Video decoder 300 may, for example, be configured to decode the video in dependence on the indication within the video data stream.

[0129] According to embodiments, video decoder 300 may, for example, be configured to reconstruct the video from the video data stream. Video decoder 300 may, for example, be configured to output or not output pictures of the video preceding a dependent random access picture in dependence on the first variable (e.g. NoOutputBeforeDrapFlag).

[0130] In embodiments, video decoder 300 may, for example, be configured to determine a value of a second variable (e.g. PictureOutputFlag) of a picture of the video preceding a dependent random access picture in dependence on the first variable (e.g. NoOutputBeforeDrapFlag), wherein the second variable (e.g. PictureOutputFlag) may, for example, indicate whether the picture should be outputted or not for said picture, and wherein apparatus 200 may, for example, be configured to output or not output said picture in dependence on the second variable (e.g. PictureOutputFlag).

[0131] According to embodiments, the picture of the video preceding a dependent random access picture may be an independent random access picture. Video decoder 300 may, for example, be configured to decode the video in dependence on the first variable (e.g. NoOutputBeforeDrapFlag) which may, for example, indicate that the independent random access picture should not be outputted.

[0132] In an embodiment, the pictures of the video preceding the DRAPs can be independent random access pictures. The video decoder 300 may, for example, be configured to decode the video according to a first variable (e.g. NoOutputBeforeDrapFlag) which may, for example, indicate that an independent random access picture should be output.

[0133] Further, a system is provided. The system comprises the apparatus 200 as described above and the video decoder 300 as described above. The video decoder 300 is configured to receive the output video data stream of the apparatus 200. Further, the video decoder 300 is configured to decode the video from the output video data stream of the apparatus 200.

[0134] According to an embodiment, the system may, for example, further comprise the video encoder 100. The apparatus 200 may, for example, be configured to receive the video data stream from the video encoder 100 as the input video data stream.

[0135] In particular, the first aspect of the present invention relates to starting a CVS at a DRAP and omitting the output of an IDR preceding the DRAP in decoding and conformance testing.

[0136] When a bitstream includes pictures marked as DRAP (i.e. in the bitstream, only the previous IRAP is used as reference for the DRAP and from there on), these DRAP pictures can be used for random access functionality with lower rate overhead. However, when a certain target DRAP is used to randomly access the stream, it is not desirable to display any initial pictures preceding the target DRAP at the decoder output (i.e. the associated IRAP of the target DRAP) as the temporal distance between these pictures when played back at the original video rate will cause the video playback to be unstable / jittery until the video is played back from the target DRAP in a smooth manner.

[0137] Therefore, it is desirable to omit the output of pictures preceding a DRAP picture. This aspect of the present invention proposes means to control the decoder accordingly.

[0138] In one embodiment, an external means for setting the PicOutputFlag variable of an IRAP picture is made available for implementation as follows:

[0139] - If a certain external means not specified in this specification is available to set the variable NoOutputBeforeDrapFlag of a picture to a value, the NoOutputBeforeDrapFlag of the picture is set equal to the value provided by the external means.

[0140] […]

[0141] - The variable PictureOutputFlag of the current picture is derived as follows:

[0142] - If sps_video_parameter_set_id is greater than 0 and the current layer is not an output layer (i.e., nuh layer id is not equal to OutputLayerldInOls[TargetOlsldx][i] for any value of i in the range of 0 to NumOutputLayersInOls[TargetOlsldx] - 1, inclusive), or one of the following conditions is true, PictureOutputFlag is set equal to 0:

[0143] - The current picture is a RASL picture and the associated IRAP picture has NoOutputBeforeRecoveryFlag equal to 1.

[0144] - The current picture is a GDR picture with NoOutputBeforeRecoveryFlag equal to 1 or a recovery picture of a GDR picture with NoOutputBeforeRecoveryFlag equal to 1.

[0145] - The current picture is an IRAP picture with NoOutputBeforeDrapFlag equal to 1.

[0146] - Otherwise, PictureOutputFlag is set equal to ph_pic_output_flag.

[0147] In another embodiment, NoOutputBeforeDrapFlag is only set by external means for the first IRAP picture in a CVS, otherwise it is set to 0.

[0148] - If some external means not specified in this specification is available to set the variable NoOutputBeforeDrapFlag of a picture to a value, NoOutputBeforeDrapFlag of the first picture in a CVS is set equal to the value provided by the external means. Otherwise, NoOutputBeforeDrapFlag is set to 0.

[0149] For the case of removing pictures between IRAP pictures and DRAP pictures, the flag NoOutputBeforeDrapFlag mentioned above can also be associated with the use of alternative HRD timing conveyed in the bitstream, e.g., the flag UseAltCpbParamsFlag in the VVC specification.

[0150] In an alternative embodiment, the IRAP picture directly preceding a DRAP picture without any non-DRAP picture in between shall have a value of 0 in the output flag ph_pic_output_flag in the picture header. In this case, whenever the extractor or player uses the DRAP for random access, i.e. it removes the intermediate pictures between the IRAP and the DRAP from the bitstream, it also needs to verify or adjust that the corresponding output flag is set to 0 and the output of the IRAP is omitted.

[0151] To make this operation simple, the original bitstream needs to be prepared accordingly. More specifically, pps_output_flag_present_flag, which determines the presence of the flag ph_pic_output_flag in the picture header, shall be equal to 1 so that the picture header can be easily changed and there is no need to change the parameter set as well. That is:

[0152] The requirement for bitstream conformance is that the value of pps_output_flag_present_flag shall be equal to 1 if the PPS is referred to by a picture within the CVSS AU with the associated DRAP AU.

[0153] In addition to the options listed above, in another embodiment, it is indicated in the parameter set PPS or SPS whether the first AU in the bitstream, i.e. the CRA or IDR that constitutes the start of the CLVS, is to be output after decoding. Thus, the system integration is simpler because, for example, when parsing a file in the file format ISOBMFF, only the parameter set needs to be adjusted, instead of needing to change the relatively low-level syntax (e.g. PHs).

[0154] An example is shown below:

[0155]

[0156] sps_pic_in_cvss_au_no_output_flag equal to 1 specifies that the pictures in the CVSS AU referring to the SPS are not output. sps_pic_in_cvss_au_no_output_flag equal to 0 specifies that the pictures in the CVSS AU referring to the SPS can or can not be output.

[0157] The requirement for bitstream conformance is that, for any SPS referred to by any output layer in the OLS, the value of sps_pic_in_cvss_au_no_output_flag shall be the same.

[0158] In 8.1.2

[0159] - The variable PictureOutputFlag for the current picture is derived as follows:

[0160] - If sps_video_parameter_set_id is greater than 0 and the current layer is not an output layer (i.e., nuh layer id is not equal to OutputLayerldInOls[TargetOlsldx][i] for any value of i in the range of 0 to NumOutputLayersInOls[TargetOlsldx] - 1, inclusive), or one of the following conditions is true, PictureOutputFlag is set equal to 0:

[0161] - The current picture is a RASL picture and the associated IRAP picture has NoOutputBeforeRecoveryFlag equal to 1.

[0162] - The current picture is a GDR picture with NoOutputBeforeRecoveryFlag equal to 1 or a recovery picture of a GDR picture with NoOutputBeforeRecoveryFlag equal to 1.

[0163] - Otherwise, if the current AU is a CVSS AU and sps_pic_in_cvss_au_no_output_flag is equal to 1, PictureOutputFlag is set equal to 0.

[0164] - Otherwise, PictureOutputFlag is set equal to ph_pic_output_flag.

[0165] NOTE - In implementations, a decoder can output pictures that do not belong to an output layer. For example, when there is only one output layer and the pictures of the output layer are not available in an AU, e.g., due to loss or layer down-switching, the decoder can set PictureOutputFlag equal to 1 for the picture among all pictures of the AU available to the decoder that has the highest value of nuh layer id and ph_pic_output_flag equal to 1, and set PictureOutputFlag equal to 0 for all other pictures of the AU available to the decoder.

[0166] In another embodiment, for example, it is required that, for example, can be defined as follows:

[0167] The requirement for bitstream conformance is that if a picture belongs to an IRAP AU and the IRAP AU is directly preceded by a DRAP AU, the value of ph_pic_output_flag shall be equal to 0.

[0168] In the following, the second aspect of the application will now be described in detail.

[0169] According to the second aspect of the application, an apparatus for receiving one or more input video data streams is provided. Each of the one or more input video data streams has an input video encoded therein. The apparatus 200 is configured to generate an output video data stream from the one or more input video data streams, the output video data stream encoding an output video, wherein the apparatus is configured to generate the output video data stream such that the output video is the input video encoded in one of the one or more input video data streams or such that the output video depends on the input video of at least one of the one or more input video data streams. Further, the apparatus 200 is configured to determine a removal time of a current picture of a plurality of pictures of the output video from an access unit of a coded picture buffer. The apparatus 200 is configured to determine whether to use coded picture buffer delay offset information for determining the removal time of the current picture from the access unit of the coded picture buffer.

[0170] According to embodiments, the apparatus 200 may, for example, be configured to discard a group of one or more pictures of the input video of a first video data stream of the one or more input video data streams to generate the output video data stream. The apparatus 200 may, for example, be configured to determine a removal time of at least one picture of the plurality of pictures of the output video from an access unit of a coded picture buffer in dependence on the coded picture buffer delay offset information.

[0171] In embodiments, the first video received by the apparatus 200 may, for example, be a pre-processed video derived from an original video from which a group of one or more pictures has been discarded to generate the pre-processed video. The apparatus 200 may, for example, be configured to determine a removal time of at least one picture of the plurality of pictures of the output video from an access unit of a coded picture buffer in dependence on the coded picture buffer delay offset information.

[0172] According to embodiments, the buffer delay offset information depends on a number of pictures of the input video that have been discarded.

[0173] In embodiments, the one or more input video data streams are two or more input video data streams. The apparatus 200 may, for example, be configured to splice the processed video with the input video of a second video data stream of the two or more input video data streams to obtain the output video, and the apparatus 200 may, for example, be configured to encode the output video into the output video data stream.

[0174] According to embodiments, the apparatus 200 may, for example, be configured to determine whether to use the coded picture buffer delay offset information for determining the access unit removal time of the current picture depending on a position of the current picture within the output video. Or, the apparatus 200 may, for example, be configured to determine whether to set the coded picture buffer delay offset value of the coded picture buffer delay offset information to 0 for determining the access unit removal time of the current picture depending on a position of the current picture within the output video.

[0175] In embodiments, the apparatus 200 may, for example, be configured to determine whether to use the coded picture buffer delay offset information for determining the access unit removal time of the current picture depending on a position of a previous non- discardable picture preceding the current picture within the output video.

[0176] According to embodiments, the apparatus 200 may, for example, be configured to determine whether to use the coded picture buffer delay offset information for determining the access unit removal time of the current picture depending on whether a previous non- discardable picture preceding the current picture within the output video may, for example, be the first picture in a previous buffering period.

[0177] In embodiments, the apparatus 200 may, for example, be configured to determine whether to use the coded picture buffer delay offset information for determining the access unit removal time of the current picture depending on the concatenation flag, the current picture being the first picture of the input video of the second video data stream.

[0178] According to embodiments, the apparatus 200 may, for example, be configured to determine the access unit removal time of the current picture depending on the removal time of the previous picture.

[0179] In embodiments, the apparatus 200 may, for example, be configured to determine the access unit removal time of the current picture depending on the initial coded picture buffer removal delay information.

[0180] According to embodiments, the apparatus 200 may, for example, be configured to update the initial coded picture buffer removal delay information depending on the clock ticks to obtain temporary coded picture buffer removal delay information for determining the access unit removal time of the current picture.

[0181] According to embodiments, if the concatenation flag is set to a first value, the apparatus 200 is configured to use the coded picture buffer delay offset information for determining one or more removal times. If the concatenation flag is set to a second value different from the first value, the apparatus 200 is configured to not use the coded picture buffer delay offset information for determining one or more removal times.

[0182] In an embodiment, the apparatus 200 may, for example, be configured to signal to the video decoder 300 whether to use the coded picture buffer delay offset information for determining the current picture removal time from the coded picture buffer access unit.

[0183] According to an embodiment, the current picture may, for example, be located at a splice point of an output video, wherein two input videos have been spliced.

[0184] Further, a video data stream is provided. The video is encoded in the video data stream. The video data stream comprises coded picture buffer delay offset information.

[0185] According to an embodiment, the video data stream may, for example, comprise a concatenation flag.

[0186] In an embodiment, the video data stream may, for example, comprise initial coded picture buffer removal delay information.

[0187] According to an embodiment, the concatenation flag indicates that the coded picture buffer delay offset information needs to be used for determining the one or more (picture or access unit) removal times if the concatenation flag is set to a first value (e.g. 0), for example when it is known that some pictures (e.g. RASL pictures) have been discarded. The concatenation flag indicates that the indicated offset is not to be used for determining the one or more (picture or access unit) removal times if the concatenation flag is set to a second value (e.g. 1) different from the first value, for example independent of the offset signaling and for example independent of whether RASL pictures have been discarded or not. If the pictures are not discarded, for example, the offset is not used.

[0188] Further, a video encoder 100 is provided. The video encoder 100 is configured to encode a video into a video data stream. The video encoder 100 is configured to generate the video data stream such that the video data stream comprises coded picture buffer delay offset information.

[0189] According to an embodiment, the video encoder 100 may, for example, be configured to generate the video data stream such that the video data stream may, for example, comprise a concatenation flag.

[0190] In an embodiment, the video encoder 100 may, for example, be configured to generate the video data stream such that the video data stream may, for example, comprise the coded picture buffer delay offset information.

[0191] Furthermore, a video decoder 300 for receiving a video data stream having a video stored therein is provided. The video decoder 300 is configured to decode the video from the video data stream. Furthermore, the video decoder 300 is configured to decode the video in dependence on a removal time of a current picture from an access unit of a coded picture buffer of a plurality of pictures of the video. The video decoder 300 is configured to decode the video in dependence on an indication indicating whether to use coded picture buffer delay offset information for determining the removal time of the current picture from the access unit of the coded picture buffer.

[0192] According to embodiments, the removal time of at least one picture of the plurality of pictures from the access unit of the coded picture buffer depends on the coded picture buffer delay offset information.

[0193] In embodiments, the video decoder 300 is configured to decode the video in dependence on determining whether to use the coded picture buffer delay offset information for determining the removal time of the current picture from the access unit in dependence on a position of the current picture within the video.

[0194] According to embodiments, the video decoder 300 may, for example, be configured to decode the video in dependence on whether a coded picture buffer delay offset value of the coded picture buffer delay offset information may, for example, be set to 0.

[0195] In embodiments, the video decoder 300 may, for example, be configured to determine whether to use the coded picture buffer delay offset information for determining the removal time of the current picture from the access unit in dependence on a position of a previous non-discardable picture within the video before the current picture.

[0196] According to embodiments, the video decoder 300 may, for example, be configured to determine whether to use the coded picture buffer delay offset information for determining the removal time of the current picture from the access unit in dependence on whether a previous non-discardable picture within the video before the current picture may, for example, be the first picture in a previous buffering period.

[0197] In embodiments, the video decoder 300 may, for example, be configured to determine whether to use the coded picture buffer delay offset information for determining the removal time of the current picture from the access unit in dependence on a concatenation flag, the current picture being a first picture of an input video of a second video data stream.

[0198] According to embodiments, the video decoder 300 may, for example, be configured to determine the removal time of the current picture from the access unit in dependence on a removal time of a previous picture.

[0199] In embodiments, the video decoder 300 may, for example, be configured to determine the removal time of the current picture from the access unit in dependence on initial coded picture buffer removal delay information.

[0200] According to embodiments, the video decoder 300 may, for example, be configured to update the initial coded picture buffer removal delay information according to the clock ticks to obtain temporary coded picture buffer removal delay information to determine the access unit removal time of the current picture.

[0201] According to embodiments, if the concatenation flag is set to a first value, the video decoder 300 is configured to use the coded picture buffer delay offset information to determine the one or more removal times. If the concatenation flag is set to a second value different from the first value, the video decoder 300 is configured to not use the coded picture buffer delay offset information to determine the one or more removal times.

[0202] Further, a system is provided. The system comprises the apparatus 200 as described above and the video decoder 300 as described above. The video decoder 300 is configured to receive the output video data stream of the apparatus 200. Further, the video decoder 300 is configured to decode the video from the output video data stream of the apparatus 200.

[0203] According to embodiments, the system may, for example, further comprise the video encoder 100. The apparatus 200 may, for example, be configured to receive the video data stream from the video encoder 100 as the input video data stream.

[0204] In particular, the second aspect of the present invention relates to the fact that prevNonDiscardable can have included an alternative offset (CpbDelayOffset) in case of alternative timing (when it is not a BP start), so for AUs with concatenation_flag == 1, CpbDelayOffset should be temporarily set to zero.

[0205] When splicing of two bitstreams occurs, the derivation of the removal time of an AU from the CPB is done differently than for non-spliced bitstreams. At the splice point, the buffering period SEI message (BP SEI message; SEI = supplemental enhancement information) includes a concatenationFlag equal to 1. The decoder then needs to check 2 values and take the larger one of the two:

[0206] • the previous non-discardable picture (prevNonDiscardablePic) removal time plus the increment signaled in the BP SEI message (auCpbRemovalDelayDeltaMinusl + 1), or

[0207] • the previous picture removal time plus the InitCpbRemovalDelay.

[0208] However, when the previous picture with BP SEI message is an AU whose removal times have been derived using the alternative timing (i.e. the second timing information used when RASL pictures or pictures up to DRAP have been discarded), an offset (CpbDelayOffset) is used to compute each removal time, which is computed as an increment of the previous picture with buffering period, i.e. AuNominalRemovalTime[firstPicInPrevBuffPeriod] plus AuCpbRemovalDelayVal - CpbDelayOffset, as Figure 4 shown.

[0209] Figure 4 The original bitstream (top of Figure 4 ) and the bitstream after discarding pictures (bottom of Figure 4 ) are shown: after discarding the AUs (lines 1, 2 and 3 in the original bitstream), the offset is taken into account in the computation of the removal delay.

[0210] The offset is added because the removal times are computed using an increment of the removal time of a picture (called firstPicInPrevBuffPeriod), after which some AUs have been discarded, so CpbDelayOffset is needed to take this into account (to compensate) for the AU discarding.

[0211] Figure 5 The splicing of two bitstreams (the first bitstream (in the middle left in Figure 5 ) and the second bitstream (in the middle right in Figure 5 )) after discarding pictures from the original first bitstream (in the middle left in Figure 5 ) is shown.

[0212] The example using the previous picture removal time as anchor instead of the previous non-discardable picture is similar, and does not require the "-3" correction factor (CpbDelayOffset) either.

[0213] However, in case the picture with BP SEI message is a picture that has been discarded (i.e. a picture that has been removed from the bitstream), the removal time is computed using the removal time of the previous picture with buffering period, i.e. AuNominalRemovalTime[firstPicInPrevBuffPeriod] plus AuCpbRemovalDelayVal, as shown in Figure 5In the stitching case shown, note that this is not necessarily the case for two derived removal times of AUs using the firstPicInPrevBuffPeriod associated with the BP SEI message. As discussed, for the stitching case, the delta is added to prevNonDiscardablePic or just the previous picture. This means that when prevNonDiscardablePic is not firstPicInPrevBuffPeriod, CpbDelayOffset cannot be used to derive the removal time of the current AU from the CPB because the removal time of prevNonDiscardablePic already takes into account the AU discard and there is no discard of any AU between prevNonDiscardablePic and the AU for which the removal time is computed. Now assuming that the previous picture removal time is used instead, for the case of the current AU (i.e., the stitching point with the new BP SEI message) having InitialCpbRemovalDelay that forces the removal time of the current AU to be after its expected removal time, this would achieve an equidistant removal time (when prevNonDiscardablePic is used instead). In this case, the removal time of the current AU cannot be less than the time computed by using the previous picture removal time plus InitCpbRemovalDelay because this can result in buffer underflow (AU not in the buffer before it needs to be removed). Therefore, as part of this invention, for this case, CpbDelayOffset is not used for the computation or considered to be equal to 0.

[0214] The embodiments summarized herein are dependent on checking whether CpbDelayOffset is used to compute the removal time of an AU when discarding an AU from the bitstream between a RASL AU or discarding an IRAP and a DRAP AU. The check to determine whether CpbDelayOffset is not used or considered to be equal to 0 is one of the following:

[0215] • prevNonDiscardablePic is not firstPicInPrevBuffPeriod

[0216] • The previous picture removal time plus InitCpbRemovalDelay is used to compute the removal time of the current AU.

[0217] The implementation in the specification can be as follows:

[0218] - When AU n is the first AU of a BP that is not initialized by the HRD, the following applies:

[0219] The nominal removal time of AU n from the CPB is specified by the following:

[0220]

[0221] Alternatively, in Figure 6 In another embodiment, illustrated in Fig. 6, CpbDelayOffset for the calculation of the AU removal time when a RASL AU is discarded from the bitstream or when an AU between an IRAP and a DRAP AU is discarded depends on different checks including checking concatenationFlag.

[0222] In this case, when concatenationFlag is set to 1, the delta in the bitstream needs to match the correct value as if CpbDelayOffset was considered (when comparing Figure 5 and Figure 6 It is clear that for this figure, CpbDelayOffset is not applied or considered as 0.

[0223] The implementation in the specification can be as follows:

[0224] - When AU n is the first AU of a BP that does not initialize the HRD, the following applies:

[0225] The nominal removal time of AU n from the CPB is specified by:

[0226]

[0227] In the following, the third aspect of the present application is now described in detail.

[0228] According to the third aspect of the present application, a video data stream is provided. The video is encoded in the video data stream. Further, the video data stream comprises an initial coded picture buffer removal delay. Further, the video data stream comprises an initial coded picture buffer removal offset. Further, the video data stream comprises information indicating whether a sum of the initial coded picture buffer removal delay and the initial coded picture buffer removal offset is defined to be constant across two or more buffering periods.

[0229] According to embodiments, the initial coded picture buffer removal delay may, for example, indicate a time that needs to elapse before a first access unit of the video data stream that initializes the video decoder 300 is transmitted to the video decoder 300.

[0230] In embodiments, the video data stream may, for example, comprise a single indication that may, for example, indicate whether a sum of the initial coded picture buffer removal delay and the initial coded picture buffer removal offset may, for example, be defined to be constant across two or more buffering periods.

[0231] According to embodiments, the video data stream may, for example, comprise a single indication, e.g. a concatenation flag, which may, for example, indicate whether the sum of the initial coded picture buffer removal delay and the initial coded picture buffer removal offset may, for example, be defined to be constant across two or more buffering periods. If the concatenation flag is equal to a first value, the sum of the initial coded picture buffer removal delay and the initial coded picture buffer removal offset is constant across two or more buffering periods. If the concatenation flag is different from the first value, the concatenation flag does not define whether the sum of the initial coded picture buffer removal delay and the initial coded picture buffer removal offset is constant across two or more buffering periods.

[0232] In embodiments, if the single indication does not indicate that the sum of the initial coded picture buffer removal delay and the initial coded picture buffer removal offset is defined to be constant across two or more buffering periods, the video data stream may, for example, comprise continuously updated information on the initial coded picture buffer removal delay information and continuously updated information on the initial coded picture buffer removal offset information.

[0233] According to embodiments, if the video data stream comprises information indicating that the sum of the initial coded picture buffer removal delay and the initial coded picture buffer removal offset is defined to be constant across two or more buffering periods, the sum of the initial coded picture buffer removal delay and the initial coded picture buffer removal offset may, for example, be defined to be constant from a current position within the video data stream.

[0234] Further, a video encoder 100 is provided. The video encoder 100 is configured to encode a video into a video data stream. Further, the video encoder 100 is configured to generate the video data stream such that the video data stream comprises an initial coded picture buffer removal delay. Further, the video encoder 100 is configured to generate the video data stream such that the video data stream comprises an initial coded picture buffer removal offset. Further, the video encoder 100 is configured to generate the video data stream such that the video data stream comprises information indicating whether a sum of the initial coded picture buffer removal delay and the initial coded picture buffer removal offset is defined to be constant across two or more buffering periods.

[0235] According to embodiments, the initial coded picture buffer removal delay may, for example, indicate a time that a first access unit of a picture of the video data stream, which initializes the video decoder 300, needs to pass before the first access unit is transmitted to the video decoder 300.

[0236] In an embodiment, the video encoder 100 may, for example, be configured to generate the video data stream such that the video data stream may, for example, comprise a single indication which may, for example, indicate whether the sum of the initial coded picture buffer removal delay and the initial coded picture buffer removal offset may, for example, be defined to be constant across two or more buffering periods.

[0237] According to an embodiment, the video encoder 100 may, for example, be configured to generate the video data stream such that the video data stream may, for example, comprise a concatenated flag as a single indication which may, for example, indicate whether the sum of the initial coded picture buffer removal delay and the initial coded picture buffer removal offset may, for example, be defined to be constant across two or more buffering periods. If the concatenated flag is equal to a first value, the sum of the initial coded picture buffer removal delay and the initial coded picture buffer removal offset is constant across two or more buffering periods. If the concatenated flag is different from the first value, the concatenated flag does not define whether the sum of the initial coded picture buffer removal delay and the initial coded picture buffer removal offset is constant across two or more buffering periods.

[0238] In an embodiment, if the single indication does not indicate that the sum of the initial coded picture buffer removal delay and the initial coded picture buffer removal offset is defined to be constant across two or more buffering periods, the video encoder 100 may, for example, be configured to generate the video data stream such that the video data stream may, for example, comprise continuously updated information on the initial coded picture buffer removal delay information and continuously updated information on the initial coded picture buffer removal offset information.

[0239] According to an embodiment, if the video data stream comprises information which may, for example, indicate that the sum of the initial coded picture buffer removal delay and the initial coded picture buffer removal offset is defined to be constant across two or more buffering periods, the sum of the initial coded picture buffer removal delay and the initial coded picture buffer removal offset is defined to be constant from a current position within the video data stream.

[0240] Furthermore, it is provided an apparatus 200 for receiving two input video data streams, as a first input video data stream and a second input video data stream. Each of the two input video data streams has an input video encoded therein. The apparatus 200 is configured to generate an output video data stream from the two input video data streams, the output video data stream encoding an output video, wherein the apparatus is configured to generate the output video data stream by concatenating the first input video data stream and the second input video data stream. Furthermore, the apparatus 200 is configured to generate the output video data stream such that the output video data stream comprises an initial coded picture buffer removal delay. Furthermore, the apparatus 200 is configured to generate the output video data stream such that the output video data stream comprises an initial coded picture buffer removal offset. Furthermore, the apparatus 200 is configured to generate the output video data stream such that the output video data stream comprises information indicating whether a sum of the initial coded picture buffer removal delay and the initial coded picture buffer removal offset is defined to be constant across two or more buffering periods.

[0241] According to embodiments, the initial coded picture buffer removal delay may, for example, indicate a time that a first access unit of the output video data stream needs to elapse before transmitting the first access unit to the video decoder 300 for initializing the video decoder 300.

[0242] In embodiments, the apparatus 200 may, for example, be configured to generate the output video data stream such that the output video data stream may, for example, comprise a single indication which may, for example, indicate whether a sum of the initial coded picture buffer removal delay and the initial coded picture buffer removal offset may, for example, be defined to be constant across two or more buffering periods.

[0243] According to embodiments, the apparatus 200 may, for example, be configured to generate the output video data stream such that the output video data stream may, for example, comprise a concatenation flag as a single indication which may, for example, indicate whether a sum of the initial coded picture buffer removal delay and the initial coded picture buffer removal offset may, for example, be defined to be constant across two or more buffering periods. If the concatenation flag is equal to a first value, the sum of the initial coded picture buffer removal delay and the initial coded picture buffer removal offset is constant across two or more buffering periods. If the concatenation flag is different from the first value, the concatenation flag does not define whether the sum of the initial coded picture buffer removal delay and the initial coded picture buffer removal offset is constant across two or more buffering periods.

[0244] In an embodiment, if the single indication does not indicate that the sum of the initial coded picture buffer removal delay and the initial coded picture buffer removal offset is defined to be constant across two or more buffering periods, the apparatus 200 is configured to: generate the output video data stream such that the output video data stream comprises a continuous update information on the initial coded picture buffer removal delay information and a continuous update information on the initial coded picture buffer removal offset information.

[0245] According to an embodiment, if the video data stream comprises information indicating that the sum of the initial coded picture buffer removal delay and the initial coded picture buffer removal offset is defined to be constant across two or more buffering periods, the sum of the initial coded picture buffer removal delay and the initial coded picture buffer removal offset is defined to be constant from a current position within the video data stream.

[0246] Further, a video decoder 300 for receiving a video data stream in which a video is stored is provided. The video decoder 300 is configured to decode the video from the video data stream. Further, the video data stream comprises an initial coded picture buffer removal delay. Further, the video data stream comprises an initial coded picture buffer removal offset. Further, the video data stream comprises information indicating whether a sum of the initial coded picture buffer removal delay and the initial coded picture buffer removal offset is defined to be constant across two or more buffering periods. Further, the video decoder 300 is configured to decode the video in dependence on the information indicating whether the sum of the initial coded picture buffer removal delay and the initial coded picture buffer removal offset is defined to be constant across two or more buffering periods.

[0247] According to an embodiment, the initial coded picture buffer removal delay may, for example, indicate a time that a first access unit needs to pass before sending the first access unit to the video decoder 300 initializing the output video data stream of the video decoder 300.

[0248] In an embodiment, the video data stream may, for example, comprise a single indication which may, for example, indicate whether the sum of the initial coded picture buffer removal delay and the initial coded picture buffer removal offset may, for example, be defined to be constant across two or more buffering periods. The video decoder 300 may, for example, be configured to decode the video in dependence on the single indication.

[0249] According to embodiments, the video data stream may, for example, comprise a concatenation flag as a single indication, which may, for example, indicate whether the sum of the initial coded picture buffer removal delay and the initial coded picture buffer removal offset may, for example, be defined to be constant across two or more buffering periods. If the concatenation flag is equal to a first value, the sum of the initial coded picture buffer removal delay and the initial coded picture buffer removal offset is constant across two or more buffering periods. If the concatenation flag is different from the first value, the concatenation flag does not define whether the sum of the initial coded picture buffer removal delay and the initial coded picture buffer removal offset is constant across two or more buffering periods. The video decoder 300 is configured to decode the video in accordance with the concatenation flag.

[0250] In embodiments, if the single indication does not indicate that the sum of the initial coded picture buffer removal delay and the initial coded picture buffer removal offset is defined to be constant across two or more buffering periods, the video data stream comprises persistent update information on initial coded picture buffer removal delay information and persistent update information on initial coded picture buffer removal offset information. The video decoder 300 is configured to decode the video in accordance with the persistent update information on initial coded picture buffer removal delay information and the persistent update information on initial coded picture buffer removal offset information.

[0251] According to embodiments, if the video data stream comprises information indicating that the sum of the initial coded picture buffer removal delay and the initial coded picture buffer removal offset is defined to be constant across two or more buffering periods, the sum of the initial coded picture buffer removal delay and the initial coded picture buffer removal offset is defined to be constant from a current position within the video data stream.

[0252] Further, a system is provided. The system comprises the apparatus 200 as described above and the video decoder 300 as described above. The video decoder 300 is configured to receive the output video data stream of the apparatus 200. Further, the video decoder 300 is configured to decode the video from the output video data stream of the apparatus 200.

[0253] According to embodiments, the system may, for example, further comprise the video encoder 100. The apparatus 200 may, for example, be configured to receive the video data stream from the video encoder 100 as the input video data stream.

[0254] In particular, the third aspect of the present invention relates to concatenation, initial CPB removal delay and initial CPB removal offset.

[0255] Currently, the specification indicates that the sum of the initial Cpb removal delay and the initial Cpb removal offset is constant within a CVS. The same constraint is formulated for the alternative timing. The initial Cpb removal delay indicates the time that the first AU in the bitstream needs to pass before the first AU can be sent to an initialization decoder for decoding. The initial Cpb removal offset is a property of the bitstream, which means that the earliest arrival time of an AU in the decoder is not necessarily equidistant to the time 0, when the first AU arrives at the decoder. It helps to determine when the first bit of an AU can arrive earliest at the decoder.

[0256] The current constraint in the VVC draft specification indicates that the sum of these two values needs to be constant within a CVS:

[0257] The sum of nal_initial_cpb_removal_delay[i][j] and nal_initial_cpb_removal_offset[i][j] should be constant for each pair of values of i and j over the entire CVS, and the sum of nal_initial_alt_cpb_removal_delay[i][j] and nal_initial_alt_cpb_removal_offset[i][j] should be constant.

[0258] A problem arises when a bitstream is edited or concatenated to form a new joint bitstream. It is also desirable to be able to indicate whether this property is fulfilled across the CVS boundaries of the bitstreams, because having different values for this sum can lead to underflow or overflow of the buffer.

[0259] Therefore, in an embodiment, an indication is carried in the bitstream that from a certain point in the bitstream (e.g. a concatenation point), the value constraint on the constant sum of InitCpbRemovalDelay and InitCpbRemovalDelayOffset (and the alternative counterparts) is reset, and the sum before and after this point in the bitstream can be different. Unless this indication is present in the bitstream, the sum remains constant.

[0260] For example:

[0261] When concatenationFlag is equal to 0, the constraint of bitstream conformance is that the sum of InitCpbRemovalDelay and InitCpbRemovalDelayOffset is constant across the buffering period.

[0262] Otherwise, the sum of InitCpbRemovalDelay and InitCpbRemovalDelayOffset does not have to be constant across buffering periods. The values of InitCpbRemovalDelay and InitCpbRemovalDelayOffset are updated to take into account the arrival time.

[0263] In embodiments, if several bitstreams are spliced, at each splice point, a concatenation flag, for example, can define whether the sum remains constant.

[0264] In the following, a fourth aspect of the present application is described in detail.

[0265] According to the fourth aspect of the present application, a video data stream is provided. The video data stream has a video encoded therein. Furthermore, the video data stream comprises an indication (e.g. general_same_pic_timing_in_all_ols_flag) indicating whether a non-scalable nested picture timing supplemental enhancement information message of a network abstraction layer unit of an access unit of a plurality of access units of a coded video sequence of one or more coded video sequences of the video data stream is defined to be applicable to all output layer sets of the plurality of output layer sets of the access unit. If the indication (e.g. general_same_pic_timing_in_all_ols_flag) has a first value, the non-scalable nested picture timing supplemental enhancement information message of the network abstraction layer unit of the access unit is defined to be applicable to all output layer sets of the plurality of output layer sets of the access unit. If the indication (e.g. general_same_pic_timing_in_all_ols_flag) has a value different from the first value, the indication does not define whether the non-scalable nested picture timing supplemental enhancement information message of the network abstraction layer unit of the access unit is applicable to all output layer sets of the plurality of output layer sets of the access unit.

[0266] According to embodiments, for example, if the indication (e.g. general_same_pic_timing_in_all_ols_flag) has the first value, the network abstraction layer unit does not comprise any other supplemental enhancement information message than the picture timing supplemental enhancement information message.

[0267] In embodiments, for example, if the indication (e.g. general_same_pic_timing_in_all_ols_flag) has the first value, the network abstraction layer unit does not comprise any other supplemental enhancement information message.

[0268] According to embodiments, for example, if the indication (e.g., general_same_pic_timing_in_all_ols_flag) has a first value, each network abstraction layer unit of a coded video sequence of one or more coded video sequences in the video data stream that includes a non-scalable-nested picture timing supplemental enhancement information message for each access unit of a plurality of access units of the coded video sequence does not include any other supplemental enhancement information message different from the picture timing supplemental enhancement information message, or does not include any other supplemental enhancement information message.

[0269] In embodiments, if the indication (e.g., general_same_pic_timing_in_all_ols_flag) has a first value, each network abstraction layer unit of a coded video sequence of one or more coded video sequences in the video data stream that includes a non-scalable-nested picture timing supplemental enhancement information message for each access unit of a plurality of access units of the coded video sequence does not include any other supplemental enhancement information message different from the picture timing supplemental enhancement information message, or does not include any other supplemental enhancement information message.

[0270] Further, for example, a video encoder 100 can be provided. The video encoder 100 is configured to encode a video into a video data stream. Further, the video encoder 100 is configured to generate the video data stream such that the video data stream comprises an indication (e.g., general_same_pic_timing_in_all_ols_flag) indicating whether a non-scalable-nested picture timing supplemental enhancement information message of a network abstraction layer unit of an access unit of a plurality of access units of a coded video sequence of one or more coded video sequences of the video data stream is defined to apply to all output layer sets of the access unit. If the indication (e.g., general_same_pic_timing_in_all_ols_flag) has a first value, the non-scalable-nested picture timing supplemental enhancement information message of the network abstraction layer unit of the access unit is defined to apply to all output layer sets of the access unit. If the indication (e.g., general_same_pic_timing_in_all_ols_flag) has a value different from the first value, the indication does not define whether the non-scalable-nested picture timing supplemental enhancement information message of the network abstraction layer unit of the access unit applies to all output layer sets of the access unit.

[0271] According to embodiments, e.g., if the indication (e.g., general_same_pic_timing_in_all_ols_flag) has the first value, video encoder 100 is configured to generate the stream of video data such that the network abstraction layer units do not include any other supplemental enhancement information messages different from the picture timing supplemental enhancement information messages.

[0272] In embodiments, e.g., if the indication (e.g., general_same_pic_timing_in_all_ols_flag) has the first value, video encoder 100 is configured to generate the stream of video data such that the network abstraction layer units do not include any other supplemental enhancement information messages.

[0273] According to embodiments, e.g., if the indication (e.g., general_same_pic_timing_in_all_ols_flag) has the first value, video encoder 100 may, e.g., be configured to generate the stream of video data such that for each network abstraction layer unit including the non-scalable nested picture timing supplemental enhancement information messages for each access unit of a plurality of access units of a coded video sequence of the one or more coded video sequences, the network abstraction layer unit does not include any other supplemental enhancement information messages different from the picture timing supplemental enhancement information messages or does not include any other supplemental enhancement information messages.

[0274] In embodiments, e.g., if the indication (e.g., general_same_pic_timing_in_all_ols_flag) has the first value, video encoder 100 may, e.g., be configured to generate the stream of video data such that for each network abstraction layer unit including the non-scalable nested picture timing supplemental enhancement information messages for each access unit of a plurality of access units of each of the one or more coded video sequences of the stream of video data, the network abstraction layer unit does not include any other supplemental enhancement information messages different from the picture timing supplemental enhancement information messages or does not include any other supplemental enhancement information messages.

[0275] Furthermore, an apparatus 200 for receiving an input video data stream is provided. Video is encoded in the input video data stream. The apparatus 200 is configured to generate a processed video data stream from the input video data stream. Furthermore, the apparatus 200 is configured to generate the processed video data stream such that the processed video data stream comprises an indication (e.g. general_same_pic_timing_in_all_ols_flag) indicating whether non-scalable nested picture timing supplemental enhancement information messages of network abstraction layer units of access units in a plurality of access units of a coded video sequence in one or more coded video sequences of the processed video data stream are defined to apply to all of a plurality of output layer sets of the access units. If the indication (e.g. general_same_pic_timing_in_all_ols_flag) has a first value, the non-scalable nested picture timing supplemental enhancement information messages of the network abstraction layer units of the access units are defined to apply to all of the plurality of output layer sets of the access units. If the indication (e.g. general_same_pic_timing_in_all_ols_flag) has a value different from the first value, the indication does not define whether the non-scalable nested picture timing supplemental enhancement information messages of the network abstraction layer units of the access units apply to all of the plurality of output layer sets of the access units.

[0276] According to embodiments, for example, if the indication (e.g. general_same_pic_timing_in_all_ols_flag) has the first value, the apparatus 200 is configured to generate the processed video data stream such that the network abstraction layer units do not comprise any other supplemental enhancement information messages than the picture timing supplemental enhancement information messages.

[0277] In embodiments, for example, if the indication (e.g. general_same_pic_timing_in_all_ols_flag) has the first value, the apparatus 200 is configured to generate the processed video data stream such that the network abstraction layer units do not comprise any other supplemental enhancement information messages.

[0278] According to embodiments, for example, if the indication (e.g., general_same_pic_timing_in_all_ols_flag) has a first value, the apparatus 200 may, for example, be configured to generate the processing of the video data stream such that for each network abstraction layer unit of each access unit of a plurality of access units of a coded video sequence of the one or more coded video sequences comprising a non-scalable nested picture timing supplemental enhancement information message, the network abstraction layer unit does not comprise any other supplemental enhancement information message different from the picture timing supplemental enhancement information message or does not comprise any other supplemental enhancement information message.

[0279] In embodiments, for example, if the indication (e.g., general_same_pic_timing_in_all_ols_flag) has a first value, the apparatus 200 may, for example, be configured to generate the processing of the video data stream such that for each network abstraction layer unit of each access unit of a plurality of access units of each of the one or more coded video sequences comprising a non-scalable nested picture timing supplemental enhancement information message, the network abstraction layer unit does not comprise any other supplemental enhancement information message different from the picture timing supplemental enhancement information message or does not comprise any other supplemental enhancement information message.

[0280] Further, a video decoder 300 for receiving a video data stream having a video stored therein is provided. The video decoder 300 is configured to decode the video from the video data stream. The video data stream comprises an indication (e.g., general_same_pic_timing_in_all_ols_flag) indicating whether a non-scalable nested picture timing supplemental enhancement information message of a network abstraction layer unit of an access unit of a plurality of access units of a coded video sequence of the one or more coded video sequences of the video data stream is defined to be applicable to all output layer sets of the access unit. If the indication (e.g., general_same_pic_timing_in_all_ols_flag) has a first value, the non-scalable nested picture timing supplemental enhancement information message of the network abstraction layer unit of the access unit is defined to be applicable to all output layer sets of the access unit. If the indication (e.g., general_same_pic_timing_in_all_ols_flag) has a value different from the first value, the indication does not define whether the non-scalable nested picture timing supplemental enhancement information message of the network abstraction layer unit of the access unit is applicable to all output layer sets of the access unit. The video decoder 300 is configured to decode the video in accordance with the indication.

[0281] According to embodiments, for example, if the indication (e.g. general_same_pic_timing_in_all_ols_flag) has a first value, the network abstraction layer unit does not comprise any other supplemental enhancement information messages than the picture timing supplemental enhancement information messages.

[0282] In embodiments, for example, if the indication (e.g. general_same_pic_timing_in_all_ols_flag) has a first value, the network abstraction layer unit does not comprise any other supplemental enhancement information messages. The video decoder 300 is configured to decode the video in dependence on the indication.

[0283] According to embodiments, for example, if the indication (e.g. general_same_pic_timing_in_all_ols_flag) has a first value, for each network abstraction layer unit comprising a non-scalable nested picture timing supplemental enhancement information message in each access unit of a plurality of access units of a coded video sequence of one or more coded video sequences, the network abstraction layer unit does not comprise any other supplemental enhancement information messages than the picture timing supplemental enhancement information messages, or does not comprise any other supplemental enhancement information messages.

[0284] In embodiments, for example, if the indication (e.g. general_same_pic_timing_in_all_ols_flag) has a first value, for each network abstraction layer unit comprising a non-scalable nested picture timing supplemental enhancement information message in each access unit of a plurality of access units of each of one or more coded video sequences of the video data stream, the network abstraction layer unit does not comprise any other supplemental enhancement information messages than the picture timing supplemental enhancement information messages, or does not comprise any other supplemental enhancement information messages.

[0285] Further, a system is provided. The system comprises the apparatus 200 as described above and the video decoder 300 as described above. The video decoder 300 is configured to receive the processed video data stream of the apparatus 200. Further, the video decoder 300 is configured to decode the video from the output video data stream of the apparatus 200.

[0286] According to embodiments, the system may, for example, further comprise the video encoder 100. The apparatus 200 may, for example, be configured to receive the video data stream from the video encoder 100 as the input video data stream.

[0287] In particular, the fourth aspect of the application relates to the constraint that PT SEI is not paired with other HRD SEI when general_same_pic_timing_in_all_ols_flag is equal to 1.

[0288] The VVC draft specification includes a flag called general_same_pic_timing_in_all_ols_flag in the general HRD parameters structure with the following semantics:

[0289] general_same_pic_timing_in_all_ols_flag equal to 1 specifies that the non-scalable-nested PT SEI messages in each AU apply to the AUs of any OLS in the bitstream and no scalable-nested PT SEI messages are present. general_same_pic_timing_in_all_ols_flag equal to 0 specifies that the non-scalable-nested PT SEI messages in each AU can or can not apply to the AUs of any OLS in the bitstream and scalable-nested PT SEI messages can be present.

[0290] In general, when extracting an OLS sub-bitstream from the original bitstream (including OLS data plus non-OLS data), the corresponding HRD-related timing / buffer information of the target OLS in the form of buffering period, picture timing and decoding unit information SEI messages encapsulated in so-called scalable-nested SEI messages is decapsulated. This decapsulated SEI message is then used to replace the non-scalable-nested HRD SEI information in the original bitstream. However, in many scenarios, the content of some messages (e.g. picture timing SEI messages) can remain unchanged when layers are discarded, i.e. from one OLS to its subset. Therefore, general_same_pic_timing_in_all_ols_flag provides a shortcut so that only the BP and DUI SEI messages are replaced, but the PT SEI in the original bitstream can remain valid, i.e. the PT SEI is not removed during extraction when general_same_pic_timing_in_all_ols_flag is equal to 1. Therefore, there is no need to encapsulate the replacement PT SEI messages in the scalable-nested SEI messages carrying the replacement BP and DUI SEI messages, and no bit rate overhead is introduced for this information.

[0291] However, in the prior art, PT SEI messages are allowed to be carried within one SEI NAL unit (NAL unit = Network Abstraction Layer unit) together with other HRD SEI messages, i.e. BP, PT and SEI messages can all be encapsulated in the same prefix SEI NAL unit. Therefore, the extractor would have to make a deeper inspection of such SEI NAL unit to understand the included messages and when only one of the included messages (PT) is kept during the extraction process, it is actually required to rewrite the display SEI NAL unit (i.e. remove the non-PT SEI messages). In order to avoid such cumbersome low level processing and to allow the extractor to operate fully at NAL unit level on the non-parameter set part of the bitstream, it is part of the present application that the bitstream constraint does not allow such bitstream construction. In one embodiment, the constraint is formulated as follows:

[0292] general_same_pic_timing_in_all_ols_flag equal to 1 specifies that the non-scalable nesting PT SEI messages in each AU apply to the AUs of any OLS in the bitstream and no scalable nesting PT SEI messages are present. general_same_pic_timing_in_all_ols_flag equal to 0 specifies that the non-scalable nesting PT SEI messages in each AU can or can not apply to the AUs of any OLS in the bitstream and scalable nesting PT SEI messages can be present. When general_same_pic_timing_in_all_ols_flag is equal to 1, the constraint on bitstream conformance is that all general SEI messages containing SEI messages (picture timing) with payload_type equal to 1 in the bitstream should not contain SEI messages with payload_type not equal to 1.

[0293] In the following, a fifth aspect of the present application is described in detail.

[0294] According to the fifth aspect of the present application, a video data stream is provided. The video is encoded in the video data stream. Furthermore, the video data stream comprises one or more scalable nesting supplemental enhancement information messages. The one or more scalable nesting supplemental enhancement information messages comprise a plurality of syntax elements. Each of one or more syntax elements of the plurality of syntax elements is defined to have the same size in each scalable nesting supplemental enhancement information message of the video data stream or a part of the video data stream.

[0295] According to an embodiment, the video data stream may, for example, comprise one or more non-scalable nested supplemental enhancement information messages. The one or more scalable nested supplemental enhancement information messages and the one or more non-scalable nested supplemental enhancement information messages comprise a plurality of syntax elements. Each syntax element of one or more syntax elements of the plurality of syntax elements is defined to have a same size in each scalable nested supplemental enhancement information message of the video data stream or the portion of the video data stream and in each non-scalable nested supplemental enhancement information message of the video data stream or the portion of the video data stream.

[0296] In an embodiment, the video data stream may, for example, comprise a plurality of access units, wherein each access unit of the plurality of access units may, for example, be assigned to one of a plurality of pictures of the video. The portion of the video data stream may, for example, be an access unit of the plurality of access units of the video data stream. Each syntax element of one or more syntax elements of the plurality of syntax elements may, for example, be defined to have a same size in each scalable nested supplemental enhancement information message of the access unit.

[0297] According to an embodiment, the video data stream may, for example, comprise one or more non-scalable nested supplemental enhancement information messages. The one or more scalable nested supplemental enhancement information messages and the one or more non-scalable nested supplemental enhancement information messages comprise a plurality of syntax elements. Each syntax element of one or more syntax elements of the plurality of syntax elements may, for example, be defined to have a same size in each scalable nested supplemental enhancement information message of the access unit and in each non-scalable nested supplemental enhancement information message of the access unit.

[0298] In an embodiment, the portion of the video data stream may, for example, be a coded video sequence of the video data stream. Each syntax element of one or more syntax elements of the plurality of syntax elements may, for example, be defined to have a same size in each scalable nested supplemental enhancement information message of the coded video sequence.

[0299] According to an embodiment, the video data stream may, for example, comprise one or more non-scalable nested supplemental enhancement information messages. The one or more scalable nested supplemental enhancement information messages and the one or more non-scalable nested supplemental enhancement information messages comprise a plurality of syntax elements. Each syntax element of one or more syntax elements of the plurality of syntax elements may, for example, be defined to have a same size in each scalable nested supplemental enhancement information message of the scalable nested supplemental enhancement information messages of the coded video sequence and in each non-scalable nested supplemental enhancement information message of the non-scalable nested supplemental enhancement information of the coded video sequence.

[0300] In an embodiment, each syntax element of one or more syntax elements of the plurality of syntax elements may, for example, be defined to have a same size in each scalable nested supplemental enhancement information message of the scalable nested supplemental enhancement information messages of the video data stream.

[0301] According to an embodiment, each syntax element of one or more syntax elements of the plurality of syntax elements may, for example, be defined to have a same size in each scalable nested supplemental enhancement information message of the scalable nested supplemental enhancement information messages of the video data stream and in each non-scalable nested supplemental enhancement information message of the non-scalable nested supplemental enhancement information of the video data stream.

[0302] In an embodiment, the video data stream or a portion of the video data stream may, for example, comprise at least one buffering period supplemental enhancement information message, wherein the buffering period supplemental enhancement information message defines a size of each syntax element of one or more syntax elements of the plurality of syntax elements.

[0303] According to an embodiment, the buffering period supplemental enhancement information message comprises at least one of the following for defining a size of each syntax element of one or more syntax elements of the plurality of syntax elements:

[0304] a bp_cpb_initial_removal_delay_length_minus1 element,

[0305] a bp_cpb_removal_delay_length_minus1 element,

[0306] a bp_dpb_output_delay_length_minus1 element,

[0307] a bp_du_cpb_removal_delay_increment_length_minus1 element,

[0308] bp_dpb_output_delay_du_length_minus1 element.

[0309] In embodiments, for each access unit of the plurality of access units of the video data stream that includes a scalable-nested buffering period supplemental enhancement information message, the access unit may, for example, further include a non-scalable-nested buffering period supplemental enhancement information message.

[0310] According to embodiments, for each single-layer access unit of the plurality of single-layer access units of the video data stream that includes a scalable-nested buffering period supplemental enhancement information message, the single-layer access unit may, for example, further include a non-scalable-nested buffering period supplemental enhancement information message.

[0311] Furthermore, a video encoder 100 is provided. The video encoder 100 is configured to encode a video into a video data stream. Furthermore, the video encoder 100 is configured to generate the video data stream such that the video data stream comprises one or more scalable-nested supplemental enhancement information messages. Furthermore, the video encoder 100 is configured to generate the video data stream such that the one or more scalable-nested supplemental enhancement information messages comprise a plurality of syntax elements. Furthermore, the video encoder 100 is configured to generate the video data stream such that each syntax element of one or more syntax elements of the plurality of syntax elements is defined to have a same size in each scalable-nested supplemental enhancement information message of the video data stream or a portion of the video data stream.

[0312] According to embodiments, the video encoder 100 may, for example, be configured to generate the video data stream such that the video data stream may, for example, comprise one or more non-scalable-nested supplemental enhancement information messages. The video encoder 100 may, for example, be configured to generate the video data stream such that the one or more scalable-nested supplemental enhancement information messages and the one or more non-scalable-nested supplemental enhancement information messages comprise a plurality of syntax elements. The video encoder 100 may, for example, be configured to generate the video data stream such that each syntax element of one or more syntax elements of the plurality of syntax elements may, for example, be defined to have a same size in each scalable-nested supplemental enhancement information message of the video data stream or a portion of the video data stream, and in each non-scalable-nested supplemental enhancement information message of the video data stream or a portion of the video data stream.

[0313] In an embodiment, the video encoder 100 may, for example, be configured to generate the video data stream such that the video data stream may, for example, comprise a plurality of access units, wherein each access unit of the plurality of access units may, for example, be assigned to one of a plurality of pictures of the video. The portion of the video data stream may, for example, be an access unit of the plurality of access units of the video data stream. The video encoder 100 may, for example, be configured to generate the video data stream such that each of the one or more syntax elements of the plurality of syntax elements may, for example, be defined to have a same size in each of the scalable-nested supplemental enhancement information messages of the access unit.

[0314] According to an embodiment, the video encoder 100 may, for example, be configured to generate the video data stream such that the video data stream may, for example, comprise one or more non-scalable-nested supplemental enhancement information messages. The video encoder 100 may, for example, be configured to generate the video data stream such that the one or more scalable-nested supplemental enhancement information messages and the one or more non-scalable-nested supplemental enhancement information messages comprise the plurality of syntax elements. The video encoder 100 may, for example, be configured to generate the video data stream such that each of the one or more syntax elements of the plurality of syntax elements may, for example, be defined to have a same size in each of the scalable-nested supplemental enhancement information messages of the access unit and in each of the non-scalable-nested supplemental enhancement information messages of the access unit.

[0315] In an embodiment, the portion of the video data stream may, for example, be a coded video sequence of the video data stream. The video encoder 100 may, for example, be configured to generate the video data stream such that each of the one or more syntax elements of the plurality of syntax elements may, for example, be defined to have a same size in each of the scalable-nested supplemental enhancement information messages of the coded video sequence.

[0316] According to an embodiment, the video encoder 100, for example, can be configured to generate the video data stream such that the video data stream, for example, can comprise one or more non-scalable nested supplemental enhancement information messages. The video encoder 100, for example, can be configured to generate the video data stream such that the one or more scalable nested supplemental enhancement information messages and the one or more non-scalable nested supplemental enhancement information messages comprise a plurality of syntax elements. The video encoder 100, for example, can be configured to generate the video data stream such that each of the one or more syntax elements of the plurality of syntax elements, for example, can be defined to have a same size in each of the scalable nested supplemental enhancement information messages of the coded video sequence and in each of the non-scalable nested supplemental enhancement information messages of the coded video sequence.

[0317] In an embodiment, the video encoder 100, for example, can be configured to generate the video data stream such that each of the one or more syntax elements of the plurality of syntax elements, for example, can be defined to have a same size in each of the scalable nested supplemental enhancement information messages of the video data stream.

[0318] According to an embodiment, the video encoder 100, for example, can be configured to generate the video data stream such that each of the one or more syntax elements of the plurality of syntax elements, for example, can be defined to have a same size in each of the scalable nested supplemental enhancement information messages of the video data stream and in each of the non-scalable nested supplemental enhancement information messages of the video data stream.

[0319] In an embodiment, the video encoder 100, for example, can be configured to generate the video data stream such that the video data stream or a portion of the video data stream, for example, can comprise at least one buffering period supplemental enhancement information message, wherein the buffering period supplemental enhancement information message defines a size of each of the one or more syntax elements of the plurality of syntax elements.

[0320] According to an embodiment, the video encoder 100, for example, can be configured to generate the video data stream such that the buffering period supplemental enhancement information message comprises at least one of the following for defining the size of each of the one or more syntax elements of the plurality of syntax elements:

[0321] a bp_cpb_initial_removal_delay_length_minus1 element,

[0322] a bp_cpb_removal_delay_length_minus1 element,

[0323] bp_dpb_output_delay_length_minus1 element,

[0324] bp_du_cpb_removal_delay_increment_length_minus1 element,

[0325] bp_dpb_output_delay_du_length_minus1 element.

[0326] In an embodiment, the video encoder 100 may, for example, be configured to generate the video data stream such that for each access unit of the plurality of access units of the video data stream comprising the scalable-nested buffering period supplemental enhancement information message, the access unit may, for example, further comprise a non-scalable-nested buffering period supplemental enhancement information message.

[0327] According to an embodiment, the video encoder 100 may, for example, be configured to generate the video data stream such that for each single-layer access unit of the plurality of single-layer access units of the video data stream comprising the scalable-nested buffering period supplemental enhancement information message, the single-layer access unit may, for example, further comprise a non-scalable-nested buffering period supplemental enhancement information message.

[0328] Further, an apparatus 200 for receiving an input video data stream is provided. Video is encoded in the input video data stream. The apparatus 200 is configured to generate an output video data stream from the input video data stream. The video data stream comprises one or more scalable-nested supplemental enhancement information messages. The one or more scalable-nested supplemental enhancement information messages comprise a plurality of syntax elements. Each syntax element of one or more syntax elements of the plurality of syntax elements is defined to have a same size in each scalable-nested supplemental enhancement information message of the scalable-nested supplemental enhancement information messages of the video data stream or a portion of the video data stream. The apparatus 200 is configured to process the one or more scalable-nested supplemental enhancement information messages.

[0329] According to embodiments, the video data stream may, for example, comprise one or more non-scalable nested supplemental enhancement information messages. The one or more scalable nested supplemental enhancement information messages and the one or more non-scalable nested supplemental enhancement information messages comprise a plurality of syntax elements. Each of the one or more syntax elements of the plurality of syntax elements is defined to have a same size in each scalable nested supplemental enhancement information message of the scalable nested supplemental enhancement information messages of the video data stream or the portion of the video data stream, and in each non-scalable nested supplemental enhancement information message of the non-scalable nested supplemental enhancement information messages of the video data stream or the portion of the video data stream. The apparatus 200 may, for example, be configured to process the one or more scalable nested supplemental enhancement information messages and the one or more non-scalable nested supplemental enhancement information messages.

[0330] In embodiments, the video data stream may, for example, comprise a plurality of access units, wherein each access unit of the plurality of access units may, for example, be assigned to one of a plurality of pictures of the video. The portion of the video data stream may, for example, be an access unit of the plurality of access units of the video data stream. Each of the one or more syntax elements of the plurality of syntax elements may, for example, be defined to have a same size in each scalable nested supplemental enhancement information message of the scalable nested supplemental enhancement information messages of the access unit.

[0331] According to embodiments, the video data stream may, for example, comprise one or more non-scalable nested supplemental enhancement information messages. The one or more scalable nested supplemental enhancement information messages and the one or more non-scalable nested supplemental enhancement information messages comprise a plurality of syntax elements. Each of the one or more syntax elements of the plurality of syntax elements may, for example, be defined to have a same size in each scalable nested supplemental enhancement information message of the scalable nested supplemental enhancement information messages of the access unit, and in each non-scalable nested supplemental enhancement information message of the non-scalable nested supplemental enhancement information of the access unit. The apparatus 200 may, for example, be configured to process the one or more scalable nested supplemental enhancement information messages and the one or more non-scalable nested supplemental enhancement information messages.

[0332] In embodiments, the portion of the video data stream may, for example, be a coded video sequence of the video data stream. Each of the one or more syntax elements of the plurality of syntax elements may, for example, be defined to have a same size in each scalable nested supplemental enhancement information message of the scalable nested supplemental enhancement information messages of the coded video sequence.

[0333] According to an embodiment, the video data stream may, for example, comprise one or more non-scalable nested supplemental enhancement information messages. The one or more scalable nested supplemental enhancement information messages and the one or more non-scalable nested supplemental enhancement information messages comprise a plurality of syntax elements. Each syntax element of one or more syntax elements of the plurality of syntax elements may, for example, be defined to have a same size in each scalable nested supplemental enhancement information message of the scalable nested supplemental enhancement information messages of the coded video sequence and in each non-scalable nested supplemental enhancement information message of the non-scalable nested supplemental enhancement information of the coded video sequence. The apparatus 200 may, for example, be configured to process one or more scalable nested supplemental enhancement information messages and one or more non-scalable nested supplemental enhancement information messages.

[0334] In an embodiment, each syntax element of one or more syntax elements of the plurality of syntax elements may, for example, be defined to have a same size in each scalable nested supplemental enhancement information message of the scalable nested supplemental enhancement information messages of the video data stream.

[0335] According to an embodiment, each syntax element of one or more syntax elements of the plurality of syntax elements may, for example, be defined to have a same size in each scalable nested supplemental enhancement information message of the scalable nested supplemental enhancement information messages of the video data stream and in each non-scalable nested supplemental enhancement information message of the non-scalable nested supplemental enhancement information of the video data stream. The apparatus 200 may, for example, be configured to process one or more scalable nested supplemental enhancement information messages and one or more non-scalable nested supplemental enhancement information messages.

[0336] In an embodiment, the video data stream or the part of the video data stream may, for example, comprise at least one buffering period supplemental enhancement information message, wherein the buffering period supplemental enhancement information message defines a size of one or more syntax elements of a plurality of syntax elements. The apparatus 200 may, for example, be configured to process at least one buffering period supplemental enhancement information message.

[0337] According to an embodiment, the buffering period supplemental enhancement information message comprises at least one of the following for defining a size of one or more syntax elements of a plurality of syntax elements:

[0338] a bp_cpb_initial_removal_delay_length_minus1 element,

[0339] a bp_cpb_removal_delay_length_minus1 element,

[0340] bp_dpb_output_delay_length_minus1 element,

[0341] bp_du_cpb_removal_delay_increment_length_minus1 element,

[0342] bp_dpb_output_delay_du_length_minus1 element.

[0343] In an embodiment, for each access unit of the plurality of access units of the video data stream comprising a scalable-nested buffering period supplemental enhancement information message, the access unit may, for example, further comprise a non-scalable-nested buffering period supplemental enhancement information message. The apparatus 200 may, for example, be configured to process scalable-nested supplemental enhancement information messages and non-scalable-nested supplemental enhancement information messages.

[0344] According to an embodiment, for each single-layer access unit of the plurality of single-layer access units of the video data stream comprising a scalable-nested buffering period supplemental enhancement information message, the single-layer access unit may, for example, further comprise a non-scalable-nested buffering period supplemental enhancement information message. The apparatus 200 may, for example, be configured to process scalable-nested supplemental enhancement information messages and non-scalable-nested supplemental enhancement information messages.

[0345] Further, a video decoder 300 for receiving a video data stream having a video stored therein is provided. The video decoder 300 is configured to decode the video from the video data stream. The video data stream comprises one or more scalable-nested supplemental enhancement information messages. The one or more scalable-nested supplemental enhancement information messages comprise a plurality of syntax elements. Each syntax element of one or more syntax elements of the plurality of syntax elements is defined to have a same size in each scalable-nested supplemental enhancement information message of the video data stream or a part of the video data stream. The video decoder 300 is configured to decode the video in accordance with the one or more syntax elements of the plurality of syntax elements.

[0346] According to an embodiment, the video data stream may, for example, comprise one or more non-scalable nested supplemental enhancement information messages. The one or more scalable nested supplemental enhancement information messages and the one or more non-scalable nested supplemental enhancement information messages comprise a plurality of syntax elements. Each syntax element of one or more syntax elements of the plurality of syntax elements may, for example, be defined to have a same size in each scalable nested supplemental enhancement information message of the scalable nested supplemental enhancement information messages of the video data stream or the portion of the video data stream and in each non-scalable nested supplemental enhancement information message of the non-scalable nested supplemental enhancement information messages of the video data stream or the portion of the video data stream.

[0347] In an embodiment, the video data stream may, for example, comprise a plurality of access units, wherein each access unit of the plurality of access units may, for example, be assigned to one of a plurality of pictures of the video. The portion of the video data stream may, for example, be an access unit of the plurality of access units of the video data stream. Each syntax element of one or more syntax elements of the plurality of syntax elements may, for example, be defined to have a same size in each scalable nested supplemental enhancement information message of the scalable nested supplemental enhancement information messages of the access unit.

[0348] According to an embodiment, the video data stream may, for example, comprise one or more non-scalable nested supplemental enhancement information messages. The one or more scalable nested supplemental enhancement information messages and the one or more non-scalable nested supplemental enhancement information messages comprise a plurality of syntax elements. Each syntax element of one or more syntax elements of the plurality of syntax elements may, for example, be defined to have a same size in each scalable nested supplemental enhancement information message of the scalable nested supplemental enhancement information messages of the access unit and in each non-scalable nested supplemental enhancement information message of the non-scalable nested supplemental enhancement information of the access unit.

[0349] In an embodiment, the portion of the video data stream may, for example, be a coded video sequence of the video data stream. Each syntax element of one or more syntax elements of the plurality of syntax elements may, for example, be defined to have a same size in each scalable nested supplemental enhancement information message of the scalable nested supplemental enhancement information messages of the coded video sequence.

[0350] According to an embodiment, the video data stream may, for example, comprise one or more non-scalable nested supplemental enhancement information messages. The one or more scalable nested supplemental enhancement information messages and the one or more non-scalable nested supplemental enhancement information messages comprise a plurality of syntax elements. Each syntax element of one or more syntax elements of the plurality of syntax elements may, for example, be defined to have a same size in each scalable nested supplemental enhancement information message of the scalable nested supplemental enhancement information messages of the coded video sequence and in each non-scalable nested supplemental enhancement information message of the non-scalable nested supplemental enhancement information of the coded video sequence.

[0351] In an embodiment, each syntax element of one or more syntax elements of the plurality of syntax elements may, for example, be defined to have a same size in each scalable nested supplemental enhancement information message of the scalable nested supplemental enhancement information messages of the video data stream.

[0352] According to an embodiment, each syntax element of one or more syntax elements of the plurality of syntax elements may, for example, be defined to have a same size in each scalable nested supplemental enhancement information message of the scalable nested supplemental enhancement information messages of the video data stream and in each non-scalable nested supplemental enhancement information message of the non-scalable nested supplemental enhancement information of the video data stream.

[0353] In an embodiment, the video data stream or the portion of the video data stream may, for example, comprise at least one buffering period supplemental enhancement information message, wherein the buffering period supplemental enhancement information message defines a size of each syntax element of one or more syntax elements of the plurality of syntax elements.

[0354] According to an embodiment, the buffering period supplemental enhancement information message comprises at least one of the following for defining a size of each syntax element of one or more syntax elements of the plurality of syntax elements:

[0355] a bp_cpb_initial_removal_delay_length_minus1 element,

[0356] a bp_cpb_removal_delay_length_minus1 element,

[0357] a bp_dpb_output_delay_length_minus1 element,

[0358] a bp_du_cpb_removal_delay_increment_length_minus1 element,

[0359] bp_dpb_output_delay_du_length_minus1 element.

[0360] In embodiments, for each access unit of the plurality of access units of the video data stream comprising a scalable-nested buffering period supplemental enhancement information message, the access unit may, for example, further comprise a non-scalable-nested buffering period supplemental enhancement information message.

[0361] According to embodiments, for each single-layer access unit of the plurality of single-layer access units of the video data stream comprising a scalable-nested buffering period supplemental enhancement information message, the single-layer access unit may, for example, further comprise a non-scalable-nested buffering period supplemental enhancement information message.

[0362] Further, a system is provided. The system comprises the apparatus 200 as described above and the video decoder 300 as described above. The video decoder 300 is configured to receive the output video data stream of the apparatus 200. Further, the video decoder 300 is configured to decode the video from the output video data stream of the apparatus 200.

[0363] According to embodiments, the system may, for example, further comprise the video encoder 100. The apparatus 200 may, for example, be configured to receive the video data stream from the video encoder 100 as the input video data stream.

[0364] In particular, the fifth aspect of the present invention relates to constraining all BP SEI messages in a bitstream to indicate the same length for certain variable length coded syntax elements and not be scalable-nested in the absence of a non-scalable-nested variant in the same AU.

[0365] Buffering period SEI messages, picture timing SEI messages, and decoding unit information SEI messages are used to provide precise timing information for NAL units within a bitstream to control their transition through a buffer of a decoder in conformance testing. Some syntax elements in PT messages and DUI SEI messages are coded in variable length, and the length of these syntax elements is conveyed in BP SEI messages. This parsing dependency is a design trade-off. For the cost of not allowing PT messages and DUI SEI messages to parse without first parsing the associated BP SEI message, the benefit of saving sending these length syntax elements at each PT or DUI SEI message is realized. Since the frequency of sending BP SEI messages (once per multiple frames) is much lower than PT messages (once per frame) or DUI SEI messages (multiple times per frame), a bit saving is realized by this common design trade-off, similar to how the picture header structure can reduce the bit cost of slice headers when many slices are used.

[0366] More specifically, the BP SEI message in the current VVC draft specification includes syntax elements that are roots of parsing dependencies:

[0367] • bp_cpb_initial_removal_delay_length_minus1 specifying the coding length of the alternative timing initial CPB removal delay of the AUs in the PT SEI message, and

[0368] • bp_cpb_removal_delay_length_minus1 specifying the coding length of the CPB removal delay and removal delay offset of the AUs in the PT SEI message, and

[0369] • bp_dpb_output_delay_length_minus1 specifying the coding length of the DPB output delay of the AUs in the PT SEI message, and

[0370] • bp_du_cpb_removal_delay_increment_length_minus1 specifying the coding length of the individual CPB removal delay and the common CPB removal delay of the DUs in the PT SEI message and the CPB removal delay of the DUs in the DUI SEI message, and

[0371] • bp_dpb_output_delay_du_length_minus1 specifying the coding length of the DPB output delay of the AUs in the PT SEI message and the DUI SEI message.

[0372] However, problems arise when the bitstream contains multiple OLSs. While the BP / PT / DUI SEI messages that apply to the OLSs representing the bitstream are carried in the bitstream in a verbatim fashion, it is trivial to keep track of the parsing dependencies, the other BP / PT / DUI SEI messages corresponding to the OLSs representing (sub-)bitstreams will be carried in encapsulated form in so-called scalable nesting SEI messages. The parsing dependencies still apply and, given that the number of OLSs can be very large, it is a considerable burden for the decoder or parser to keep track of the correctly encapsulated BP SEI messages for the parsing dependencies when processing the encapsulated PT messages and DUI SEI messages. In particular, since these messages can also be encapsulated in different scalable nesting SEI messages.

[0373] Accordingly, as part of the present application, in one embodiment, the following bitstream constraint is established: the coded value of the respective syntax element describing the length must be identical in all scalable-nested and non-scalable-nested BP SEI messages in an AU. Accordingly, the decoder or parser only needs to store the respective length value upon parsing the first non-scalable BP SEI message in an AU and can resolve all parsing dependencies of PT messages and DUI SEI messages within the buffering period starting at the respective AU, regardless of whether these messages are encapsulated in scalable-nested SEI messages or not. The following is an example of the respective specification text:

[0374] The bitstream conformance requirement is that all scalable-nested and non-scalable-nested buffering period SEI messages in an AU have identical respective values of the syntax elements bp_cpb_initial_removal_delay_length_minus1, bp_cpb_removal_delay_length_minus1, bp_dpb_output_delay_length_minus1, bp_du_cpb_removal_delay_increment_length_minus1, bp_dpb_output_delay_du_length_minus1.

[0375] In another embodiment, the constraint is only formulated for scalable-nested BP SEI messages within the buffering period determined by the current non-scalable-nested BP SEI message as follows:

[0376] The bitstream conformance requirement is that all scalable-nested buffering period SEI messages within the buffering period have identical respective values of the syntax elements bp_cpb_initial_removal_delay_length_minus1, bp_cpb_removal_delay_length_minus1, bp_dpb_output_delay_length_minus1, bp_du_cpb_removal_delay_increment_length_minus1, bp_dpb_output_delay_du_length_minus1, followed by the non-scalable-nested buffering period SEI messages of the buffering period.

[0377] Here, the BP of the bitstream defines the constraint range of a scalable-nested BP from one scalable-nested BP to the next scalable-nested BP.

[0378] In another embodiment, the constraint is formulated for all AUs of the bitstream as follows:

[0379] The requirement for bitstream conformance is that all scalable-nested and non-scalable-nested buffering period SEI messages in the bitstream have the same respective values of syntax elements bp_cpb_initial_removal_delay_length_minus1, bp_cpb_removal_delay_length_minus1, bp_dpb_output_delay_length_minus1, bp_du_cpb_removal_delay_increment_length_minus1, bp_dpb_output_delay_du_length_minus1.

[0380] In another embodiment, the constraint is only formulated for the AU in the CVS, so that a smart encoder can still be able to facilitate the difference in duration of BPs in the bitstream for the encoding of the related delay and offset syntax elements. The normative text would be as follows:

[0381] The requirement for bitstream conformance is that all scalable-nested and non-scalable-nested buffering period SEI messages in the CVS have the same respective values of syntax elements bp_cpb_initial_removal_delay_length_minus1, bp_cpb_removal_delay_length_minus1, bp_dpb_output_delay_length_minus1, bp_du_cpb_removal_delay_increment_length_minus1, bp_dpb_output_delay_du_length_minus1.

[0382] Here, the constraint scope is the CVS.

[0383] More specifically, the buffering period or BP SEI message defines a so-called buffering period in which the timing of individual pictures uses the picture at the buffering period start as an anchor point. For example, the start of the buffering period facilitates the testing of the conformance of the random access functionality in the bitstream.

[0384] Figure 7 Two sets of HRD SEI (scalable-nested SEI and non-scalable-nested SEI) in a two-layer bitstream according to an embodiment are shown.

[0385] In as Figure 7In the multi-layer scenario shown, for example, the scalable-nested HRD SEI provides a different buffering period setting (through BP at POC 0 and POC 3) than the non-scalable-nested SEI (POC 0 only) to be used when only extracting and playing layer L0 starting from POC 3.

[0386] However, this also adds the complexity cost of tracking the parsing dependencies between PT and individual BP messages as described above, which is undesirable. Therefore, as part of the present application, in one embodiment, it is prohibited to have a scalable-nested BP SEI message in an AU that does not contain a non-scalable-nested BP SEI message, as follows:

[0387] The bitstream conformance requirement is that a scalable-nested BP SEI message shall not be in an AU that does not contain a non-scalable-nested BP SEI message.

[0388] Since the use case described above is limited to multi-layer bitstreams, in another embodiment, the related constraint is limited to single-layer bitstreams, as follows:

[0389] The bitstream conformance requirement is that a scalable-nested BP SEI message shall not be in a single-layer AU that does not contain a non-scalable-nested BP SEI message.

[0390] Although some aspects have been described in the context of an apparatus, it is clear that other aspects also represent a description, although using different terminology, of corresponding methods. Similarily, some aspects described in the context of a method also represent a description, although using different terminology, of corresponding apparatuses, or corresponding means for performing the steps of the method. Some or all of the method steps can be executed by (or using) a hardware apparatus, like for example, a microprocessor, a programmable computer or electronic circuit. In some embodiments, one or more of the method steps can be executed by such an apparatus.

[0391] Depending on certain implementation requirements, embodiments of the application can be implemented in hardware or in software, or in a combination of hardware and software. The implementation can be performed using a digital storage medium, for example a floppy disk, a DVD, a Blu-Ray, a CD, a ROM, a PROM, an EPROM, an EEPROM or a FLASH memory, having electronically readable control signals stored thereon, which cooperate (or are capable of cooperating) with a programmable computer system such that the respective method is performed. Therefore, the digital storage medium can be computer readable.

[0392] Some embodiments according to the application comprise a data carrier having electronically readable control signals, which are capable of cooperating with a programmable computer system, such that one of the methods described herein is performed.

[0393] Generally, an embodiment of the inventive method can be implemented as a computer program product having a program code, the program code being operative for performing one of the methods when the computer program product runs on a computer. The program code can for example be stored on a machine readable carrier.

[0394] A further embodiment comprises a computer having installed thereon the computer program for performing one of the methods described herein.

[0395] In other words, an embodiment of the inventive method is, therefore, a computer program having a program code for performing one of the methods, when the computer program runs on a computer.

[0396] A further embodiment of the inventive methods is, therefore, a data carrier (or a digital storage medium, or a computer-readable medium) comprising, recorded thereon, the computer program for performing one of the methods described herein. The data carrier, the digital storage medium or the recorded medium are typically tangible and / or non- transitory.

[0397] A further embodiment of the inventive method is, therefore, a data stream or a sequence of signals representing the computer program for performing one of the methods described herein. The data stream or the sequence of signals can for example be configured to be transferred via a data communication connection, for example via the Internet.

[0398] A further embodiment comprises a processing means, for example a computer, or a programmable logic device, configured to or adapted for performing one of the methods described herein.

[0399] A further embodiment comprises a computer having installed thereon the computer program for performing one of the methods described herein.

[0400] A further embodiment according to the application comprises an apparatus or a system configured to transfer (for example, electronically or optically) a computer program for performing one of the methods described herein to a receiver. The receiver may, for example, be a computer, a mobile device, a memory device or the like. The apparatus or system may, for example, comprise a file server for transferring the computer program to the receiver.

[0401] In some embodiments, a programmable logic device (for example a field programmable gate array) can be used to perform some or all of the functionalities of the methods described herein. In some embodiments, a field programmable gate array can cooperate with a microprocessor in order to perform one of the methods described herein. Generally, the methods are preferably performed by any hardware apparatus.

[0402] The apparatus described herein can be implemented using a hardware apparatus, or using a computer, or using a combination of hardware and computer.

[0403] The methods described herein can be performed using hardware apparatus, or using a computer, or using a combination of hardware apparatus and computer.

[0404] The above-described embodiments are merely illustrative of the principles of the application. It is understood that modifications and variations of the arrangements and details described herein will be apparent to others skilled in the art. It is the intent, therefore, to limit the application only as indicated by the scope of the claims appended hereto and not by the specifics of the embodiments described above.

[0405] References

[0406] [1] ISO / IEC, ITU-T. High efficiency video coding. ITU-T Recommendation H.265 | ISO / IEC 23008-10 (HEVC), edition 1, 2013; edition 2, 2014.

Claims

1. A video decoder configured to receive a video data stream having a video stored therein, wherein the video decoder being configured to decode the video from the video data stream, wherein the video data stream comprises a scalable-nested supplemental enhancement information message, wherein the scalable-nested supplemental enhancement information message comprises a plurality of syntax elements, wherein each of one or more syntax elements of the plurality of syntax elements is defined to have a same length in each of the scalable-nested supplemental enhancement information messages of the video data stream or the portion of the video data stream, wherein the video decoder is configured to decode the video in accordance with the one or more syntax elements of the plurality of syntax elements.

2. The video decoder according to claim 1, wherein the video data stream or the portion of the video data stream comprises at least one buffering period supplemental enhancement information message, wherein the buffering period supplemental enhancement information message defines a length of each of the one or more syntax elements of the plurality of syntax elements.

3. The video decoder according to claim 1 or 2, wherein for each access unit of a plurality of access units of the video data stream comprising a scalable-nested buffering period supplemental enhancement information message, the access unit further comprises a non-scalable-nested buffering period supplemental enhancement information message.

4. The video decoder according to claim 2, wherein for each single-layer access unit of a plurality of single-layer access units of the video data stream comprising a scalable-nested buffering period supplemental enhancement information message, the single-layer access unit further comprises a non-scalable-nested buffering period supplemental enhancement information message.

5. The video decoder according to any one of claims 1 to 4, wherein the video data stream comprises one or more non-scalable-nested supplemental enhancement information messages, wherein the scalable-nested supplemental enhancement information message and the one or more non-scalable-nested supplemental enhancement information messages comprise the plurality of syntax elements, wherein each of the one or more syntax elements of the plurality of syntax elements is defined to have a same length in each of the scalable-nested supplemental enhancement information messages of the video data stream or the portion of the video data stream, and in each of the non-scalable-nested supplemental enhancement information messages of the video data stream or the portion of the video data stream.

6. The video decoder according to claim 3, wherein the video data stream comprises a plurality of access units, wherein each access unit of the plurality of access units is assigned to one of a plurality of pictures of the video, wherein the portion of the video data stream is an access unit of the plurality of access units of the video data stream, and wherein each of the one or more of the plurality of syntax elements is defined to have the same length in each of the scalable-nested supplemental enhancement information messages of the access unit.

7. The video decoder of claim 6, wherein, the video data stream comprises one or more non-scalable-nested supplemental enhancement information messages, wherein the scalable-nested supplemental enhancement information messages and the one or more non-scalable-nested supplemental enhancement information messages comprise the plurality of syntax elements, wherein each of the one or more of the plurality of syntax elements is defined to have the same length in each of the scalable-nested supplemental enhancement information messages of the access unit and in each of the non-scalable-nested supplemental enhancement information messages of the access unit.

8. The video decoder of any of claims 1 to 4, wherein, the portion of the video data stream is a coded video sequence of the video data stream, and wherein each of the one or more of the plurality of syntax elements is defined to have the same length in each of the scalable-nested supplemental enhancement information messages of the coded video sequence.

9. The video decoder of claim 8, wherein the video data stream comprises one or more non-scalable-nested supplemental enhancement information messages, wherein the scalable-nested supplemental enhancement information messages and the one or more non-scalable-nested supplemental enhancement information messages comprise the plurality of syntax elements, wherein each of the one or more of the plurality of syntax elements is defined to have the same length in each of the scalable-nested supplemental enhancement information messages of the coded video sequence and in each of the non-scalable-nested supplemental enhancement information messages of the coded video sequence.

10. A video encoder (100), wherein, the video encoder (100) is configured to encode video into a video data stream, wherein the video encoder (100) is configured to generate the video data stream such that the video data stream comprises a scalable-nested supplemental enhancement information message, wherein the video encoder (100) is configured to generate the video data stream such that the scalable-nested supplemental enhancement information message comprises a plurality of syntax elements, wherein the video encoder (100) is configured to generate the video data stream such that each of one or more of the plurality of syntax elements is defined to have the same length in each of the scalable-nested supplemental enhancement information messages of the video data stream or a portion of the video data stream.

11. The video encoder (100) of claim 10, wherein The video encoder (100) is configured to generate the video data stream such that the video data stream or the portion of the video data stream comprises at least one buffering period supplemental enhancement information message, wherein the buffering period supplemental enhancement information message defines a length of each of the one or more of the plurality of syntax elements.

12. The video encoder (100) of claim 10 or 11, wherein The video encoder (100) is configured to generate the video data stream such that for each access unit of a plurality of access units of the video data stream comprising a scalable-nested buffering period supplemental enhancement information message, the access unit further comprises a non-scalable-nested buffering period supplemental enhancement information message.

13. The video encoder (100) of claim 11, wherein The video encoder (100) is configured to generate the video data stream such that for each single-layer access unit of a plurality of single-layer access units of the video data stream comprising a scalable-nested buffering period supplemental enhancement information message, the single-layer access unit further comprises a non-scalable-nested buffering period supplemental enhancement information message.

14. The video encoder (100) of any of claims 10 to 13, wherein The video encoder (100) is configured to generate the video data stream such that the video data stream comprises one or more non-scalable-nested supplemental enhancement information messages, wherein the video encoder (100) is configured to generate the video data stream such that the scalable-nested supplemental enhancement information message and the one or more non-scalable-nested supplemental enhancement information messages comprise the plurality of syntax elements, wherein the video encoder (100) is configured to generate the video data stream such that each of the one or more of the plurality of syntax elements is defined to have a same length in each of scalable-nested supplemental enhancement information messages of the video data stream or the portion of the video data stream and in each of non-scalable-nested supplemental enhancement information messages of the video data stream or the portion of the video data stream.

15. The video encoder (100) of claim 12, wherein The video encoder (100) is configured to generate the video data stream such that the video data stream comprises a plurality of access units, wherein each access unit of the plurality of access units is assigned to one of a plurality of pictures of the video, wherein the portion of the video data stream is an access unit of the plurality of access units of the video data stream, and wherein the video encoder (100) is configured to generate the video data stream such that the one or more of the plurality of syntax elements are defined for the access unit of the video data stream. wherein the video encoder (100) is configured to generate the video data stream such that each of the one or more of the plurality of syntax elements is defined to have the same length in each of the scalable-nested supplemental enhancement information messages of the access unit.

16. The video encoder (100) of claim 15, wherein, the video encoder (100) is configured to generate the video data stream such that the video data stream comprises one or more non-scalable-nested supplemental enhancement information messages, wherein the video encoder (100) is configured to generate the video data stream such that the scalable-nested supplemental enhancement information messages and the one or more non-scalable-nested supplemental enhancement information messages comprise the plurality of syntax elements, wherein the video encoder (100) is configured to generate the video data stream such that each of the one or more of the plurality of syntax elements is defined to have the same length in each of the scalable-nested supplemental enhancement information messages of the access unit and in each of the non-scalable-nested supplemental enhancement information messages of the access unit.

17. The video encoder (100) of any of claims 10 to 13, wherein the portion of the video data stream is a coded video sequence of the video data stream, and wherein the video encoder (100) is configured to generate the video data stream such that each of the one or more of the plurality of syntax elements is defined to have the same length in each of the scalable-nested supplemental enhancement information messages of the coded video sequence.

18. The video encoder (100) of claim 17, wherein the video encoder (100) is configured to generate the video data stream such that the video data stream comprises one or more non-scalable-nested supplemental enhancement information messages, wherein the video encoder (100) is configured to generate the video data stream such that the scalable-nested supplemental enhancement information messages and the one or more non-scalable-nested supplemental enhancement information messages comprise the plurality of syntax elements, wherein the video encoder (100) is configured to generate the video data stream such that each of the one or more of the plurality of syntax elements is defined to have the same length in each of the scalable-nested supplemental enhancement information messages of the coded video sequence and in each of the non-scalable-nested supplemental enhancement information messages of the coded video sequence.

19. A non-transitory computer-readable medium storing a video data stream, wherein the video data stream having video encoded therein, wherein the video data stream comprises one or more scalable-nested supplemental enhancement information messages, wherein the one or more scalable-nested supplemental enhancement information messages comprise a plurality of syntax elements, wherein each of the one or more syntax elements of the plurality of syntax elements is defined to have a same length in each of the scalable-nested supplemental enhancement information messages of the video data stream or the portion of the video data stream.

20. The non-transitory computer-readable medium storing a video data stream of claim 19, wherein the video data stream or the portion of the video data stream comprises at least one buffering period supplemental enhancement information message, wherein the buffering period supplemental enhancement information message defines a length of each of the one or more syntax elements of the plurality of syntax elements.

21. The non-transitory computer-readable medium storing a video data stream of claim 19, wherein for each access unit of a plurality of access units of the video data stream that comprises a scalable-nested buffering period supplemental enhancement information message, the access unit further comprises a non-scalable-nested buffering period supplemental enhancement information message.

22. The non-transitory computer-readable medium storing a video data stream of claim 20, wherein, for each single-layer access unit of a plurality of single-layer access units of the video data stream that comprises a scalable-nested buffering period supplemental enhancement information message, the single-layer access unit further comprises a non-scalable-nested buffering period supplemental enhancement information message.

23. The non-transitory computer-readable medium storing a video data stream of claim 19, wherein the video data stream comprises one or more non-scalable-nested supplemental enhancement information messages, wherein the scalable-nested supplemental enhancement information messages and the one or more non-scalable-nested supplemental enhancement information messages comprise the plurality of syntax elements, wherein each of the one or more syntax elements of the plurality of syntax elements is defined to have a same length in each of the scalable-nested supplemental enhancement information messages of the video data stream or the portion of the video data stream, and in each of the non-scalable-nested supplemental enhancement information messages of the video data stream or the portion of the video data stream.

24. The non-transitory computer-readable medium storing a video data stream of claim 21, wherein, the video data stream comprises a plurality of access units, wherein each access unit of the plurality of access units is assigned to one of a plurality of pictures of the video, wherein the portion of the video data stream is an access unit of the plurality of access units of the video data stream, and wherein each of the one or more syntax elements of the plurality of syntax elements is defined to have a same length in each of the scalable-nested supplemental enhancement information messages of the access unit.

25. The non-transitory computer-readable medium storing a stream of video data of claim 24, wherein the stream of video data including one or more non-scalably-nested supplemental enhancement information messages, wherein the scalably-nested supplemental enhancement information messages and the one or more non-scalably-nested supplemental enhancement information messages include the plurality of syntax elements, wherein each of the one or more of the plurality of syntax elements is defined to have a same length in each of the scalably-nested supplemental enhancement information messages of the access unit and in each of the non-scalably-nested supplemental enhancement information messages of the access unit.

26. The non-transitory computer-readable medium storing a stream of video data of claim 19, wherein the portion of the stream of video data being a coded video sequence of the stream of video data, and wherein each of the one or more of the plurality of syntax elements is defined to have a same length in each of the scalably-nested supplemental enhancement information messages of the coded video sequence.

27. The non-transitory computer-readable medium storing a stream of video data of claim 26, wherein the stream of video data including one or more non-scalably-nested supplemental enhancement information messages, wherein the scalably-nested supplemental enhancement information messages and the one or more non-scalably-nested supplemental enhancement information messages include the plurality of syntax elements, wherein each of the one or more of the plurality of syntax elements is defined to have a same length in each of the scalably-nested supplemental enhancement information messages of the coded video sequence and in each of the non-scalably-nested supplemental enhancement information messages of the coded video sequence.

Images