Video encoder, video decoder, methods for encoding and decoding and video data stream for realizing advanced video coding concepts

Abstract

An apparatus (200) for receiving an input video data stream according to an embodiment is provided. Video is encoded in the input video data stream. An apparatus (200) is configured to generate an output video data stream from an input video data stream.

Description

(19) State Intellectual Property Office (12) Invention Patent Application (10) Application Publication Number (43) Application Publication Date (21) Application Number 202511329556.X (22) Application Date 2021.05.21 (30) Priority Data 20176178.0 2020.05.22 EP 20176206.9 2020.05.22 EP (62) Divisional Application Data 202180036764.4 2021.05.21 (71) Applicant Dolby Video Compression LLC Address USA (72) Inventors Iago Sanchez de la Fuente Karlsten Zuling Cornelius Helch Thomas Scher Robert Scubin Thomas Wigand (74) Patent Agency China Patent & Trademark Agency Co., Ltd. 11021 Patent Attorneys Luo Songmei and Pan Jianying (51) Int.Cl. H04N 19 / 70 (2014.01) H04N 19 / 503 (2014.01) (54) Title of Invention Video Encoder, Video Decoder, Encoding and Decoding Method and Video Data Stream for Implementing Advanced Video Coding Concepts (57) Abstract An apparatus (200) for receiving an input video data stream according to an embodiment is provided. The input video data stream contains encoded video. The apparatus (200) is configured to generate an output video data stream from the input video data stream. Claims (3 pages), Description (41 pages), Drawings (10 pages), CN 121099071 A, 2025.12.09, CN 1 21 09 90 71 A. 1. An apparatus for receiving one or more input video data streams, wherein each of the one or more input video data streams encodes an input video, wherein the apparatus is configured to generate an output video data stream from the one or more input video data streams, the output video data stream encoding the output video, wherein the apparatus is configured to: generate the output video data stream such that the output video is the input video encoded within one of the one or more input video data streams, or such that the output video depends on the input video of at least one of the one or more input video data streams, wherein the apparatus is configured to: determine the removal time of a current image from an access unit of an encoded image buffer among a plurality of images of the output video, wherein the apparatus is configured to: determine, at least according to a concatenation flag, whether to use encoded image buffer delay offset information to determine the removal time of the current image from an access unit of the encoded image buffer. The device is configured to determine whether to determine the access unit removal time of the current image based on the position of a previously non-discardable image in the video preceding the current image.2. The apparatus of claim 1, wherein the apparatus is configured to: discard a group of one or more images of the input video of the first video data stream in the one or more input video data streams to generate the output video data stream, wherein the apparatus is configured to: determine, based on the encoded image buffer delay offset information, the removal time of at least one image of the plurality of images of the output video from the access unit of the encoded image buffer. 3. The apparatus of claim 1, wherein the first video received by the apparatus is a preprocessed video obtained from an original video, from which a group of one or more images has been discarded to generate the preprocessed video, wherein the apparatus is configured to: determine, based on the encoded image buffer delay offset information, the removal time of at least one image of the plurality of images of the output video from the access unit of the encoded image buffer. 4. The apparatus of claim 2, wherein the buffer delay offset information depends on the number of images of the input video that have been discarded. 5. The apparatus of claim 2, wherein the one or more input video data streams are two or more input video data streams, and wherein the apparatus is configured to concatenate the processed video with the input video of a second video data stream among the two or more input video data streams to obtain the output video, and the apparatus is configured to encode the output video into the output video data stream. 6. The apparatus of claim 5, wherein the apparatus is configured to: determine whether to use encoded image buffer delay offset information to determine the access unit removal time of the current image based on the position of the current image within the output video, or wherein the apparatus is configured to: determine whether to set the encoded image buffer delay offset value of the encoded image buffer delay offset information to 0 to determine the access unit removal time of the current image based on the position of the current image within the output video. 7. The apparatus of claim 1, wherein the apparatus is configured to: determine the access unit removal time of the current image based on the removal time of the previous image. Claims 1 / 3 Page 2 CN 121099071 A 8. The apparatus of claim 1, wherein the apparatus is configured to: determine the access unit removal time of the current image based on initial encoded image buffer removal delay information. 9. The apparatus of claim 8, wherein the apparatus is configured to: update the initial encoded image buffer removal delay information according to a clock tick to obtain temporary encoded image buffer removal delay information to determine the access unit removal time of the current image.10. The apparatus of claim 1, wherein if the concatenation flag is set to a first value, the apparatus is configured to use the encoded image buffer delay offset information to determine one or more removal times, and wherein if the concatenation flag is set to a second value different from the first value, the apparatus is configured not to use the encoded image buffer delay offset information to determine the one or more removal times. 11. The apparatus of claim 1, wherein the apparatus is configured to: signal a video decoder whether to use the encoded image buffer delay offset information to determine the removal time of the current image from the access unit of the encoded image buffer. 12. The apparatus of claim 11, wherein the current image is located at the splicing point of the output video, wherein two input videos are spliced ​​at the splicing point. 13. A non-transitory digital storage medium for storing a video data stream, wherein the video data stream encodes video, wherein the video data stream includes encoded image buffer delay offset information and a concatenation flag, wherein determining whether to use the encoded image buffer delay offset information to determine the removal time of an image from the encoded image buffer depends at least on the concatenation flag, wherein the video data stream includes information about the position of a previously non-discardable image preceding the current image within the video, the information being usable to determine the access unit removal time of the current image. 14. The non-transitory digital storage medium for storing a video data stream according to claim 13, wherein the video data stream includes initial encoded image buffer removal delay information. 15. The non-transitory digital storage medium for storing a video data stream according to claim 13, wherein if the concatenation flag is set to a first value, the concatenation flag indicates that the encoded image buffer delay offset information needs to be used to determine one or more removal times, and wherein if the concatenation flag is set to a second value different from the first value, the concatenation flag indicates that the indicated offset is not used to determine the one or more removal times.16. A video encoder, wherein the video encoder is configured to encode video into a video data stream, wherein the video encoder is configured to: generate the video data stream such that the video data stream includes encoded image buffer delay offset information and a concatenation flag, wherein the video encoder is configured to: determine, at least based on the concatenation flag, whether to use the encoded image buffer delay offset information to determine the removal time of an image from the encoded image buffer, wherein the video encoder (100) is configured to generate the video data stream such that the video data stream includes information about the position of a previously non-discardable image within the video prior to the current image, the information being usable for determining the access unit removal time of the current image. 17. The video encoder of claim 16, wherein the video encoder is configured to: generate the video data stream such that the video data stream includes initial encoded image buffer delay offset information. 18. The video encoder of claim 16, wherein if the concatenation flag is set to a first value, the concatenation flag indicates that the encoded image buffer delay offset information needs to be used to determine one or more removal times, and wherein if the concatenation flag is set to a second value different from the first value, the concatenation flag indicates that the indicated offset is not used to determine the one or more removal times. 19. A video decoder for receiving a video data stream storing a video, wherein the video decoder is configured to decode the video from the video data stream, wherein the video decoder is configured to: decode the video based on the removal time of the current image from the access unit of the encoded image buffer among a plurality of images of the video, wherein the video decoder is configured to: decode the video at least based on a concatenation flag indicating whether the encoded image buffer delay offset information is used to determine the removal time of the current image from the access unit of the encoded image buffer, wherein the video decoder (300) is configured to: determine whether to determine the removal time of the access unit of the current image based on the position of a previously non-discardable image within the video preceding the current image. 20. The video decoder of claim 19, wherein the removal time of at least one of the plurality of images of the video from the access unit of the encoded image buffer depends on the encoded image buffer delay offset information. 21. The video decoder of claim 19, wherein the video decoder is configured to decode the video based on whether the encoded image buffer delay offset value of the encoded image buffer delay offset information is set to 0.22. The video decoder of claim 19, wherein the video decoder is configured to: determine the access unit removal time of the current image based on the removal time of the previous image. 23. The video decoder of claim 19, wherein the video decoder is configured to: determine the access unit removal time of the current image based on initial encoded image buffer removal delay information. 24. The video decoder of claim 23, wherein the video decoder is configured to: update the initial encoded image buffer removal delay information based on a clock tick to obtain temporary encoded image buffer removal delay information to determine the access unit removal time of the current image. 25. The video decoder of claim 19, wherein if the concatenation flag is set to a first value, the video decoder is configured to use the encoded image buffer delay offset information to determine one or more removal times, and wherein if the concatenation flag is set to a second value different from the first value, the video decoder is configured not to use the encoded image buffer delay offset information to determine the one or more removal times. Claims 3 / 3 Page 4 CN 121099071 A Video encoder, video decoder, encoding and decoding method and video data stream for implementing advanced video coding concepts

[0001] This application is a divisional application of the invention patent application filed on May 21, 2021, with application number 202180036764.4 and invention title "Video encoder, video decoder, encoding and decoding method and video data stream for implementing advanced video coding concepts". Technical Field

[0002] The present invention relates to video coding and video decoding, and more particularly, to video encoders, video decoders, encoding and decoding methods and video data streams for implementing advanced video coding concepts. Background Art

[0003] H.265 / HEVC (HEVC = High Efficiency Video Coding) is a video codec that has provided tools for improving or even implementing parallel processing at the encoder and / or decoder. For example, HEVC supports the subdivision of a picture into a set of independently encoded tiles. Another concept supported by HEVC relates to WPP, according to which CTU rows or lines of an image can be processed in parallel from left to right, for example, in stripes, provided that some minimum CTU offset (CTU = coding tree unit) is observed when processing consecutive CTU rows. However, it would be advantageous to have a video codec that supports even more efficient parallel processing capabilities for the video encoder and / or video decoder.

[0004] Hereinafter, an introduction to VCL segmentation (VCL = video coding layer) according to the prior art is described.

[0005] Typically, in video coding, the encoding process of image samples requires small segments, where samples are divided into rectangular regions for joint processing, such as prediction or transform coding. Therefore, images are segmented into blocks of a specific size, which remains constant during the encoding of the video sequence. In the H.264 / AVC (AVC = Advanced Video Coding) standard, fixed-size blocks of 16x16 samples, known as macroblocks, are used.

[0006] In the prior art HEVC standard (see [1]), there are Code Tree Blocks (CTBs) or Code Tree Units (CTUs) with a maximum size of 64x64 samples. In a further description of HEVC, the more common term CTU is used for such blocks.

[0007] CTUs are processed in raster scan order (starting with the top-left CTU, processing the CTUs line by line in the image until the bottom-right CTU).

[0008] The encoded CTU data is organized into a container called a slice. Initially, in previous video coding standards, a slice referred to a segment containing one or more consecutive CTUs of an image. Slices were used to segment encoded data. From another perspective, a complete image can also be defined as a large segment; therefore, historically, the term slice remains applicable. In addition to the encoded image sample, a slice includes additional information related to the encoding process of the slice itself, which is placed in a so-called slice header.

[0009] According to the prior art, the VCL (Video Coding Layer) also includes techniques for segmentation and spatial partitioning. For example, such partitioning can be applied to video coding for various reasons, including parallel processing load balancing, CTU size matching in network transmission, error mitigation, etc.

[0010] Other examples relate to RoI (RoI = Region of Interest) encoding, where, for example, there is an observer-selectable region in the middle of the image, e.g., through a zoom-in operation (decoding only the RoI) or progressive decoder refresh (GDR), where the intra-frame data (typically placed in a frame of a video sequence) is temporally distributed across several consecutive frames, e.g., as a series of intra-frame blocks that slide across the picture plane and reset the temporal prediction chain locally in the same way that intra-frame images reset the temporal prediction chain for the entire picture plane. For the latter, there are two regions in each image, one most recently reset and one potentially affected by errors and error propagation.

[0011] Reference image resampling (RPR) is a technique for video coding that adapts the video quality / rate not only by using coarser quantization parameters but also by adapting to the resolution of each possible transmitted image. Therefore, the reference used for inter-frame prediction can have a different size than the image currently being predicted for encoding.Basically, RPR requires a resampling process in the prediction loop, such as defining upsampling and downsampling filters.

[0012] Depending on the style, RPR can result in a change in the size of the encoded image at any image, or it can be limited to a specific image, for example, only at a specific location bounded by, for example, a segment boundary adaptive HTTP stream. Summary of the Invention

[0013] The object of the present invention is to provide an improved concept for video encoding and video decoding.

[0014] The object of the present invention is achieved by the subject matter of the independent claims.

[0015] According to a first aspect of the invention, an apparatus for receiving an input video data stream is provided. The input video data stream contains encoded video. The apparatus is configured to generate an output video data stream from the input video data stream. Furthermore, the apparatus determines whether the video should be output before the image dependent on random access.

[0016] Furthermore, a video data stream is provided. The video data stream contains encoded video. The video data stream includes an indication of whether the video should be output before the image dependent on random access.

[0017] Furthermore, a video encoder is provided. The video encoder is configured to encode video into the video data stream. The video encoder is configured to generate a video data stream such that the video data stream includes an indication of whether the video should be output before a random access image.

[0018] Furthermore, a video decoder is provided for receiving a video data stream containing a video. The video decoder is configured to decode the video from the video data stream. The video decoder is configured to decode the video based on an indication of whether the video before a random access image should be output.

[0019] Furthermore, a method is provided for receiving an input video data stream. The input video data stream contains encoded video. The method includes generating an output video data stream from the input video data stream. Furthermore, the method includes determining whether the video before a random access image should be output.

[0020] Furthermore, a method is provided for encoding video into a video data stream. The method includes generating a video data stream such that the video data stream includes an indication of whether the video should be output before a random access image.

[0021] Furthermore, a method is provided for receiving a video data stream containing a video. The method includes decoding the video from the video data stream. The video is decoded based on an indication of whether to output the video from the image preceding the image that relies on random access.

[0022] Furthermore, a computer program is provided for implementing one of the above methods when the computer program is executed by a computer or a signal processor.

[0023] According to a second aspect of the invention, an apparatus for receiving one or more input video data streams is provided.Each of the one or more input video data streams encodes an input video. The apparatus is configured to generate an output video data stream from the one or more input video data streams, the output video data stream encoding an output video, wherein the apparatus is configured to generate the output video data stream such that the output video is an input video encoded within one or more input video data streams, or such that the output video depends on the input video of at least one of the one or more input video data streams. Furthermore, the apparatus is configured to determine the access unit removal time in the current image among a plurality of images of the output video from the encoded image buffer. The apparatus is configured to determine whether to use encoded image buffer delay offset information to determine the access unit removal time of the current image from the encoded image buffer.

[0024] Furthermore, a video data stream is provided. The video data stream encodes video. The video data stream includes encoded image buffer delay offset information.

[0025] Furthermore, a video decoder is provided for receiving a video data stream containing video. The video decoder is configured to decode the video from the video data stream. Furthermore, the video decoder is configured to decode the video based on the access unit removal time of the current image among a plurality of images of the video from the encoded image buffer. The video decoder is configured to decode the video based on an indication indicating whether to use encoded image buffer delay offset information to determine the access unit removal time of the current image from the encoded image buffer.

[0026] Furthermore, a method for receiving one or more input video data streams is provided. Each of the one or more input video data streams encodes an input video. The method includes generating an output video data stream from the one or more input video data streams, the output video data stream encoding an output video, wherein generating the output video data stream is performed such that the output video is an input video encoded within one or more input video data streams, or such that the output video depends on the input video of at least one of the one or more input video data streams. Furthermore, the method includes determining the access unit removal time in the current image among a plurality of images of the output video from the encoded image buffer. Furthermore, the method includes determining whether to use encoded image buffer delay offset information to determine the access unit removal time of the current image from the encoded image buffer.

[0027] Furthermore, a method for encoding video into a video data stream is provided according to an embodiment. The method includes: generating a video data stream such that the video data stream includes encoded image buffer delay offset information.

[0028] Furthermore, a method for receiving a video data stream storing video is provided.The method includes decoding the video from a video data stream. The video is decoded based on the access unit removal time of the current image among a plurality of images of the video from an encoded image buffer. Furthermore, the decoding of the video is performed based on an indication indicating whether to use encoded image buffer delay offset information to determine the access unit removal time of the current image from the encoded image buffer.

[0029] Furthermore, a computer program is provided for implementing one of the above methods when executed by a computer or signal processor.

[0030] According to a third aspect of the invention, a video data stream is provided. Video is encoded in the video data stream. Furthermore, the video data stream includes an initial encoded image buffer removal delay. Furthermore, the video data stream includes an initial encoded image buffer removal offset. Furthermore, the video data stream includes information indicating whether the sum of the initial encoded image buffer removal delay and the initial encoded image buffer removal offset is defined as constant over two or more buffering cycles.

[0031] Furthermore, a video encoder is provided. The video encoder is configured to encode video into a video data stream. Furthermore, the video encoder is configured to generate a video data stream such that the video data stream includes an initial encoded image buffer removal delay. Furthermore, the video encoder is configured to generate a video data stream such that the video data stream includes an initial encoded image buffer removal offset. Furthermore, the video encoder (100) is configured to generate a video data stream such that the video data stream includes information indicating whether the sum of the initial encoded image buffer removal delay and the initial encoded image buffer removal offset is defined as constant over two or more buffering cycles. Specification 3 / 41 page 7 CN 121099071 A

[0032] Furthermore, an apparatus is provided for receiving two input video data streams, the two input video data streams being a first input video data stream and a second input video data stream. Each of the two input video data streams encodes an input video. The apparatus is configured to generate an output video data stream from the two input video data streams, the output video data stream encoding an output video, wherein the apparatus is configured to generate the output video data stream by concatenating the first input video data stream and the second input video data stream. Furthermore, the apparatus is configured to generate an output video data stream such that the output video data stream includes an initial encoded image buffer removal delay. Furthermore, the apparatus is configured to: generate an output video data stream such that the output video data stream includes an initial encoded image buffer removal offset. Furthermore, the apparatus is configured to: generate an output video data stream such that the output video data stream includes information indicating whether the sum of the initial encoded image buffer removal delay and the initial encoded image buffer removal offset is defined as being constant over two or more buffering cycles.

[0033] Furthermore, a video decoder is provided for receiving a video data stream containing a stored video. The video decoder is configured to decode the video from the video data stream. Furthermore, the video data stream includes an initial encoded image buffer removal delay. Furthermore, the video data stream includes an initial encoded image buffer removal offset. Furthermore, the video data stream includes information indicating whether the sum of the initial encoded image buffer removal delay and the initial encoded image buffer removal offset is defined as constant over two or more buffering cycles. Furthermore, the video decoder is configured to decode the video depending on the information indicating whether the sum of the initial encoded image buffer removal delay and the initial encoded image buffer removal offset is defined as constant over two or more buffering cycles.

[0034] Furthermore, a method for encoding video into a video data stream is provided. The method includes: generating a video data stream such that the video data stream includes an initial encoded image buffer removal delay. The method includes: generating a video data stream such that the video data stream includes an initial encoded image buffer removal offset. Furthermore, the method includes: generating a video data stream such that the video data stream includes information indicating whether the sum of the initial encoded image buffer removal delay and the initial encoded image buffer removal offset is defined to be constant over two or more buffering cycles.

[0035] Furthermore, a method for receiving two input video data streams is provided, the two input video data streams being a first input video data stream and a second input video data stream. Each of the two input video data streams encodes an input video. The method includes: generating an output video data stream from the two input video data streams, the output video data stream encoding an output video, wherein the apparatus is configured to: generate the output video data stream by concatenating the first input video data stream and the second input video data stream. Furthermore, the method includes: generating an output video data stream such that the output video data stream includes an initial encoded image buffer removal delay. Furthermore, the method includes: generating an output video data stream such that the output video data stream includes an initial encoded image buffer removal offset. Furthermore, the method includes: generating an output video data stream such that the output video data stream includes information indicating whether the sum of the initial encoded image buffer removal delay and the initial encoded image buffer removal offset is defined as constant over two or more buffering cycles.

[0036] Furthermore, a method for receiving a video data stream containing video is provided. The method includes decoding the video from the video data stream. The video data stream includes an initial encoded image buffer removal delay. Furthermore, the video data stream includes an initial encoded image buffer removal offset. Furthermore, the video data stream includes information indicating whether the sum of the initial encoded image buffer removal delay and the initial encoded image buffer removal offset is defined as constant over two or more buffering cycles.The method includes decoding the video based on information indicating whether the sum of the initial encoded image buffer removal delay and the initial encoded image buffer removal offset is defined as being constant over two or more buffering cycles.

[0037] Furthermore, a computer program is provided for implementing one of the above methods when the computer program is executed by a computer or signal processor.

[0038] According to a fourth aspect of the invention, a video data stream is provided. The video data stream has encoded video. Furthermore, the video data stream includes an indication indicating whether a non-scalable nested picture timing supplementation enhancement information message of a network abstraction layer unit of a plurality of access units of one or more encoded video sequences of the video data stream is defined as being applied to all output layer sets of a plurality of output layer sets of the access unit. If the indication has a first value, then the non-scalable nested picture timing supplementation enhancement information message of the network abstraction layer unit of the access unit is defined as being applied to all output layer sets of a plurality of output layer sets of the access unit. If the indication has a value different from the first value, then the indication does not define whether the non-scalable nested image timing supplementation enhancement information message of the network abstraction layer unit of the access unit is applicable to all output layer sets in the multiple output layer sets of the access unit.

[0039] In addition, a video encoder is provided. The video encoder is configured to encode video into a video data stream. In addition, the video encoder is configured to generate a video data stream such that the video data stream includes an indication indicating whether the non-scalable nested image timing supplementation enhancement information message of the network abstraction layer unit of one or more encoded video sequences of the video data stream is defined as being applied to all output layer sets in the multiple output layer sets of the access unit. If the indication has a first value, then the non-scalable nested image timing supplementation enhancement information message of the network abstraction layer unit of the access unit is defined as being applied to all output layer sets in the multiple output layer sets of the access unit. If the indication has a value different from the first value, then the indication does not define whether the non-scalable nested image timing supplementation enhancement information message of the network abstraction layer unit of the access unit is applicable to all output layer sets in the multiple output layer sets of the access unit.

[0040] Furthermore, an apparatus is provided for receiving an input video data stream. The input video data stream contains encoded video. The apparatus is configured to generate a processed video data stream from the input video data stream.Furthermore, the apparatus is configured to generate a processed video data stream, such that the processed video data stream includes an indication indicating whether a non-scalable nested picture timing supplementation enhancement message of a network abstraction layer unit of a plurality of access units of one or more coded video sequences of the processed video data stream is defined as being applied to all output layer sets of a plurality of output layer sets of the access unit. If the indication has a first value, then the non-scalable nested picture timing supplementation enhancement message of the network abstraction layer unit of the access unit is defined as being applied to all output layer sets of a plurality of output layer sets of the access unit. If the indication has a value different from the first value, then the indication does not define whether the non-scalable nested picture timing supplementation enhancement message of the network abstraction layer unit of the access unit is applicable to all output layer sets of a plurality of output layer sets of the access unit.

[0041] Furthermore, a video decoder for receiving a video data stream containing a video is provided. The video decoder is configured to decode the video from the video data stream. The video data stream includes an indication indicating whether a non-scalable nested picture timing supplementation enhancement message of a network abstraction layer unit of one or more coded video sequences of the video data stream is defined as being applied to all output layer sets in the multiple output layer sets of the access unit. If the indication has a first value, then the non-scalable nested picture timing supplementation enhancement message of the network abstraction layer unit of the access unit is defined as being applied to all output layer sets in the multiple output layer sets of the access unit. If the indication has a value different from the first value, then the indication does not define whether the non-scalable nested picture timing supplementation enhancement message of the network abstraction layer unit of the access unit is applicable to all output layer sets in the multiple output layer sets of the access unit.

[0042] Furthermore, a method for encoding video into a video data stream is provided. The method includes: generating a video data stream (see page 5 / 41 of CN 121099071 A), such that the video data stream includes an indication indicating whether a non-scalable nested picture timing supplementation enhancement message of a network abstraction layer unit of one or more coded video sequences of the video data stream is defined as being applied to all output layer sets in a plurality of output layer sets of said access unit. If the indication has a first value, then the non-scalable nested picture timing supplementation enhancement message of the network abstraction layer unit of said access unit is defined as being applied to all output layer sets in a plurality of output layer sets of said access unit.If the indication has a value different from the first value, then the indication does not define whether the non-scalable nested picture timing supplementation enhancement information message of the network abstraction layer unit of the access unit is applicable to all output layer sets in the plurality of output layer sets of the access unit.

[0043] Furthermore, a method for receiving an input video data stream is provided. The input video data stream is encoded with video. The method includes generating a processed video data stream from the input video data stream. Furthermore, the method includes: generating a processed video data stream such that the processed video data stream includes an indication indicating whether the non-scalable nested picture timing supplementation enhancement information message of the network abstraction layer unit of one or more encoded video sequences of the processed video data stream is defined as being applied to all output layer sets in the plurality of output layer sets of the access unit. If the indication has a first value, then the non-scalable nested picture timing supplementation enhancement information message of the network abstraction layer unit of the access unit is defined as being applied to all output layer sets in the plurality of output layer sets of the access unit. If the indication has a value different from the first value, then the indication does not define whether the non-scalable nested image timing supplementation enhancement information message of the network abstraction layer unit of the access unit is applicable to all output layer sets in the plurality of output layer sets of the access unit.

[0044] Furthermore, a method for receiving a video data stream storing video is provided. The method includes decoding the video from the video data stream. The video data stream includes an indication indicating whether the non-scalable nested image timing supplementation enhancement information message of the network abstraction layer unit of one or more encoded video sequences of the video data stream is defined as being applied to all output layer sets in the plurality of output layer sets of the access unit. If the indication has a first value, then the non-scalable nested image timing supplementation enhancement information message of the network abstraction layer unit of the access unit is defined as being applied to all output layer sets in the plurality of output layer sets of the access unit. If the indication has a value different from the first value, then the indication does not define whether the non-scalable nested image timing supplementation enhancement information message of the network abstraction layer unit of the access unit is applicable to all output layer sets in the plurality of output layer sets of the access unit.

[0045] Furthermore, a computer program is provided for implementing one of the above methods when executed by a computer or signal processor.

[0046] According to a fifth aspect of the invention, a video data stream is provided. Video is encoded in the video data stream. Furthermore, the video data stream includes one or more scalable nested supplemental enhancement information messages. The one or more scalable nested supplemental enhancement information messages include a plurality of syntax elements.Each of the multiple syntax elements is defined as having the same size in each scalable nested supplementary enhancement information message in the video data stream or a portion thereof.

[0047] Furthermore, a video encoder is provided. The video encoder is configured to encode video into a video data stream. Furthermore, the video encoder is configured to generate a video data stream such that the video data stream includes one or more scalable nested supplementary enhancement information messages. Furthermore, the video encoder is configured to generate a video data stream such that one or more scalable nested supplementary enhancement information messages include multiple syntax elements. Furthermore, the video encoder is configured to generate a video data stream such that each of the multiple syntax elements is defined as having the same size in each scalable nested supplementary enhancement information message in the video data stream or a portion thereof.

[0048] Furthermore, an apparatus for receiving an input video data stream is provided. The input video data stream contains encoded video. The apparatus is configured to generate an output video data stream from an input video data stream. The video data stream includes one or more scalable nested supplemental enhancement information messages. The one or more scalable nested supplemental enhancement information messages include multiple syntax elements. Each syntax element of the one or more syntax elements is defined as having the same size in each scalable nested supplemental enhancement information message in the video data stream or a portion of the video data stream. The apparatus is configured to process the one or more scalable nested supplemental enhancement information messages.

[0049] Furthermore, a video decoder is provided for receiving a video data stream containing a video. The video decoder is configured to decode the video from the video data stream. The video data stream includes one or more scalable nested supplemental enhancement information messages. The one or more scalable nested supplemental enhancement information messages include multiple syntax elements. Each syntax element of the one or more syntax elements is defined as having the same size in each scalable nested supplemental enhancement information message in the video data stream or a portion of the video data stream. The video decoder is configured to decode the video depending on one or more syntax elements of the multiple syntax elements.

[0050] Furthermore, a method for encoding video into a video data stream is provided. The method includes: generating a video data stream such that the video data stream includes one or more scalable nested supplemental enhancement information messages. Furthermore, the method includes: generating a video data stream such that the one or more scalable nested supplemental enhancement information messages include multiple syntax elements.Furthermore, the method includes: generating a video data stream such that each of one or more syntax elements of a plurality of syntax elements is defined to have the same size in each scalable nested supplementary enhancement information message in the video data stream or a portion thereof.

[0051] Furthermore, a method for receiving an input video data stream is provided. The input video data stream contains encoded video. The method includes generating an output video data stream from the input video data stream. The video data stream includes one or more scalable nested supplementary enhancement information messages. The one or more scalable nested supplementary enhancement information messages include a plurality of syntax elements. Each of one or more syntax elements of the plurality of syntax elements is defined to have the same size in each scalable nested supplementary enhancement information message in the video data stream or a portion thereof. The method includes processing the one or more scalable nested supplementary enhancement information messages.

[0052] Furthermore, a method for receiving a video data stream containing stored video is provided. The method includes decoding the video from the video data stream. The video data stream includes one or more scalable nested supplementary enhancement information messages. The one or more scalable nested supplementary enhancement information messages include a plurality of syntax elements. Each of the plurality of syntax elements is defined as having the same size in each of the scalable nested supplementary enhancement information messages in the video data stream or a portion thereof. The video is decoded depending on one or more of the plurality of syntax elements.

[0053] Furthermore, a computer program is provided for implementing one of the above methods when the computer program is executed by a computer or signal processor.

[0054] Preferred embodiments are provided in the dependent claims. Specification 7 / 41 pages 11 CN 121099071 A Brief Description of the Drawings

[0055] Hereinafter, embodiments of the invention will be described in detail with reference to the accompanying drawings, in which:

[0056] FIG1 shows a video encoder for encoding video into a video data stream according to an embodiment.

[0057] FIG2 shows an apparatus for receiving an input video data stream according to an embodiment.

[0058] FIG3 shows a video decoder for receiving a video data stream in which video is stored according to an embodiment.

[0059] Figure 4 shows the original bitstream (shown at the top of Figure 4) and the bitstream after the images have been discarded (shown at the bottom of Figure 4) according to an embodiment.

[0060] Figure 5 shows the concatenation of the two bitstreams according to an embodiment after the images have been discarded from one of the two bitstreams.

[0061] Figure 6 shows the concatenation of the two bitstreams according to another embodiment.

[0062] Figure 7 illustrates two sets of HRD SEIs (scalable nested SEI and non-scalable nested SEI) in a two-layer bitstream according to an embodiment.

[0063] Figure 8 illustrates a video encoder.

[0064] Figure 9 illustrates a video decoder.

[0065] Figure 10 illustrates the relationship between a reconstructed signal (e.g., a reconstructed picture) on one hand and a combination of a prediction residual signal and a prediction signal signal signal notified by a signal in the data stream on the other hand. Detailed Description

[0066] The following description of the figures begins with the presentation of a description of an encoder and decoder for encoding video pictures using block-based prediction encoding and decoding, in order to form an example of an encoding framework in which embodiments of the present invention can be incorporated. The corresponding encoder and decoder are described with respect to Figures 8 through 10. Hereinafter, the description of embodiments of the inventive concept is presented together with a description of how such a concept is incorporated into the encoder and decoder of Figures 8 and 9, respectively, although the embodiments described using Figures 1 through 3 and below can also be used to form encoders and decoders that do not operate according to the encoding framework under the encoder and decoder of Figures 8 and 9.

[0067] FIG8 illustrates a video encoder, an apparatus for predictively encoding image 12 into data stream 14 using transform-based residual coding, exemplarily. This apparatus or encoder is indicated by reference numeral 10. FIG9 illustrates a corresponding video decoder 20, an apparatus 20 configured to also predictively decode image 12' from data stream 14 using transform-based residual coding, wherein an apostrophe is used to indicate that image 12' reconstructed by decoder 20 deviates from image 12 originally encoded by apparatus 10 in terms of coding loss introduced by the quantization of the predictive residual signal. FIG8 and FIG9 exemplarily use transform-based predictive residual coding, but embodiments of this application are not limited to this predictive residual coding. As will be outlined below, so too will the other details described with respect to FIG8 and FIG9.

[0068] Encoder 10 is configured to perform a spatial-to-spectral transform on the predictive residual signal and encode the resulting predictive residual signal into data stream 14. Similarly, decoder 20 is configured to decode the prediction residual signal from data stream 14 and perform a spectral-to-spatial transformation on the resulting prediction residual signal.

[0069] Internally, encoder 10 may include a prediction residual signal former 22 that generates a prediction residual 24 to measure the deviation of the prediction signal 26 from the original signal (e.g., from image 12). The prediction residual signal former 22 may be, for example, a subtractor that subtracts the prediction signal from the original signal (e.g., from image 12). Encoder 10 then includes a transformer 28 that performs a spatial-to-spectral transformation on the prediction residual signal 24 to obtain a spectral domain prediction residual signal 24', which is then quantized by a quantizer 32, which is also included in encoder 10.Therefore, the quantized prediction residual signal 24'' is encoded into bitstream 14. For this purpose, encoder 10 may optionally include entropy encoder 34, which entropy-encodes the transformed and quantized prediction residual signal into data stream 14. Prediction signal 26 is generated by prediction stage 36 of encoder 10 based on the prediction residual signal 24'' encoded into data stream 14 and decodeable from data stream 14. To this end, as shown in Figure 8, prediction stage 36 may internally include a dequantizer 38 followed by an inverse transformer 40. The dequantizer 38 dequantizes the prediction residual signal 24'' to obtain a spectral domain prediction residual signal 24''' corresponding to the signal 24' except for the quantization loss. The inverse transformer 40 performs an inverse transform (e.g., a spectral-to-spatial transform) on the latter's prediction residual signal 24''' to obtain a prediction residual signal 24'''' corresponding to the original prediction residual signal 24 except for the quantization loss. Combiner 42 of prediction stage 36 then recombines the prediction signal 26 and the prediction residual signal 24'''', for example, by addition, to obtain a reconstructed signal 46, for example, a reconstruction of the original signal 12. The reconstructed signal 46 may correspond to signal 12'. Prediction module 44 of prediction stage 36 then generates prediction signal 26 based on signal 46 by using, for example, spatial prediction (e.g., intra-picture prediction) and / or temporal prediction (e.g., inter-picture prediction).

[0070] Similarly, as shown in FIG9, decoder 20 may internally include components corresponding to prediction stage 36 and interconnected in a manner corresponding to prediction stage 36. Specifically, entropy decoder 50 of decoder 20 may entropy decode the quantized spectral domain prediction residual signal 24'' from the data stream, and then dequantizer 52, inverse transformer 54, combiner 56 and prediction module 58, interconnected and cooperating in the manner described above with respect to prediction stage 36, recover the reconstructed signal based on the prediction residual signal 24'', so that, as shown in FIG9, the output of combiner 56 produces the reconstructed signal (i.e., picture 12').

[0071] Although not specifically described above, it is clear that encoder 10 may set some coding parameters, including, for example, prediction mode, motion parameters, etc., according to some optimization scheme (e.g., in a manner that optimizes some rate and distortion correlation criteria (e.g., coding cost)). For example, encoder 10 and decoder 20 and their respective corresponding modules 44, 58 may support different prediction modes such as intra-frame coding mode and inter-frame coding mode. The granularity at which the encoder and decoder switch between these prediction mode types can correspond to subdividing the image 12' and 12' into coded segments or coded blocks, respectively. For example, using these coded segments as units, the image can be subdivided into blocks that are intra-coded and blocks that are inter-coded. Intra-coded blocks are predicted based on the spatial context of the corresponding block and the already encoded / decoded neighborhood, as outlined in more detail below.Several intra-coding modes may exist and be selected for corresponding intra-coded segments that include directional or angular intra-coding modes. Depending on the directional or angular intra-coding mode, the corresponding segment is filled by extrapolating sample values ​​from the neighborhood along a specific direction specific to the directional intra-coding mode into the corresponding intra-coded segment. For example, the intra-coding modes may also include one or more other modes, such as: a DC coding mode, according to which the prediction of the corresponding intra-coded block assigns DC values ​​to all samples within the corresponding intra-coded segment; and / or a planar intra-coding mode, according to which the prediction of the corresponding block is approximated or determined as the spatial distribution of sample values ​​at sample locations within the corresponding intra-coded block, described by a two-dimensional linear function, where the driving tilt and offset of the plane are defined by the two-dimensional linear function based on neighboring samples. In contrast, inter-coded blocks can be predicted, for example, temporally. For inter-coded blocks, a motion vector can be signaled within the data stream, indicating the spatial displacement of a portion of the previously encoded / decoded image of the video to which image 12 belongs, where the previously encoded / decoded image is sampled to obtain the prediction signal for the corresponding inter-coded block. This means that, in addition to the residual signal encoding included in data stream 14 (e.g., the entropy-coded transform coefficient level representing the quantized spectral domain prediction residual signal 24''), data stream 14 may have already encoded therein coding mode parameters for assigning coding modes to various blocks, prediction parameters for some blocks (e.g., motion parameters of inter-frame coded segments), and optional other parameters, such as parameters for controlling and signaling the subdivision of images 12 and 12' into segments respectively. Decoder 20 uses these parameters to subdivide the images in the same manner as encoder, to assign the same prediction modes to segments, and to perform the same predictions to produce the same prediction signals.

[0072] Figure 10 illustrates the relationship between the reconstructed signal (e.g., reconstructed image 12') on one side and the combination of the prediction residual signal 24'''' and prediction signal 26 signaled in the data stream on the other side. As described above, this combination can be additive. The prediction signal 26 is shown in Figure 10 as subdividing the image region into intra-coded blocks indicated by shading and inter-coded blocks not indicated by shading. The subdivision can be arbitrary, such as regularly subdividing the image region into rows and columns of square or non-square blocks, or subdividing the image 12 from a root block into multiple leaf blocks of variable size, such as quadtree subdivision, etc. Figure 10 shows a mixture of these, where the image region is first subdivided into rows and columns of root blocks, and then further subdivided into one or more leaf blocks according to recursive multi-tree subdivision.

[0073] Similarly, data stream 14 may have intra-coding modes encoded into intra-coding blocks 80, which assign one of several supported intra-coding modes to the corresponding intra-coding blocks 80. For inter-coding blocks 82, data stream 14 may have one or more motion parameters encoded into them. Generally, inter-coding blocks 82 are not limited to being encoded temporally. Alternatively, inter-coding blocks 82 may be any block predicted from a previously encoded portion other than the current picture 12 itself, such as a previously encoded picture of the video to which picture 12 belongs, or a picture of another view or a lower-level layer in the case that the encoder and decoder are scalable encoders and decoders, respectively.

[0074] The prediction residual signal 24'''' in FIG. 10 is also shown as subdividing the picture region into blocks 84. These blocks may be referred to as transform blocks in order to distinguish them from the encoded blocks 80 and 82. In practice, Figure 10 shows that encoder 10 and decoder 20 can use two different subdivisions, which subdivide images 12 and 12' into blocks respectively: one subdividing into coding blocks 80 and 82, and the other subdividing into transform blocks 84. The two subdivisions can be the same; for example, each coding block 80 and 82 can simultaneously form transform block 84. However, Figure 10 shows the following situation: for example, subdividing into transform block 84 forms an extension of subdividing into coding blocks 80 and 82, such that any boundary between the two blocks of blocks 80 and 82 covers the boundary between the two blocks 84, or in other words, each block 80, 82 coincides with one of the transform blocks 84, or with a cluster of transform blocks 84. However, the subdivisions can also be determined or selected independently of each other, such that transform block 84 can alternatively span the block boundary between blocks 80, 82. Regarding the subdivision into transform blocks 84, statements similar to those regarding subdivision into blocks 80 and 82 are therefore correct. For example, block 84 can be the result of regularly subdividing an image region into blocks (arranged or not arranged in rows and columns), the result of recursively subdividing an image region into multiple trees, or a combination thereof, or any other type of blockation. Incidentally, it should be noted that blocks 80, 82, and 84 are not limited to squares, rectangles, or any other shape.

[0075] FIG10 also shows that the combination of prediction signal 26 and prediction residual signal 24'''' directly produces reconstructed signal 12'. However, it should be noted that, according to alternative embodiments, more than one prediction signal 26 can be combined with prediction residual signal 24'''' to produce image 12'.

[0076] In FIG10, transform blocks 84 should have the following meaning. Transformer 28 and inverse transformer 54 perform their transforms in units of these transform blocks 84. For example, many codecs use some kind of DST or DCT for all transform blocks 84. Some codecs allow skipping the transform, so that for some transform blocks 84, the predicted residual signal is directly encoded in the spatial domain.However, according to the embodiments described below, encoder 10 and decoder 20 are configured in a way that they support several transformations. For example, the transforms supported by encoder 10 and decoder 20 may include:

[0077] oDCT-II (or DCT-III), where DCT stands for Discrete Cosine Transform

[0078] oDST-IV, where DST stands for Discrete Sine Transform

[0079] oDCT-IV

[0080] oDST-VII

[0081] oIdentity Transform (IT)

[0082] Of course, while transformer 28 will support all forward transform versions of these transforms, decoder 20 or inverse transformer 54 will support their corresponding backward or inverse versions: Specification 10 / 41 Page 14 CN 121099071 A

[0083] oInverse DCT-II (or Inverse DCT-III)

[0084] oInverse DST-IV

[0085] oInverse DCT-IV

[0086] oInverse DST-VII

[0087] oIdentity Transform (IT)

[0088] The following description provides further details about which transformations the encoder 10 and decoder 20 can support. In any case, it should be noted that the set of supported transformations may include only one transformation, such as a spectrum-to-space or space-to-spectrum transformation.

[0089] As outlined above, Figures 8 through 10 have been presented as examples in which the inventive concepts described further below can be implemented to form specific examples of encoders and decoders according to this application. In this regard, the encoders and decoders of Figures 8 and 9 may respectively represent possible implementations of the encoders and decoders described below. However, Figures 8 and 9 are merely examples. However, the encoder of an embodiment according to this application may perform block-based encoding of picture 12 using the more detailed outline below, and may differ from the encoder of Figure 8, for example, in that the encoder is not a video encoder but a still picture encoder, does not support inter-frame prediction, or is subdivided into blocks 80 in a manner different from that illustrated in Figure 10. Similarly, the decoder according to embodiments of this application can perform block-based decoding of image 12' from data stream 14 using the coding concept further outlined below, but may differ from, for example, the decoder 20 of FIG. 9, in that the decoder is not a video decoder but a still image decoder, does not support intra-frame prediction, or subdivides image 12' into blocks in a manner different from that described with respect to FIG. 10, and / or the decoder derives prediction residuals from data stream 14 in the spatial domain rather than in the transform domain.

[0090] FIG. 1 illustrates a video encoder 100 according to an embodiment for encoding video into a video data stream. The video encoder 100 is configured to generate a video data stream such that the video data stream includes an indication of whether video should be output before relying on random access images.

[0091] FIG2 illustrates an apparatus 200 for receiving an input video data stream according to an embodiment. The input video data stream contains encoded video. The apparatus 200 is configured to generate an output video data stream from the input video data stream.

[0092] FIG3 illustrates a video decoder 300 for receiving a video data stream in which video is stored, according to an embodiment. The video decoder 300 is configured to decode the video from the video data stream. The video decoder 300 is configured to decode the video according to an indication of whether the video should be output from a picture preceding a picture dependent on random access.

[0093] Furthermore, a system according to an embodiment is provided. The system includes the apparatus of FIG2 and the video decoder of FIG3. The video decoder 300 of FIG3 is configured to receive the output video data stream of the apparatus of FIG2. The video decoder 300 of FIG3 is configured to decode the video from the output video data stream of the apparatus 200 of FIG2.

[0094] In an embodiment, the system may, for example, also include the video encoder 100 of FIG1. The apparatus 200 of FIG2 may, for example, be configured to receive a video data stream as an input video data stream from the video encoder 100 of FIG1. ​​

[0095] An (optional) intermediate device 210 of apparatus 200 may, for example, be configured to receive a video data stream as an input video data stream from the video encoder 100 and generate an output video data stream from the input video data stream. For example, the intermediate device may, for example, be configured to modify the (header / metadata) information of the input video data stream, and / or may, for example, be configured to delete images from the input video data stream, and / or may be configured to mix / concatenate the input video data stream with an additional second bitstream having a second video encoded therein.

[0096] An (optional) video decoder 221 may, for example, be configured to decode video from the output video data stream.

[0097] (Optional) It is assumed that the reference decoder 222 may be configured, for example, to determine the timing information of the video based on the output video data stream, as per page 11 / 41 of the specification 15 CN 121099071 A, or may be configured, for example, to determine buffer information of the buffer to store the video or a portion of the video.

[0098] The system includes a video encoder 101 of FIG1 and a video decoder 151 of FIG2.

[0099] The video encoder 101 is configured to generate an encoded video signal. The video decoder 151 is configured to decode the encoded video signal to reconstruct pictures of the video.

[0100] Hereinafter, a first aspect of the invention will be described in detail.

[0101] According to a first aspect of the invention, an apparatus 200 for receiving an input video data stream is provided. The input video data stream contains encoded video. The apparatus 200 is configured to generate an output video data stream from the input video data stream. Furthermore, the apparatus 200 is configured to determine whether the pictures of the video prior to the pictures dependent on random access should be output.

[0102] According to an embodiment, the device 200 may be configured, for example, to determine a first variable (e.g., NoOutputBeforeDrapFlag) indicating whether the images of the video preceding the random access images should be output.

[0103] In an embodiment, the device 200 may be configured, for example, to generate an output video data stream such that the output video data stream may include, for example, an indication indicating whether the images of the video preceding the random access images should be output.

[0104] According to an embodiment, the device 200 may be configured, for example, to generate an output video data stream such that the output video data stream may include, for example, supplementary enhancement information including, for example, an indication indicating whether the images of the video preceding the random access images should be output.

[0105] In an embodiment, the images of the video preceding the random access images may be independent random access images. The apparatus 200 may, for example, be configured to: generate an output video data stream such that the output video data stream may, for example, include a flag (e.g., ph_pic_output_flag) with a predefined value (e.g., 0) in the image header of an independent random access image, such that the predefined value (e.g., ph_pic_output_flag) of the flag (e.g., ph_pic_output_flag) may, for example, indicate that the independent random access image is directly before the dependent random access image in the video data stream and should not be output.

[0106] According to an embodiment, the flag may, for example, be a first flag, wherein the apparatus 200 may, for example, be configured to: generate an output video data stream such that the output video data stream may, for example, include another flag in the image parameter set of the video data stream, wherein the other flag may, for example, indicate whether the first flag (e.g., ph_pic_output_flag) exists in the image header of the independent random access image.

[0107] In an embodiment, the device 200 may be configured, for example, to generate an output video data stream, such that the output video data stream may include, for example, the following flags as indicators of whether the video should be output before the random access image: a supplementary enhancement information flag in the supplementary enhancement information of the output video data stream, or a picture parameter set flag in the picture parameter set of the output video data stream, or a sequence parameter set flag in the sequence parameter set of the output video data stream, or an external device flag, wherein the value of the external device flag may be set, for example, by an external unit located outside the device 200.

[0108] According to an embodiment, the device 200 may, for example, be configured to determine the value of a second variable (e.g., PictureOutputFlag) of the video before the image that depends on the random access image based on a first variable (e.g., NoOutputBeforeDrapFlag), wherein the second variable (e.g., PictureOutputFlag) may, for example, indicate whether the image should be output for the image, and wherein the device 200 may, for example, be configured to output or not output the image based on the second variable (e.g., PictureOutputFlag). Specification 12 / 41 pages 16 CN 121099071 A

[0109] In an embodiment, the image of the video before the image that depends on the random access image may be an independent random access image. The first variable (e.g., NoOutputBeforeDrapFlag) may, for example, indicate that an independent random access image should not be output.

[0110] According to an embodiment, the image of the video before the image that depends on the random access image may be an independent random access image. Device 200 may be configured, for example, to set a first variable (e.g., NoOutputBeforeDrapFlag) such that the first variable (e.g., NoOutputBeforeDrapFlag) may indicate, for example, that an independent random access image should be output.

[0111] In an embodiment, device 200 may be configured, for example, to signal to video decoder 300 whether the images preceding the dependent random access images of the video should be output.

[0112] Furthermore, a video data stream is provided. Video is encoded in the video data stream. The video data stream includes an indication of whether the images preceding the dependent random access images of the video should be output.

[0113] According to an embodiment, the video data stream includes supplementary enhancement information, which includes, for example, an indication of whether the images preceding the dependent random access images of the video should be output.

[0114] In an embodiment, the images preceding the dependent random access images of the video may be independent random access images. The video data stream may include, for example, a flag (e.g., ph_pic_output_flag) with a predefined value (e.g., 0) in the image header of the standalone random access image, such that the predefined value (e.g., ph_pic_output_flag) of the flag (e.g., ph_pic_output_flag) may indicate, for example, that the standalone random access image is directly before the dependent random access image in the video data stream and should not be output.

[0115] According to an embodiment, the flag may be, for example, a first flag, wherein the video data stream may include, for example, another flag in the picture parameter set of the video data stream, wherein the other flag may indicate, for example, whether the first flag (e.g., ph_pic_output_flag) exists in the picture header of the independent random access picture.

[0116] In an embodiment, the video data stream may include, for example, the following flags as an indication of, for example, whether the video should be output before the random access picture: a supplemental enhancement information flag in the supplemental enhancement information of the output video data stream, or a picture parameter set flag in the picture parameter set of the output video data stream, or a sequence parameter set flag in the sequence parameter set of the output video data stream, or an external device flag, wherein the value of the external device flag may be set, for example, by an external unit located outside the device 200.

[0117] Furthermore, a video encoder 100 is provided. The video encoder 100 may be configured, for example, to encode video into a video data stream. Furthermore, the video encoder 100 may be configured, for example, to generate a video data stream such that the video data stream includes an indication of whether the video should be output before the dependent random access image.

[0118] According to an embodiment, the video encoder 100 may be configured, for example, to generate a video data stream such that the video data stream may include, for example, supplementary enhancement information, which includes, for example, an indication of whether the video should be output before the dependent random access image.

[0119] In an embodiment, the image before the dependent random access image may be an independent random access image. The video encoder 100 may be configured, for example, to generate a video data stream such that the video data stream may include, for example, a flag (e.g., ph_pic_output_flag) with a predefined value (e.g., 0) in the image header of the independent random access image, such that the predefined value (e.g., ph_pic_output_flag) of the flag (e.g., ph_pic_output_flag) may, for example, indicate that the independent random access image is directly before the dependent random access image in the video data stream and that the independent random access image should not be output.

[0120] According to an embodiment, the flag may be, for example, a first flag, wherein the video encoder 100 may be configured to generate a video data stream such that the video data stream may include, for example, another flag in the picture parameter set of the video data stream, wherein the other flag may indicate, for example, whether the first flag (e.g., ph_pic_output_flag) exists in the picture header of an independently randomly accessed picture.

[0121] In an embodiment, the video encoder 100 may be configured, for example, to generate a video data stream such that the video data stream may include, for example, the following flags as indications of whether the video should be output before a random access-dependent image: a supplementary enhancement information flag within the supplementary enhancement information of the output video data stream, or a picture parameter set flag within the picture parameter set of the output video data stream, or a sequence parameter set flag within the sequence parameter set of the output video data stream, or an external device flag, wherein the value of the external device flag may be set, for example, by an external unit located outside the device 200.

[0122] Furthermore, a video decoder 300 is provided for receiving a video data stream in which video is stored. The video decoder 300 is configured to decode video from the video data stream. The video decoder 300 is configured to decode the video according to an indication of whether the video should be output before a random access-dependent image.

[0123] According to an embodiment, the video decoder 300 may be configured, for example, to decode the video based on a first variable (e.g., NoOutputBeforeDrapFlag), which indicates whether the video should output images preceding the random access images.

[0124] In an embodiment, the video data stream may include, for example, an indication that may indicate whether the video should output images preceding the random access images. The video decoder 300 may be configured, for example, to decode the video based on the indication within the video data stream.

[0125] According to an embodiment, the video data stream includes supplemental enhancement information, which includes, for example, an indication that may indicate whether the video should output images preceding the random access images. The video decoder 300 may be configured, for example, to decode the video based on the supplemental enhancement information.

[0126] In an embodiment, the images preceding the random access images may be independent random access images. The video data stream may, for example, include a flag (e.g., ph_pic_output_flag) with a predefined value (e.g., 0) in the image header of the independent random access image, such that the predefined value (e.g., ph_pic_output_flag, 0) of the flag (e.g., ph_pic_output_flag) may indicate, for example, that the independent random access image is directly preceding the dependent random access image within the video data stream and should not be output. The video decoder 300 may, for example, be configured to decode the video based on this flag.

[0127] According to an embodiment, the flag may be, for example, a first flag, wherein the video data stream may include, for example, another flag in the picture parameter set of the video data stream, wherein the other flag may indicate, for example, whether the first flag (e.g., ph_pic_output_flag) exists in the picture header of the independent random access picture. The video decoder 300 may be configured, for example, to decode the video according to the other flag.

[0128] In an embodiment, the video data stream may include, for example, the following flags as an indication of, for example, whether the video should be output before the random access picture: a supplemental enhancement information flag in the supplemental enhancement information of the output video data stream, or a picture parameter set flag in the picture parameter set of the output video data stream, or a sequence parameter set flag in the sequence parameter set of the output video data stream, or an external device flag, wherein the value of the external device flag may be set, for example, by an external unit located outside the device 200. The video decoder 300 may be configured, for example, to decode the video according to the indication in the video data stream.

[0129] According to an embodiment, the video decoder 300 may, for example, be configured to reconstruct the video from a video data stream. The video decoder 300 may, for example, be configured to output or not output the images of the video preceding the random access images based on a first variable (e.g., NoOutputBeforeDrapFlag). Specification 14 / 41 pages 18 CN 121099071 A

[0130] In an embodiment, the video decoder 300 may, for example, be configured to determine the value of a second variable (e.g., PictureOutputFlag) of the images of the video preceding the random access images based on the first variable (e.g., NoOutputBeforeDrapFlag), wherein the second variable (e.g., PictureOutputFlag) may, for example, indicate whether the image should be output for the image, and wherein the device 200 may, for example, be configured to output or not output the image based on the second variable (e.g., PictureOutputFlag).

[0131] According to an embodiment, the images of the video preceding the random access images may be independent random access images. The video decoder 300 may be configured, for example, to decode the video based on a first variable (e.g., NoOutputBeforeDrapFlag) that may indicate that independent random access images should not be output.

[0132] In an embodiment, the images in the video prior to the images that depend on the random access images may be independent random access images.The video decoder 300 may be configured, for example, to decode the video according to a first variable (e.g., NoOutputBeforeDrapFlag) that may indicate that an independent random access picture should be output.

[0133] Furthermore, a system is provided. The system includes the device 200 as described above and the video decoder 300 as described above. The video decoder 300 is configured to receive the output video data stream of the device 200. Furthermore, the video decoder 300 is configured to decode video from the output video data stream of the device 200.

[0134] According to an embodiment, the system may also include, for example, a video encoder 100. The device 200 may be configured, for example, to receive a video data stream from the video encoder 100 as an input video data stream.

[0135] Specifically, a first aspect of the invention relates to starting CVS at DRAP and omitting IDR output in decoding and conformance testing.

[0136] When the bitstream includes pictures marked as DRAP (i.e., in the bitstream, only the previous IRAP is used as a reference for DRAP and starts from there), these DRAP pictures can be used for random access functionality with a lower rate overhead. However, when using a target DRAP to randomly access a stream, it is not desirable to display any initial images preceding the target DRAP (i.e., the associated IRAP of the target DRAP) at the decoder output, because the temporal distance between these images will cause video playback instability / jitter when played back at the rate of the original video until the video plays back smoothly from the target DRAP.

[0137] Therefore, it is desirable to omit the output of images preceding the DRAP images. This aspect of the invention proposes means for controlling the decoder accordingly.

[0138] In one embodiment, an external means for setting the PicOutputFlag variable of the IRAP image is available for implementation as follows:

[0139] – If an external means not specified in the specification is available to set the NoOutputBeforeDrapFlag variable of the image to a value, then the NoOutputBeforeDrapFlag of the image is set to equal to the value provided by the external means.

[0140] [...]

[0141] – The variable PictureOutputFlag for the current image is derived as follows:

[0142] – If sps_video_parameter_set_id is greater than 0 and the current layer is not an output layer (i.e., for any value of i in the range of 0 to NumOutputLayersInOls[TargetOlsIdx]-1 (inclusive of 0 and NumOutputLayersInOls[TargetOlsIdx]-1), nuh_layer_id is not equal to OutputLayerIdInOls[TargetOlsIdx][i]), or one of the following conditions is true, then PictureOutputFlag is set to equal to 0:

[0143] – The current image is a RASL image, and the NoOutputBeforeRecoveryFlag of the associated IRAP image is equal to 1.

[0144] – The current image is a GDR image with NoOutputBeforeRecoveryFlag equal to 1, or a recovered image of a GDR image with NoOutputBeforeRecoveryFlag equal to 1 as described on page 15 / 41 of the specification (CN 121099071 A).

[0145] – The current image is an IRAP image with NoOutputBeforeDrapFlag equal to 1.

[0146] – Otherwise, PictureOutputFlag is set to equal to ph_pic_output_flag.

[0147] In another embodiment, NoOutputBeforeDrapFlag is set by an external means only for the first IRAP image in the CVS, otherwise it is set to 0.

[0148] – If an external means not specified in the specification is available to set the image variable NoOutputBeforeDrapFlag to a value, then the NoOutputBeforeDrapFlag of the first image in the CVS is set to equal to the value provided by the external means. Otherwise, NoOutputBeforeDrapFlag is set to 0.

[0149] In the case of removing images between IRAP and DRAP images, the aforementioned flag NoOutputBeforeDrapFlag can also be associated with the use of alternative HRD timings transmitted in the bitstream, such as the flag UseAltCpbParamsFlag in the VVC specification.

[0150] In an alternative embodiment, it is a constraint that the output flag ph_pic_output_flag in the image header should have a value of 0 for an IRAP image that is directly before the DRAP image and in between without any non-DRAP images. In this case, whenever the extractor or player uses DRAP for random access, i.e., when it removes the intermediate image between IRAP and DRAP from the bitstream, it is also necessary to verify or adjust that the corresponding output flag is set to 0 and the IRAP output is omitted.

[0151] To simplify this operation, the original bitstream needs to be prepared accordingly. More specifically, pps_output_flag_present_flag (which determines the presence of the flag ph_pic_output_flag in the image header) should be equal to 1, so that the image header can be easily changed without changing the parameter set. That is:

[0152] The requirement for bitstream consistency is that if the PPS is referenced by an image within a CVSS AU with an associated DRAP AU, then the value of pps_output_flag_present_flag should be equal to 1.

[0153] In addition to the options listed above, in another embodiment, the parameter set PPS or SPS indicates whether the first AU in the bitstream, i.e., the CRA or IDR that constitutes the start of CLVS, should be output after decoding. Therefore, system integration is simpler because, for example, when parsing a file in ISOBMFF format, only the parameter set needs to be adjusted, rather than changing the relatively low-level syntax (e.g., PHs).

[0154] An example is shown below: Specification 16 / 41 pages 20 CN 121099071 A

[0155]

[0156] sps_pic_in_cvss_au_no_output_flag equal to 1 specifies that the picture referencing the SPS in the CVSS AU should not be output. sps_pic_in_cvss_au_no_output_flag equal to 0 specifies that the picture referencing the SPS in the CVSS AU may or may not be output.

[0157] The requirement for bitstream consistency is that the value of sps_pic_in_cvss_au_no_output_flag should be the same for any SPS referenced by any output layer in OLS.

[0158] In 8.1.2

[0159] – The variable PictureOutputFlag for the current picture is derived as follows:

[0160] – If sps_video_parameter_set_id is greater than 0 and the current layer is not an output layer (i.e., for any value of i in the range of 0 to NumOutputLayersInOls[TargetOlsIdx]-1 (inclusive of 0 and NumOutputLayersInOls[TargetOlsIdx]-1), then nuh_layer_id is not equal to OutputLayerIdInOls[TargetOlsIdx][i]), or one of the following conditions is true, then PictureOutputFlag is set to equal to 0:

[0161] – The current picture is a RASL picture, and the NoOutputBeforeRecoveryFlag of the associated IRAP picture is equal to 1.

[0162] – The current image is either a GDR image with NoOutputBeforeRecoveryFlag equal to 1, or a recovered image of a GDR image with NoOutputBeforeRecoveryFlag equal to 1.

[0163] – Otherwise, if the current AU is a CVSS AU and sps_pic_in_cvss_au_no_output_flag equals 1, then PictureOutputFlag is set to 0.

[0164] – Otherwise, PictureOutputFlag is set to ph_pic_output_flag.

[0165] Note – In the implementation, the decoder can output images that do not belong to the output layer. For example, when there is only one output layer and the image of the output layer is unavailable in the AU, for example due to loss or layer downswitching, the decoder can set PictureOutputFlag to 1 for the image with the highest nuh_layer_id value and ph_pic_output_flag equal to 1 among all images available to the decoder in the AU, and set PictureOutputFlag to 0 for all other images available to the decoder in the AU.

[0166] In another embodiment, for example, a requirement can be defined as follows:

[0167] The requirement for bitstream consistency is that if the image belongs to the IRAP AU and the IRAP AU is directly before the DRAP AU, then the value of ph_pic_output_flag should be equal to 0.

[0168] Hereinafter, the second aspect of the invention will be described in detail.

[0169] According to a second aspect of the invention, an apparatus is provided for receiving one or more input video data streams. Each of the one or more input video data streams encodes an input video. The apparatus 200 is configured to generate an output video data stream from the one or more input video data streams, the output video data streams encoding the output video, wherein the apparatus is configured to: generate the output video data stream such that the output video is an input video encoded within one of the one or more input video data streams, or such that the output video depends on the input video of at least one of the one or more input video data streams. Furthermore, the apparatus 200 is configured to determine the removal time of the current image from the access unit of the encoded image buffer among a plurality of images of the output video. The apparatus 200 is configured to: determine whether to use encoded image buffer delay offset information to determine the removal time of the current image from the access unit of the encoded image buffer.

[0170] According to an embodiment, the apparatus 200 may, for example, be configured to: discard a group of one or more images of the input video of a first video data stream in one or more input video data streams to generate the output video data stream. The apparatus 200 may, for example, be configured to: determine, based on encoded image buffer delay offset information, the time when at least one image of a plurality of images in the output video is removed from the access unit of the encoded image buffer.

[0171] In an embodiment, the first video received by the apparatus 200 may, for example, be a preprocessed video obtained from an original video, from which one or more images have been discarded to generate the preprocessed video. The apparatus 200 may, for example, be configured to: determine, based on encoded image buffer delay offset information, the time when at least one image of a plurality of images in the output video is removed from the access unit of the encoded image buffer.

[0172] According to an embodiment, the buffer delay offset information depends on the number of images that have been discarded in the input video.

[0173] In an embodiment, one or more input video data streams are two or more input video data streams. The apparatus 200 may, for example, be configured to concatenate the processed video with the input video of a second video data stream from two or more input video data streams to obtain an output video, and the apparatus 200 may, for example, be configured to encode the output video into the output video data stream.

[0174] According to an embodiment, the device 200 may be configured, for example, to determine whether to use encoded image buffer delay offset information to determine the access unit removal time of the current image based on the position of the current image in the output video. Alternatively, the device 200 may be configured, for example, to determine whether to set the encoded image buffer delay offset value of the encoded image buffer delay offset information to 0 to determine the access unit removal time of the current image based on the position of the current image in the output video.

[0175] In an embodiment, the device 200 may, for example, be configured to: determine whether to use encoded image buffer delay offset information to determine the access unit removal time of the current image based on the position of a previous non-discardable image preceding the current image in the output video.

[0176] In an embodiment, the device 200 may, for example, be configured to: determine whether to use encoded image buffer delay offset information to determine the access unit removal time of the current image based on whether a previous non-discardable image preceding the current image in the output video is, for example, the first image in a previous buffer cycle.

[0177] In an embodiment, the device 200 may, for example, be configured to: determine whether to use encoded image buffer delay offset information to determine the access unit removal time of the current image based on a concatenation flag, wherein the current image is the first image of the input video of the second video data stream.

[0178] In an embodiment, the device 200 may, for example, be configured to: determine the access unit removal time of the current image based on the removal time of the previous image.

[0179] In an embodiment, the device 200 may be configured, for example, to determine the access unit removal time of the current image based on the initial encoded image buffer removal delay information.

[0180] In an embodiment, the device 200 may be configured, for example, to update the initial encoded image buffer removal delay information based on a clock tick to obtain temporary encoded image buffer removal delay information to determine the access unit removal time of the current image.

[0181] In an embodiment, if the concatenation flag is set to a first value, the device 200 is configured to use the encoded image buffer delay offset information to determine one or more removal times. If the concatenation flag is set to a second value different from the first value, the device 200 is configured not to use the encoded image buffer delay offset information to determine one or more removal times.

[0182] In an embodiment, the device 200 may be configured, for example, to signal to the video decoder 300 whether to use the encoded image buffer delay offset information to determine the access unit removal time of the current image from the encoded image buffer.

[0183] In an embodiment, the current image may, for example, be located at the splicing point of the output video, where two input videos have been spliced.

[0184] In addition, a video data stream is provided. This video data stream contains encoded video. The video data stream includes encoded image buffer delay offset information.

[0185] According to an embodiment, the video data stream may, for example, include a concatenation flag.

[0186] In an embodiment, the video data stream may, for example, include initial encoded image buffer removal delay information.

[0187] According to an embodiment, for example, when it is known that some images (e.g., RASL images) have been discarded, if the concatenation flag is set to a first value (e.g., 0), the concatenation flag indicates that encoded image buffer delay offset information needs to be used to determine the removal time of one or more (images or access units). If the concatenation flag is set to a second value different from the first value (e.g., 1), the concatenation flag indicates that the indicated offset is not used to determine the removal time of one or more (images or access units), for example, regardless of offset signaling and for example, regardless of whether the RASL images have been discarded. If the images have not been discarded, then for example, the offset is not used.

[0188] Furthermore, a video encoder 100 is provided. The video encoder 100 is configured to encode video into a video data stream. The video encoder 100 is configured to generate a video data stream such that the video data stream includes encoded image buffer delay offset information.

[0189] According to an embodiment, the video encoder 100 may, for example, be configured to generate a video data stream such that the video data stream may, for example, include a concatenation flag.

[0190] In an embodiment, the video encoder 100 may be configured, for example, to generate a video data stream such that the video data stream may include, for example, encoded image buffer delay offset information.

[0191] Furthermore, a video decoder 300 is provided for receiving a video data stream in which video is stored. The video decoder 300 is configured to decode the video from the video data stream. Furthermore, the video decoder 300 is configured to decode the video based on the removal time of the current image from the access unit of the encoded image buffer among the multiple images of the video (page 19 / 41 of the specification, 23 CN 121099071 A). The video decoder 300 is configured to decode the video based on an indication indicating whether the encoded image buffer delay offset information is used to determine the removal time of the current image from the access unit of the encoded image buffer.

[0192] According to an embodiment, the removal time of at least one image among the multiple images of the video from the access unit of the encoded image buffer depends on the encoded image buffer delay offset information.

[0193] In an embodiment, the video decoder 300 is configured to decode the video based on the following: determining whether to use encoded image buffer delay offset information to determine the access unit removal time of the current image based on the position of the current image within the video.

[0194] According to an embodiment, the video decoder 300 may, for example, be configured to decode the video based on whether the encoded image buffer delay offset value of the encoded image buffer delay offset information can, for example, be set to 0.

[0195] In an embodiment, the video decoder 300 may, for example, be configured to determine whether to use encoded image buffer delay offset information to determine the access unit removal time of the current image based on the position of a previously non-discardable image within the video preceding the current image.

[0196] According to an embodiment, the video decoder 300 may, for example, be configured to: determine whether to use encoded image buffer delay offset information to determine the access unit removal time of the current image based on whether a previous non-discardable image preceding the current image in the video is, for example, a first image in a previous buffer cycle.

[0197] In an embodiment, the video decoder 300 may, for example, be configured to: determine whether to use encoded image buffer delay offset information to determine the access unit removal time of the current image based on a concatenation flag, wherein the current image is a first image of the input video of a second video data stream.

[0198] According to an embodiment, the video decoder 300 may, for example, be configured to: determine the access unit removal time of the current image based on the removal time of the previous image.

[0199] In an embodiment, the video decoder 300 may, for example, be configured to: determine the access unit removal time of the current image based on initial encoded image buffer removal delay information.

[0200] According to an embodiment, the video decoder 300 may, for example, be configured to: update the initial encoded image buffer removal delay information based on a clock tick to obtain temporary encoded image buffer removal delay information to determine the access unit removal time of the current image.

[0201] According to an embodiment, if the concatenation flag is set to a first value, the video decoder 300 is configured to use encoded image buffer delay offset information to determine one or more removal times. If the concatenation flag is set to a second value different from the first value, the video decoder 300 is configured not to use encoded image buffer delay offset information to determine one or more removal times.

[0202] Furthermore, a system is provided. The system includes the apparatus 200 as described above and the video decoder 300 as described above. The video decoder 300 is configured to receive the output video data stream of the apparatus 200. Furthermore, the video decoder 300 is configured to decode video from the output video data stream of the apparatus 200.

[0203] According to an embodiment, the system may also include, for example, a video encoder 100. The apparatus 200 may be configured, for example, to receive a video data stream from the video encoder 100 as an input video data stream.

[0204] Specifically, the second aspect of the invention relates to the possibility that prevNonDiscardable may already include the alternative offset (CpbDelayOffset) when an alternative is selected (when it is not the start of BP), therefore, for an AU with concatenation_flag == 1, CpbDelayOffset should be temporarily set to zero.

[0205] When concatenation of two bitstreams occurs, the time for removing the AU from the CPB is different from that for non-concatenated bitstreams. (See specification 20 / 41, page 24, CN 121099071 A)At the splicing point, the buffered periodic SEI message (BP SEI message; SEI = Supplemental Enhancement Information) includes a concatenationFlag equal to 1. The decoder then needs to check two values ​​and take the larger of the two:

[0206] ● The removal time of the previous non-discardable picture (prevNonDiscardablePic) plus the increment of the signal notification in the BP SEI message (auCpbRemovalDelayDeltaMinus1+1), or

[0207] ● The removal time of the previous picture plus InitCpbRemovalDelay.

[0208] However, when the previous picture with the BP SEI message is an AU for which the removal time has been derived using an alternative timing (i.e., the second timing information used when the RASL picture or the picture until DRAP has been discarded), an offset (CpbDelayOffset) is used to calculate each removal time, which is calculated as the increment of the previous picture with the buffer period, i.e., AuNominalRemovalTime[firstPicInPrevBuffPeriod] plus AuCpbRemovalDelayVal – CpbDelayOffset, as shown in Figure 4.

[0209] Figure 4 shows the original bitstream (top of Figure 4) and the bitstream after the picture is discarded (bottom of Figure 4): after the discarded AU (lines 1, 2, and 3 in the original bitstream), the offset is incorporated into the calculation of the removal delay.

[0210] The offset is added because the removal time is calculated using the increment of the removal time of the image (called firstPicInPrevBuffPeriod), after which some AUs have been discarded, so CpbDelayOffset is needed to account for (compensate) the AU discarding.

[0211] Figure 5 shows the concatenation of two bitstreams (first bitstream (in Figure 5, in Figure 5, in the middle left) and second bitstream (in Figure 5, in the middle right)) after the image is discarded from the original first bitstream (in Figure 5, in the middle left) at different positions.

[0212] The example of using the removal time of the previous image as the anchor point instead of the previous non-discardable image is similar and also does not require a "-3" correction factor (CpbDelayOffset).

[0213] However, in the concatenation case shown in Figure 5, note that this is not necessarily the case where two exports use the removal time (firstPicInPrevBuffPeriod) of the AU associated with the BP SEI message. As discussed, for splicing cases, the increment is added to prevNonDiscardablePic or simply the previous image.This means that when `prevNonDiscardablePic` is not `firstPicInPrevBuffPeriod`, `CpbDelayOffset` cannot be used to derive the removal time of the current AU from the CPB, because the removal time of `prevNonDiscardablePic` already takes into account AU discarding, and no AU is discarded between `prevNonDiscardablePic` and the AU whose removal time is calculated. Now, assuming the previous image removal time is used instead, for the current AU (i.e., the splice point with the new BPSEI message) with `InitialCpbRemovalDelay` (which forces the removal time of the current AU to be after its expected removal time), this will achieve equidistant removal times (when using `prevNonDiscardablePic` instead). In this case, the removal time of the current AU cannot be less than the time calculated by adding `InitCpbRemovalDelay` to the previous image removal time, because this could lead to buffer underloading (the AU is not in the buffer before it needs to be removed). Therefore, as part of this invention, for this case, `CpbDelayOffset` is not used for calculation or is considered equal to 0.

[0214] The embodiments summarized herein depend on checking the use of CpbDelayOffset to calculate the AU removal time when a RASL AU is dropped from the bitstream or an AU between an IRAP and a DRAP AU is dropped. The checks used to determine whether CpbDelayOffset is not used or is considered equal to 0 are one of the following:

[0215] ●prevNonDiscardablePic is not firstPicInPrevBuffPeriod

[0216] ●The previous image removal time plus InitCpbRemovalDelay is used to calculate the removal time of the current AU.

[0217] The implementation in the specification may be as follows:

[0218] – When AU n is the first AU of the BP that is not initialized with HRD, the following applies: Specification 21 / 41 pages 25 CN 121099071 A

[0219] The nominal removal time of AU n from the CPB is specified by the following:

[0220]

[0221] Alternatively, in another embodiment shown in FIG6, the CpbDelayOffset used to calculate the AU removal time when the RASL AU is dropped from the bitstream or the AU between the IRAP and DRAP AUs is dropped depends on different checks including checking the concatenationFlag.

[0222] In this case, when concatenationFlag is set to 1, the increment in the bitstream needs to match the value of the correct specification page 22 / 41, 26 CN 121099071 A, as if CpbDelayOffset were taken into account (which is obvious when comparing Figures 5 and 6), since for that figure, CpbDelayOffset is not applied or is treated as 0.

[0223] The implementation in the specification may be as follows:

[0224] – When AU n is the first AU of the BP that is not initialized with HRD, the following applies:

[0225] The nominal removal time of AU n from the CPB is specified by the following:

[0226]

[0227] Hereinafter, the third aspect of the invention will be described in detail.

[0228] According to the third aspect of the invention, a video data stream is provided. Video is encoded in the video data stream. Furthermore, the video data stream includes an initial encoded picture buffer removal delay. Furthermore, the video data stream includes an initial encoded picture buffer removal offset. Furthermore, the video data stream includes information indicating whether the sum of the initial encoded image buffer removal delay and the initial encoded image buffer removal offset (page 23 / 41, CN 121099071 A) is defined as constant across two or more buffer cycles.

[0229] According to an embodiment, the initial encoded image buffer removal delay may, for example, indicate the time required for a first access unit before sending the first access unit of the image that initializes the video data stream of the video decoder 300 to the video decoder 300.

[0230] In an embodiment, the video data stream may, for example, include a single indication, which may, for example, indicate whether the sum of the initial encoded image buffer removal delay and the initial encoded image buffer removal offset can, for example, be defined as constant across two or more buffer cycles.

[0231] According to an embodiment, the video data stream may, for example, include a concatenation flag as a single indication, which may, for example, indicate whether the sum of the initial encoded image buffer removal delay and the initial encoded image buffer removal offset can, for example, be defined as constant across two or more buffer cycles. If the concatenation flag equals the first value, then the sum of the initial encoded image buffer removal delay and the initial encoded image buffer removal offset is constant across two or more buffer cycles. If the concatenation flag differs from the first value, then the concatenation flag does not define whether the sum of the initial encoded image buffer removal delay and the initial encoded image buffer removal offset is constant across two or more buffer cycles.

[0232] In an embodiment, if the single indication does not indicate that the sum of the initial encoded image buffer removal delay and the initial encoded image buffer removal offset is defined as constant across two or more buffering cycles, then the video data stream may, for example, include continuously updated information regarding the initial encoded image buffer removal delay and continuously updated information regarding the initial encoded image buffer removal offset.

[0233] According to an embodiment, if the video data stream includes information indicating that the sum of the initial encoded image buffer removal delay and the initial encoded image buffer removal offset is defined as constant across two or more buffering cycles, then the sum of the initial encoded image buffer removal delay and the initial encoded image buffer removal offset may, for example, be defined as constant starting from the current position within the video data stream.

[0234] Furthermore, a video encoder 100 is provided. The video encoder 100 is configured to encode video into a video data stream. Furthermore, the video encoder 100 is configured to generate a video data stream such that the video data stream includes the initial encoded image buffer removal delay. Furthermore, the video encoder 100 is configured to generate a video data stream such that the video data stream includes the initial encoded image buffer removal offset. Furthermore, the video encoder 100 is configured to generate a video data stream such that the video data stream includes information indicating whether the sum of the initial encoded image buffer removal delay and the initial encoded image buffer removal offset is defined as constant across two or more buffer cycles.

[0235] According to an embodiment, the initial encoded image buffer removal delay may, for example, indicate the time required for a first access unit before sending a first access unit of the image that initializes the video data stream of the video decoder 300 to the video decoder 300.

[0236] In an embodiment, the video encoder 100 may, for example, be configured to generate a video data stream such that the video data stream may, for example, include a single indication, which may, for example, indicate whether the sum of the initial encoded image buffer removal delay and the initial encoded image buffer removal offset can, for example, be defined as constant across two or more buffer cycles.

[0237] According to an embodiment, the video encoder 100 may, for example, be configured to generate a video data stream such that the video data stream may, for example, include a concatenated flag as a single indication, which may, for example, indicate whether the sum of the initial encoded image buffer removal delay and the initial encoded image buffer removal offset can, for example, be defined as constant across two or more buffer cycles. If the concatenation flag is equal to the first value, the sum of the initial encoded image buffer removal delay and the initial encoded image buffer removal offset is constant across two or more buffer cycles.If the concatenation flag differs from the first value, then the concatenation flag does not define whether the sum of the initial encoded image buffer removal delay and the initial encoded image buffer removal offset is constant across two or more buffer cycles.

[0238] In an embodiment, if the single indication does not indicate that the sum of the initial encoded image buffer removal delay and the initial encoded image buffer removal offset is defined as constant across two or more buffer cycles, then the video encoder 100 may, for example, be configured to generate a video data stream such that the video data stream may, for example, include continuously updated information about the initial encoded image buffer removal delay and continuously updated information about the initial encoded image buffer removal offset.

[0239] According to an embodiment, if the video data stream includes, for example, information that may indicate that the sum of the initial encoded image buffer removal delay and the initial encoded image buffer removal offset is defined as constant across two or more buffer cycles, then the sum of the initial encoded image buffer removal delay and the initial encoded image buffer removal offset is defined as constant from the current position within the video data stream.

[0240] Furthermore, an apparatus 200 is provided for receiving two input video data streams (as a first input video data stream and a second input video data stream). Each of the two input video data streams contains an input video. The apparatus 200 is configured to generate an output video data stream from the two input video data streams, the output video data stream encoding the output video, wherein the apparatus is configured to generate the output video data stream by concatenating the first input video data stream and the second input video data stream. Furthermore, the apparatus 200 is configured to generate the output video data stream such that the output video data stream includes an initial encoded image buffer removal delay. Furthermore, the apparatus 200 is configured to generate the output video data stream such that the output video data stream includes an initial encoded image buffer removal offset. Furthermore, the apparatus 200 is configured to generate the output video data stream such that the output video data stream includes information indicating whether the sum of the initial encoded image buffer removal delay and the initial encoded image buffer removal offset is defined as constant across two or more buffer cycles.

[0241] According to an embodiment, the initial encoded image buffer removal delay may, for example, indicate the time required for the first access unit to elapse before sending the first access unit of the image that initializes the output video data stream of the video decoder 300 to the video decoder 300.

[0242] In an embodiment, the apparatus 200 may, for example, be configured to generate an output video data stream such that the output video data stream may, for example, include a single indication, which may indicate whether, for example, the sum of the initial encoded image buffer removal delay and the initial encoded image buffer removal offset can be defined as constant across two or more buffer cycles.

[0243] According to an embodiment, the apparatus 200 may, for example, be configured to: generate an output video data stream such that the output video data stream may include, for example, a concatenation flag as a single indication, which may indicate, for example, whether the sum of the initial encoded image buffer removal delay and the initial encoded image buffer removal offset can be defined as constant across two or more buffer cycles. If the concatenation flag is equal to a first value, then the sum of the initial encoded image buffer removal delay and the initial encoded image buffer removal offset is constant across two or more buffer cycles. If the concatenation flag is different from the first value, then the concatenation flag does not define whether the sum of the initial encoded image buffer removal delay and the initial encoded image buffer removal offset is constant across two or more buffer cycles.

[0244] In an embodiment, if the single indication does not indicate that the sum of the initial encoded image buffer removal delay and the initial encoded image buffer removal offset is defined as constant across two or more buffer cycles, then the apparatus 200 is configured to: generate an output video data stream such that the output video data stream includes continuously updated information regarding the initial encoded image buffer removal delay and continuously updated information regarding the initial encoded image buffer removal offset.

[0245] According to an embodiment, if the video data stream includes information indicating that the sum of the initial encoded image buffer removal delay and the initial encoded image buffer removal offset is defined as constant across two or more buffering cycles, then the sum of the initial encoded image buffer removal delay and the initial encoded image buffer removal offset is defined as constant starting from the current position within the video data stream (page 25 / 41 of the specification, CN 121099071 A).

[0246] Furthermore, a video decoder 300 is provided for receiving a video data stream in which video is stored. The video decoder 300 is configured to decode video from the video data stream. Furthermore, the video data stream includes an initial encoded image buffer removal delay. Furthermore, the video data stream includes an initial encoded image buffer removal offset. Furthermore, the video data stream includes information indicating whether the sum of the initial encoded image buffer removal delay and the initial encoded image buffer removal offset is defined as constant across two or more buffering cycles. Furthermore, the video decoder 300 is configured to decode the video based on information indicating whether the sum of the initial encoded image buffer removal delay and the initial encoded image buffer removal offset is defined as constant across two or more buffer cycles.

[0247] According to an embodiment, the initial encoded image buffer removal delay may, for example, indicate the time required for the first access unit to elapse before sending the first access unit of the image that initializes the output video data stream of the video decoder 300 to the video decoder 300.

[0248] In an embodiment, the video data stream may, for example, include a single indication, which may indicate whether the sum of the initial encoded image buffer removal delay and the initial encoded image buffer removal offset can, for example, be defined as constant across two or more buffering cycles. The video decoder 300 may, for example, be configured to decode the video according to this single indication.

[0249] According to an embodiment, the video data stream may, for example, include a concatenation flag as a single indication, which may, for example, indicate whether the sum of the initial encoded image buffer removal delay and the initial encoded image buffer removal offset can, for example, be defined as constant across two or more buffering cycles. If the concatenation flag is equal to a first value, then the sum of the initial encoded image buffer removal delay and the initial encoded image buffer removal offset is constant across two or more buffering cycles. If the concatenation flag is different from the first value, then the concatenation flag does not define whether the sum of the initial encoded image buffer removal delay and the initial encoded image buffer removal offset is constant across two or more buffering cycles. The video decoder 300 is configured to decode the video according to this concatenation flag.

[0250] In an embodiment, if the single indication does not indicate that the sum of the initial encoded image buffer removal delay and the initial encoded image buffer removal offset is defined as constant across two or more buffering cycles, then the video data stream includes continuously updated information regarding the initial encoded image buffer removal delay and continuously updated information regarding the initial encoded image buffer removal offset. The video decoder 300 is configured to decode the video based on the continuously updated information regarding the initial encoded image buffer removal delay and the continuously updated information regarding the initial encoded image buffer removal offset.

[0251] According to an embodiment, if the video data stream includes information indicating that the sum of the initial encoded image buffer removal delay and the initial encoded image buffer removal offset is defined as constant across two or more buffering cycles, then the sum of the initial encoded image buffer removal delay and the initial encoded image buffer removal offset is defined as constant from the current position within the video data stream.

[0252] Furthermore, a system is provided. The system includes the apparatus 200 as described above and the video decoder 300 as described above. The video decoder 300 is configured to receive the output video data stream of the apparatus 200. Furthermore, the video decoder 300 is configured to decode video from the output video data stream of the device 200.

[0253] According to an embodiment, the system may also include, for example, a video encoder 100. The device 200 may be configured, for example, to receive a video data stream from the video encoder 100 as an input video data stream.

[0254] Specifically, the third aspect of the invention relates to splicing, initial Cpb removal delay, and initial Cpb removal offset.

[0255] Currently, the specification indicates that the sum of the initial Cpb removal delay and the initial Cpb removal offset is constant within the CVS. The same constraint is sometimes described for page 26 / 41 of the specification, 30 CN 121099071 A. This initial Cpb removal delay indicates the time required for the first AU in the bitstream used to initialize the decoder to be used for decoding. This initial Cpb removal offset is a property of the bitstream, meaning that the earliest arrival time of an AU in the decoder is not necessarily equidistant from the time 0 when the first AU arrives at the decoder. It helps determine when the first bit of an AU can arrive at the decoder earliest.

[0256] The current constraint in the VVC draft specification indicates that the sum of these two values ​​needs to be constant within the CVS:

[0257] For each pair of values ​​i and j across the entire CVS, the sum of nal_initial_cpb_removal_delay[i][j] and nal_initial_cpb_removal_offset[i][j] should be constant, and the sum of nal_initial_alt_cpb_removal_delay[i][j] and nal_initial_alt_cpb_removal_offset[i][j] should be constant.

[0258] Problems arise when editing or splicing bitstreams to form new joint bitstreams. It is also desirable to be able to indicate whether this property is satisfied across the CVS boundaries of the bitstream, as different values ​​of this sum can lead to buffer underloading or overflow.

[0259] Therefore, in an embodiment, an indication is carried in the bitstream, and starting from a point in the bitstream (e.g., a concatenation point), the constraint on the constant sum of InitCpbRemovalDelay and InitCpbRemovalDelayOffset (and alternative counterparts) is reset, and the sum before and after that point in the bitstream may differ. The sum remains constant unless the indication is present in the bitstream.

[0260] For example:

[0261] When concatenationFlag equals 0, the constraint on bitstream consistency is that the sum of InitCpbRemovalDelay and InitCpbRemovalDelayOffset is constant across buffer periods.

[0262] Otherwise, the sum of InitCpbRemovalDelay and InitCpbRemovalDelayOffset does not necessarily have to be constant across buffer periods. The values ​​of InitCpbRemovalDelay and InitCpbRemovalDelayOffset are updated to account for arrival times.

[0263] In an embodiment, if several bitstreams are concatenated, at each concatenation point, a concatenation flag may, for example, define whether the sum remains constant.

[0264] Hereinafter, a fourth aspect of the invention is described in detail.

[0265] According to a fourth aspect of the invention, a video data stream is provided. Video is encoded in the video data stream. Furthermore, the video data stream includes an indication (e.g., general_same_pic_timing_in_all_ols_flag) indicating whether a non-scalable nested picture timing supplementation enhancement information message of a network abstraction layer unit of a plurality of access units of an access unit in one or more encoded video sequences of the video data stream is defined as applicable to all output layer sets in a plurality of output layer sets of the access unit. If the indication (e.g., general_same_pic_timing_in_all_ols_flag) has a first value, then the non-scalable nested picture timing supplementation enhancement information message of the network abstraction layer unit of the access unit is defined as applicable to all output layer sets in a plurality of output layer sets of the access unit. If the indication (e.g., general_same_pic_timing_in_all_ols_flag) has a value different from the first value, then the indication does not define whether the non-scalable nested picture timing supplementary enhancement information message of the network abstraction layer unit of the access unit is applicable to all output layer sets in the multiple output layer sets of the access unit.

[0266] According to an embodiment, for example, if the indication (e.g., general_same_pic_timing_in_all_ols_flag) has a first value, then the network abstraction layer unit does not include any other supplementary enhancement information message different from the picture timing supplementary enhancement information message.

[0267] In an embodiment, for example, if the indication (e.g., general_same_pic_timing_in_all_ols_flag) has a first value, then the network abstraction layer unit does not include any other supplementary enhancement information message.

[0268] According to an embodiment, for example, if the indication (e.g., general_same_pic_timing_in_all_ols_flag) has a first value, then for each of the multiple access units of the encoded video sequence in one or more encoded video sequences, each network abstraction layer unit includes a non-scalable nested picture timing supplemental enhancement information message, the network abstraction layer unit does not include any other supplemental enhancement information message different from the picture timing supplemental enhancement information message, or does not include any other supplemental enhancement information message.

[0269] In an embodiment, if the indication (e.g., general_same_pic_timing_in_all_ols_flag) has a first value, then for each of the plurality of access units in one or more encoded video sequences of a video data stream, each network abstraction layer unit including a non-scalable nested picture timing supplemental enhancement information message, the network abstraction layer unit does not include any other supplemental enhancement information message different from the picture timing supplemental enhancement information message, or does not include any other supplemental enhancement information message.

[0270] Furthermore, for example, a video encoder 100 may be provided. The video encoder 100 is configured to encode video into a video data stream. Furthermore, the video encoder 100 is configured to generate a video data stream such that the video data stream includes an indication (e.g., general_same_pic_timing_in_all_ols_flag) indicating whether the non-scalable nested picture timing supplemental enhancement information message of the network abstraction layer unit of the access unit in one or more encoded video sequences of the video data stream is defined as being applied to all output layer sets in a plurality of output layer sets of the access unit. If the indicator (e.g., general_same_pic_timing_in_all_ols_flag) has a first value, then the non-scalable nested picture timing supplementary enhancement message of the network abstraction layer unit of the access unit is defined as applicable to all output layer sets in the plurality of output layer sets of the access unit. If the indicator (e.g., general_same_pic_timing_in_all_ols_flag) has a value different from the first value, then the indicator does not define whether the non-scalable nested picture timing supplementary enhancement message of the network abstraction layer unit of the access unit is applicable to all output layer sets in the plurality of output layer sets of the access unit.

[0271] According to an embodiment, for example, if the indicator (e.g., general_same_pic_timing_in_all_ols_flag) has a first value, then the video encoder 100 is configured to generate a video data stream such that the network abstraction layer unit does not include any other supplementary enhancement message different from the picture timing supplementary enhancement message.

[0272] In an embodiment, for example, if the indication (e.g., general_same_pic_timing_in_all_ols_flag) has a first value, the video encoder 100 is configured to generate a video data stream such that the network abstraction layer unit does not include any other supplementary enhancement information messages.

[0273] According to an embodiment, for example, if the indication (e.g., general_same_pic_timing_in_all_ols_flag) has a first value, the video encoder 100 may be configured, for example, to generate a video data stream such that each network abstraction layer unit, for each of a plurality of access units of a encoded video sequence in one or more encoded video sequences, includes a non-scalable nested picture timing supplemental enhancement information message, the network abstraction layer unit not including any other supplemental enhancement information message different from the picture timing supplemental enhancement information message, or not including any other supplemental enhancement information message.

[0274] In an embodiment, for example, if the indication (e.g., general_same_pic_timing_in_all_ols_flag) has a first value, the video encoder 100 may be configured, for example, to generate a video data stream such that each of the plurality of access units of each of one or more encoded video sequences in the video data stream includes a non-scalable nested picture timing supplemental enhancement information message in each network abstraction layer unit, the network abstraction layer unit not including any other supplemental enhancement information message different from the picture timing supplemental enhancement information message, or not including any other supplemental enhancement information message as described on pages 28 / 41 of the specification 32 CN 121099071 A.

[0275] Furthermore, an apparatus 200 is provided for receiving an input video data stream. The input video data stream contains encoded video. The apparatus 200 is configured to generate a processed video data stream from the input video data stream. Furthermore, the apparatus 200 is configured to generate a processed video data stream such that the processed video data stream includes an indication (e.g., general_same_pic_timing_in_all_ols_flag) indicating whether a non-scalable nested picture timing supplementation enhancement message of a network abstraction layer unit of a plurality of access units of an access unit in one or more coded video sequences of the processed video data stream is defined as applicable to all output layer sets in a plurality of output layer sets of the access unit. If the indication (e.g., general_same_pic_timing_in_all_ols_flag) has a first value, then the non-scalable nested picture timing supplementation enhancement message of the network abstraction layer unit of the access unit is defined as applicable to all output layer sets in a plurality of output layer sets of the access unit.If the indication (e.g., general_same_pic_timing_in_all_ols_flag) has a value different from the first value, then the indication does not define whether the non-scalable nested picture timing supplementary enhancement information message of the network abstraction layer unit of the access unit is applicable to all output layer sets in the multiple output layer sets of the access unit.

[0276] According to an embodiment, for example, if the indication (e.g., general_same_pic_timing_in_all_ols_flag) has a first value, then the apparatus 200 is configured to generate a processed video data stream such that the network abstraction layer unit does not include any other supplementary enhancement information messages different from the picture timing supplementary enhancement information message.

[0277] In an embodiment, for example, if the indication (e.g., general_same_pic_timing_in_all_ols_flag) has a first value, then the apparatus 200 is configured to generate a processed video data stream such that the network abstraction layer unit does not include any other supplementary enhancement information messages.

[0278] According to an embodiment, for example, if the indication (e.g., general_same_pic_timing_in_all_ols_flag) has a first value, then the apparatus 200 may be configured, for example, to generate a processed video data stream such that for each of the plurality of access units of the encoded video sequences in one or more encoded video sequences, each network abstraction layer unit includes a non-scalable nested picture timing supplemental enhancement information message, the network abstraction layer unit does not include any other supplemental enhancement information message different from the picture timing supplemental enhancement information message, or does not include any other supplemental enhancement information message.

[0279] In an embodiment, for example, if the indication (e.g., general_same_pic_timing_in_all_ols_flag) has a first value, then the apparatus 200 may be configured, for example, to generate a processed video data stream such that for each of the plurality of access units of each of the processed video data streams in one or more encoded video sequences, each network abstraction layer unit includes a non-scalable nested picture timing supplemental enhancement information message, the network abstraction layer unit does not include any other supplemental enhancement information message different from the picture timing supplemental enhancement information message, or does not include any other supplemental enhancement information message.

[0280] In addition, a video decoder 300 is provided for receiving a video data stream in which video is stored. The video decoder 300 is configured to decode video from the video data stream.The video data stream includes an indication (e.g., general_same_pic_timing_in_all_ols_flag) that indicates whether a non-scalable nested picture timing supplementation enhancement message for a network abstraction layer unit of a network abstraction layer unit of a plurality of access units in one or more coded video sequences of the video data stream is defined as applicable to all output layer sets in a plurality of output layer sets of the access unit. If the indication (e.g., general_same_pic_timing_in_all_ols_flag) has a first value, then the non-scalable nested picture timing supplementation enhancement message for the network abstraction layer unit of the access unit is defined as applicable to all output layer sets in a plurality of output layer sets of the access unit. If the indication (e.g., general_same_pic_timing_in_all_ols_flag, page 29 / 41, CN 121099071 A) has a value different from the first value, then the indication does not define whether the non-scalable nested picture timing supplementation enhancement message for the network abstraction layer unit of the access unit is applicable to all output layer sets in a plurality of output layer sets of the access unit. The video decoder 300 is configured to decode the video according to the instruction.

[0281] According to an embodiment, for example, if the instruction (e.g., general_same_pic_timing_in_all_ols_flag) has a first value, the network abstraction layer unit does not include any other supplementary enhancement information messages different from the picture timing supplementary enhancement information message.

[0282] In an embodiment, for example, if the instruction (e.g., general_same_pic_timing_in_all_ols_flag) has a first value, the network abstraction layer unit does not include any other supplementary enhancement information messages. The video decoder 300 is configured to decode the video according to the instruction.

[0283] According to an embodiment, for example, if the indication (e.g., general_same_pic_timing_in_all_ols_flag) has a first value, then for each of the multiple access units of the encoded video sequence in one or more encoded video sequences, each network abstraction layer unit includes a non-scalable nested picture timing supplemental enhancement information message, the network abstraction layer unit does not include any other supplemental enhancement information message different from the picture timing supplemental enhancement information message, or does not include any other supplemental enhancement information message.

[0284] In an embodiment, for example, if the indication (e.g., general_same_pic_timing_in_all_ols_flag) has a first value, then for each of the plurality of access units in one or more encoded video sequences of a video data stream, each network abstraction layer unit including a non-scalable nested picture timing supplemental enhancement information message, the network abstraction layer unit does not include any other supplemental enhancement information message different from the picture timing supplemental enhancement information message, or does not include any other supplemental enhancement information message.

[0285] Furthermore, a system is provided. The system includes the apparatus 200 as described above and the video decoder 300 as described above. The video decoder 300 is configured to receive a processed video data stream from the apparatus 200. Furthermore, the video decoder 300 is configured to decode video from the output video data stream of the apparatus 200.

[0286] According to an embodiment, the system may also include, for example, a video encoder 100. The apparatus 200 may be configured, for example, to receive a video data stream from the video encoder 100 as an input video data stream.

[0287] Specifically, the fourth aspect of the invention relates to constraining the PT SEI from being paired with other HRD SEIs when general_same_pic_timing_in_all_ols_flag equals 1.

[0288] The VVC draft specification includes a flag called general_same_pic_timing_in_all_ols_flag in the general HRD parameter structure, the semantics of which are as follows:

[0289] general_same_pic_timing_in_all_ols_flag equal to 1 indicates that non-scalable nested PT SEI messages in each AU are applicable to AUs of any OLS in the bitstream and that no scalable nested PT SEI messages exist. general_same_pic_timing_in_all_ols_flag equal to 0 indicates that non-scalable nested PT SEI messages in each AU may or may not be applicable to AUs of any OLS in the bitstream and that scalable nested PT SEI messages may exist.

[0290] Typically, when extracting an OLS sub-bitstream from the original bitstream (including OLS data plus non-OLS data), the corresponding HRD-related timing / buffer information of the target OLS, which is encapsulated in the form of a so-called scalable nested SEI message containing buffer periods, picture timing, and decoding unit information SEI messages, is decapsulated. This decapsulated SEI message is then used to replace the non-scalable nested HRD SEI information in the original bitstream. However, in many scenarios, the content of some messages (e.g., picture timing SEI messages) can remain unchanged when the layer is discarded, i.e., from one OLS to its subset.Therefore, the general_same_pic_timing_in_all_ols_flag provides a shortcut to replace only the BP and DUI SEI messages, but the PT SEI in the original bitstream can remain valid, i.e., when general_same_pic_timing_in_all_ols_flag equals 1, the PT SEI is not removed during extraction. Therefore, it is not necessary to encapsulate the replacement PT SEI message in the scalable nested SEI message carrying the replacement BP and DUI SEI messages, and no bit rate overhead is introduced for this information.

[0291] However, in the prior art, the PT SEI message is allowed to be carried together with other HRD SEI messages in a single SEI NAL unit (NAL unit = Network Abstraction Layer Unit), i.e., the BP, PT and SEI messages can all be encapsulated in the same prefix SEI NAL unit. Therefore, the extractor would have to perform a more in-depth examination of such SEI NAL units to understand the included messages, and would actually need to rewrite the explicit SEI NAL units (i.e., remove non-PT SEI messages) when only one of the included messages (PT) is retained during the extraction process. To avoid this cumbersome low-level processing and to allow the extractor to operate entirely at the NAL unit level on the non-parametric set portion of the bitstream, a bitstream constraint prohibiting such bitstream construction is part of this invention. In one embodiment, this constraint is stated as follows:

[0292] general_same_pic_timing_in_all_ols_flag equal to 1 indicates that non-scalable nested PT SEI messages in each AU are applicable to any AU of OLS in the bitstream and that no scalable nested PT SEI messages exist. general_same_pic_timing_in_all_ols_flag equal to 0 indicates that non-scalable nested PT SEI messages in each AU may or may not be applicable to any AU of OLS in the bitstream and that scalable nested PT SEI messages may exist. When general_same_pic_timing_in_all_ols_flag equals 1, the constraint on bitstream consistency is: all general SEI messages in the bitstream that contain a payload_type equal to 1 (picture timing) should not contain SEI messages that contain a payload_type not equal to 1.

[0293] The fifth aspect of the invention is described in detail below.

[0294] According to the fifth aspect of the invention, a video data stream is provided. Video is encoded in the video data stream. Furthermore, the video data stream includes one or more scalable nested supplemental enhancement information messages.The one or more scalable nested supplemental enhancement messages include multiple syntax elements. Each of the multiple syntax elements is defined to have the same size in each scalable nested supplemental enhancement message in the video data stream or a portion of the video data stream.

[0295] According to an embodiment, the video data stream may, for example, include one or more non-scalable nested supplemental enhancement messages. The one or more scalable nested supplemental enhancement messages and the one or more non-scalable nested supplemental enhancement messages include multiple syntax elements. Each of the multiple syntax elements is defined to have the same size in each scalable nested supplemental enhancement message in the video data stream or a portion of the video data stream, and in each non-scalable nested supplemental enhancement message in the video data stream or a portion of the video data stream.

[0296] In an embodiment, the video data stream may, for example, include multiple access units, wherein each of the multiple access units may, for example, be assigned to one of a plurality of pictures of the video. The portion of the video data stream may, for example, be an access unit among the multiple access units of the video data stream. Each of the plurality of syntax elements may, for example, be defined as having the same size in each of the scalable nested supplementary enhancement information messages in the scalable nested supplementary enhancement information messages of the access unit.

[0297] According to an embodiment, the video data stream may, for example, include one or more non-scalable nested supplementary enhancement information messages. The one or more scalable nested supplementary enhancement information messages and the one or more non-scalable nested supplementary enhancement information messages include a plurality of syntax elements. Each of the plurality of syntax elements may, for example, be defined as having the same size in each of the scalable nested supplementary enhancement information messages in the scalable nested supplementary enhancement information messages of the access unit, and in each of the non-scalable nested supplementary enhancement information messages in the non-scalable nested supplementary enhancement information messages of the access unit.

[0298] In an embodiment, this portion of the video data stream may, for example, be an encoded video sequence of the video data stream. Each of the plurality of syntax elements may, for example, be defined as having the same size in each of the scalable nested supplementary enhancement information messages in the scalable nested supplementary enhancement information messages of the encoded video sequence.

[0299] According to an embodiment, the video data stream may include, for example, one or more non-scalable nested supplemental enhancement information messages.The one or more scalable nested supplemental enhancement messages and the one or more non-scalable nested supplemental enhancement messages include a plurality of syntax elements. Each of the one or more syntax elements may be defined, for example, as having the same size in each scalable nested supplemental enhancement message of the encoded video sequence and in each non-scalable nested supplemental enhancement message of the encoded video sequence.

[0300] In an embodiment, each of the one or more syntax elements may be defined, for example, as having the same size in each scalable nested supplemental enhancement message of the video data stream.

[0301] According to an embodiment, each of the one or more syntax elements may be defined, for example, as having the same size in each scalable nested supplemental enhancement message of the video data stream and in each non-scalable nested supplemental enhancement message of the video data stream.

[0302] In an embodiment, the video data stream or a portion thereof may include, for example, at least one buffered periodicity enhancement message, wherein the buffered periodicity enhancement message defines the size of each of one or more syntax elements among a plurality of syntax elements.

[0303] According to an embodiment, the buffered periodicity enhancement message includes at least one of the following for defining the size of each of one or more syntax elements among a plurality of syntax elements:

[0304] bp_cpb_initial_removal_delay_length_minus1 element,

[0305] bp_cpb_removal_delay_length_minus1 element,

[0306] bp_dpb_output_delay_length_minus1 element,

[0307] bp_du_cpb_removal_delay_increment_length_minus1 element,

[0308] bp_dpb_output_delay_du_length_minus1 element.

[0309] In an embodiment, for each of the multiple access units of a video data stream that includes a scalable nested buffer periodic supplementation enhancement information message, the access unit may also include, for example, a non-scalable nested buffer periodic supplementation enhancement information message.

[0310] According to an embodiment, for each of the multiple single-level access units of a video data stream that includes a scalable nested buffer periodic supplemental enhancement information message, the single-level access unit may, for example, also include a non-scalable nested buffer periodic supplemental enhancement information message.

[0311] Furthermore, a video encoder 100 is provided. The video encoder 100 is configured to encode video into a video data stream. Furthermore, the video encoder 100 is configured to: generate a video data stream such that the video data stream includes one or more scalable nested supplemental enhancement information messages. Furthermore, the video encoder 100 is configured to: generate a video data stream such that one or more scalable nested supplemental enhancement information messages include multiple syntax elements. Furthermore, the video encoder 100 is configured to: generate a video data stream such that each syntax element of one or more of the multiple syntax elements is defined to have the same size in each scalable nested supplemental enhancement information message in the scalable nested supplemental enhancement information message of the video data stream or a part of the video data stream.

[0312] According to an embodiment, the video encoder 100 may be configured, for example, to generate a video data stream such that the video data stream may include, for example, one or more non-scalable nested supplemental enhancement messages. The video encoder 100 may be configured, for example, to generate a video data stream such that one or more scalable nested supplemental enhancement messages and one or more non-scalable nested supplemental enhancement messages include multiple syntax elements. The video encoder 100 may be configured, for example, to generate a video data stream such that each syntax element of one or more of the multiple syntax elements may be defined, for example, as having the same size in each scalable nested supplemental enhancement message in the video data stream or a portion thereof, and in each non-scalable nested supplemental enhancement message in the video data stream or a portion thereof.

[0313] In an embodiment, the video encoder 100 may be configured, for example, to generate a video data stream such that the video data stream may include, for example, multiple access units, wherein each of the multiple access units may be assigned to one of a plurality of pictures of the video. The portion of the video data stream may, for example, be an access unit among the multiple access units of the video data stream. The video encoder 100 can be configured, for example, to generate a video data stream such that each of one or more syntax elements among a plurality of syntax elements can be defined, for example, to have the same size in each of the scalable nested supplementary enhancement messages in the access unit.

[0314] According to an embodiment, the video encoder 100 may be configured, for example, to generate a video data stream such that the video data stream may include, for example, one or more non-scalable nested supplemental enhancement information messages. The video encoder 100 may be configured, for example, to generate a video data stream such that one or more scalable nested supplemental enhancement information messages and one or more non-scalable nested supplemental enhancement information messages include a plurality of syntax elements. The video encoder 100 may be configured, for example, to generate a video data stream such that each syntax element of one or more of the plurality of syntax elements may be defined, for example, as having the same size in each scalable nested supplemental enhancement information message of the access unit and in each non-scalable nested supplemental enhancement information message of the access unit.

[0315] In an embodiment, this portion of the video data stream may, for example, be an encoded video sequence of the video data stream. The video encoder 100 may be configured, for example, to generate a video data stream such that each syntax element of one or more of the plurality of syntax elements may be defined, for example, as having the same size in each scalable nested supplemental enhancement information message of the encoded video sequence.

[0316] According to an embodiment, the video encoder 100 may be configured, for example, to generate a video data stream such that the video data stream may include, for example, one or more non-scalable nested supplemental enhancement information messages. The video encoder 100 may be configured, for example, to generate a video data stream such that one or more scalable nested supplemental enhancement information messages and one or more non-scalable nested supplemental enhancement information messages include a plurality of syntax elements. The video encoder 100 may be configured, for example, to generate a video data stream such that each syntax element of one or more of the plurality of syntax elements may be defined, for example, as having the same size in each scalable nested supplemental enhancement information message of the encoded video sequence and in each non-scalable nested supplemental enhancement information message of the encoded video sequence.

[0317] In an embodiment, the video encoder 100 may be configured, for example, to generate a video data stream such that each syntax element of one or more of the plurality of syntax elements may be defined, for example, as having the same size in each scalable nested supplemental enhancement information message of the video data stream.Specification page 33 / 41 37 CN 121099071 A

[0318] According to an embodiment, the video encoder 100 may be configured, for example, to generate a video data stream such that each syntax element of one or more syntax elements among a plurality of syntax elements may be defined, for example, as having the same size in each scalable nested supplementary enhancement information message in the scalable nested supplementary enhancement information message of the video data stream and in each non-scalable nested supplementary enhancement information message in the non-scalable nested supplementary enhancement information message of the video data stream.

[0319] In an embodiment, the video encoder 100 may be configured, for example, to generate a video data stream such that the video data stream or a portion thereof may include, for example, at least one buffered periodic supplementary enhancement information message, wherein the buffered periodic supplementary enhancement information message defines the size of each syntax element of one or more syntax elements among a plurality of syntax elements.

[0320] According to an embodiment, the video encoder 100 may be configured, for example, to generate a video data stream such that the buffer period supplement enhancement information message includes at least one of the following for defining the size of each of one or more syntax elements among a plurality of syntax elements:

[0321] bp_cpb_initial_removal_delay_length_minus1 element,

[0322] bp_cpb_removal_delay_length_minus1 element,

[0323] bp_dpb_output_delay_length_minus1 element,

[0324] bp_du_cpb_removal_delay_increment_length_minus1 element,

[0325] bp_dpb_output_delay_du_length_minus1 element.

[0326] In an embodiment, the video encoder 100 may be configured, for example, to generate a video data stream such that each of a plurality of access units of the video data stream includes a scalable nested buffer periodic supplementation enhancement information message, the access unit further including, for example, a non-scalable nested buffer periodic supplementation enhancement information message.

[0327] According to an embodiment, the video encoder 100 may be configured, for example, to generate a video data stream such that each of a plurality of single-level access units of the video data stream includes a scalable nested buffer periodic supplementation enhancement information message, the single-level access unit further including, for example, a non-scalable nested buffer periodic supplementation enhancement information message.

[0328] Furthermore, an apparatus 200 for receiving an input video data stream is provided. The input video data stream contains encoded video. The apparatus 200 is configured to generate an output video data stream from the input video data stream.The video data stream includes one or more scalable nested supplemental enhancement messages. Each of the one or more scalable nested supplemental enhancement messages includes multiple syntax elements. Each syntax element of one or more of the multiple syntax elements is defined to have the same size in each scalable nested supplemental enhancement message in the video data stream or a portion thereof. The apparatus 200 is configured to process one or more scalable nested supplemental enhancement messages.

[0329] According to an embodiment, the video data stream may, for example, include one or more non-scalable nested supplemental enhancement messages. Both the one or more scalable nested supplemental enhancement messages and the one or more non-scalable nested supplemental enhancement messages include multiple syntax elements. Each syntax element of one or more of the multiple syntax elements is defined to have the same size in each scalable nested supplemental enhancement message in the video data stream or a portion thereof, and in each non-scalable nested supplemental enhancement message in the video data stream or a portion thereof. The apparatus 200 is configured to process one or more scalable nested supplemental enhancement messages and one or more non-scalable nested supplemental enhancement messages.

[0330] In an embodiment, the video data stream may, for example, include a plurality of access units, wherein each of the plurality of access units may, for example, be assigned to one of a plurality of images of the video. This portion of the video data stream may, for example, be an access unit among the plurality of access units of the video data stream. Each of one or more of the plurality of syntax elements may, for example, be defined as having the same size in each of the scalable nested supplementary enhancement information messages in the scalable nested supplementary enhancement information messages of the access unit.

[0331] According to an embodiment, the video data stream may, for example, include one or more non-scalable nested supplementary enhancement information messages. The one or more scalable nested supplementary enhancement information messages and the one or more non-scalable nested supplementary enhancement information messages include a plurality of syntax elements. Each of one or more of the plurality of syntax elements may, for example, be defined as having the same size in each of the scalable nested supplementary enhancement information messages in the scalable nested supplementary enhancement information messages of the access unit and in each of the non-scalable nested supplementary enhancement information messages in the non-scalable nested supplementary enhancement information messages of the access unit. The device 200 may be configured, for example, to process one or more scalable nested supplemental enhancement messages and one or more non-scalable nested supplemental enhancement messages.

[0332] In an embodiment, the portion of the video data stream may, for example, be an encoded video sequence of the video data stream. Each of the plurality of syntax elements may, for example, be defined as having the same size in each scalable nested supplementary enhancement message in the scalable nested supplementary enhancement message of the encoded video sequence.

[0333] According to an embodiment, the video data stream may, for example, include one or more non-scalable nested supplementary enhancement messages. The one or more scalable nested supplementary enhancement messages and the one or more non-scalable nested supplementary enhancement messages include a plurality of syntax elements. Each of the plurality of syntax elements may, for example, be defined as having the same size in each scalable nested supplementary enhancement message in the scalable nested supplementary enhancement message of the encoded video sequence and in each non-scalable nested supplementary enhancement message of the non-scalable nested supplementary enhancement message of the encoded video sequence. The apparatus 200 may, for example, be configured to process one or more scalable nested supplementary enhancement messages and one or more non-scalable nested supplementary enhancement messages.

[0334] In an embodiment, each of the plurality of syntax elements may, for example, be defined as having the same size in each of the scalable nested supplementary enhancement information messages in the scalable nested supplementary enhancement information messages of the video data stream.

[0335] According to an embodiment, each of the plurality of syntax elements may, for example, be defined as having the same size in each of the scalable nested supplementary enhancement information messages in the scalable nested supplementary enhancement information messages of the video data stream and in each of the non-scalable nested supplementary enhancement information messages in the non-scalable nested supplementary enhancement information messages of the video data stream. The apparatus 200 may, for example, be configured to process one or more scalable nested supplementary enhancement information messages and one or more non-scalable nested supplementary enhancement information messages.

[0336] In an embodiment, the video data stream or the portion thereof may, for example, include at least one buffered periodic supplementary enhancement information message, wherein the buffered periodic supplementary enhancement information message defines the size of one or more of the plurality of syntax elements. The apparatus 200 may, for example, be configured to process at least one buffered periodic supplementary enhancement information message.

[0337] According to an embodiment, the buffer periodicity enhancement message includes at least one of the following for defining the size of one or more syntax elements among a plurality of syntax elements:

[0338] bp_cpb_initial_removal_delay_length_minus1 element,

[0339] bp_cpb_removal_delay_length_minus1 element,

[0340] bp_dpb_output_delay_length_minus1 element,

[0341] bp_du_cpb_removal_delay_increment_length_minus1 element, Specification 35 / 41 pages 39 CN 121099071 A

[0342] bp_dpb_output_delay_du_length_minus1 element.

[0343] In an embodiment, for each of the plurality of access units of a video data stream that includes a scalable nested buffer periodicity enhancement message, the access unit may also include, for example, a non-scalable nested buffer periodicity enhancement message. The apparatus 200 may, for example, be configured to process scalable nested supplemental enhancement information messages and non-scalable nested supplemental enhancement information messages.

[0344] According to an embodiment, for each of a plurality of single-level access units of a video data stream that includes a scalable nested buffer periodic supplemental enhancement information message, the single-level access unit may, for example, also include a non-scalable nested buffer periodic supplemental enhancement information message. The apparatus 200 may, for example, be configured to process scalable nested supplemental enhancement information messages and non-scalable nested supplemental enhancement information messages.

[0345] Furthermore, a video decoder 300 is provided for receiving a video data stream in which video is stored. The video decoder 300 is configured to decode video from the video data stream. The video data stream includes one or more scalable nested supplemental enhancement information messages. The one or more scalable nested supplemental enhancement information messages include a plurality of syntax elements. Each of the one or more syntax elements of the plurality of syntax elements is defined to have the same size in each scalable nested supplemental enhancement information message in the video data stream or a portion thereof. The video decoder 300 is configured to decode the video based on one or more syntax elements of the plurality of syntax elements.

[0346] According to an embodiment, the video data stream may, for example, include one or more non-scalable nested supplemental enhancement messages. The one or more scalable nested supplemental enhancement messages and the one or more non-scalable nested supplemental enhancement messages each include multiple syntax elements.Each of the plurality of syntax elements may, for example, be defined as having the same size in each scalable nested supplementary enhancement message in the scalable nested supplementary enhancement message of the video data stream or the portion thereof, and in each non-scalable nested supplementary enhancement message of the video data stream or the portion thereof.

[0347] In an embodiment, the video data stream may, for example, include a plurality of access units, wherein each of the plurality of access units may, for example, be assigned to one of a plurality of pictures of the video. The portion of the video data stream may, for example, be an access unit among the plurality of access units of the video data stream. Each of the plurality of syntax elements may, for example, be defined as having the same size in each scalable nested supplementary enhancement message of the access unit.

[0348] According to an embodiment, the video data stream may, for example, include one or more non-scalable nested supplementary enhancement messages. The one or more scalable nested supplementary enhancement messages and the one or more non-scalable nested supplementary enhancement messages include a plurality of syntax elements. Each of the plurality of syntax elements may, for example, be defined as having the same size in each scalable nested supplementary enhancement information message in the scalable nested supplementary enhancement information message of the access unit and in each non-scalable nested supplementary enhancement information message of the access unit.

[0349] In an embodiment, the portion of the video data stream may, for example, be an encoded video sequence of the video data stream. Each of the plurality of syntax elements may, for example, be defined as having the same size in each scalable nested supplementary enhancement information message of the encoded video sequence.

[0350] According to an embodiment, the video data stream may, for example, include one or more non-scalable nested supplementary enhancement information messages. The one or more scalable nested supplementary enhancement information messages and the one or more non-scalable nested supplementary enhancement information messages include a plurality of syntax elements. Each of the multiple syntax elements can be defined, for example, as having the same size in each scalable nested supplemental enhancement message in the scalable nested supplemental enhancement message of the encoded video sequence, and in each non-scalable nested supplemental enhancement message of the scalable video sequence.

[0351] In an embodiment, each of the plurality of syntax elements may, for example, be defined as having the same size in each of the scalable nested supplementary enhancement information messages in the scalable nested supplementary enhancement information messages of the video data stream.

[0352] According to an embodiment, each of the plurality of syntax elements may, for example, be defined as having the same size in each of the scalable nested supplementary enhancement information messages in the scalable nested supplementary enhancement information messages of the video data stream and in each of the non-scalable nested supplementary enhancement information messages in the non-scalable nested supplementary enhancement information messages of the video data stream.

[0353] In an embodiment, the video data stream or the portion thereof may, for example, include at least one buffered periodic supplementary enhancement information message, wherein the buffered periodic supplementary enhancement information message defines the size of each of the plurality of syntax elements.

[0354] According to an embodiment, the buffer periodicity enhancement information message includes at least one of the following for defining the size of each of one or more syntax elements among a plurality of syntax elements:

[0355] bp_cpb_initial_removal_delay_length_minus1 element,

[0356] bp_cpb_removal_delay_length_minus1 element,

[0357] bp_dpb_output_delay_length_minus1 element,

[0358] bp_du_cpb_removal_delay_increment_length_minus1 element,

[0359] bp_dpb_output_delay_du_length_minus1 element.

[0360] In an embodiment, for each of a plurality of access units of a video data stream that includes a scalable nested buffer periodicity enhancement information message, the access unit may, for example, also include a non-scalable nested buffer periodicity enhancement information message.

[0361] According to an embodiment, for each of the plurality of single-level access units of a video data stream that includes a scalable nested buffer periodic supplementation enhancement information message, the single-level access unit may, for example, also include a non-scalable nested buffer periodic supplementation enhancement information message.

[0362] Furthermore, a system is provided. The system includes the apparatus 200 as described above and the video decoder 300 as described above. The video decoder 300 is configured to receive the output video data stream of the apparatus 200. Furthermore, the video decoder 300 is configured to decode the video from the output video data stream of the apparatus 200.

[0363] According to an embodiment, the system may, for example, also include a video encoder 100.Apparatus 200 may, for example, be configured to receive a video data stream from video encoder 100 as an input video data stream.

[0364] Specifically, the fifth aspect of the invention relates to constraining all BP SEI messages in a bitstream to indicate certain variable-length encoded syntax elements of the same length and not scalable nested in the absence of non-scalable nested variants in the same AU.

[0365] Buffer period SEI messages, picture timing SEI messages, and decoding unit information SEI messages are used to provide precise timing information for NAL units within the bitstream to control their transition through the decoder's buffer during conformance testing. Some syntax elements in PT messages and DUI SEI messages are encoded with variable lengths, and the lengths of these syntax elements are transmitted in the BP SEI messages on pages 37 / 41 of the specification, CN 121099071 A. This resolution dependency is a design trade-off. The benefit of saving the transmission of these length syntax elements at each PT or DUI SEI message is realized at the cost of not allowing the PT and DUI SEI messages to be resolved without first resolving the associated BP SEI messages. Since BP SEI messages (once per multiple frames) are sent much less frequently than PT messages (once per frame) or DUI SEI messages (multiple times per frame), bit savings are achieved through this common design trade-off, similar to how the image header structure can reduce the bit cost of the slice header when using many slices.

[0366] More specifically, the BP SEI message in the current VVC draft specification includes the following syntax elements as the root of the resolution relevance:

[0367] ●bp_cpb_initial_removal_delay_length_minus1, which specifies the encoded length of the candidate initial CPB removal delay of the AU in the PT SEI message, and

[0368] ●bp_cpb_removal_delay_length_minus1, which specifies the encoded length of the CPB removal delay and removal delay offset of the AU in the PT SEI message, and

[0369] ●bp_dpb_output_delay_length_minus1, which specifies the encoded length of the DPB output delay of the AU in the PT SEI message, and

[0370] ●bp_du_cpb_removal_delay_increment_length_minus1, which specifies the encoded length of the individual CPB removal delay and common CPB removal delay of the DU in the PT SEI message and the CPB removal delay of the DU in the DUI SEI message, and

[0371] ●bp_dpb_output_delay_du_length_minus1, which specifies the encoded length of the DPB output delay of the AU in the PT SEI message and the DU SEI message.

[0372] However, problems arise when the bitstream contains multiple OLSs. Although the BP / PT / DUI SEI messages applicable to the OLS representing the bitstream are carried verbatim in the bitstream, tracking the parsing dependencies is trivial, and other pairs of BP / PT / DUI SEI messages corresponding to the OLS representing the (sub)bitstream will be carried in encapsulated form in so-called scalable nested SEI messages. Parsing dependencies still apply, and given that the number of OLSs can be very large, it is a considerable burden for the decoder or parser to track the correctly encapsulated BP SEI messages for parsing dependencies when processing encapsulated PT messages and DUI SEI messages. In particular, since these messages can also be encapsulated in different scalable nested SEI messages.

[0373] Therefore, as part of the present invention, in one embodiment, the following bitstream constraint is established: the encoded value of the corresponding syntax element describing the length must be the same in all scalable nested and non-scalable nested BP SEI messages in the AU. Thus, the decoder or parser only needs to store the corresponding length value when parsing the first non-scalable BP SEI message in the AU, and can resolve the parsing dependencies of all PT messages and DUI SEI messages within a buffer period starting at the corresponding AU, regardless of whether these messages are encapsulated in scalable nested SEI messages.The following is an example of the corresponding specification text:

[0374] The requirement for bitstream consistency is that all scalable nested and non-scalable nested buffered periodic SEI messages in the AU have the same corresponding values ​​for the syntax elements bp_cpb_initial_removal_delay_length_minus1, bp_cpb_removal_delay_length_minus1, bp_dpb_output_delay_length_minus1, bp_du_cpb_removal_delay_increment_length_minus1, and bp_dpb_output_delay_du_length_minus1.

[0375] In another embodiment, the constraint applies only to scalable nested BP SEI messages within the buffer period determined by the current non-scalable nested BP SEI message, as stated below:

[0376] The requirement for bitstream consistency is that all scalable nested buffer period SEI messages within the buffer period have the same corresponding values ​​for the syntax elements bp_cpb_initial_removal_delay_length_minus1, bp_cpb_removal_delay_length_minus1, bp_dpb_output_delay_length_minus1, bp_du_cpb_removal_delay_increment_length_minus1, bp_dpb_output_delay_du_length_minus1then, followed by the non-scalable nested buffer period SEI messages of the buffer period.

[0377] Here, the BP of the bitstream defines the constraint range of the scalable nested BP from one scalable nested BP to the next.

[0378] In another embodiment, the constraint applies to all AU representations of the bitstream, for example, as follows:

[0379] The requirement for bitstream consistency is that all scalable nested and non-scalable nested buffered periodic SEI messages in the bitstream have the same corresponding values ​​for the syntax elements bp_cpb_initial_removal_delay_length_minus1, bp_cpb_removal_delay_length_minus1, bp_dpb_output_delay_length_minus1, bp_du_cpb_removal_delay_increment_length_minus1, and bp_dpb_output_delay_du_length_minus1.

[0380] In another embodiment, the constraint applies only to AU representations in the CVS, so the smart encoder can still facilitate differences in the duration of BPs in the bitstream for encoding relevant delay and offset syntax elements. The specification text will be as follows:

[0381] The requirement for bitstream consistency is that all scalable nested and non-scalable nested buffered period SEI messages in CVS have the same corresponding values ​​for the syntax elements bp_cpb_initial_removal_delay_length_minus1, bp_cpb_removal_delay_length_minus1, bp_dpb_output_delay_length_minus1, bp_du_cpb_removal_delay_increment_length_minus1, and bp_dpb_output_delay_du_length_minus1.

[0382] Here, the constraint scope is CVS.

[0383] More specifically, a buffered period or BP SEI message defines a so-called buffered period in which the timing of each picture uses the picture at the beginning of the buffered period as an anchor. For example, the beginning of the buffered period helps to test the consistency of random access functionality in the bitstream.

[0384] Figure 7 shows two sets of HRD SEIs (scalable nested SEIs and non-scalable nested SEIs) in a two-layer bitstream according to an embodiment.

[0385] In the multi-layered scenario shown in Figure 7, for example, the scalable nested HRD SEI provides a different buffer cycle setting (through BP at POC 0 and POC 3) than the non-scalable nested SEI (POC 0 only) used when only layer L0 is extracted and played from POC 3.

[0386] However, this also increases the complexity cost of tracking the parsing correlation between PT and individual BP messages as described above, which is undesirable.Therefore, as part of the present invention, in one embodiment, scalable nested BP SEI messages are prohibited in AUs that do not contain non-scalable nested BP SEI messages, as follows:

[0387] The requirement for bitstream consistency is that scalable nested BP SEI messages should not be present in AUs that do not contain non-scalable nested BP SEI messages.

[0388] Since the above use case is limited to multi-level bitstreams, in another embodiment, the relevant constraints are limited to single-level bitstreams, as follows:

[0389] The requirement for bitstream consistency is that scalable nested BP SEI messages should not be present in single-level AUs that do not contain non-scalable nested BP SEI messages.

[0390] Although some aspects have been described in the context of apparatus, it will be clear that these aspects also represent descriptions of corresponding methods, wherein blocks or devices correspond to method steps or features of method steps. Similarly, aspects described in the context of method steps also represent descriptions of features of corresponding blocks or items or corresponding apparatuses. Some or all of the method steps may be performed by (or using) hardware devices (such as microprocessors, programmable computers, or electronic circuits). In some embodiments, such as page 39 / 41 of CN 121099071 A, one or more of the most important method steps can be performed by such an apparatus.

[0391] According to certain implementation requirements, embodiments of the invention can be implemented in hardware or software, or at least partially in hardware or at least partially in software. Implementation can be performed using a digital storage medium (e.g., floppy disk, DVD, Blu-ray, CD, ROM, PROM, EPROM, EEPROM, or flash memory) having electronically readable control signals stored thereon, which cooperate with (or are capable of cooperating with) a programmable computer system to perform the corresponding methods. Thus, the digital storage medium can be computer-readable.

[0392] Some embodiments of the invention include a data carrier having electronically readable control signals capable of cooperating with a programmable computer system to perform one of the methods described herein.

[0393] Generally, embodiments of the invention can be implemented as a computer program product having program code operable to perform one of the methods when the computer program product is run on a computer. The program code can be stored, for example, on a machine-readable medium.

[0394] Other embodiments include a computer program stored on a machine-readable medium for performing one of the methods described herein.

[0395] In other words, embodiments of the methods of the invention are therefore computer programs having program code for performing one of the methods described herein when the computer program is run on a computer.

[0396] Therefore, another embodiment of the method of the present invention is a data carrier (or digital storage medium or computer-readable medium) on which a computer program is recorded, the computer program being used to perform one of the methods described herein. The data carrier, digital storage medium, or recorded medium is generally tangible and / or non-transitory.

[0397] Therefore, another embodiment of the method of the present invention is a data stream or signal sequence representing a computer program being used to perform one of the methods described herein. The data stream or signal sequence may, for example, be configured to be transmitted via a data communication connection (e.g., via the Internet).

[0398] Another embodiment includes a processing means, such as a computer or programmable logic device, configured or adapted to perform one of the methods described herein.

[0399] Another embodiment includes a computer on which a computer program is installed, the computer program being used to perform one of the methods described herein.

[0400] According to another embodiment of the invention, an apparatus or system is configured to transmit a computer program to a receiver (e.g., electronically or optically), the computer program being used to perform one of the methods described herein. The receiver may be, for example, a computer, a mobile device, a storage device, etc. The apparatus or system may, for example, include a file server for transmitting computer programs to a receiver.

[0401] In some embodiments, a programmable logic device (e.g., a field-programmable gate array) may be used to perform some or all of the functions described herein. In some embodiments, the field-programmable gate array may cooperate with a microprocessor to perform one of the methods described herein. Generally, the methods are preferably performed by any hardware device.

[0402] The apparatus described herein may be implemented using hardware devices, or using a computer, or using a combination of hardware devices and a computer.

[0403] The methods described herein may be performed using hardware devices, or using a computer, or using a combination of hardware devices and a computer.

[0404] The above embodiments are merely illustrative of the principles of the invention. It should be understood that modifications and variations of the arrangements and details described herein will be apparent to others skilled in the art. Therefore, it is intended to limit the scope of the appended claims only and not to the specific details given by way of the description and explanation of the embodiments herein. (Pages 40 / 41, CN 121099071 A)

[0405] References

[0406] [1] ISO / IEC, ITU-T. High efficiency video coding. ITU-T Recommendation H.265 | ISO / IEC 23008 10 (HEVC), edition 1, 2013; edition 2, 2014.Instruction manual page 41 / 41 45 CN 121099071 A Figure 1 Instruction manual Figure 1 / 10 page 46 CN 121099071 A Figure 2 Instruction manual Figure 2 / 10 page 47 CN 121099071 A Figure 3 Instruction manual Figure 3 / 10 page 48 CN 121099071 A Figure 4 Instruction manual Figure 4 / 10 page 49 CN 121099071 A Figure 5 Instruction manual Figure 5 / 10 page 50 CN 121099071 A Figure 6 Instruction manual Figure 6 / 10 page 51 CN 121099071 A Figure 7 Instruction manual Figure 7 / 10 page 52 CN 121099071 A Figure 8 Instruction manual Figure 8 / 10 page 53 CN 121099071 A Figure 9 Instruction manual Figure 9 / 10 page 54 CN 121099071 A Figure 10 Description Figure 10 / 10 Page 55 CN 121099071 A Abstract An apparatus (200) for receiving an input video data stream according to an embodiment is provided. The input video data stream has a video encoded thereinto. The apparatus (200) is configured to generate an output video data stream from the input video data stream..

Claims

1. An apparatus for receiving one or more input video data streams, wherein, Each of the one or more input video data streams contains an input video encoded in it. The apparatus is configured to generate an output video data stream from the one or more input video data streams, the output video data stream encoding an output video, wherein the apparatus is configured to: generate the output video data stream such that the output video is the input video encoded within one of the one or more input video data streams, or such that the output video depends on the input video of at least one of the one or more input video data streams. The device is configured to: determine the time when the current image among a plurality of images of the output video is removed from the access unit of the encoded image buffer. The device is configured to: at least based on a concatenation flag, determine whether to use encoded image buffer delay offset information to determine the time when the current image is removed from the access unit of the encoded image buffer. The device is configured to determine whether to determine the access unit removal time of the current image based on the position of a previously non-discardable image in the video preceding the current image.

2. The apparatus according to claim 1, in, The apparatus is configured to: discard a group of one or more images of the input video of a first video data stream from the one or more input video data streams to generate the output video data stream. The device is configured to: determine, based on the encoded image buffer delay offset information, the removal time of at least one of the plurality of images of the output video from the access unit of the encoded image buffer.

3. The apparatus according to claim 1, in, The first video received by the device is a preprocessed video obtained from the original video, which was generated by discarding one or more images from the original video. The device is configured to: determine, based on the encoded image buffer delay offset information, the removal time of at least one of the plurality of images of the output video from the access unit of the encoded image buffer.

4. The apparatus according to claim 2, in, The buffer delay offset information depends on the number of images that have been discarded from the input video.

5. The apparatus according to claim 2, in, The one or more input video data streams are two or more input video data streams, and The apparatus is configured to concatenate the processed video with the input video of a second video data stream from the two or more input video data streams to obtain the output video, and the apparatus is configured to encode the output video into the output video data stream.

6. The apparatus according to claim 5, in, The device is configured to: determine, based on the position of the current image within the output video, whether to use encoded image buffer delay offset information to determine the access unit removal time of the current image, or The device is configured to: determine, based on the position of the current image within the output video, whether to set the encoded image buffer delay offset value of the encoded image buffer delay offset information to 0 to determine the access unit removal time of the current image.

7. The apparatus according to claim 1, in, The device is configured to determine the access unit removal time of the current image based on the removal time of the previous image.

8. The apparatus according to claim 1, in, The device is configured to determine the access unit removal time of the current image based on the initial encoded image buffer removal delay information.

9. The apparatus according to claim 8, in, The device is configured to update the initial encoded image buffer removal delay information according to a clock tick, so as to obtain temporary encoded image buffer removal delay information to determine the access unit removal time of the current image.

10. The apparatus according to claim 1, in, If the concatenation flag is set to a first value, the device is configured to use the encoded image buffer delay offset information to determine one or more removal times, and If the concatenation flag is set to a second value different from the first value, the device is configured to determine the one or more removal times without using the encoded image buffer delay offset information.

11. The apparatus according to claim 1, in, The device is configured to signal a video decoder whether to use encoded image buffer delay offset information to determine the time when the current image is removed from the access unit of the encoded image buffer.

12. The apparatus according to claim 11, in, The current image is located at the splicing point of the output video, where the two input videos are spliced ​​together at the splicing point.

13. A non-transitory digital storage medium for storing video data streams, in, The video data stream contains encoded video. The video data stream includes encoded image buffer delay offset information and concatenation flags. Specifically, determining whether to use the encoded image buffer delay offset information to determine the removal time of the image from the encoded image buffer depends at least on the concatenation flag. The video data stream includes information about the location of previously non-discardable images within the video preceding the current image, and this information can be used to determine the access unit removal time of the current image.

14. The non-transitory digital storage medium for storing video data streams according to claim 13, in, The video data stream includes initial encoded image buffer removal delay information.

15. The non-transitory digital storage medium for storing video data streams according to claim 13, in, If the concatenation flag is set to a first value, the concatenation flag indicates that the encoded image buffer delay offset information needs to be used to determine one or more removal times, and If the cascading flag is set to a second value different from the first value, the cascading flag indicates that the indicated offset is not used to determine the one or more removal times.

16. A video encoder, in, The video encoder is configured to encode video into a video data stream. The video encoder is configured to generate the video data stream, such that the video data stream includes encoded image buffer delay offset information and a concatenation flag. The video encoder is configured to: at least based on the concatenation flag, determine whether to use the encoded image buffer delay offset information to determine the removal time of the image from the encoded image buffer. The video encoder (100) is configured to generate the video data stream such that the video data stream includes information about the location of a previously non-discardable image within the video before the current image, the information being usable to determine the access unit removal time of the current image.

17. The video encoder according to claim 16, in, The video encoder is configured to generate the video data stream such that the video data stream includes initial encoded image buffer delay offset information.

18. The video encoder according to claim 16, in, If the concatenation flag is set to a first value, the concatenation flag indicates that the encoded image buffer delay offset information needs to be used to determine one or more removal times, and If the cascading flag is set to a second value different from the first value, the cascading flag indicates that the indicated offset is not used to determine the one or more removal times.

19. A video decoder for receiving a video data stream containing stored video data. in, The video decoder is configured to decode the video from the video data stream. The video decoder is configured to decode the video based on the removal time of the current image from the access unit of the encoded image buffer among a plurality of images in the video. The video decoder is configured to decode the video at least based on a concatenated flag indicating whether encoded image buffer delay offset information is used to determine the removal time of the current image from the access unit of the encoded image buffer. The video decoder (300) is configured to determine whether to determine the access unit removal time of the current image based on the position of a previously non-discardable image in the video preceding the current image.

20. The video decoder according to claim 19, in, The time at which at least one of the plurality of images in the video is removed from the access unit of the encoded image buffer depends on the delay offset information of the encoded image buffer.

21. The video decoder according to claim 19, in, The video decoder is configured to decode the video based on whether the encoded image buffer delay offset value of the encoded image buffer delay offset information is set to 0.

22. The video decoder according to claim 19, in, The video decoder is configured to determine the access unit removal time of the current image based on the removal time of the previous image.

23. The video decoder according to claim 19, in, The video decoder is configured to determine the access unit removal time of the current image based on the initial encoded image buffer removal delay information.

24. The video decoder according to claim 23, in, The video decoder is configured to update the initial encoded image buffer removal delay information according to the clock tick, so as to obtain the temporary encoded image buffer removal delay information to determine the access unit removal time of the current image.

25. The video decoder according to claim 19, in, If the concatenation flag is set to a first value, the video decoder is configured to use the encoded image buffer delay offset information to determine one or more removal times, and If the concatenation flag is set to a second value different from the first value, the video decoder is configured not to use the encoded image buffer delay offset information to determine the one or more removal times.

Images