Video data stream, video encoder, apparatus and methods for a hypothetical reference decoder and for output layer sets

Abstract

The invention relates to a video data stream, a video encoder, an apparatus and a method for a hypothetical reference decoder and for outputting a set of layers. An apparatus for receiving a video data stream as an input bitstream is provided. The video data stream has video encoded therein, where for a playback speed modification factor (Nx), means are configured to determine a decoding capability requirement in dependence on a decoding capability requirement limit, where the playback speed modification factor (Nx) is either a forward playback speed modification factor or a backward playback speed modification factor.

Description

(19) State Intellectual Property Office (12) Invention Patent Application (10) Application Publication Number (43) Application Publication Date (21) Application Number 202511179258.7 (22) Application Date 2020.12.18 (30) Priority Data 19219037.9 2019.12.20 EP (62) Divisional Application Data 202080088769.7 2020.12.18 (71) Applicant Fraunhofer Association for the Promotion of Applied Research Address Munich, Germany (72) Inventors Y. Sanchez de la Feente K. Schüring C. Helleg T. Skirle R. Skupin T. Wigand (74) Patent Agency China Patent Agency (Hong Kong) Limited 72001 Patent Attorney Lv Chuanqi (51) Int.Cl. H04N 19 / 20 (2014.01) H04N 19 / 42 (2014.01) (54) Title of Invention: Video Data Stream, Video Encoder, Apparatus, and Method for Hypothetical Reference Decoder and for Output Layer Set (57) Abstract This application relates to a video data stream, video encoder, apparatus, and method for hypothetical reference decoder and for output layer set. An apparatus is provided for receiving a video data stream as an input bit stream. The video data stream has video encoded therein, wherein, for a playback speed modification factor (Nx), the apparatus is used to determine a decoding capability requirement based on a decoding capability requirement constraint, wherein the playback speed modification factor (Nx) is a forward playback speed modification factor or a backward playback speed modification factor. Claims (2 pages), Description (32 pages), Drawings (8 pages), CN 121037573 A, 2025.11.28, CN 1 21 03 75 73 A 1. A method for receiving a video data stream as an input bit stream, wherein the video data stream has video encoded therein, wherein, for a playback speed modification factor (Nx), the method includes determining a decoding capability requirement based on a decoding capability requirement constraint, wherein the playback speed modification factor (Nx) is a forward playback speed modification factor or a backward playback speed modification factor. 2. A method for encoding video into a video data stream, such that the video data stream has the video encoded therein, wherein the method includes generating the video data stream such that the video data stream includes information regarding a sampling rate constraint, and / or wherein the method includes generating the video data stream such that the video data stream includes information regarding a bit rate constraint. 3. A video data stream having a video encoded therein, wherein the video data stream includes a plurality of access units, wherein a group of access units in the video data stream includes the plurality of access units, wherein the video data stream includes an indication for each access unit of a subset of access units in the access unit.4. A video encoder for encoding video into a video data stream, such that the video data stream has the video encoded therein, wherein the video encoder is used to generate the video data stream such that the video data stream includes a plurality of access units, wherein the access unit group of the video data stream includes the plurality of access units, wherein the video encoder is used to generate the video data stream such that the video data stream includes an indication for each access unit of the access unit subset, indicating that a seamless stitching point exists after the access unit, wherein the video encoder is used to generate the video data stream such that the access unit subset is an appropriate subset of the access unit group of the video data stream. 5. An apparatus for receiving a video data stream as an input bit stream, wherein the video data stream has video encoded therein, wherein the apparatus is used to process the video data stream, wherein the video data stream includes a plurality of access units, wherein a group of access units of the video data stream includes the plurality of access units, wherein the video data stream includes an indication for each access unit of a subset of access units, indicating the existence of a seamless splice point after the access unit, wherein the subset of access units is an appropriate subset of the group of access units of the video data stream, wherein the apparatus is used to process the indication. 6. A method for encoding video into a video data stream, such that the video data stream has the video encoded therein, wherein the method includes generating the video data stream such that the video data stream includes a plurality of access units, wherein a group of access units of the video data stream includes the plurality of access units, (Claim 1 / 2 page 2 CN 121037573 A) wherein the method includes generating the video data stream such that the video data stream includes an indication for each access unit of a subset of access units, indicating the existence of a seamless splice point after the access unit, wherein the method includes generating the video data stream such that the subset of access units is an appropriate subset of the group of access units of the video data stream. 7. A method for receiving a video data stream as an input bit stream, wherein the video data stream has video encoded therein, wherein the method includes processing the video data stream, wherein the video data stream includes a plurality of access units, wherein a group of access units of the video data stream includes the plurality of access units, wherein the video data stream includes an indication for each access unit of a subset of access units, indicating the existence of a seamless splicing point after the access unit, wherein the subset of access units is an appropriate subset of the group of access units of the video data stream.The method includes processing the instruction. 8. An apparatus for receiving a video data stream as an input bit stream, wherein the video data stream has video encoded therein, wherein the apparatus is configured to process the input bit stream in relation to an output layer set to obtain a sub-bit stream, wherein if the output layer set does not include a predefined layer (vps_layer_id[0]) present in a plurality of layers in the video data stream, the apparatus is configured to remove at least one non-expandable nested supplemental enhancement information message allocated with the predefined layer (vps_layer_id[0]), and wherein if the output layer set includes the predefined layer (vps_layer_id[0]), the apparatus is configured not to remove any non-expandable nested supplemental enhancement information messages allocated with the predefined layer (vps_layer_id[0]). 9. A video data stream having a video encoded therein, wherein the video data stream includes a video parameter set, wherein the video parameter set includes a plurality of output layer sets, wherein if none of the plurality of output layer sets includes all layers present in the plurality of layers in the video data stream, then the video data stream does not include any non-scalable nested supplementary enhancement information messages for assuming a reference decoder. 10. A video encoder for encoding video into a video data stream such that the video data stream has the video encoded therein, wherein the video encoder is used to generate the video data stream such that the video data stream includes a video parameter set, wherein the video encoder is used to generate the video data stream such that the video parameter set includes a plurality of output layer sets, wherein if none of the plurality of output layer sets includes all layers present in the plurality of layers in the video data stream, then the video encoder is used to generate the video data stream such that the video data stream does not include any non-scalable nested supplementary enhancement information messages for assuming a reference decoder. Claims 2 / 2 Page 3 CN 121037573 A Video data stream, video encoder, apparatus and method for hypothetical reference decoder and for output layer set Technical Field

[0001] This application is a divisional application of application number: 202080088769.7, entitled: Video data stream, video encoder, apparatus and method for hypothetical reference decoder and for output layer set. The present invention relates to video encoding and video decoding, and particularly to video encoders, video decoders, methods for encoding and decoding, and video data streams for implementing advanced video decoding concepts. Background Art

[0002] H.265 / HEVC (HEVC = High Efficiency Video Decoding) is a video codec that has been provided for improving or even enhancing video decoding capabilities.This enables tools that allow parallel processing at the encoder and / or decoder. For example, HEVC supports subdividing an image into an array of independently encoded tiles. Another concept supported by HEVC involves WPP, under which CTU rows or CTU units of an image can be processed in parallel from left to right (e.g., in stripes) as long as a minimum CTU offset (CTU = Decode Tree Unit) is observed during the processing of consecutive CTU rows. However, it would be advantageous to have video codecs that support parallel processing of the video encoder and / or video decoder even more efficiently.

[0003] Typically, in video decoding, the decoding process of image samples requires smaller partitions, where samples are divided into rectangular regions for joint processing, such as prediction or transform decoding. Thus, images are divided into blocks of a specific size that remain constant during the encoding of the video sequence. In the H.264 / AVC standard, fixed-size blocks of 16x16 samples are used, known as macroblocks (AVC = Advanced Video Decoding).

[0004] In the current HEVC standard (see [1]), there exists a maximum size of 64×64 samples in the decode tree block (CTB) or decode tree unit (CTU). In a further description of HEVC, the more common term CTU is used for this type of block.

[0005] CTUs are processed in raster scan order, starting from the top left CTU and processing the CTUs in the image line by line, down to the bottom right CTU.

[0006] The decoded CTU data is organized into a container called a slice. Originally, in previous video decoding standards, a slice was a segment that included one or more consecutive CTUs of an image. Slices are used to segment the decoded data. From another perspective, a complete image can also be defined as a large segment, and therefore, historically, the term slice still applies. In addition to the decoded image samples, slices also include additional information related to the decoding process of the slice itself, which is placed in a so-called slice header.

[0007] According to current technology, the VCL (Video Decoding Layer) also includes techniques for fragmentation and spatial partitioning. For example, such partitioning can be applied to video decoding for various reasons, including processing load balancing in parallelization, CTU size matching in network transmission, error mitigation, etc.

[0008] The bitstream specified in the video decoding standard has information associated with HRD consistency. This consistency consists of a hypothetical reference decoder (HRD) that includes a buffer model that assumes NAL units enter the decoded picture buffer (CPB) before the decoder and are removed from it during specific timing intervals to ensure that the CPB size does not change.Exceeding (buffer overrun) or NAL units arriving no later than when they need to be removed (buffer underrun). Furthermore, the model consists of a decoded picture buffer (DPB), from which the decoded picture is output when it is predicted that a picture no longer needs to be decoded. In many implementations, the size of the decoded picture is also limited. Timing information of the HRD is transmitted in the bitstream via so-called SEI messages, specifically, a buffer period (BP) SEI message defining the timing information of a buffer period (multiple access units or AUs), a picture timing (PT) SEI message transmitting the timing information of a single associated AU, and a decoded unit information (DUI) SEI message transmitting the timing information of an associated subset of AUs (i.e., decoded units or DUs). Summary of the Invention

[0009] The object of the present invention is to provide an improved concept for video encoding and video decoding.

[0010] This object is achieved by the subject matter of the independent claims.

[0011] Preferred embodiments are provided in the dependent claims.

[0012] According to an embodiment, an apparatus is provided for receiving a video data stream as an input bit stream, wherein the video data stream has video encoded therein. For a playback speed modification factor (Nx), the apparatus is used to determine a decoding capability requirement based on decoding capability requirement constraints. The playback speed modification factor (Nx) is either a forward playback speed modification factor or a backward playback speed modification factor.

[0013] Furthermore, according to an embodiment, a video data stream having video encoded therein is provided. The video data stream includes information regarding a sampling rate constraint, and / or the video data stream includes information regarding a bitrate constraint.

[0014] Furthermore, according to an embodiment, a video encoder is provided for encoding video into a video data stream, such that the video data stream has video encoded therein. The video encoder is used to generate the video data stream, such that the video data stream includes information regarding a sampling rate constraint, and / or the video encoder is used to generate the video data stream, such that the video data stream includes information regarding a bitrate constraint.

[0015] Furthermore, according to an embodiment, a method is provided for receiving a video data stream as an input bit stream, wherein the video data stream has video encoded therein. For the playback speed modification factor (Nx), the method includes determining the decoding capability requirement based on decoding capability requirement constraints. The playback speed modification factor (Nx) is either a forward playback speed modification factor or a backward playback speed modification factor.

[0016] Furthermore, according to an embodiment, a method for encoding video into a video data stream is provided, such that the video data stream has video encoded therein. The method includes generating the video data stream such that the video data stream includes a key...Information regarding sampling rate limitations, and / or the method includes generating the video data stream such that the video data stream includes information regarding bitrate limitations.

[0017] Furthermore, a computer program is provided for implementing one of the methods as described above when executed on a computer or signal processor.

[0018] Furthermore, according to an embodiment, a video data stream having video encoded therein is provided. The video data stream includes a plurality of access units, wherein a group of access units of the video data stream includes the plurality of access units. Furthermore, the video data stream includes an indication for each access unit of a subset of access units, indicating the existence of a seamless splicing point after the access unit. The subset of access units is an appropriate subset of the group of access units of the video data stream.

[0019] Furthermore, according to an embodiment, a video encoder is provided for encoding video into a video data stream such that the video data stream has video encoded therein. The video encoder is used to generate the video data stream such that the video data stream includes a plurality of access units, wherein a group of access units of the video data stream includes the plurality of access units. Furthermore, the video encoder is used to generate the video data stream such that the video data stream includes an indication for each access unit of the subset of access units, indicating the existence of a seamless stitching point after the access unit. Additionally, the encoder is used to generate the video data stream such that the subset of access units is an appropriate subset of the group of access units of the video data stream.

[0020] Furthermore, according to an embodiment, an apparatus for receiving a video data stream as an input bit stream is provided, wherein the video data stream has video encoded therein. The apparatus is used to process the video data stream. The video data stream includes a plurality of access units, wherein a group of access units of the video data stream includes the plurality of access units. Furthermore, the video data stream includes an indication for each access unit of the subset of access units, indicating the existence of a seamless stitching point after the access unit. The subset of access units is an appropriate subset of the group of access units of the video data stream. The apparatus is used to process the instruction.

[0021] Furthermore, according to an embodiment, a method for encoding video into a video data stream is provided, such that the video data stream has video encoded therein. The method includes generating the video data stream such that the video data stream includes a plurality of access units, wherein a group of access units of the video data stream includes the plurality of access units. Furthermore, the method includes generating the video data stream such that the video data stream includes an indication for each access unit of a subset of access units, indicating the existence of a seamless stitching point after the access unit. Additionally, the method includes generating the...A video data stream, such that the subset of access units is a suitable subset of the group of access units of the video data stream.

[0022] Furthermore, according to an embodiment, a method for receiving a video data stream as an input bit stream is provided, wherein the video data stream has video encoded therein. The method includes processing the video data stream, the video data stream including a plurality of access units, wherein the group of access units of the video data stream includes the plurality of access units. Furthermore, the video data stream includes an indication for each access unit of the subset of access units, indicating the existence of a seamless splicing point after the access unit. The subset of access units is a suitable subset of the group of access units of the video data stream. The method includes processing the indication.

[0023] Furthermore, a computer program is provided for implementing one of the methods as described above when executed on a computer or signal processor.

[0024] Furthermore, according to an embodiment, an apparatus for receiving a video data stream as an input bit stream is provided, wherein the video data stream has video encoded therein. The apparatus is used to process the input bit stream in relation to an output layer set to obtain sub-bits. If the output layer set does not include a predefined layer among the multiple layers present in the video data stream, the apparatus is configured to remove at least one non-scalable nested supplementary enhancement information message allocated with the predefined layer. If the output layer set includes the predefined layer, the apparatus is configured not to remove any non-scalable nested supplementary enhancement information messages allocated with the predefined layer.

[0025] Furthermore, according to an embodiment, a video data stream having video encoded therein is provided. The video data stream includes a video parameter set. The video parameter set includes multiple output layer sets. If none of the multiple output layer sets includes all layers present in the multiple layers of the video data stream, the video data stream does not include any non-scalable nested supplementary enhancement information messages for assuming a reference decoder.

[0026] Furthermore, according to an embodiment, a video encoder for encoding video into a video data stream is provided, such that the video data stream has video encoded therein. The video encoder is used to generate the video data stream such that the video data stream includes a video parameter set. Furthermore, the video encoder is used to generate the video data stream such that the video parameter set includes multiple output layer sets. If none of the plurality of output layer sets includes all layers present in the plurality of layers in the video data stream, the video encoder generates the video data stream such that the video data stream does not include any non-extendable nested supplementary enhancement information messages used for assuming a reference decoder.

[0027] Furthermore, according to an embodiment, a method for receiving a video data stream as an input bit stream is provided, wherein, video specification 3 / 32 pages 6 CN 121037573A data stream has video encoded therein. The method includes processing the input bitstream based on an output layer set to obtain a sub-bitstream. If the output layer set does not include a predefined layer (vps_layer_id[0]) among a plurality of layers present in the video data stream, the method includes removing at least one non-expandable nested supplemental enhancement information message allocated with the predefined layer (vps_layer_id[0]). If the output layer set includes the predefined layer (vps_layer_id[0]), the method includes not removing any non-expandable nested supplemental enhancement information messages allocated with the predefined layer (vps_layer_id[0]).

[0028] Furthermore, according to an embodiment, a method for encoding video into a video data stream is provided such that the video data stream has video encoded therein. The method includes generating the video data stream such that the video data stream includes a video parameter set. Furthermore, the method includes generating the video data stream such that the video parameter set includes a plurality of output layer sets. If none of the plurality of output layer sets includes all layers present in the plurality of layers in the video data stream, the method includes generating the video data stream such that the video data stream does not include any non-expandable nested supplementary enhancement information messages used for assuming a reference decoder.

[0029] Furthermore, according to an embodiment, an apparatus is provided for receiving a video data stream as an input bit stream, wherein the video data stream has video encoded therein. The video data stream includes initial profile hierarchy information. Furthermore, the video data stream includes a first set of two or more layers. The apparatus is configured to process the input bit stream based on an output layer set of a plurality of output layer sets to generate an output bit stream by removing at least one layer from the first set of two or more layers to obtain a second set of one or more layers, such that the output bit stream includes the second set of one or more layers, but does not include the at least one layer that has been removed from the first set. Furthermore, the apparatus is configured to remove portions of the initial profile hierarchy information that are not related to at least one layer in the second set of one or more layers from the initial profile hierarchy information.

[0030] Furthermore, according to an embodiment, an apparatus is provided for receiving a video data stream as an input bit stream, wherein the video data stream has video encoded therein. The video data stream includes expandable nested supplemental enhancement information, the expandable nested supplemental enhancement information including decoding parameter set information for each of a plurality of output layer sets. The apparatus is configured to process the input bit stream based on the output layer sets of the plurality of output layer sets to obtain an output bit stream, such that the output bit stream includes the decoding parameter set information for the output layer sets, and such that the output bit stream does not include the decoding parameter set information for any other output layer set among the plurality of output layer sets.

[0031] Furthermore, according to an embodiment, a video data stream having video encoded therein is provided. The video data stream includes scalable nested supplemental enhancement information, which includes decoding parameter set information for each of a plurality of output layer sets.

[0032] Furthermore, according to an embodiment, a video encoder is provided for encoding video into a video data stream, such that the video data stream has video encoded therein. The video encoder is used to generate the video data stream, such that the video data stream includes scalable nested supplemental enhancement information, which includes decoding parameter set information for each of a plurality of output layer sets.

[0033] Furthermore, according to an embodiment, a video data stream having video encoded therein is provided. The video data stream includes indications of a plurality of operation points. Furthermore, the video data stream includes a first mapping for each of the plurality of operation points, the first mapping assigning one or more profile levels from a plurality of profile levels to the operation point. Furthermore, the video data stream includes a plurality of second mappings, wherein each of the plurality of mappings assigns one of the plurality of operation points to an output level set from a plurality of output level sets.

[0034] Furthermore, according to an embodiment, a video encoder is provided for encoding video into a video data stream, such that the video data stream has video encoded therein. The video encoder is used to generate the video data stream such that the video data stream includes indications of a plurality of operation points. Furthermore, the video encoder is used to generate the video data stream such that the video data stream includes a first mapping for each of the plurality of operation points, the first mapping assigning one or more of a plurality of profile levels to the operation point. Furthermore, the video encoder is used to generate the video data stream such that the video data stream includes a plurality of second mappings, wherein each of the plurality of mappings assigns one of the plurality of operation points to an output layer set of a plurality of output layer sets.

[0035] Furthermore, according to an embodiment, an apparatus is provided for receiving a video data stream as an input bit stream, wherein the video data stream has video encoded therein. The video data stream includes indications of a plurality of operation points. Furthermore, the video data stream includes a first mapping for each of the plurality of operation points, the first mapping mapping one or more profile levels from a plurality of profile levels to the operation point. Additionally, the video data stream includes a plurality of second mappings, wherein each of the plurality of mappings maps one of the plurality of operation points to an output level from a plurality of output level sets. The apparatus is used to depend on the output level sets in the plurality of output level sets and to map the output level sets.At least one of the second mappings of one of the plurality of operation points is used to process the input bitstream to produce an output bitstream.

[0036] Furthermore, according to an embodiment, a method for receiving a video data stream as an input bitstream is provided, wherein the video data stream has video encoded therein. The video data stream includes initial profile hierarchy information. Furthermore, the video data stream includes a first set of two or more layers. The method includes processing the input bitstream based on an output layer set that depends on a plurality of output layer sets to generate an output bitstream by removing at least one layer from the first set of two or more layers to obtain a second set of one or more layers, such that the output bitstream includes the second set of one or more layers, but does not include the at least one layer that has been removed from the first set. Furthermore, the method includes removing portions of the initial profile hierarchy information that are not related to at least one layer in the second set of one or more layers from the initial profile hierarchy information.

[0037] Furthermore, according to an embodiment, a method for receiving a video data stream as an input bitstream is provided, wherein the video data stream has video encoded therein. The video data stream includes expandable nested supplemental enhancement information, which includes decoding parameter set information for each of a plurality of output layer sets. The method includes processing the input bitstream based on the output layer sets of the plurality of output layer sets to obtain an output bitstream, such that the output bitstream includes the decoding parameter set information for the output layer sets, and such that the output bitstream does not include the decoding parameter set information for any other output layer set of the plurality of output layer sets.

[0038] Furthermore, according to an embodiment, a method for encoding video into a video data stream is provided, such that the video data stream has video encoded therein. The method includes generating the video data stream, such that the video data stream includes expandable nested supplemental enhancement information, which includes decoding parameter set information for each of a plurality of output layer sets.

[0039] Furthermore, according to an embodiment, a method for encoding video into a video data stream is provided, such that the video data stream has video encoded therein. The method includes generating the video data stream, such that the video data stream includes indications of a plurality of operation points. Furthermore, the method includes generating the video data stream such that the video data stream includes a first mapping for each of the plurality of operation points, the first mapping assigning one or more profile levels from a plurality of profile levels to the operation point. Furthermore, the method includes generating the video data stream such that the video data stream includes a plurality of second mappings, wherein each of the plurality of mappings assigns one of the plurality of operation points to an output layer set from a plurality of output layer sets.

[0040] Furthermore, according to an embodiment, a method for receiving a video data stream as an input bit stream is provided, wherein the video data stream has video encoded therein. The video data stream includes indications of a plurality of operation points. Furthermore, the video data stream includes a first mapping for each of the plurality of operation points, the first mapping mapping one or more profile levels from a plurality of profile levels to the operation point. Furthermore, the video data stream includes a plurality of second mappings, wherein each of the plurality of mappings maps one of the plurality of operation points to an output level from a plurality of output level sets. The method includes processing the input bit stream based on at least one of the second mappings that depends on the output level set in the plurality of output level sets and on an operation point from the plurality of operation points mapping the output level set to produce an output bit stream.

[0041] Furthermore, a computer program is provided for implementing one of the methods as described above when executed on a computer or signal processor.

[0042] According to an embodiment, a video data stream having video encoded therein is provided. The video data stream includes a sequence parameter set and a decoding parameter set. The sequence parameter set includes a first indication value, which indicates the maximum number of sub-layers or the maximum number of sub-layers minus a constant value based on information stored in the sequence parameter set. The decoding parameter set includes a second indication value, which indicates the maximum number of sub-layers or the maximum number of sub-layers minus the constant value based on information stored in the decoding parameter set. The first indication value is less than or equal to the second indication value.

[0043] Furthermore, according to an embodiment, a video encoder is provided for encoding video into a video data stream, such that the video data stream has video encoded therein. The video encoder is used to generate the video data stream, such that the video data stream includes a sequence parameter set and a decoding parameter set. The video encoder is used to generate the video data stream, such that the sequence parameter set includes a first indication value, which indicates the maximum number of sub-layers or the maximum number of sub-layers minus a constant value based on information stored in the sequence parameter set. Furthermore, the video encoder generates the video data stream, such that the decoding parameter set includes a second indication value, which indicates the maximum number of sub-layers or the maximum number of sub-layers minus the constant value based on information stored in the decoding parameter set. Furthermore, the video encoder is used to generate the video data stream such that the first indication value is less than or equal to the second indication value.

[0044] Furthermore, according to an embodiment, a video encoder is provided for encoding video into a video data stream, such that the video data stream has video encoded therein. The video encoder is used to generate the video data stream such that the video...The video data stream includes a sequence parameter set and a decoding parameter set. Furthermore, the video encoder generates the video data stream such that the sequence parameter set includes a first indication value, which indicates the maximum number of sublayers or the maximum number of sublayers minus a constant value based on information stored in the sequence parameter set. Furthermore, the video encoder generates the video data stream such that the decoding parameter set includes a second indication value, which indicates the maximum number of sublayers or the maximum number of sublayers minus the constant value based on information stored in the decoding parameter set. The second indication value of the decoding parameter set is an upper boundary prior to the first indication value of the sequence parameter set.

[0045] Furthermore, according to an embodiment, an apparatus is provided for receiving a video data stream as an input bitstream, wherein the video data stream has video encoded therein. The video data stream includes a sequence parameter set and a decoding parameter set. The sequence parameter set includes a first indication value, which indicates the maximum number of sublayers or the maximum number of sublayers minus a constant value based on information stored in the sequence parameter set. The decoding parameter set includes a second indication value, which indicates the maximum number of sub-layers or the maximum number of sub-layers minus the constant value based on information stored in the decoding parameter set. If the first indication value is greater than the second indication value, the apparatus is used to process the input bitstream to generate an output bitstream such that the output bitstream includes the second indication value as an indication of the maximum number of sub-layers or the maximum number of sub-layers minus the constant value. Specification 6 / 32 Page 9 CN 121037573 A

[0046] Furthermore, according to an embodiment, a method for encoding video into a video data stream is provided, such that the video data stream has video encoded therein. The method includes generating the video data stream such that the video data stream includes a sequence parameter set and a decoding parameter set. Furthermore, the method includes generating the video data stream such that the sequence parameter set includes a first indication value, which indicates the maximum number of sub-layers or the maximum number of sub-layers minus the constant value based on information stored in the sequence parameter set. Furthermore, the method includes generating the video data stream such that the decoding parameter set includes a second indication value, the second indication value indicating the maximum number of sub-layers or the maximum number of sub-layers minus the constant value based on information stored in the decoding parameter set. The method includes generating the video data stream such that the first indication value is less than or equal to the second indication value.

[0047] Furthermore, according to an embodiment, a method for encoding video into a video data stream is provided such that the video data stream has video encoded therein. The method includes generating the video data stream such that the video data stream includes...A sequence parameter set and a decoding parameter set. Furthermore, the method includes generating the video data stream such that the sequence parameter set includes a first indication value, the first indication value indicating a maximum number of sub-layers or the maximum number of sub-layers minus a constant value based on information stored in the sequence parameter set. Furthermore, the method includes generating the video data stream such that the decoding parameter set includes a second indication value, the second indication value indicating the maximum number of sub-layers or the maximum number of sub-layers minus the constant value based on information stored in the decoding parameter set. The second indication value of the decoding parameter set is an upper boundary prior to the first indication value of the sequence parameter set.

[0048] Furthermore, according to an embodiment, a method for receiving a video data stream as an input bit stream is provided, wherein the video data stream has video encoded therein. The video data stream includes a sequence parameter set and a decoding parameter set. The sequence parameter set includes a first indication value, the first indication value indicating a maximum number of sub-layers or the maximum number of sub-layers minus a constant value based on information stored in the sequence parameter set. The decoding parameter set includes a second indication value, which indicates the maximum number of sublayers or the maximum number of sublayers minus the constant value based on information stored in the decoding parameter set. If the first indication value is greater than the second indication value, the method includes processing the input bitstream to generate an output bitstream such that the output bitstream includes the second indication value as an indication of the maximum number of sublayers or the maximum number of sublayers minus the constant value.

[0049] Furthermore, a computer program is provided for implementing the methods described above when executed on a computer or signal processor. Brief Description of the Drawings

[0050] FIG1 illustrates a video encoder for encoding video into a video data stream according to an embodiment.

[0051] FIG2 illustrates an apparatus for receiving an input video data stream according to an embodiment.

[0052] FIG3 illustrates a video decoder for receiving a video data stream in which video is stored according to an embodiment.

[0053] FIG4 illustrates an example of using reference picture resampling for temporal scalability.

[0054] Figure 5 illustrates two stitching point options, where option I (top) maintains the frame rate throughout the example, and option II (bottom) has a frame rate decrease in the last picture group of the bitstream.

[0055] Figure 6 illustrates the video encoder.

[0056] Figure 7 illustrates the video decoder.

[0057] Figure 8 illustrates the relationship between a reconstructed signal (e.g., reconstructed picture) on one side and a combination of a prediction residual signal and a prediction signal, such as a signal notified in the data stream on the other side. Specification 7 / 32 pages 10 CN 121037573 A Detailed Description

[0058] The following description of the accompanying drawings begins with a description of the encoder and decoder of a block-based predictive codec used to decode images of a video to form an example of a decoding framework in which embodiments of the present invention can be built. Corresponding encoders and decoders are described with respect to Figures 6 through 8. Thereafter, a description of embodiments of the concepts of the present invention is presented together with a description of how these concepts can be built into the encoders and decoders of Figures 6 and 7, although the embodiments described with Figures 1 through 3 and below can also be used to form encoders and decoders that do not operate according to the decoding framework that forms the basis of the encoders and decoders of Figures 6 and 7.

[0059] Figure 6 illustrates a video encoder, an apparatus for predictively decoding an image 12 into a data stream 14 using transform-based residual decoding, exemplarily. This apparatus or encoder is indicated by reference numeral 10. Figure 7 illustrates a corresponding video decoder 20, for example, an apparatus 20 configured to also use transform-based residual decoding to predictively decode image 12' from data stream 14, where an apostrophe has been used to indicate that image 12' reconstructed by decoder 20 deviates from image 12 originally encoded by encoder 10 in terms of decoding loss introduced by the quantization of the predictive residual signal. Figures 6 and 7 exemplarily use transform-based predictive residual decoding, although embodiments of this application are not limited to this type of predictive residual decoding. Other details described with respect to Figures 6 and 7 are also as will be outlined below.

[0060] Encoder 10 is configured to perform a spatial-to-spectral transform on the predictive residual signal and encode the resulting predictive residual signal into data stream 14. Similarly, decoder 20 is configured to decode the predictive residual signal from data stream 14 and perform a spectral-to-spatial transform on the resulting predictive residual signal.

[0061] Internally, encoder 10 may include a prediction residual signal former 22 that generates a prediction residual 24 to measure the deviation of the prediction signal 26 from the original signal, such as the deviation from image 12. The prediction residual signal former 22 may be, for example, a subtractor that subtracts the prediction signal from the original signal (e.g., from image 12). Encoder 10 then includes a transformer 28 that performs a space-to-spectral transformation on the prediction residual signal 24 to obtain a spectral domain prediction residual signal 24', which is then quantized by a quantizer 32 also included by encoder 10. The quantized prediction residual signal 24'' is thus decoded into bitstream 14. For this purpose, encoder 10 may optionally include an entropy decoder 34 that performs entropy decoding on the transformed and quantized prediction residual signal into datastream 14. The prediction signal 26 is generated by the prediction stage 36 of the encoder 10 based on the prediction residual signal 24 encoded into and decodeable from the data stream 14. For this purpose, as shown in Figure 6, the prediction...Prediction stage 36 may internally include a dequantizer 38 that dequantizes the prediction residual signal 24'' to obtain a spectral domain prediction residual signal 24''' corresponding to signal 24' excluding quantization loss. This is followed by an inverse transformer 40 that performs an inverse transform (e.g., spectral-to-spatial transform) on the latter prediction residual signal 24''' to obtain a prediction residual signal 24'''' corresponding to the original prediction residual signal 24 excluding quantization loss. Then, a combiner 42 of prediction stage 36 recombines the prediction signal 26 and the prediction residual signal 24'''', for example by addition, to obtain a reconstructed signal 46, for example, a reconstruction of the original signal 12. The reconstructed signal 46 may correspond to signal 12'. Then, prediction module 44 of prediction stage 36 generates the prediction signal 26 based on signal 46 using, for example, spatial prediction (e.g., intra-picture prediction) and / or temporal prediction (e.g., inter-picture prediction).

[0062] Similarly, as shown in FIG7, decoder 20 may internally consist of components corresponding to prediction stage 36 and interconnected in a manner corresponding to prediction stage 36. In particular, entropy decoder 50 of decoder 20 may entropy decode the quantized spectral domain prediction residual signal 24'' from the data stream, so that the dequantizer 52, inverse transformer 54, combiner 56 and prediction module 58, which are interconnected and cooperate in the manner described above with respect to the modules of prediction stage 36, recover the reconstructed signal based on the prediction residual signal 24'', so that, as shown in FIG7, the output of combiner 56 results in the reconstructed signal, i.e., picture 12'.

[0063] Although not specifically described above, it is readily apparent that encoder 10 may set some decoding parameters, including, for example, prediction mode, motion parameters, etc., according to some optimization schemes (see specification page 8 / 32 11 CN 121037573 A, such as optimizing a certain rate and distortion correlation criterion (e.g., decoding cost)). For example, encoder 10 and decoder 20, along with corresponding modules 44 and 58, can each support different prediction modes, such as intra-frame decoding modes and inter-frame decoding modes. The encoder and decoder, by the granularity of their switching between these prediction mode types, can respectively correspond to subdividing images 12 and 12' into decoding segments or decoding blocks. For example, using these decoding segments as units, the image can be subdivided into blocks to be intra-frame decoded and blocks to be inter-frame decoded. As outlined in more detail below, intra-frame decoded blocks are predicted based on the spatial, already decoded / decoded neighborhood of the corresponding block. Several intra-frame decoding modes can exist, and these modes are selected for corresponding intra-frame decoded segments that include intra-frame decoding modes of direction or angle. According to these modes, the corresponding segments are filled by extrapolating sample values ​​of the neighborhood along a specific direction specific to the intra-frame decoding mode of the corresponding direction. The intra-frame decoding mode can, for example, also include a...One or more other modes, such as DC decoding mode (in which the prediction of the corresponding intra-decoded block assigns DC values ​​to all samples within the corresponding intra-decoded segment), and / or planar intra-decoded mode (in which the prediction of the corresponding block is approximated or determined as a spatial distribution of sample values ​​described by a two-dimensional linear function at the sample locations of the corresponding intra-decoded block, having a driving tilt and offset of a plane defined by the two-dimensional linear function based on adjacent samples). In contrast, inter-decoded blocks can be predicted, for example, temporally. For inter-decoded blocks, a motion vector can be signaled within the data stream, indicating the spatial displacement of a portion of a previously decoded image of the video to which image 12 belongs, in which the previously decoded / decoded image is sampled to obtain the prediction signal for the corresponding inter-decoded block. This means that, in addition to the residual signal decoding included in data stream 14 (such as the entropy decoding transform coefficient level representing the quantized spectral domain prediction residual signal 24''), data stream 14 may also encode the following: decoding mode parameters for assigning decoding modes to the individual blocks; prediction parameters for some of the blocks (such as motion parameters for inter-frame coded segments); and optional other parameters (such as parameters for controlling and signaling the subdivision of images 12 and 12' into segments, respectively). Decoder 20 uses these parameters to subdivide the images in the same way as the encoder has done, to assign the same prediction modes to the segments, and to perform the same predictions to result in the same prediction signals.

[0064] Figure 8 illustrates the relationship between the reconstructed signal (e.g., reconstructed image 12') on one side and the combination of the prediction residual signal 24'''' and the prediction signal 26, as signaled in data stream 14 on the other side. As already stated above, this combination may be additive. The prediction signal 26 is illustrated in FIG8 as subdividing the image region into intra-frame decoding blocks illustratively indicated by shading and inter-frame decoding blocks illustratively indicated by non-shading. This subdivision can be any subdivision, such as regularly subdividing the image region into rows and columns of square or non-square blocks, or subdividing the image 12 from a root block multi-tree into multiple leaf blocks of different sizes, such as quadtree subdivision, etc., wherein a mixture of these is illustrated in FIG8, wherein the image region is first subdivided into rows and columns of root blocks, and then further subdivided into one or more leaf blocks according to the recursive multi-tree subdivision.

[0065] Similarly, for intra-frame decoding block 80, data stream 14 may have an intra-frame decoding mode decoded therein, which assigns one of several supported intra-frame decoding modes to the corresponding intra-frame decoding block 80. For inter-frame decoding block 82, data stream 14 may have one or more motion parameters encoded therein. Generally speaking, inter-frame decoding block 82 is not limited.The image is decoded in time. Alternatively, the inter-frame decoding block 82 may be any block predicted from a previously decoded portion other than the current image 12 itself, such as a previously decoded image of the video to which image 12 belongs, or another view or a lower-level image in the case where the encoder and decoder are scalable encoders and decoders, respectively.

[0066] The prediction residual signal 24'''' in FIG8 is also illustrated as subdividing the image region into blocks 84. These blocks may be referred to as transform blocks in order to distinguish them from the decoding blocks 80 and 82. In fact, FIG8 illustrates that the encoder 10 and decoder 20 may use two different subdivisions to subdivide images 12 and 12' into blocks, one subdividing them into decoding blocks 80 and 82 respectively, and the other subdividing them into transform blocks 84 respectively. These two subdivisions can be identical; for example, each decoded block 80 and 82 can simultaneously form transform block 84. However, Figure 8 illustrates a situation where, for example, the subdivision into transform block 84 forms an extension of the subdivision into decoded blocks 80 and 82, such that any boundary between the two blocks of 80 and 82 covers the boundary between the two blocks 84. Alternatively, each block 80 and 82 either coincides with one of the transform blocks 84 or with a cluster of transform blocks 84. However, the subdivisions can also be determined or selected independently so that transform block 84 can alternatively span the block boundary between blocks 80 and 82. Regarding the subdivision into transform block 84, similar statements are correct due to those previously made regarding the subdivision into blocks 80 and 82; for example, block 84 can be the result of subdividing a picture region rule into blocks (with or without arrangement in rows and columns), the result of a recursive multi-way tree subdivision of the picture region, or a combination thereof, or any other type of blocking. Incidentally, note that blocks 80, 82, and 84 are not limited to quadratic, rectangular, or any other shape.

[0067] FIG8 further illustrates that the combination of the prediction signal 26 and the prediction residual signal 24'''' directly results in the reconstructed image 12'. However, it should be noted that, according to alternative embodiments, more than one prediction signal 26 can be combined with the prediction residual signal 24'''' to result in image 12'.

[0068] In FIG8, transform blocks 84 should have the following meaning. Transformers 28 and inverse transformers 54 perform their transforms on a per-block basis of these transform blocks 84. For example, many codecs use some kind of DST or DCT for all transform blocks 84. Some codecs allow skipping transforms so that the prediction residual signal can be decoded directly in the spatial domain for some of the transform blocks 84. However, according to the embodiments described below, encoder 10 and decoder 20 are configured to support several transforms. For example, transforms supported by encoder 10 and decoder 20 may include:o DCT-II (or DCT-III), where DCT stands for Discrete Cosine Transform o DST-IV, where DST stands for Discrete Sine Transform o DCT-IV o DST-VII o Identity Transform (IT) Naturally, while transformer 28 will support all forward versions of these transforms, decoder 20 or inverse transformer 54 will support their corresponding backward or inverse versions: o Inverse DCT-II (or Inverse DCT-III) o Inverse DST-IV o Inverse DCT-IV o Inverse DST-VII o Identity Transform (IT) The following description provides more details about which transforms encoder 10 and decoder 20 are capable of supporting. In any case, it should be noted that the set of supported transforms may include only one transform, such as a spectrum-to-space or space-to-spectrum transform.

[0069] As already outlined above, Figures 6 through 8 have been presented as examples in which the inventive concepts further described below can be implemented to form specific examples of encoders and decoders according to this application. So far, the encoders and decoders of Figures 6 and 7 can represent possible implementations of the encoder and decoder described below, respectively. However, Figures 6 and 7 are merely examples. However, the encoder according to embodiments of this application can perform block-based encoding of image 12 using concepts outlined in more detail below, and differs from the encoder of Figure 6 in that it is a still image encoder rather than a video encoder, does not support inter-frame prediction, or performs block segmentation in a manner different from that illustrated in Figure 8. Similarly, the decoder according to embodiments of this application can perform block-based decoding of image 12' from data stream 14 using decoding concepts further outlined below, but may differ from, for example, the decoder 20 of Figure 7 in that it is a still image decoder rather than a video decoder, does not support intra-frame prediction, or divides image 12' into blocks in a manner different from that described with respect to Figure 8 and / or derives prediction residuals from data stream 14, for example, not in the transform domain but in the spatial domain.

[0070] FIG1 illustrates a video encoder 100 according to an embodiment for encoding video into a video data stream. The video encoder 100 is configured to generate a video data stream.

[0071] FIG2 illustrates an apparatus 200 according to an embodiment for receiving an input video data stream. The input video data stream has video encoded therein. The apparatus 200 is configured to generate an output video data stream from the input video data stream.

[0072] FIG3 illustrates a video decoder 300 according to an embodiment for receiving a video data stream in which video is stored. The video decoder 300 is configured to decode video from the video data stream.

[0073] Furthermore, a system according to an embodiment is provided. The system includes the apparatus of FIG2 and the video decoder of FIG3. The video decoder (300) of FIG3 is configured to receive the output video data stream of the apparatus (200) of FIG2. The video decoder 300 of FIG3 is configured to decode video from the output video data stream of the apparatus 200 of FIG2.

[0074] In an embodiment, the system may, for example, further include the video encoder 100 of FIG1. ​​For example, the apparatus 200 of FIG2 may be configured to receive a video data stream from the video encoder 100 of FIG1 as an input video data stream.

[0075] An (optional) intermediate device 210 of the apparatus 200 may, for example, be configured to receive a video data stream from the video encoder 100 as an input video data stream and generate an output video data stream from the input video data stream. For example, the intermediate device may, for example, be configured to modify the (header / metadata) information of the input video data stream and / or may, for example, be configured to delete pictures from the input video data stream and / or may be configured to mix / concatenate the input video data stream with an additional second bitstream having a second video encoded therein.

[0076] (Optional) The video decoder 221 may be configured, for example, to decode video from the output video data stream.

[0077] (Optional) The hypothetical reference decoder 222 may be configured, for example, to determine timing information of the video based on the output video data stream, or may be configured, for example, to determine buffer information of a buffer in which the video or a portion of the video is to be stored.

[0078] The system includes a video encoder 101 of FIG1 and a video decoder 151 of FIG2.

[0079] The video encoder 101 is configured to generate an encoded video signal. The video decoder 151 is configured to decode the encoded video signal to reconstruct a picture of the video.

[0080] Hereinafter, trick modes and fast playback are described.

[0081] According to an embodiment, an apparatus is provided for receiving a video data stream as an input bit stream, wherein the video data stream has video encoded therein. For a playback speed modification factor (Nx), the apparatus is used to determine the decoding capability requirement based on the decoding capability requirement constraint. The playback speed modification factor (Nx) is either a forward playback speed modification factor or a backward playback speed modification factor.

[0082] In an embodiment, the decoding capability requirement may, for example, be a modified sampling rate. The decoding capability requirement limitation may, for example, be a sampling rate limitation.

[0083] According to an embodiment, the video data stream may, for example, include information about the sampling rate limitation.

[0084] In an embodiment, the apparatus may, for example, be configured to determine the modified sampling rate based on the image size of the video's frames.

[0085] According to an embodiment, the apparatus may, for example, also be configured to depend on the frame rate and the playback speed modification factor. (Specification 11 / 32 pages 14 CN)121037573 A (Nx) determines the modified sampling rate. The video data stream may, for example, include information about the frame rate.

[0086] In an embodiment, the frame rate depends on the sublayers of one or more sublayers. The video data stream may, for example, include sublayer-specific frame rate information for each of the one or more sublayers. The apparatus may, for example, be configured to obtain the frame rate from the sublayer-specific frame rate information for one of the one or more sublayers.

[0087] According to an embodiment, the apparatus may, for example, be configured to determine that the modified sampling rate is the highest image size among a plurality of images of the video, depending on the image size of the images of the video.

[0088] In an embodiment, the video data stream may, for example, include an application factor. The apparatus may, for example, be configured to calculate the modified sampling rate using the highest layer image size and the application factor.

[0089] According to an embodiment, the highest image size depends on the sublayers of one or more sublayers. The video data stream may, for example, include sublayer-specific maximum image size information for each of the one or more sublayers. The apparatus may, for example, be configured to obtain the highest image size from the sublayer-specific maximum image size information for one of the one or more sublayers.

[0090] In an embodiment, the video data stream may, for example, include an application factor. The apparatus may, for example, be configured to use the sampling rate of the video and use the application factor to determine a modified sampling rate.

[0091] According to an embodiment, the decoding capability requirement may, for example, be a modified bitrate. The decoding capability requirement limitation may, for example, be a bitrate limitation.

[0092] According to an embodiment, the video data stream may, for example, include information about a bitrate limitation.

[0093] According to an embodiment, the apparatus may, for example, be configured to determine a modified bitrate based on an initial bitrate of the video data stream and based on a playback speed modification factor (Nx).

[0094] In an embodiment, the initial bitrate depends on a sub-layer video data stream in one or more sub-layers, including sub-layer-specific initial bitrate information that may, for example, be configured to be per sub-layer in one or more sub-layers. The apparatus may, for example, be configured to obtain the initial bitrate from sub-layer-specific frame rate information for one of the one or more sub-layers.

[0095] According to an embodiment, the video data stream may, for example, include an IRAP-only flag (IRAP = Intra-Frame Random Access Point), which indicates whether decoding of IRAP-only pictures is possible when the initial playback speed increases. Depending on the IRAP-only flag, the apparatus may, for example, be configured to read decoding capability requirements from the video data stream.

[0096] In an embodiment, the video data stream may, for example, include a reference picture-only flag, which indicates whether decoding of reference pictures is possible when the initial playback speed increases. Depending on the reference picture-only flag, the apparatus may, for example, be configured to read decoding capability requirements from the video data stream.

[0097] Furthermore, according to an embodiment, a video data stream having video encoded therein is provided. The video data stream may, for example, include information about sampling rate limits, and / or the video data stream may, for example, include information about bitrate limits.

[0098] According to an embodiment, the video data stream may, for example, include information about frame rate.

[0099] In an embodiment, the video data stream may, for example, include sub-layer-specific frame rate information for each of one or more sub-layers.

[0100] According to an embodiment, the video data stream may, for example, include an application factor for determining a modified sampling rate.

[0101] In an embodiment, the video data stream may, for example, include sub-layer-specific maximum picture size information for each of one or more sub-layers.

[0102] According to an embodiment, the video data stream may, for example, include sub-layer-specific initial bitrate information for each of one or more sub-layers.

[0103] In an embodiment, the video data stream may, for example, include an IRAP-only flag, which indicates whether IRAP-only picture decoding is possible when the initial playback speed increases.

[0104] According to an embodiment, the video data stream may, for example, include a reference-only image flag, the IRAP-only flag indicating whether reference-only decoding is possible when the initial playback speed increases.

[0105] In an embodiment, depending on the IRAP-only flag, the video data stream may, for example, include level information; and / or depending on the reference-only image flag, the video data stream may, for example, include level information.

[0106] Furthermore, according to an embodiment, a video encoder is provided for encoding video into a video data stream, such that the video data stream has video encoded therein. The video encoder is used to generate the video data stream such that the video data stream includes information about sampling rate limits, and / or the video encoder is used to generate the video data stream such that the video data stream includes information about bitrate limits.

[0107] In an embodiment, the video encoder may, for example, be configured to generate the video data stream such that the video data stream may, for example, include information about frame rate limits.

[0108] According to an embodiment, the video encoder may, for example, be configured to generate the video data stream such that the video data stream may, for example, include sub-layer-specific frame rate information for each of one or more sub-layers.

[0109] In an embodiment, the video encoder may, for example, be configured to generate a video data stream such that the video data stream may, for example, include an application factor for determining a modified sampling rate.

[0110] According to an embodiment, the video encoder may, for example, be configured to generate a video data stream such that the video data stream may, for example, include sublayer-specific maximum image size information for each of one or more sublayers.

[0111] In an embodiment, the video encoder may be configured, for example, to generate a video data stream such that the video data stream may include, for example, sublayer-specific initial bitrate information for each of one or more sublayers.

[0112] According to an embodiment, the video encoder may be configured, for example, to generate a video data stream such that the video data stream may include, for example, an IRAP-only flag indicating whether IRAP-only image decoding is possible when the initial playback speed can be increased, for example.

[0113] In an embodiment, the video encoder may be configured, for example, to generate a video data stream such that the video data stream may include, for example, a reference image flag indicating whether reference image decoding is possible when the initial playback speed can be increased, for example.

[0114] According to an embodiment, the video encoder may be configured, for example, to generate a video data stream such that, depending on the IRAP-only flag, the video data stream may include, for example, level information; and / or the video encoder may be configured, for example, to generate a video data stream such that, depending on the reference image flag, the video data stream may include, for example, level information.

[0115] In an embodiment, the apparatus may be configured, for example, to decode an input bitstream to decode video.

[0116] Furthermore, according to an embodiment, a system for encoding video into a video data stream and for decoding video is provided. The system includes a video encoder as described above and means as described above. The video encoder is used to encode video into a video data stream such that the video data stream has video encoded therein. The means is used to receive the video data stream as an input bit stream. Furthermore, the means is used to decode the input bit stream to decode the video.

[0117] Fast forward operation or reverse operation (fast backward replay) is a typical operation performed in video applications. Typically, these operations involve decoding only a subset of the bit stream and decoding it at a higher speed (e.g., frame rate) than indicated by the bit stream. Furthermore, even for non-stunt mode operation, in some scenarios, playing the video at a higher speed may be of interest. Although there is not a significant difference between fast forward and fast replay, the difference considered in this description could be that the second objective is continuous and smooth playback (even potentially with audio playback and synchronization), while the first objective is not; for example, fast forward might be a discontinuous playback with many "jumps" in its content. In any case, the result is the content replayed at Nx speed specification page 13 / 32 16 CN 121037573 A.

[0118] Obviously, replaying the content at a higher speed also requires decoding the bitstream at a higher speed, which leads to higher decoding capability requirements (i.e., a higher level), as a result of the higher sampling rate and higher bitrate indicated in the HRD parameters.

[0119] When reference picture resampling (RPR) is not used, i.e., all pictures in the CVS or bitstream have the same size,The sampling rate can be easily calculated as the image size in the sample divided by the frame rate and multiplied by the speedup factor Nx. Similarly, the bitrate will be multiplied by the speedup factor Nx.

[0120] In the first embodiment, the two values ​​described above, derived according to the level limit in the VVC, are checked, and the level to which the Nx speedup belongs is calculated as the minimum derived value less than the level limit.

[0121] Note that the frame rate or bitrate is sublayer specific, meaning that regardless of what sublayer is being replayed (e.g., fast-forwarding is only performed by decoding temporal level 0), the corresponding value is calculated using the sublayer-specific signaled frame rate or bitrate.

[0122] However, when using Reference Picture Resampling (RPR), the sampling rate cannot be easily derived, i.e., only the worst-case scenario with the highest image size can be used, which will result in a worst-case sampling rate. Figure 4 illustrates an example of how RPR can be used in conjunction with temporal scalability.

[0123] Figure 4 illustrates an example of using Reference Picture Resampling in conjunction with temporal scalability.

[0124] In this example, the image size of the lowest sublayer might be 1920x1080, and the image size of the highest sublayer might be 960x540. By using only the maximum image size and assuming a 60fps bitstream, a sampling rate of approximately 124*106 samples per second is derived, while the actual value will be approximately 78*106, i.e., the worst case will be 1.6 times the actual value.

[0125] In one embodiment, the bitstream contains an indication of a factor that needs to be applied to the sampling rate derived with the worst-case value. Using the signaled factor, the worst-case sampling rate can be derived by dividing it by the signaled factor and multiplying it by a speed decoding factor to calculate the actual sampling rate by accelerating the decoding operation.

[0126] In another embodiment, each sublayer indicates a maximum image size value, which will result in a better approximation or even a more accurate value if the image size does not change within the sublayer.

[0127] Furthermore, there are some cases where sublayers are not fully utilized. Sometimes, fast-forwarding is performed by only decoding and replaying IRAP images (e.g., IDR) or images that are not non-reference images. However, it is unclear what level is required to decode such a sub-bit stream when decoding at Nx speed. In another embodiment, additional level signaling is indicated for either of the following two cases: • IRAP only: Only IRAP is decoded.

[0128] • Reference only: Non-reference images are discarded, and the rest are decoded.

[0129] The syntax for such an embodiment may be as follows.

[0130] For profile_tier_level: Similar to the irap case, only reference images can be added to the syntax.

[0131] The bitrate and CPB size can be similarly signaled as follows: (See specification 15 / 32, page 18, CN 121037573 A). Hereinafter, protection for delay_for_concatenation_ensured_flag is described.

[0132] According to an embodiment, a video data stream having video encoded therein is provided. The video data stream includes a plurality of access units, wherein a group of access units of the video data stream includes a plurality of access units. Furthermore, the video data stream includes an indication for each access unit of a subset of access units, indicating the presence of a seamless splicing point after the access unit. The subset of access units is an appropriate subset of the group of access units of the video data stream.

[0133] In an embodiment, the video data stream may, for example, include a plurality of temporal sublayers. The indication that a seamless splicing point exists after an access unit exists only for access units of a first group of temporal sublayers among the plurality of temporal sublayers, but does not exist for access units of a second group of temporal sublayers among the plurality of temporal sublayers.

[0134] According to an embodiment, the video data stream may, for example, include buffer period supplementation enhancement information. The buffer period supplementation enhancement information indicates a subset of access units among a plurality of access units for which a seamless stitching point exists after the access unit. Specification 16 / 32 pages 19 CN 121037573 A

[0135] In an embodiment, the video data stream may, for example, include a divisor. A delay value may, for example, be assigned to each of the plurality of access units of the video data stream. The indication that a seamless stitching point exists in an access unit exists only for those access units among the plurality of access units of the video data stream that have the assigned delay value, and the delay value may, for example, be equal to the remainder when divided by the divisor.

[0136] According to an embodiment, the remainder may, for example, be 0.

[0137] In an embodiment, the video data stream may, for example, include a remainder.

[0138] According to an embodiment, the indication for each access unit of the subset of access units that a seamless stitching point exists after the access unit may, for example, be a flag.

[0139] In an embodiment, the flag may be, for example, delay_for_concatenation_ensured_flag.

[0140] According to an embodiment, if, for example, a constant frame rate for the output images obtained from the decoded video data stream cannot be guaranteed, the flag exhibits a first value. If, for example, a constant frame rate for the output images obtained from the decoded video data stream can be guaranteed, the flag may, for example, exhibit a second value different from the first value.

[0141] In an embodiment, if the output images obtained from the decoded video data stream have equidistant output times, the flag exhibits a second value.

[0142] Furthermore, according to an embodiment, a video encoder is provided for encoding video into a video data stream, such that the video data stream has video encoded therein. The video encoder is used to generate a video data stream such that the video data stream includes a plurality of access units, wherein a group of access units in the video data stream includes a plurality of access units. Additionally, the video encoder is used to generate a video data stream such that the video data stream includes an indication for each access unit of a subset of access units, indicating the presence of a seamless stitching point after the access unit. Furthermore, the video encoder is used to generate a video data stream such that the subset of access units is an appropriate subset of the group of access units in the video data stream.

[0143] In an embodiment, the video encoder may, for example, be configured to generate a video data stream such that the video data stream may, for example, include a plurality of temporal sublayers. The video encoder may, for example, be configured to generate a video data stream such that the indication of a seamless stitching point after an access unit exists only for access units of a first group of temporal sublayers among the plurality of temporal sublayers, but the video encoder may, for example, be configured to generate a video data stream such that the indication of a seamless stitching point after an access unit does not exist for access units of a second group of temporal sublayers among the plurality of temporal sublayers.

[0144] According to an embodiment, a video encoder may be configured, for example, to generate a video data stream such that the video data stream may include, for example, buffer periodicity enhancement information. The video encoder may be configured, for example, to generate a video data stream such that the buffer periodicity enhancement information indicates a subset of access units among a plurality of access units for which a seamless stitching point exists after the access unit.

[0145] In an embodiment, a video encoder may be configured, for example, to generate a video data stream such that the video data stream may include, for example, a divisor. The video encoder may be configured, for example, to generate a video data stream such that a delay value may be assigned, for example, to each of the plurality of access units of the video data stream. The video encoder may be configured, for example, to generate a video data stream such that the indication that a seamless stitching point exists exists only for those access units among the plurality of access units of the video data stream that have the assigned delay value, the delay value being, for example, equal to the remainder when divided by the divisor.

[0146] According to an embodiment, a video encoder may be configured, for example, to generate a video data stream such that the remainder may be, for example, 0.

[0147] In an embodiment, the video encoder may be configured, for example, to generate a video data stream such that the video data stream may include, for example, a remainder value.

[0148] According to an embodiment, the video encoder may be configured, for example, to generate a video data stream such that an indication of a seamless stitching point following each access unit in the set of access unit sub-specifications (pages 17 / 32 of 20 CN 121037573 A) may be, for example, a flag.

[0149] In an embodiment, the video encoder may be configured, for example, to generate a video data stream such that an indication of a seamless stitching point following each access unit of a subset of access units may be, for example, a flag.

[0150] According to an embodiment, if, for example, a constant frame rate for the output image obtained from the decoded video data stream cannot be ensured, the video encoder generates the video data stream such that the flag exhibits a first value. If, for example, a constant frame rate for the output image obtained from the decoded video data stream can be ensured, the video encoder may be configured, for example, to generate the video data stream such that the flag exhibits a second value different from the first value.

[0151] In an embodiment, if the output image obtained from the decoded video data stream has equidistant output times, the video encoder may be configured, for example, to generate the video data stream such that the flag exhibits a second value.

[0152] Furthermore, according to an embodiment, an apparatus is provided for receiving a video data stream as an input bit stream, wherein the video data stream has video encoded therein. The apparatus is used to process the video data stream. The video data stream includes a plurality of access units, wherein a group of access units of the video data stream includes a plurality of access units. Furthermore, the video data stream includes an indication for each access unit of a subset of access units, indicating the existence of a seamless stitching point after the access unit. The subset of access units is an appropriate subset of the group of access units in the video data stream. The apparatus is used to process the indication.

[0153] In an embodiment, the video data stream may, for example, include multiple temporal sublayers. The indication that a seamless stitching point exists after an access unit exists only for access units of a first group of temporal sublayers among the multiple temporal sublayers, but the indication that a seamless stitching point exists after an access unit does not exist for access units of a second group of temporal sublayers among the multiple temporal sublayers.

[0154] According to an embodiment, the video data stream may, for example, include buffer period supplementation enhancement information. The buffer period supplementation enhancement information indicates a subset of access units among the multiple access units for which a seamless stitching point exists after the access unit. The apparatus may, for example, be configured to process the buffer period supplementation enhancement information.

[0155] In an embodiment, the video data stream may, for example, include a divisor. The apparatus may, for example, be configured to receive the divisor. A delay value may, for example, be assigned to each of the multiple access units of the video data stream. The indication that a seamless stitching point exists in an access unit applies only to those access units among a plurality of access units in the video data stream that have an assigned delay value, which may, for example, be equal to the remainder when divided by the divisor.

[0156] According to an embodiment, the remainder may, for example, be 0.

[0157] In an embodiment, the video data stream may, for example, include a remainder.

[0158] The apparatus may, for example, be configured to receive the remainder.

[0159] According to an embodiment, for each access unit of a subset of access units, a seamless stitching point exists after the access unit.The indicator of the contact can be, for example, a flag. The apparatus can be configured, for example, to process the flag.

[0160] In an embodiment, the flag can be, for example, delay_for_concatenation_ensured_flag.

[0161] According to an embodiment, if a constant frame rate for the output picture obtained from the decoded video data stream cannot be guaranteed, for example, the flag exhibits a first value. If a constant frame rate for the output picture obtained from the decoded video data stream can be guaranteed, for example, the flag exhibits a second value different from the first value.

[0162] In an embodiment, if the output picture obtained from the decoded video data stream has equidistant output times, the flag exhibits a second value.

[0163] According to an embodiment, the apparatus can be, for example, configured to decode an input bitstream to decode video.

[0164] Furthermore, according to an embodiment, a system for encoding video into a video data stream and for decoding video is provided. The system includes a video encoder as described above and an apparatus as described above. A video encoder is used to encode a video into a video data stream, such that the video data stream has video encoded therein. An apparatus is used to receive the video data stream as an input bitstream. Furthermore, the apparatus is used to decode the input bitstream to decode the video.

[0165] In the current VVC draft specification, there are indications that allow splicing devices, i.e., devices that concatenate portions of bitstream A and bitstream B to find AUs in bitstream A that allow seamless splicing of bitstream B. Seamless splicing means that the CPB removal time distance between the last AU of bitstream A and the first AU of bitstream B is the same as the CPB removal time of two consecutive AUs of bitstream A. Conversely, non-seamless splicing would mean that the decoding (CPB removal) of the first AU of bitstream B would be delayed because the first AU of bitstream B cannot be sent to the decoder's CPB at the correct time (i.e., the last AU of bitstream A takes longer to send than expected), and the AU cannot be decoded as quickly as expected. The current picture timing SEI message in the VVC draft specification includes a flag called `delay_for_concatenation_ensured_flag`. When set to 1, this flag indicates that for a particular associated AU, concatenation at that bitstream position will occur as long as the following AU (i.e., the start of bitstream B) has a BP SEI message with `delay_for_concatenation_ensured_flag` equal to 1, and the selected `InitCpbRemovalDelay` is less than or equal to the value of `max_initial_removal_delay_for_concatination` indicated in bitstream A.It will be seamless. Therefore, for all these AUs of bitstream A, delay_for_concatenation_ensured_flag represents the variable option for seamless splicing.

[0166] The problem is that not all of these access units that give the above characteristics will also represent meaningful splicing points, and even for meaningless splicing points, delay_for_concatenation_ensured_flag will be signaled, which may cause some poorly implemented splicing devices to splice at such a less-than-optimal position. In addition, the indication of delay_for_concatenation_ensured_flag for all access units, even for access units that are not meaningful splicing points, causes bitrate overhead and processing overhead on the encoder side in order to set the flag value correctly, even when the flag value is not useful to the splicing device. In this context, meaningless splicing points mean that even if the frame rate is maintained at the actual splicing point, some of these splicing points may omit a part of bitstream A, resulting in discontinuous playback of bitstream A before the splicing point, for example, splicing in the middle of a hierarchical GOP. Figure 5 shows an example where two bitstreams (A and B) are concatenated at two locations (I and II). The first option (top) results in a constant frame rate throughout the example, while the second option (bottom) results in a decreased frame rate due to the selection of an incorrect concatenation point.

[0167] Figure 5 illustrates two concatenation point options, where option I (top) maintains the frame rate throughout the example, while option II (bottom) results in a decreased frame rate in the last GOP of bitstream A.

[0168] Therefore, the present invention reduces signaling overhead and avoids misleading the concatenation device by selectively signaling delay_for_concatenation_ensured_flag for a subset of access units.

[0169] In one embodiment, delay_for_concatenation_ensured_flag is signaled only for AUs of specific inter-time sublayers, where the specific inter-time sublayer is indicated in the associated BP SEI message.

[0170] In another embodiment, delay_for_concatenation_ensured_flag is signaled only at a subset of all AUs within the buffer period, where the subset of AUs is identified by information from the associated BP SEI message.

[0171] In one embodiment, the subset of AUs is indicated by signaling the divisor and an optional remainder value. Only when the value of the associated AU (e.g., cpb_removal_delay or dpb_output_delay in the PT SEI message) is divided by the divisor, etc.The delay_for_concatenation_ensured_flag in the PT SEI message of the buffer period is signaled only when there is an optional remainder, or when it is equal to zero in the absence of a divisor.

[0172] In another embodiment, if the output picture cannot guarantee a constant frame rate as shown in the figure, the value of delay_for_concatenation_ensured_ is set to 0. Note that the current specification currently only focuses on decoding time and ensures that the difference between the last arrival time (the AU has fully arrived at the CPB) and the CPB removal delay satisfies the condition (i.e., > threshold) for AUs with the flag set to 1. Therefore, equidistant decoding times for consecutive AUSs are achieved. Here, the present invention will add that if the flag is set to 1, the output times of consecutive AUSs in bit stream A also have equidistant output times.

[0173] Hereinafter, HRDs specific to the output layer set are described.

[0174] According to an embodiment, an apparatus is provided for receiving a video data stream as an input bit stream, wherein the video data stream has video encoded therein. The apparatus is configured to process the input bit stream in relation to an output layer set to obtain sub-bits. If the output layer set does not include a predefined layer (vps_layer_id[0]) present in a plurality of layers in the video data stream, the apparatus is configured to remove at least one non-expandable nested supplemental enhancement information message allocated with the predefined layer (vps_layer_id[0]). If the output layer set includes the predefined layer (vps_layer_id[0]), the apparatus is configured not to remove any non-expandable nested supplemental enhancement information message allocated with the predefined layer (vps_layer_id[0]).

[0175] Wherein, if an output layer set consisting of all layers in the plurality of layers does not exist in the current decoded video sequence of the video data stream, then, for example, a non-expandable nested supplemental enhancement information message having at least one of a buffer period payload, a picture timing payload, and a decoding unit information payload may not exist.

[0176] According to an embodiment, if there is no output layer set consisting of all layers in the plurality of layers in the current decoded video sequence of the video data stream, then for example, there may be no non-expandable nested supplementary enhancement information message having at least one of the buffer period payload and the picture timing payload and the decoding unit information payload and the sub-picture level information payload.

[0177] In an embodiment, if there is no output layer set consisting of all layers in the plurality of layers in the current decoded video sequence of the video data stream, then for example, there may be no payload type equal to indicating the buffer period.A non-expandable nested supplementary enhancement information message whose payload type is equal to a first value, or equal to a second value indicating the picture timing payload, or equal to a third value indicating the decoding unit information payload, or equal to a fourth value indicating the sub-picture level information payload.

[0178] According to an embodiment, for a non-expandable nested supplementary enhancement information message whose payload type is equal to the first value, or equal to the second value, or equal to the third value, or equal to the fourth value, when present, the non-expandable nested supplementary enhancement information message is applied to all output layer sets consisting of all layers in the current decoded video sequence in the entire bitstream.

[0179] In an embodiment, if there is no output layer set consisting of all layers in the plurality of layers in the current decoded video sequence of the video data stream, then for example, there may be no non-expandable nested supplementary enhancement information message whose payload type is equal to 0 indicating the buffer period payload, or equal to 1 indicating the picture timing payload, or equal to 130 indicating the decoding unit payload, or equal to 203 indicating the sub-picture level information payload.

[0180] According to an embodiment, for a non-expandable nested supplemental enhancement message whose payload type is equal to 0, 1, 130, or 203, when present, the non-expandable nested supplemental enhancement message is applied to all output layer sets consisting of all layers in the current decoded video sequence in the entire bitstream.

[0181] In an embodiment, if the output layer set does not include the predefined layer (vps_layer_id[0]), the apparatus may, for example, be configured to remove all nested supplemental enhancement messages allocated with the predefined layer (vps_layer_id[0]).

[0182] Furthermore, according to an embodiment, a video data stream having video encoded therein is provided. The video data stream specification 20 / 32 pages 23 CN 121037573 A includes a video parameter set. The video parameter set includes a plurality of output layer sets. If none of the plurality of output layer sets includes all layers present in the plurality of layers in the video data stream, the video data stream does not include any non-expandable nested supplemental enhancement messages for assuming a reference decoder.

[0183] Wherein, if there is no output layer set consisting of all the layers in the current decoded video sequence of the video data stream, then for example, there may be no non-scalable nested supplementary enhancement information message having at least one of buffer period payload, picture timing payload, and decoding unit information payload.

[0184] According to an embodiment, if there is no output layer set consisting of all the layers in the current decoded video sequence of the video data stream, then there may be no non-scalable nested supplementary enhancement information message having at least one of buffer period payload, picture timing payload, and decoding unit information payload.The output layer set consisting of all layers in the current decoded video sequence of the video data stream may, for example, lack a non-expandable nested supplementary enhancement information message having at least one of the buffer period payload, the picture timing payload, the decoding unit information payload, and the sub-picture level information payload.

[0185] In an embodiment, if the output layer set consisting of all layers in the plurality of layers in the current decoded video sequence of the video data stream does not exist, the non-expandable nested supplementary enhancement information message may, for example, lack a payload type equal to a first value indicating the buffer period payload, or equal to a second value indicating the picture timing payload, or equal to a third value indicating the decoding unit information payload, or equal to a fourth value indicating the sub-picture level information payload.

[0186] According to an embodiment, for the non-expandable nested supplementary enhancement information message whose payload type is equal to the first value, or equal to the second value, or equal to the third value, or equal to the fourth value, when present, the non-expandable nested supplementary enhancement information message is applied to all the output layer sets consisting of all layers in the current decoded video sequence of the entire bitstream.

[0187] In an embodiment, if there is no output layer set consisting of all layers in the current decoded video sequence of the video data stream, then for example, there may be no non-expandable nested supplemental enhancement information message whose payload type is equal to 0 indicating the buffer period payload, or equal to 1 indicating the picture timing payload, or equal to 130 indicating the decoding unit payload, or equal to 203 indicating the sub-picture level information payload.

[0188] According to an embodiment, for a non-expandable nested supplemental enhancement information message whose payload type is equal to 0, 1, 130, or 203, when present, the non-expandable nested supplemental enhancement information message is applied to all output layer sets consisting of all layers in the current decoded video sequence of the entire bit stream.

[0189] In an embodiment, if the video data stream includes at least one non-expandable nested supplemental enhancement information message for the hypothetical reference decoder, then at least one of the plurality of output layer sets may, for example, include all layers present in the plurality of layers of the video data stream.

[0190] According to an embodiment, all supplementary enhancement information messages used for the hypothetical reference decoder are scalable nested supplementary enhancement information messages, which are independent of the output layer sets of the plurality of output layer sets, which may, for example, include all layers present in the plurality of layers in the video data stream.

[0191] In an embodiment, all non-scalable nested supplementary enhancement information messages used for the hypothetical reference decoder are...The output layer sets in the plurality of output layer sets are associated, and the output layer sets may, for example, include all the layers present in the plurality of layers in the video data stream.

[0192] Furthermore, according to an embodiment, a video encoder is provided for encoding video into a video data stream, such that the video data stream has video encoded therein. The video encoder is used to generate the video data stream such that the video data stream includes a video parameter set. Furthermore, the video encoder is used to generate the video data stream such that the video parameter set includes a plurality of output layer sets. If none of the plurality of output layer sets includes all the layers present in the plurality of layers in the video data stream, then the video encoder is used to generate the video data stream such that the video data stream does not include any non-scalable nested supplementary enhancement information messages for assuming a reference decoder.

[0193] Wherein, if, for example, an output layer set consisting of all the layers in the plurality of layers may not exist in the current decoded video sequence of the video data stream, then, for example, a non-scalable nested supplementary enhancement information message having at least one of a buffer period payload, a picture timing payload, and a decoding unit information payload may not exist.

[0194] According to an embodiment, if, for example, an output layer set consisting of all layers in the plurality of layers does not exist in the current decoded video sequence of the video data stream, then, for example, a non-scalable nested supplementary enhancement information message having at least one of the buffer period payload, the picture timing payload, the decoding unit information payload, and the sub-picture level information payload may not exist.

[0195] In an embodiment, if an output layer set consisting of all layers in the plurality of layers in the current decoded video sequence of the video data stream does not exist, then, for example, a non-scalable nested supplementary enhancement information message with a payload type equal to a first value indicating the buffer period payload, or equal to a second value indicating the picture timing payload, or equal to a third value indicating the decoding unit information payload, or equal to a fourth value indicating the sub-picture level information payload may not exist.

[0196] According to an embodiment, for a non-expandable nested supplemental enhancement information message whose payload type is equal to the first value, or equal to the second value, or equal to the third value, or equal to the fourth value, when present, the non-expandable nested supplemental enhancement information message is applied to all output layer sets consisting of all layers in the current decoded video sequence of the entire bitstream.

[0197] In an embodiment, if there is no output layer set consisting of all layers in the plurality of layers in the current decoded video sequence of the video data stream, then there is no payload type equal to indicating that the buffer period is valid.Non-expandable nested supplemental enhancement information messages with a payload type of 0, or equal to 1 indicating the picture timing payload, or equal to 130 indicating the decoding unit payload, or equal to 203 indicating the sub-picture level information payload.

[0198] According to an embodiment, for non-expandable nested supplemental enhancement information messages with a payload type equal to 0, 1, 130, or 203, when present, the non-expandable nested supplemental enhancement information message is applied to all output layer sets consisting of all layers in the current decoded video sequence in the entire bitstream.

[0199] In an embodiment, if the video data stream may include, for example, at least one non-expandable nested supplemental enhancement information message for assuming a reference decoder, the video encoder may be configured, for example, to generate the video data stream such that at least one of the plurality of output layer sets includes all layers present in the plurality of layers in the video data stream.

[0200] According to an embodiment, the video encoder may be configured, for example, to generate the video data stream such that all supplemental enhancement information messages for the hypothetical reference decoder are scalable nested supplemental enhancement information messages, all of which are independent of the output layer sets of the plurality of output layer sets, which may include, for example, all layers present in the plurality of layers in the video data stream.

[0201] In an embodiment, the video encoder may be configured, for example, to generate the video data stream such that all non-scalable nested supplemental enhancement information messages for the hypothetical reference decoder are associated with the output layer sets of the plurality of output layer sets, which may include, for example, all layers present in the plurality of layers in the video data stream. Specification 22 / 32 pages 25 CN 121037573 A

[0202] According to an embodiment, the apparatus may be configured, for example, to decode the input bitstream to decode the video.

[0203] The current VVC draft specification provides a means to carry HRD timing information (BP SEI message, PT SEI message, DUI SEI message) in a nested form within a bitstream that applies only to sub-bitstreams (i.e., OLSs). Other HRD SEI messages carried directly in the bitstream (as non-expandable nested SEI messages) need to be applied to the 0th OLS, i.e., the single-layer sub-bitstream indicated in the VPS by the syntax element vps_layer_id[0]. However, as is typical of layered coding, removing this layer or any other layer from the bitstream does not require rewriting the vp, so it is even possible for a layer with nuh_layer_id equal to vps_layer_id[0] to not exist.

[0204] The problem with this requirement is that when extracting OLS other than the 0th OLS, no non-expandable nested SEI messages are allowed, even if the extracted OLS does include layers where nuh_layer_id equals vps_layer_id[0]. Such non-expandable nested messages can only be preserved in the bitstream after rewriting the VPS(vps_layer_id[0]) and the corresponding values ​​in the NAL cell headers of each layer.

[0205] Two aspects of the invention that solve this problem are described below.

[0206] 1) As the first solution, the complexity described above is not touched, but the extraction process is adjusted in such a way that when the extracted OLS does not include layers where nuh_layer_id equals vps_layer_id[0], the extraction process only removes non-expandable nested SEI messages applied to layers where nuh_layer_id equals vps_layer_id[0]. Therefore, it is not necessary to unnecessarily delete non-expandable nested SEI messages (or in other words, it is not necessary to unnecessarily copy each OLS that includes layers where nuh_layer_id equals vps_layer_id[0]).

[0207] 2) As a second solution, when no OLS corresponding to the complete bitstream (i.e., all layers in it) is defined in the VPS, non-expandable nested SEI messages (BP, PT, DUI) are not allowed in the bitstream. Otherwise, when an OLS corresponding to all existing layers in the bitstream exists, individual HRD SEI messages are allowed in the bitstream in a non-expandable nested manner. All other HRD SEI messages (applied to other OLS) must be in expandable nested SEI messages. Conversely, all non-expandable nested SEI messages correspond as a whole to the current bitstream.

[0208] For non-expandable nested supplemental enhancement messages, when payloadType equals 0 (indicating buffer period, BP / content), 1 (indicating picture timing, PT, content), 130 (indicating decoding unit information, DUI, content), or 203 (indicating subpicture level information, SLI, content), non-expandable nested supplemental enhancement messages are applied to all output layer sets, which, when present, consist of all layers in the current encoded video sequence in the entire bitstream.

[0209] When there is no output layer set consisting of all layers in the current encoded video sequence of the entire bitstream, there will be no non-expandable nested supplemental enhancement message with payloadType equal to 0 (BP), 1 (PT), 130 (DUI), or 203 (SLI).

[0210] Hereinafter, the rewritten sublayer PTL in DPS is described.

[0211] According to an embodiment, an apparatus for receiving a video data stream as an input bitstream is provided, wherein the video data streamThe video stream has video encoded therein. The video data stream includes initial profile hierarchy information. Furthermore, the video data stream includes a first group of two or more layers. The apparatus is configured to process the input bitstream using an output layer set that depends on a plurality of output layer sets to generate an output bitstream by removing at least one layer from the first group of two or more layers to obtain a second group of one or more layers, such that the output bitstream includes the second group of one or more layers but excludes the at least one layer removed from the first group. Furthermore, the apparatus is configured to remove portions of the initial profile hierarchy information that are irrelevant to at least one layer in the second group of one or more layers.

[0212] According to an embodiment, the video data stream may, for example, include initial decoding parameter set profile hierarchy information as the initial profile hierarchy information. The apparatus may, for example, be configured to remove portions of the initial decoding parameter set profile hierarchy information that are irrelevant to at least one layer in the second group of one or more layers. (See page 23 / 32 of specification 26 CN 121037573 A)

[0213] Furthermore, according to an embodiment, an apparatus is provided for receiving a video data stream as an input bit stream, wherein the video data stream has video encoded therein.

[0214] The video data stream includes expandable nested supplemental enhancement information, the expandable nested supplemental enhancement information including decoding parameter set information for each of a plurality of output layer sets. The apparatus is used to process the input bit stream based on the output layer sets of the plurality of output layer sets to obtain an output bit stream, such that the output bit stream includes the decoding parameter set information for the output layer sets, and such that the output bit stream does not include the decoding parameter set information for any other output layer set of the plurality of output layer sets.

[0215] In an embodiment, the video data stream may, for example, include a raw byte sequence payload. The raw byte sequence payload may, for example, include the decoding parameter set information for each of the plurality of output layer sets.

[0216] According to an embodiment, the video data stream may, for example, include an expandable nested supplemental enhancement message, the expandable nested supplemental enhancement information may, for example, include the decoding parameter set information for each of the plurality of output layer sets.

[0217] In an embodiment, the video data stream may, for example, include a nesting type flag (nesting_type), which includes one or more bits.

[0218] According to an embodiment, if the nesting type flag (nesting_type) exhibits a first value, this indicates that the Scalable Nested Supplement Enhancement Message may, for example, include one or more sets of decoding parameters, the one or more decoding parameters...The set includes the decoding parameter set information and indicates that no additional supplementary enhancement messages are nested within the Scalable Nested Supplementary Enhancement Message.

[0219] In an embodiment, if the nesting type flag (nesting_type) exhibits a second value, this indicates that the one or more supplementary enhancement messages nested within the Scalable Nested Supplementary Enhancement Message are applied to the output layer set.

[0220] According to an embodiment, if the nesting type flag (nesting_type) exhibits a third value, this indicates that the one or more supplementary enhancement messages nested within the Scalable Nested Supplementary Enhancement Message are applied to a subset of the output layer set.

[0221] In an embodiment, if the nesting type flag (nesting_type) exhibits a fourth value, this indicates that the Scalable Nested Supplementary Enhancement Message may, for example, include one or more decoding parameter sets, the one or more decoding parameter sets including the decoding parameter set information.

[0222] Furthermore, according to an embodiment, a video data stream having video encoded therein is provided. The video data stream may, for example, include Scalable Nested Supplementary Enhancement Information, the Scalable Nested Supplementary Enhancement Information including decoding parameter set information for each of a plurality of output layer sets.

[0223] In an embodiment, the video data stream may, for example, include a raw byte sequence payload. The raw byte sequence payload may, for example, include the decoding parameter set information for each of the plurality of output layer sets.

[0224] According to an embodiment, the video data stream may, for example, include an expandable nested supplemental enhancement message, which may, for example, include the expandable nested supplemental enhancement information, which may, for example, include the decoding parameter set information for each of the plurality of output layer sets.

[0225] In an embodiment, the video data stream may, for example, include a nesting type flag (nesting_type), which includes one or more bits.

[0226] According to an embodiment, if the nesting type flag (nesting_type) exhibits a first value, this indicates that the expandable nested supplemental enhancement message may, for example, include one or more decoding parameter sets, which include the decoding parameter set information, and indicates that no other supplemental enhancement message is nested within the expandable nested supplemental enhancement message.

[0227] In an embodiment, if the nesting type flag (nesting_type) exhibits a second value, this indicates that the one or more supplementary enhancement messages nested within the expandable nested supplementary enhancement messages are applied to the output layer set.

[0228] According to an embodiment, if the nesting type flag (nesting_type) exhibits a third value, this indicates that one or more supplementary enhancement messages nested in the Scalable Nested Supplemental Enhancement Message are applied to a subset of the output layer set, for example, where one or more sub-pictures are assigned to the subset.

[0229] In an embodiment, if the nesting type flag (nesting_type) exhibits a fourth value, this indicates that the Scalable Nested Supplemental Enhancement Message may, for example, include one or more decoding parameter sets, the one or more decoding parameter sets including decoding parameter set information.

[0230] Furthermore, according to an embodiment, a video encoder is provided for encoding video into a video data stream, such that the video data stream has video encoded therein. The video encoder is used to generate the video data stream such that the video data stream includes Scalable Nested Supplemental Enhancement Information, the Scalable Nested Supplemental Enhancement Information including decoding parameter set information for each output layer set in a plurality of output layer sets.

[0231] In an embodiment, the video encoder may, for example, be configured to generate the video data stream such that the video data stream may, for example, include the original byte sequence payload. A video encoder may be configured, for example, to generate the video data stream such that the original byte sequence payload may include, for example, the decoding parameter set information for each of the plurality of output layer sets.

[0232] According to an embodiment, a video encoder may be configured, for example, to generate the video data stream such that the video data stream may include, for example, an extensible nested supplemental enhancement message, the extensible nested supplemental enhancement message may include, for example, the extensible nested supplemental enhancement information, the extensible nested supplemental enhancement information may include, for example, the decoding parameter set information for each of the plurality of output layer sets.

[0233] In an embodiment, a video encoder may be configured, for example, to generate a video data stream such that the video data stream includes a nesting type flag (nesting_type), the nesting type flag (nesting_type) may include, for example, one or more bits.

[0234] According to an embodiment, the video encoder may be configured, for example, to generate the video data stream such that if the nesting type flag (nesting_type) exhibits a first value, this indicates that the Scalable Nested Supplemental Enhancement Message includes one or more decoding parameter sets, which may include, for example, decoding parameter set information, and indicates that no other supplemental enhancement messages are nested within the Scalable Nested Supplemental Enhancement Message.

[0235] In an embodiment, the video encoder may be configured, for example, to generate the video data stream such that if the nesting type flag (nesting_type) exhibits a second value, this indicates that the Scalable Nested Supplemental Enhancement Message is nested within the Scalable Nested Supplemental Enhancement Message.One or more supplementary enhancement messages within the message are applied to the output layer set.

[0236] According to an embodiment, the video encoder may be configured, for example, to generate the video data stream such that if the nesting type flag (nesting_type) exhibits a third value, this indicates that one or more supplementary enhancement messages nested within the expandable nested supplementary enhancement messages are assigned to a subset of the output layer set, for example, where one or more sub-pictures are assigned to the subset.

[0237] In an embodiment, the video encoder may, for example, generate the video data stream such that if the nesting type flag (nesting_type) exhibits a fourth value, this indicates that the expandable nested supplementary enhancement messages may, for example, include one or more decoding parameter sets, the one or more decoding parameter sets including decoding parameter set information. Specification 25 / 32 pages 28 CN 121037573 A

[0238] Furthermore, according to an embodiment, a video data stream having video encoded therein is provided. The video data stream includes indications of a plurality of operation points. Furthermore, the video data stream includes a first mapping for each of the plurality of operation points, the first mapping assigning one or more of a plurality of profile levels to the operation point. Furthermore, the video data stream includes a plurality of second mappings, wherein each of the plurality of mappings assigns one of the plurality of operation points to one output level set in a plurality of output level sets.

[0239] In an embodiment, the video data stream may, for example, include a set of decoding parameters. The set of decoding parameters may, for example, include indications of a plurality of operation points.

[0240] According to an embodiment, the video data stream may, for example, include a number of layer indications indicating the number of layers in the video data stream; and / or, the video data stream may, for example, include a number of sub-layer indications indicating the number of sub-layers in the video data stream.

[0241] In an embodiment, the number of layers in the video data stream may, for example, be constant; and / or the number of sub-layers in the video data stream may, for example, be constant.

[0242] According to an embodiment, the video data stream may, for example, include scalable nested supplemental enhancement information, the scalable nested supplemental enhancement information including decoding parameter set information for each output level set in a plurality of output level sets.

[0243] Furthermore, according to an embodiment, a video encoder for encoding video into a video data stream is provided, such that the video data stream has video encoded therein. The video encoder is used to generate the video data stream such that the video data stream includes indications of multiple operation points. Furthermore, the video encoder is used to generate the video data stream such that the video data stream includes a first mapping for each of the multiple operation points, the first mapping assigning one or more profile levels from a plurality of profile levels to the operation point. Additionally, the video encoder is used to generate...The video data stream is described such that the video data stream includes a plurality of second mappings, wherein each of the plurality of mappings assigns one of the plurality of operation points to an output layer set of a plurality of output layer sets.

[0244] In an embodiment, the video encoder may be configured, for example, to generate the video data stream such that the video data stream may include, for example, a set of decoding parameters. The video encoder may be configured, for example, to generate the video data stream such that the set of decoding parameters may include, for example, the indications of the plurality of operation points.

[0245] According to an embodiment, the video encoder may be configured, for example, to generate the video data stream such that the video data stream may include, for example, a number of layer indications indicating the number of layers in the video data stream; and / or the video encoder may be configured, for example, to generate the video data stream such that the video data stream may include, for example, a number of sublayer indications indicating the number of sublayers in the video data stream.

[0246] In an embodiment, the video encoder may be configured, for example, to generate the video data stream such that the number of layers in the video data stream may be, for example, constant; and / or the video encoder may be configured, for example, to generate the video data stream such that the number of sublayers in the video data stream may be, for example, constant.

[0247] According to an embodiment, a video encoder may be configured, for example, to generate the video data stream such that the video data stream may include, for example, scalable nested supplemental enhancement information, the scalable nested supplemental enhancement information including decoding parameter set information for each of a plurality of output layer sets.

[0248] Furthermore, according to an embodiment, an apparatus is provided for receiving a video data stream as an input bit stream, wherein the video data stream has video encoded therein. The video data stream includes indications of a plurality of operation points. Furthermore, the video data stream includes a first mapping for each of the plurality of operation points, the first mapping mapping one or more of a plurality of profile levels to the operation point. Furthermore, the video data stream includes a plurality of second mappings, wherein each of the plurality of mappings maps one of the plurality of operation points to an output level in a plurality of output level sets. The apparatus is used to process the input bit stream to produce an output bit stream based on at least one of the second mappings that map one of the plurality of operation points in the plurality of output layer sets.

[0249] In an embodiment, the video data stream may, for example, include a set of decoding parameters. The set of decoding parameters may, for example, include indications of a plurality of operation points. The apparatus is used to process the set of decoding parameters.

[0250] According to an embodiment, the video data stream may, for example, include layer indications indicating the number of layers in the video data stream.The number of layers in the video data stream; and / or, the video data stream may, for example, include a number of sub-layer indications indicating the number of sub-layers in the video data stream.

[0251] In an embodiment, the number of layers in the video data stream may, for example, be constant; and / or the number of sub-layers in the video data stream may, for example, be constant.

[0252] According to an embodiment, the video data stream may, for example, include expandable nested supplemental enhancement information, the expandable nested supplemental enhancement information including decoding parameter set information for each of a plurality of output layer sets. The apparatus may, for example, be configured to process the expandable nested supplemental enhancement information.

[0253] According to an embodiment, the apparatus may, for example, be configured to decode the input bitstream to decode the video.

[0254] Furthermore, according to an embodiment, a system for encoding video into a video data stream and for decoding the video is provided. The system includes a video encoder as described above and a decoding apparatus as described above. The video encoder is used to encode the video into the video data stream such that the video data stream has the video encoded therein. The apparatus is used to receive the video data stream as an input bitstream. Furthermore, the apparatus is used to decode the input bitstream to decode the video.

[0255] The VVC draft specification includes a set of decoding parameters (DPS) that describes the constraints (PTL) of the bitstream. It can be used for capability exchange and negotiation of SDPs in such cases as RTSP / SIP, or for stream selection in adaptive streaming scenarios. Therefore, its purpose is to indicate the maximum capability required for the local stream (a concatenation of CVSs), and thus its scope is larger than all the parameter sets defined in the corresponding previous generation codec generation (HEVC and AVC) that only had the scope of coded video sequences (CVSs). Now, the VVC draft specification describes the constraints of the CVSS concatenation, i.e., the bitstream. The DPS mainly carries multiple profile hierarchy information (PTLs) including all PTLs used in the bitstream, i.e., all PTLs in this list need to be supported in order to successfully decode the corresponding bitstream.

[0256] When bitstream extraction or pruning is performed, some of these layers are removed. After processing as described on pages 27 / 32 of this specification, 30 CN 121037573 A, the resulting bitstream may no longer carry the entire set of layers; for example, extracting layer 3 containing the version (corresponding OLS) from a layer 6 bitstream. Therefore, in cases where the DPS cannot accurately describe the PTL of the content of the pruned bitstream, the DPS can no longer serve its purpose of capability negotiation.

[0257] Therefore, as part of the present invention, three options for mitigating this problem and allowing capability negotiation using the DPS are described.

[0258] Hereinafter, DPS rewriting during the extraction process is described.

[0259] During extraction, the PTL information in the DPS is adjusted because only the PTL information corresponding to the remaining layers after extraction is retained in the DPS, while other PTL information is removed.

[0260] Hereinafter, sub-bitstream-specific DPS nesting is described.

[0261] An expandable nested SEI message carrying an OLS-specific DPS RBSP and extraction is defined, which extracts the corresponding DPS RBSP through the bitstream extraction process and writes it into a new DPS in the output bitstream.

[0262] In an embodiment, a new nested SEI message carrying a parameter set is used.

[0263] In another embodiment, a single expandable nested SEI message is used to carry a parameter set or other SEI messages. In this example, syntax elements are added to the expandable nested SEI message to indicate what is being performed internally.

[0264] In the syntax above, it is assumed that no other SEI is included when a parameter set (nesting_type == 1) is included. In another embodiment, the parameter set and other SEIs can be included in the nested SEI message simultaneously.

[0265] In another embodiment, nesting_type can be used for several purposes. Instructions: Page 28 / 32 of the specification 31 CN 121037573 A · SEI messages nested within nested SEIs are applied to OLS · SEI messages nested within nested SEIs are applied to a subset of OLS, e.g., sub-pictures within OLS · A set of parameters is nested within a nested SEI. The operation points in the DPS are described below.

[0266] DPS rewriting during extraction and sub-bitstream-specific DPS nesting may work in certain situations (e.g., constrained bitstreams). For example, DPS rewriting during extraction. It may only work if all VPSs of the bitstream are known in advance and the number of layers / sublayers does not change from CVS to CVS. A similar situation occurs in the case of sub-bitstream-specific DPS nesting, because in principle, the OLS (e.g., their number and id) may change from CVS to CVS.

[0267] In another embodiment, operation points are defined. They may be defined in the DPS and provided as a list for each PTL. In addition to the OLS index, an operation point index is added as input to the decoding process to describe and select the appropriate operation point. Then, some association between the operation point and the OLS is signaled in the VPS.

[0268] The syntax elements of the DPS will be changed by removing the number of sublayers as follows, since this may change from CVS to CVS and include operation points.

[0269] In another embodiment, num_sub_layer and the number of layers remain constant in the bitstream. This indication is added to the bitstream and indicates the mapping between the operation point and sublayers or multiple layers.

[0270] In another embodiment, the Scalable Nested SEI message defined in 4.2 is applied to the operation point and in the bitstream...The mapping to OLS is also provided.

[0271] Hereinafter, the MaxSubLayer in the Param set is described.

[0272] According to an embodiment, a video data stream having video encoded therein is provided. The video data stream includes a sequence parameter set and a decoding parameter set. The sequence parameter set includes a first indication value, which indicates the maximum number of sublayers or the maximum number of sublayers minus a constant value based on information stored in the sequence parameter set. The decoding parameter set includes a second indication value, which indicates the maximum number of sublayers or the maximum number of sublayers minus the constant value based on information stored in the decoding parameter set. The first indication value is less than or equal to the second indication value.

[0273] In an embodiment, the video data stream may, for example, include a sequence parameter set and a decoding parameter set. The sequence parameter set may, for example, include a first indication value, which indicates the maximum number of sublayers or the maximum number of sublayers minus a constant value based on information stored in the sequence parameter set. The decoding parameter set may, for example, include a second indication value, which indicates the maximum number of sublayers or the maximum number of sublayers minus the constant value based on information stored in the decoding parameter set. The second indication value of the decoding parameter set is an upper boundary prior to the first indication value of the sequence parameter set.

[0274] According to an embodiment, the constant value may, for example, be 1.

[0275] Furthermore, according to an embodiment, a video encoder is provided for encoding video into a video data stream, such that the video data stream has video encoded therein. The video encoder is used to generate the video data stream, such that the video data stream includes a sequence parameter set and a decoding parameter set. The video encoder is used to generate the video data stream, such that the sequence parameter set includes a first indication value, which indicates the maximum number of sublayers or the maximum number of sublayers minus the constant value based on information stored in the sequence parameter set. Furthermore, the video encoder generates the video data stream, such that the decoding parameter set includes a second indication value, which indicates the maximum number of sublayers or the maximum number of sublayers minus the constant value based on information stored in the decoding parameter set. Furthermore, the video encoder is used to generate the video data stream such that the first indication value is less than or equal to the second indication value.

[0276] Furthermore, according to an embodiment, a video encoder is provided for encoding video into a video data stream, such that the video data stream has video encoded therein. The video encoder is used to generate the video data stream such that the video data stream includes a sequence parameter set and a decoding parameter set. Furthermore, the video encoder is used to generate the video data stream,The sequence parameter set includes a first indication value, which indicates the maximum number of sublayers or the maximum number of sublayers minus a constant value based on information stored in the sequence parameter set. Furthermore, the video encoder generates the video data stream such that the decoding parameter set includes a second indication value, which indicates the maximum number of sublayers or the maximum number of sublayers minus the constant value based on information stored in the decoding parameter set. The second indication value of the decoding parameter set is an upper boundary prior to the first indication value of the sequence parameter set.

[0277] In an embodiment, the constant value may, for example, be 1. The video encoder may be configured, for example, to generate the video data stream such that the sequence parameter set may include, for example, the first indication value, which indicates the maximum number of sublayers or the maximum number of sublayers minus 1 based on information stored in the sequence parameter set. The video encoder may be configured, for example, to generate the video data stream such that the decoding parameter set may include, for example, the second indication value, which indicates the maximum number of sublayers or the maximum number of sublayers minus 1 based on information stored in the decoding parameter set.

[0278] Furthermore, according to an embodiment, an apparatus is provided for receiving a video data stream as an input bit stream, wherein the video data stream has video encoded therein. The video data stream includes a sequence parameter set and a decoding parameter set. The sequence parameter set includes a first indication value, which indicates a maximum number of sub-layers or the maximum number of sub-layers minus a constant value based on information stored in the sequence parameter set. The decoding parameter set includes a second indication value, which indicates the maximum number of sub-layers or the maximum number of sub-layers minus the constant value based on information stored in the decoding parameter set. If the first indication value is greater than the second indication value, the apparatus is used to process the input bit stream to generate an output bit stream, such that the output bit stream includes the second indication value as an indication of the maximum number of sub-layers or the maximum number of sub-layers minus the constant value.

[0279] In an embodiment, the constant value may, for example, be 1. The sequence parameter set may, for example, include a first indication value, which indicates the maximum number of sublayers or the maximum number of sublayers minus 1, based on information stored in the sequence parameter set. The decoding parameter set may, for example, include a second indication value, which indicates the maximum number of sublayers or the maximum number of sublayers minus 1, based on information stored in the decoding parameter set. If the first indication value can, for example, be greater than the second indication value, the apparatus may, for example, be configured to process...The input bitstream is described to generate an output bitstream such that the output bitstream may include, for example, the second indication value as the maximum number of sublayers minus 1.

[0280] When bitstream pruning (also known as sublayer extraction) is performed and NAL units belonging to a specific temporal ID are discarded from the bitstream, the parameter set is generally not modified, although some high-level parameter sets, such as DPS, may be rewritten.

[0281] This leads to inconsistencies within the bitstream, making the decoding process unclear or more complicated. For example, if DPS is rewritten as described in the previous section, and the actual number of sublayers is indicated as dps_max_sublayers_minus1 (excluding discarded temporal IDs), but the value at SPS is not modified (which is not intended), sps_max_sublayers_minus1 will have a higher value than the value at DPS. The decoder will not really know which value to trust.

[0282] In one embodiment, the bitstream constraint is that sps_max_sublayers_minus1 must <= dps_max_sublayers_minus1. Therefore, if such a "problem" is encountered, some type of error resilience can be built. With such a constraint, it can be clearly seen that when parsing a "bitstream" that violates the condition, the SPS with a higher value will simply correspond to the new bitstream (e.g., not detected due to the missing end-of-bitstream (EOB) NAL unit). This will mean that when bitstream pruning is performed, dps_max_sublayers_minus1 cannot be changed unless sps_max_sublayers_minus1 is also changed.

[0283] In another embodiment, this constraint is not required, and DPS signaling is used to set the operation point or output layer set of the bitstream sent to the decoder. That is, when dps_max_sublayers_minus1 is less than sps_max_sublayers_minus1, the value signaled in the DPS takes precedence over the value of dps_max_sublayers_minus1 indicated in the SPS for the bitstream sent to the decoder.

[0284] Since the number of sublayers may differ between different layers when there are more than one layer, dps_max_sublayers_minus1 applies to all output layers, while all non-output layers have dps_max_sublayers_minus1 or its corresponding sps_max_sublayers_minus1 (if the latter is less than dps_max_sublayers_minus1).

[0285] Although some aspects have been described in the context of the apparatus, it is clear that these aspects also represent descriptions of the corresponding methods.In this description, a block or device corresponds to a method step or a feature of a method step. Similarly, aspects described in the context of method steps also represent descriptions of corresponding blocks or items or features of corresponding devices. Some or all of the method steps can be executed by (or using) hardware devices such as, for example, microprocessors, programmable computers, or electronic circuits. In some embodiments, one or more of the most important method steps can be executed by such devices.

[0286] Depending on certain implementation requirements, embodiments of the invention can be implemented in hardware or in software or at least partially in hardware or at least partially in software. Implementations can be carried out using digital storage media (e.g., floppy disks, DVDs, Blu-rays, CDs, ROMs, PROMs, EPROMs, EEPROMs, or flash memory) having electronically readable control signals stored thereon that cooperate with (or are capable of cooperating with) a programmable computer system to perform the corresponding method. Thus, the digital storage medium can be computer-readable.

[0287] Some embodiments of the invention include a data carrier having electronically readable control signals that are capable of cooperating with a programmable computer system to perform one of the methods described herein.

[0288] Generally, embodiments of the present invention can be implemented as a computer program product having program code that is operational so as to perform one of the methods when the computer program product is run on a computer. The program code may, for example, be stored on a machine-readable medium. Specification 31 / 32 pages 34 CN 121037573 A

[0289] Other embodiments include a computer program stored on a machine-readable medium for performing one of the methods described herein.

[0290] In other words, therefore, an embodiment of the inventive method is a computer program having program code for performing one of the methods described herein when the computer program is run on a computer.

[0291] Therefore, another embodiment of the inventive method is a data carrier (or digital storage medium or computer-readable medium) including a computer program recorded thereon for performing one of the methods described herein. The data carrier, digital storage medium, or recorded medium is typically tangible and / or non-transitory.

[0292] Therefore, another embodiment of the inventive method is a data stream or signal sequence representing a computer program for performing one of the methods described herein. This data stream or signal sequence may, for example, be configured to be transferred via a data communication connection (e.g., via the Internet).

[0293] Further embodiments include processing components configured or adapted to perform one of the methods described herein, such as a computer or programmable logic device.

[0294] Another embodiment includes a computer having a computer program installed thereon for performing one of the methods described herein.

[0295] Another embodiment of the invention includes an apparatus or system configured to transfer (e.g., electronically or optically) a computer program for performing one of the methods described herein to a receiver. The receiver may be, for example, a computer, a mobile device, a memory device, etc. The apparatus or system may include, for example, a file server for transferring the computer program to the receiver.

[0296] In some embodiments, a programmable logic device (e.g., a field-programmable gate array) may be used to perform some or all of the functionality of the methods described herein. In some embodiments, the field-programmable gate array may cooperate with a microprocessor to perform one of the methods described herein. Generally, the method is preferably performed by any hardware device.

[0297] The apparatus described herein may be implemented using a hardware device or using a computer or a combination of a hardware device and a computer.

[0298] The methods described herein may be implemented using a hardware device or using a computer or a combination of a hardware device and a computer.

[0299] The above embodiments are merely illustrative of the principles of the invention. It is understood that modifications and variations of the arrangements and details described herein will be readily apparent to others skilled in the art. Therefore, the intent of the invention is limited only by the scope of the outgoing patent claims and not by the specific details presented in the manner of describing and interpreting the embodiments herein.

[0300] References [1] ISO / IEC, ITU-T. High-efficiency video decoding. ITU-T Recommendation H.265 | ISO / IEC 23008 10 (HEVC), 1st edition, 2013; 2nd edition, 2014. Instruction manual, 32 / 32 pages; 35 CN 121037573 A; Figure 1; Instruction manual, Figure 1 / 8 pages; 36 CN 121037573 A; Figure 2; Instruction manual, Figure 2 / 8 pages; 37 CN 121037573 A; Figure 3; Instruction manual, Figure 3 / 8 pages; 38 CN 121037573 A; Figure 4; Instruction manual, Figure 4 / 8 pages; 39 CN 121037573 A; Figure 5; Instruction manual, Figure 5 / 8 pages; 40 CN 121037573 A; Figure 6; Instruction manual, Figure 6 / 8 pages; 41 CN 121037573 A; Figure 7; Instruction manual, Figure 7 / 8 pages; 42 CN 121037573 A; Figure 8; Instruction manual, Figure 8 / 8 pages; 43 CN 121037573 A; Abstract Abdominalultrasound examination method, system and device HKP2520746 Title: Video data stream, video encoder, apparatus and methods for a hypothetical reference decoder and for output layer sets Abstract: The invention relates to a video data stream, a video encoder, an apparatus and a method for a hypothetical reference decoder and for outputting a set of layers. An apparatus for receiving a video data stream as an input bitstream is provided. The video data stream has video encoded therein, wherein, for a playback speed modification factor (Nx), the apparatus is to determine a decoding capability requirement depending on a decoding capability requirement limit, wherein the playback speed modification factor (Nx) is a forward playback speed modification factor or is a backward playback speed modification factor.

Claims

1. A method for receiving a video data stream as an input bit stream, wherein, Video data streams contain video encoded into them. Specifically, for the playback speed modification factor (Nx), the method includes determining the decoding capability requirement based on decoding capability requirement constraints. The playback speed modification factor (Nx) is either a forward playback speed modification factor or a backward playback speed modification factor.

2. A method for encoding video into a video data stream, such that the video data stream has the video encoded therein. in, The method includes generating the video data stream such that the video data stream includes information about sampling rate limits, and / or The method includes generating the video data stream such that the video data stream includes information about bitrate limits.

3. A video data stream having video encoded therein, in, The video data stream includes multiple access units, wherein the access unit group of the video data stream includes the multiple access units. The video data stream includes an indication for each access unit in the subset of access units, indicating the existence of a seamless stitching point after the access unit. The access unit subset is an appropriate subset of the access unit group of the video data stream.

4. A video encoder for encoding video into a video data stream, such that the video data stream has the video encoded therein. in, The video encoder is used to generate the video data stream, such that the video data stream includes multiple access units, wherein the access unit group of the video data stream includes the multiple access units. The video encoder is used to generate the video data stream, such that the video data stream includes an indication for each access unit of the subset of access units, indicating the existence of a seamless stitching point after the access unit. The video encoder is used to generate the video data stream such that the subset of access units is an appropriate subset of the group of access units in the video data stream.

5. An apparatus for receiving a video data stream as an input bit stream, wherein, The video data stream contains video encoded within it. The device is used to process the video data stream. The video data stream includes multiple access units, and the access unit group of the video data stream includes the multiple access units. The video data stream includes an indication for each access unit in the subset of access units, indicating the existence of a seamless stitching point after the access unit. Wherein, the subset of access units is an appropriate subset of the group of access units in the video data stream. The device is used to process the instruction.

6. A method for encoding video into a video data stream, such that the video data stream has the video encoded therein. in, The method includes generating the video data stream such that the video data stream includes a plurality of access units, wherein the access unit group of the video data stream includes the plurality of access units. The method includes generating the video data stream such that the video data stream includes an indication for each access unit of the access unit subset, indicating the existence of a seamless stitching point after the access unit. The method includes generating the video data stream such that the subset of access units is an appropriate subset of the group of access units in the video data stream.

7. A method for receiving a video data stream as an input bit stream, wherein, The video data stream contains video encoded within it. The method includes processing the video data stream. The video data stream includes multiple access units, and the access unit group of the video data stream includes the multiple access units. The video data stream includes an indication for each access unit in the subset of access units, indicating the existence of a seamless stitching point after the access unit. Wherein, the subset of access units is an appropriate subset of the group of access units in the video data stream. The method includes processing the instruction.

8. An apparatus for receiving a video data stream as an input bit stream, wherein, Video data streams contain video encoded into them. The device is used to process the input bitstream in relation to an output layer set to obtain a sub-bitstream. Wherein, if the output layer set does not include a predefined layer (vps_layer_id[0]) among the multiple layers existing in the video data stream, the device is used to remove at least one non-expandable nested supplementary enhancement information message allocated together with the predefined layer (vps_layer_id[0]), and If the output layer set includes the predefined layer (vps_layer_id[0]), the device is configured not to remove any non-expandable nested supplementary enhancement information messages allocated together with the predefined layer (vps_layer_id[0]).

9. A video data stream having video encoded therein, in, The video data stream includes a set of video parameters. The video parameter set includes multiple output layer sets. If none of the plurality of output layer sets includes all layers present in the plurality of layers in the video data stream, then the video data stream does not include any non-scalable nested supplementary enhancement information messages used for assuming a reference decoder.

10. A video encoder for encoding video into a video data stream, such that the video data stream has the video encoded therein. in, The video encoder is used to generate the video data stream, such that the video data stream includes a set of video parameters. The video encoder is used to generate the video data stream, such that the video parameter set includes multiple output layer sets. If none of the plurality of output layer sets includes all layers present in the plurality of layers in the video data stream, then the video encoder generates the video data stream such that the video data stream does not include any non-expandable nested supplementary enhancement information messages used for assuming a reference decoder.

Images