File parser and method for reconstructing video bitstream

Abstract

The invention relates to a file parser, a file generator, an encoder, a decoder, a client, a server, and a method using a parameter set for encoding a video sequence. The parameter set may include decoding parameter information and may be transmitted in-band or out-of-band. Some embodiments provide a solution to the problem related to correctly initializing a decoder used to decode an encoded video sequence in an open GOP switching scenario. Some embodiments may provide hierarchical track switching levels.

Description

Description of the case

[0001] This application is a divisional application of the invention patent application with the application date of December 20, 2021, application number 202180092392.7, and invention name "File parser, file generator, encoder, decoder, client, server and method using parameter sets for encoding video sequences". Technical Field

[0002] The present invention relates to video coding. Embodiments of the present disclosure relate to file parsers, file generators, encoders, decoders, clients, servers, and corresponding methods for selecting parameter sets for encoding video sequences. Some embodiments relate to sample entry selection for stream switching. Background Art

[0003] In the field of video coding, parameter sets can be used, for example, to initialize the decoding behavior of a decoder. These parameter sets may include decoding parameter information to be used at the beginning of an encoded video sequence in order to correctly decode the pictures contained in the encoded video sequence. In some cases, a change in a parameter set may trigger the start of a new encoded video sequence.

[0004] ISOBMFF allows parameter sets to be stored in two different ways. The first way consists of so-called out-of-band parameter set integration, which means that the parameter set is not stored together with the other non-VCL and VCL NAL units of the AU (samples for ISOBMFF), but is stored in the sample entry of the track. Basically, the sample entry gives detailed information about the type of encoding used and any initialization information required for that encoding. This information includes the parameter set. A specific track can have several sample entries applicable to different parts of the bitstream. For example, if the bitstream consists of two coded video sequences (CVSs), each CVS referring to a different SPS with the same ID (with different content), two sample entries are required, in which the SPS with the same ID is stored. The samples within the track point to the correct sample entry used, whether the SampleToChunkBox "stcs" is used or when the samples appear in fragmented tracks, such as when MPEG-DASH fragments are used, the track fragment header of the track contained in, for example, a DASH fragment ("tfhd") contains a sample_description_entry pointing to the sample entry used. Therefore, the correct out-of-band parameters can be used to decode the sample identified by the correct sample description index.

[0005] Another alternative is to transmit the parameter sets together with the samples as so called in-band parameter sets, so they are already present in the bitstream within the AU as needed.

[0006] However, especially for track switching or representation switching in DASH, some problems may occur if not handled properly.

[0007] Specifically, VVC allows open GOP resolution switching, which is advantageous because higher efficiency can be achieved compared to closed GOP encoding, but several issues need to be considered in the adaptive HTTP streaming environment.

[0008] It is therefore an object of the present invention to provide a solution for parameter selection in a video coding environment, for example in case switching between bitstreams, tracks or representations occurs. Summary of the invention

[0009] According to the invention, these solutions are provided by a file parser, a file generator, an encoder, a decoder, a client, a server and corresponding methods according to independent embodiments. Advantageous implementations and embodiments are subject matter of the dependent embodiments.

[0010] A first aspect relates to hierarchical track grouping (e.g., in a sample entry) for initialization parameter selection, and hierarchical entity grouping (e.g., in a sample entry) for initialization parameter selection, and track grouping (e.g., in a sample entry) with additional / alternative initialization parameters. In this first aspect, the initialization parameters may be transmitted out-of-band in the sample entry.

[0011] According to this first aspect, a file parser for reconstructing a video bitstream from a video file is proposed, wherein the video file may include different tracks with at least two input video bitstreams embedded therein, wherein the video content may be differently encoded in the at least two input video bitstreams, and wherein each input video bitstream may include one or more coded video sequences (CVS) with random access points. The file parser is configured to retrieve switching information from the video file (e.g., for each representation), the switching information indicating whether the different tracks include one or more switching to candidates for the corresponding track and which track of the different tracks is the one or more switching to candidates for the corresponding track. The term "for each representation" refers to each DASH representation that can currently be processed by the file parser or the client. This means that the file parser can perform the steps described herein for each representation currently processed by the file parser, wherein the file parser can only process a single representation at a time. For example, if a representation switching is performed, it can be switched from a first representation to a second representation. Therefore, the file parser processes the first representation at a first time instance and performs all the steps as described herein for the first representation. After the switch, the file parser processes the second representation at a second time instance and performs all the steps as described herein for the second representation. Therefore, the file parser performs the steps as described herein "for each representation", in this example for two representations. Of course, if switching is performed between more than two representations, the file parser can also perform the steps as described herein for the third representation, the fourth representation, etc. However, each single representation can only be processed by the file parser at a single time instance, that is, "for each representation" should not be interpreted as causing the file parser to process multiple representations simultaneously. If representation switching is not performed, that is, if there is only one representation, the file parser will only process this one representation. Therefore, in this case, the file parser can perform the steps as described herein "for each representation" (that is, for this one representation). The file parser is configured to retrieve one or more initialization parameter sets to be inserted into the reconstructed bitstream and referenced by its samples from the video file (for example, for each representation). The reconstructed bitstream includes any one of an input video bitstream embedded in the corresponding track or a hybrid video bitstream, which is generated from a switch between an input video bitstream embedded in the corresponding track and an input video bitstream embedded in one or more switch-to candidates belonging to the corresponding track. The file parser is further configured to provide the decoder with a predetermined set of initialization parameters retrieved from the video file of the track of interest, and to feed the decoder with an input video bitstream embedded in the corresponding track and / or a hybrid video bitstream resulting from switching between the input video bitstream embedded in the corresponding track and an input video bitstream embedded in one or more switch-to candidates belonging to the corresponding track.

[0012] Further with respect to the first aspect, a file generator for generating a video file from at least two input video bitstreams is proposed, wherein video content is differently encoded in the at least two input video bitstreams, and wherein each input video bitstream includes one or more coded video sequences (CVS) with random access points. The file generator is configured to embed the at least two input video bitstreams into different tracks of the video file, and the tracks can be switched between each other at the random access points of the embedded input video bitstreams. The file generator is also configured to provide switching information in the video file for each track, the switching information indicating whether the different tracks include one or more switch-to candidates for the corresponding track and which track in the different tracks is the one or more switch-to candidates for the corresponding track. The file generator is also configured to provide one or more initialization parameter sets to be inserted into the reconstructed bitstream and referenced by its samples in the video file. The reconstructed bitstream includes any one of the input video bitstream embedded in the corresponding track or a hybrid video bitstream, the hybrid video bitstream being generated from the switching between the input video bitstream embedded in the corresponding track and the input video bitstream embedded in one or more switch-to candidates belonging to the corresponding track.

[0013] Still further with respect to the first aspect, a method for reconstructing a video bitstream from a video file is proposed, wherein the video file comprises different tracks having at least two input video bitstreams embedded therein, wherein the video content is differently encoded in the at least two input video bitstreams, and wherein each input video bitstream comprises one or more coded video sequences (CVS) having random access points. The method comprises the following steps: (e.g. for each representation) retrieving from the video file switching information indicating whether the different tracks comprise one or more switch-to candidates for the corresponding track and which of the different tracks is the one or more switch-to candidates for the corresponding track. The method comprises the following further steps: (e.g. for each representation) retrieving from the video file one or more initialization parameter sets to be inserted into the reconstructed bitstream and referenced by its samples, wherein the reconstructed bitstream comprises any of the input video bitstream embedded in the corresponding track or a hybrid video bitstream resulting from a switch between the input video bitstream embedded in the corresponding track and the input video bitstream embedded in the one or more switch-to candidates belonging to the corresponding track. The method further comprises the steps of providing to the decoder a predetermined set of initialization parameters retrieved from a video file of the track of interest, and feeding to the decoder an input video bitstream embedded in the corresponding track and / or a hybrid video bitstream resulting from switching between the input video bitstream embedded in the corresponding track and an input video bitstream embedded in one or more switch-to candidates belonging to the corresponding track.

[0014] Still further with respect to the first aspect, a method for generating a video file from at least two input video bitstreams is proposed, wherein video content is encoded differently in the at least two input video bitstreams, and wherein each input video bitstream includes one or more coded video sequences (CVS) with random access points. The method comprises the following steps: embedding the at least two input video bitstreams into different tracks of the video file, the tracks being switchable between each other at random access points of the embedded input video bitstreams; and providing switching information in the video file for each track, the switching information indicating whether the different tracks include one or more switch-to candidates for the corresponding track and which track of the different tracks is the one or more switch-to candidates for the corresponding track. The method further comprises the following steps: providing in the video file for each track one or more initialization parameter sets to be inserted into a reconstructed bitstream and referenced by its samples, wherein the reconstructed bitstream includes any one of the input video bitstream embedded in the corresponding track and a hybrid video bitstream generated from switching between the input video bitstream embedded in the corresponding track and the input video bitstream embedded in one or more switch-to candidates belonging to the corresponding track.

[0015] Still further to the first aspect, a computer program is proposed for implementing the method of the first aspect when executed on a computer or a signal processor.

[0016] Still further with respect to the first aspect, a client network device for retrieving media content from a server using a streaming protocol is proposed, wherein the client network device comprises a file parser according to the first aspect.

[0017] Still further directed to the first aspect, a server network device for providing media content to a client using a streaming protocol, wherein the server network device comprises the file generator according to the first aspect.

[0018] Still further with respect to the first aspect, a system is proposed, comprising a client network device and a server network device according to the first aspect.

[0019] A second aspect relates to a streaming system, such as DASH, for example. According to this second aspect, a client network device for retrieving media content from a server using a streaming protocol is proposed, wherein the media content is contained in one or more media streams, each of the one or more media streams containing one or more representations, wherein each representation includes one or more tracks, the one or more tracks containing samples of pictures of reference video content, wherein the pictures are encoded in one or more coded video sequences (CVS) of one or more bitstreams. Each representation is associated with an initialization segment, the initialization segment comprising one or more initialization parameter sets (e.g., sample description boxes) for those tracks contained in the corresponding representation, wherein each initialization parameter set contains decoding parameter information to be inserted into a reconstructed bitstream at the beginning of the coded video sequence and referenced by its samples. The client network device is configured to derive initialization segment information from the one or more media streams, the initialization segment information indicating that a predetermined initialization segment of a predetermined representation can be used for another representation.

[0020] Further with respect to this second aspect, a corresponding server network device for providing media content to a client using a streaming protocol is proposed, wherein the media content is transmitted in one or more media streams, each of the one or more media streams containing one or more representations, wherein each representation includes one or more tracks, the one or more tracks containing samples of pictures of reference video content, wherein the pictures are encoded in one or more coded video sequences (CVS) of one or more bitstreams. Each representation is associated with an initialization segment, the initialization segment including one or more initialization parameter sets (e.g., sample description boxes) for those tracks contained in the corresponding representation, wherein each initialization parameter set contains decoding parameter information to be inserted into a reconstructed bitstream at the beginning of the coded video sequence and referenced by its samples. The server network device is configured to transmit initialization segment information together with the one or more media streams, the initialization segment information indicating that a predetermined initialization segment of a predetermined representation can be used for another representation.

[0021] Still further with respect to the second aspect, a system is proposed, comprising the client network device according to the second aspect and the server network device according to the second aspect.

[0022] Still further with respect to the second aspect, a method for retrieving media content from a server using a streaming protocol is proposed, wherein the media content is contained in one or more media streams, each of the one or more media streams containing one or more representations, wherein each representation includes one or more tracks, the one or more tracks containing samples of pictures of reference video content, wherein the pictures are encoded in one or more coded video sequences (CVS) of one or more bitstreams, each representation is associated with an initialization segment, the initialization segment including one or more initialization parameter sets (e.g., sample description boxes) for those tracks contained in the corresponding representation, wherein each initialization parameter set contains decoding parameter information to be inserted into a reconstructed bitstream at the beginning of the coded video sequence and referenced by its samples. The method further comprises the steps of: deriving initialization segment information from the one or more media streams, the initialization segment information indicating that a predetermined initialization segment of a predetermined representation can be used for another representation.

[0023] Still further with respect to the second aspect, a method for providing media content to a server using a streaming protocol is proposed, wherein the media content is transmitted in one or more media streams, each of the one or more media streams containing one or more representations, wherein each representation includes one or more tracks, the one or more tracks containing samples of pictures of reference video content, wherein the pictures are encoded in one or more coded video sequences (CVS) of one or more bitstreams. Each representation is associated with an initialization segment, which includes one or more initialization parameter sets (e.g., sample description boxes) for those tracks contained in the corresponding representation, wherein each initialization parameter set contains decoding parameter information to be inserted into the reconstructed bitstream at the beginning of the coded video sequence and referenced by its samples. The method also includes the following steps: transmitting initialization segment information together with the one or more media streams, the initialization segment information indicating that a predetermined initialization segment of a predetermined representation can be used for another representation.

[0024] A third aspect relates to grouping samples of alternative initialization parameter sets transmitted in-band. According to the third aspect, a file parser for reconstructing a video bitstream from a video file is proposed, wherein the video file comprises one or more tracks, each track having one or more input video bitstreams embedded therein, wherein each of the one or more input video bitstreams comprises one or more coded video sequences (CVS). The file parser is configured to retrieve (e.g. for each representation) from the video file one or more sample groups to which one or more samples of the one or more tracks belong and an indication of which samples belong to which sample group. Each sample group is associated with at least one initialization parameter set to be inserted into the reconstructed bitstream and referenced by its samples. The file parser is also configured to provide a predetermined initialization parameter set for a predetermined sample group to a decoder.

[0025] Further with respect to the third aspect, a file generator for generating a video file from one or more input video bitstreams, into which video content is encoded, is proposed, wherein the video file comprises one or more tracks in which the one or more input video bitstreams are embedded, wherein each of the one or more input video bitstreams comprises one or more coded video sequences (CVSs). The file generator is configured to group one or more samples of the track into at least one sample group and mark the one or more samples as belonging to the at least one sample group, wherein each sample group is associated with at least one initialization parameter to be used at the beginning of the one or more coded video sequences (CVSs).

[0026] Still further with respect to the third aspect, a system is proposed, comprising a file parser according to the third aspect and a file generator according to the third aspect.

[0027] Still further with respect to the third aspect, a client network device comprising a file parser according to the third aspect is proposed, wherein the client is configured to download a video file from a server network device, and wherein the file parser is configured to generate a reconstructed video bitstream from the file downloaded from the server network device.

[0028] Still further with respect to the third aspect, a server network device comprising a file generator according to the third aspect is proposed, wherein the file generator is to generate a video file, and wherein the server is configured to send the video file to a client network device.

[0029] Still further with respect to the third aspect, a system is proposed comprising a server according to the third aspect and a client according to the third aspect, wherein the server is configured to generate a video file, and wherein the client downloads the video file from the server.

[0030] Still further with respect to the third aspect, a method for reconstructing a video bitstream from a video file is proposed, wherein the video file comprises one or more tracks, each track having one or more input video bitstreams embedded therein, wherein each of the one or more input video bitstreams comprises one or more coded video sequences (CVS). The method comprises the step of retrieving from the video file (e.g. for each representation) one or more sample groups to which one or more samples of the one or more tracks belong and an indication of which samples belong to which sample group. Each sample group is associated with at least one initialization parameter set to be inserted into the reconstructed bitstream and referenced by its samples. The method further comprises the step of providing a predetermined initialization parameter set for a predetermined sample group to a decoder.

[0031] Still further with respect to the third aspect, a method for generating a video file from one or more input video bitstreams into which video content is encoded is proposed, wherein the video file comprises one or more tracks in which the one or more input video bitstreams are embedded, wherein each of the one or more input video bitstreams comprises one or more coded video sequences (CVS). The method comprises the steps of grouping one or more samples of the track into at least one sample group and marking the one or more samples as belonging to the at least one sample group, wherein each sample group is associated with at least one initialization parameter to be used at the beginning of the one or more coded video sequences (CVS).

[0032] A fourth aspect relates to signaling additional decoding requirements. According to this fourth aspect, a decoder for decoding pictures of video content contained in an input video bitstream is proposed, wherein the decoder is configured to initialize its decoding behavior using predetermined decoding parameter information before starting to decode a coded video sequence for decoding pictures of the video content, wherein the pictures are arranged in one or more consecutively arranged coded video sequences (CVSs), each coded video sequence (CVS) being associated with its own sequence-dependent initialization parameter set carrying sequence-dependent decoding parameter information. The decoder is configured to derive a signal (dci_extension_flag) indicating to the decoder to use global bitstream-dependent decoding parameter information (dci_extension) instead of sequence-dependent decoding parameter information for its initialization, the global bitstream-dependent decoding parameter information indicating a predetermined global decoding capability for all coded video sequences (CVSs) contained in the entire input video bitstream, wherein the global bitstream-dependent decoding parameter information is transmitted in a metadata NAL unit (DCI) or SEI message.

[0033] Further with respect to the fourth aspect, an encoder for encoding pictures of video content into a video bitstream is proposed, wherein the pictures are arranged in one or more coded video sequences (CVSs), wherein the encoder is configured to provide predetermined decoding parameter information to be used by a decoder at the beginning of the coded video sequence (CVS) to initialize its decoding behavior for decoding the pictures contained in the decoded video sequence (CVS), wherein each coded video sequence (CVS) is associated with its own sequence-dependent initialization parameter set (SPS) carrying sequence-dependent decoding parameter information. The encoder is configured to encode a signal (dci_extension_flag) into the bitstream, the signal indicating to the decoder to use global bitstream-dependent decoding parameter information (dci_extension) instead of sequence-dependent decoding parameter information for its initialization, the global bitstream-dependent decoding parameter information indicating a predetermined global decoding capability for all coded video sequences (CVSs) contained in the entire input video bitstream, wherein the global bitstream-dependent decoding parameter information is transmitted in a metadata NAL unit (DCI) or SEI message.

[0034] Still further with respect to the fourth aspect, a method for decoding pictures of video content contained in an input video bitstream is proposed, the method comprising initializing a decoding behavior of a decoder using predetermined decoding parameter information before starting to decode a coded video sequence for decoding pictures of the video content, wherein the pictures are arranged in one or more consecutively arranged coded video sequences (CVSs), each coded video sequence (CVS) being associated with its own sequence-dependent initialization parameter set carrying sequence-dependent decoding parameter information. The method further comprises the step of deriving a signal (dci_extension_flag) indicating that the decoding behavior of the decoder is initialized using global bitstream-dependent decoding parameter information (dci_extension) instead of sequence-dependent decoding parameter information, the global bitstream-dependent decoding parameter information indicating a predetermined global decoding capability for all coded video sequences (CVSs) contained in the entire input video bitstream. The global bitstream-dependent decoding parameter information is transmitted in a metadata NAL unit (DCI) or a SEI message.

[0035] Still further with respect to the fourth aspect, a method for encoding pictures of video content into a video bitstream is proposed, wherein the pictures are arranged in one or more coded video sequences (CVSs). The method comprises the following steps: providing predetermined decoding parameter information to be used by a decoder at the beginning of a coded video sequence (CVS) to initialize its decoding behavior for decoding pictures contained in the decoded video sequence (CVS), wherein each coded video sequence (CVS) is associated with its own sequence-dependent initialization parameter set (SPS) carrying sequence-dependent decoding parameter information. The method further comprises the following steps: encoding a signal (dci_extension_flag) into the bitstream, the signal indicating to the decoder to use global bitstream-dependent decoding parameter information (dci_extension) instead of sequence-dependent decoding parameter information for its initialization, the global bitstream-dependent decoding parameter information indicating a predetermined global decoding capability for all coded video sequences (CVSs) contained in the entire input video bitstream. The global bitstream-dependent decoding parameter information is transmitted in a metadata NAL unit (DCI) or an SEI message.

[0036] Still further with respect to the fourth aspect, it is suggested to provide a bit stream to be obtained by the method according to the fourth aspect.

[0037] Still further with respect to the fourth aspect, a computer program is proposed for implementing the method of decoding and encoding according to the fourth aspect when executed on a computer or a signal processor.

[0038] According to a fifth aspect, it is recommended or required that the client / player assigns a maximum value of at least one initialization parameter instead of signaling the corresponding parameter value. According to the fifth aspect, a player is recommended, the player comprising a decoder for decoding pictures of video content contained in an input video bitstream, wherein the pictures are arranged in one or more consecutively arranged coded video sequences (CVSs), each coded video sequence (CVS) being associated with its own sequence-related initialization parameter set (SPS) carrying sequence-related decoding parameter information. The decoder is configured to initialize its decoding behavior for decoding pictures of the video content by using at least one initialization parameter of the signaled global bitstream-related decoding parameter information (dci_extension) instead of the sequence-related decoding parameter information (SPS) before starting to decode a particular coded video sequence (CVS). It is recommended or required that the player assigns a maximum value of at least one initialization parameter of the global bitstream-related decoding parameter information (dci_extension) to define maximum decoding requirements with which the decoder can decode all coded video sequences (CVSs) in a bitstream that meet these maximum decoding requirements without reinitialization. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Advantageous implementations are the subject of dependent embodiments. Preferred embodiments of the present application are described below with reference to the accompanying drawings, in which:

[0040] Figure 1A A diagram showing a video data stream in which a video is encoded using an open GOP structure is shown, wherein: Figure 1A A picture showing a video data stream in presentation time order;

[0041] Figure 1B A schematic diagram showing the path of pictures of a video data stream in which an open GOP structure is used to present the pictures in temporal order and in the lower half to encode the pictures in decoding order, wherein the open GOP structure corresponds to Figure 1A The open GOP structure is for example only;

[0042] Figure 1C A schematic diagram showing two separately / independently encoded video data streams which are temporally subdivided into segments for adaptive streaming in the upper part and arrive at the client in the form of a spliced ​​data stream in the lower part, thereby enabling a transition from a lower quality to a higher quality;

[0043] Figure 1D A schematic diagram illustrating an exemplary server network device is shown;

[0044] Figure 1E Shown is shown by Figure 1DSchematic diagram of layered video streaming and increased spatial resolution video streaming used by a server network device;

[0045] Figure 1F A schematic diagram illustrating an exemplary client network device is shown;

[0046] Figure 1G Shows Shows Figure 1F A schematic diagram of an exemplary client network device operating mode;

[0047] Figure 1H A schematic diagram showing a file generator according to an embodiment of the present application is shown;

[0048] Fig. 1I A schematic diagram illustrating a file parser according to an embodiment of the present invention is shown;

[0049] Figure 2 An example of an open GOP bitstream comprising two bitstreams having different resolutions is shown;

[0050] Figure 3 Shown in Figure 2 An example of switching between segments from two bitstreams;

[0051] Figure 4 Shown for Figure 2 An example of storage of sample entries of SPS and in-band PPS of two bitstreams;

[0052] Figure 5 An example of the resulting bitstream variants when a switch is to be performed is shown;

[0053] Figure 6 An example of in-band storage of all parameter sets is shown;

[0054] Figure 7 shows an example of an inconsistent bitstream after switching with in-band SPS storage;

[0055] Figure 8 shows a consistent bitstream after switching with in-band SPS storage according to an embodiment of the present invention;

[0056] Fig. 9 shows the use of track groups with switching levels according to an embodiment of the present invention; and

[0057] Fig.10 An indication of additional / alternative initialization parameter sets (eg, in a sample entry) for initialization-free switching according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0058] In the following description, the same or equivalent elements or elements having the same or equivalent functions are denoted by the same or equivalent reference numerals.

[0059] The method steps depicted by the block diagrams and described with reference to the block diagrams may also be performed in an order different from that depicted and / or described. Furthermore, method steps relating to certain features of an apparatus may be replaced with features of the apparatus and vice versa.

[0060] The description of the present application with reference to the accompanying drawings starts with the introduction of an open GOP structure in video streaming using an adaptive streaming protocol such as DASH, wherein switching between representations of videos associated with different spatial resolutions is enabled.

[0061] 1. Introduction

[0062] Compared to a closed GOP structure with the same random access period, an open GOP structure allows for more efficient compression of video. Figure 1A As shown, when randomly accessing a stream encoded with an open GOP structure, there are some pictures (in Figure 1A RASL in the text file), therefore, reproduction / output / display cannot be performed.

[0063] Figure 1A The open GOP structure is shown in output order using HEVC and VVC nomenclature when indicating the picture type. Here, the random access point is a clean random access (CRA) picture, and the random access skip leading (RASL) picture is a random access dependent picture that cannot be displayed to the user because the reference P picture shown in the left figure is lost when random access is performed at the CRA picture.

[0064] To make this description easier to present, refer to Figure 1B ,exist Figure 1B The upper part of FIG. 1 shows nine pictures 10 of a video 12 in the order of presentation time. The pictures 10 are numbered 1 to 9 in the order of presentation time. Figure 1B At the bottom of FIG. 1 , picture 10 is shown in its decoding order, in which the picture 10 is encoded into the data stream. Figure 1B Shows the use of Figure 1A The open GOP structure shown encodes picture 10 into the data stream. Figure 1B The numbering of the pictures 10 in the lower half of shows that the pictures 10 are rearranged in time, ie the decoding order deviates from the presentation time order.

[0065] Specifically, Figure 1BThe fifth picture 10 in the presentation time order is shown to be encoded as a random access point (RAP) picture. That is, the fifth picture or picture number 5 in the presentation time order is encoded independently of any picture that precedes it at another presentation time and in decoding order. Figure 1B As shown, pictures numbered 2 to 4 are encoded in a manner of directly or indirectly referencing picture numbered 5 (i.e., random access point picture) and another picture (here, picture numbered 1) through temporal prediction, and the other picture precedes the random access point picture in terms of both presentation temporal order and decoding order. For example, picture numbered 3 directly refers to picture numbered 5 and picture numbered 1 through temporal prediction. That is, picture numbered 3 is temporally predicted through motion compensated prediction, for example, based on picture numbered 5 and picture numbered 1. Picture numbered 4 does not directly reference picture 1 through temporal prediction, but indirectly references picture 1 via picture numbered 3. That is, the set of pictures numbered 2, 3, and 4 have the following in common: 1) they directly or indirectly reference the random access point picture through temporal prediction, here exemplified as picture numbered 5, and 2) they directly or indirectly reference the reference picture that precedes in terms of presentation temporal order and decoding order through temporal prediction, and the directly or indirectly referenced random access point picture is in Figure 1B The example is the reference picture numbered 1. If the random access point picture numbered 5 is used to randomly access a video data stream that encodes video 10 using an open GOP structure, the picture set may be skipped because, for the picture sets numbered 2, 3, and 4, the picture numbered 1 as a reference picture of the set is located upstream of the random access point picture numbered 5 in the decoding order, so that the reference picture numbered 1 may be lost.

[0066] Besides using open GOP structure in typical broadcast scenarios, where skipping some pictures such as RASL pictures at random access (e.g. during channel switching between programs) is acceptable, open GOP structure has proven to be valuable for adaptive streaming (e.g. DASH), where switching to one or the other stream with different quality is done without skipping pictures aligned with random access point pictures such as CRA. As long as the resolution is the same and the streams are authored carefully, it is possible to switch between two streams and concatenate or splice two streams with different qualities and obtain a compliant bitstream that forms a single video sequence from the point of view of the video codec specification.

[0067] refer to Figure 1C The latter case is shown, Figure 1C In its upper part, two representations Q1 and Q2 of the video are shown, and in particular, two consecutive time segments Seg#1 and Seg#2 of the video are shown. Figure 1CThe lower part of the diagram shows these time slices in a connected manner, which are actually retrieved by the client from the server. Figure 1C As shown in the example in , the client has chosen to retrieve time segment Seg#1 from representation Q1 and the subsequent time segment Seg#2 from representation Q2. In other words, Figure 1C An example is shown in which a client downloads a first segment Seg#1 in a first representation (eg, in a first quality Q1 ), and then downloads a second segment Seg#2 in a second representation (eg, in a second quality Q2 ).

[0068] and Figure 1A and Figure 1B Same situation, Figure 1C The interdependencies between pictures are shown by means of arrows from the predictively coded picture to the corresponding reference picture, which is referenced by temporal prediction. In decoding order, each slice starts with a CRA picture (i.e., a random access point picture), but Figure 1C In the presentation time order of the pictures shown, the RASL picture precedes the random access point picture of each fragment. Figure 1B This explains the situation.

[0069] By switching from a first representation (e.g. from quality Q1) to a second representation (e.g. switching to quality Q2), reference pictures of the RASL pictures of the second fragment of data stream Q2 are not lost: within stream Q2, the RASL pictures of the second fragment of stream Q2 refer to pictures P of the first fragment of stream Q2, and within the spliced ​​data stream in which the second fragment of stream Q2 follows the first fragment of stream Q1, these RASL pictures refer to the low-quality pictures P that are temporally aligned with the first fragment of stream Q1 as a replacement. Figure 1C The effect of changing the reference picture is shown. Specifically, Figure 1C Pictures representing Q2 are depicted as shaded, while pictures representing Q1 are depicted as unshaded. In a spliced ​​or concatenated stream where a fragment of quality Q2 follows a first fragment of quality Q1, the RASL pictures of the second fragment of quality Q2 are depicted as half unshaded and the other half with shading, indicating that the result of decoding these RASL pictures is not the corresponding RASL pictures when decoding a continuous stream of Q2 or Q1. However, in addition to being compliant with the specification, if they are written appropriately, the quality degradation relative to Q2 is not significant. This can already be signaled with the attribute @mediaStreamStructure in the Media Presentation Description (MPD) of DASH (i.e., within the manifest file).

[0070] When the different qualities Q1 and Q2 do not have the same resolution, additional measures need to be taken at the decoder, since the reference pictures required for open GOP switching may not be available at the appropriate resolution. This means that when the decoder is initially initialized with initialization parameters for a predetermined quality, it may not always be possible to perform open GOP switching with a resolution change, e.g., if the decoder is initially initialized with initialization parameters belonging to a lower quality, it may not be possible to switch from a low quality stream to a high quality stream without reinitializing the decoder.

[0071] First, in order to facilitate understanding of the following more detailed description of various embodiments of the present application, Figure 1D A device for outputting video to a client using an adaptive streaming protocol is shown. Figure 1D 20 is represented as an output device and is represented by reference numeral 20. Therefore, the output device 20 acts as a streaming server, and the adaptive streaming protocol used by the device 20 may be DASH or any other adaptive streaming protocol. Therefore, the device 20 may also be referred to as a server network device in this article. The device 20 may be implemented in the form of hardware, firmware or software. When implemented in hardware, the device 20 may be, for example, an integrated circuit. If implemented in firmware, the device 20 may be an FPGA, and if implemented in software, the device 20 may include one or more processes programmed by an appropriate computer program.

[0072] Device 20 supports switching between at least outputting video 12 at a first spatial resolution and outputting video 12 at a second spatial resolution. That is, stream 22 output by output device 20 to a client may represent video 12 at a spatial resolution or have encoded therein video 24 at a spatial resolution that varies over time and, for example, switches between a first temporal spatial resolution and a second spatial resolution that is higher than the first spatial resolution. For example, "spatial resolution" is measured in terms of the number of samples per picture. For example, Figure 1D An output device 20 is shown outputting a video 12 via a stream 22 at a first spatial resolution during a first time interval 26 and at a second spatial resolution during a second time interval 28. During the first time interval 26, the stream 22 represents pictures 10 of the video 12 at the first spatial resolution, and during the second time interval 28, the stream 22 represents pictures 10 at the second spatial resolution.

[0073] The fact that the output device 20 supports switching between outputting the video 12 at a first spatial resolution and outputting the video 12 at a second spatial resolution may, for example, be manifested as a capability of the client to retrieve the video 12 from the output device 20 at a different spatial resolution by requesting a specific representation from the output device 20. As will be explained later, the output device 20 may, for example, be a combination of a memory 32 storing, on the one hand, a suitably conceptualized data structure and, on the other hand, a manifest provider 34. The manifest provider 34 may, for example, provide a manifest to the client describing how the client can access the memory 32 by a corresponding request. In doing so, the client may, based on the manifest, select between temporal segments that have been encoded into the video 12 at the first spatial resolution and temporal segments that have been encoded into the video 12 at the second spatial resolution.

[0074] Figure 1E The use of an open GOP structure to represent and encode video 12 at increased spatial resolutions while avoiding the need for switching between spatial representations. Figure 1B The random access shown here relies on the loss of the picture, which can avoid causing the loss of the picture. Specifically, Figure 1E A switch in time is shown between outputting video in the form of a low resolution video stream 36 and an increased spatial resolution video stream 38. That is, the stream 22 output by the output device 20 changes between a time interval 26, in which the output stream 22 is the low resolution video stream 36, and a time interval 28, in which the output stream 22 is the increased spatial resolution video stream 38. For example, the low resolution video stream 36 output during the interval 26 represents the video during the interval 26, and is connected or spliced ​​with the increased spatial resolution video stream 38, which represents the video at, for example, the temporally subsequent interval 28 or has been encoded therein.

[0075] The problem of switching from the low resolution video stream 36 to the increased resolution video stream 38 is solved by exploiting the RPR functionality of the decoder on the client side: the decoder is able to quasi-dynamically upsample the 66 reference pictures of the resolution in addition to the reference pictures. The only problem to be solved is the correct initialization of the decoder and its correct preparation, since the decoder is prepared to process pictures of a certain maximum size. Therefore, no picture can be larger than a certain picture size initially indicated to the decoder.

[0076] Figure 1F shows a method for using an adaptive streaming protocol from a source such as Figure 1D The server retrieves the video from the output device of the device. Figure 1FThe device is represented as client device 50, and may be implemented in hardware, firmware, or software as with respect to output device 20. That is, device 50 may be an integrated circuit, a suitably programmed FPGA, or one or more processes suitably programmed by a suitable computer program.

[0077] The client device 50 (which may also be referred to as a client network device) supports switching between retrieving video at a first spatial resolution and retrieving video at a second spatial resolution. To do this, the client device 50 retrieves a stream 52 of time segments from the server, which is selected from different versions of the video or different streams representing the video at different bit rates according to the time segments. For example, the stream 52 may be Figure 1D and Figure 1E 22 , the client device 50 switches between retrieving video via the low-resolution video stream 36 and via the increased spatial resolution video stream 38 , the increased spatial resolution video stream 38 corresponding to a higher bit rate than the layered video stream 36 .

[0078] Internally, the client device 50 may include a requester 54, which is responsible for, for example, requesting the above-mentioned list from the server and sending a request to the server to obtain the time segments of the streams provided by the server (such as the time segments of the streams 36 and 38), and the requester 54 requests to switch between the above-mentioned time segments to, for example, avoid buffer overflow or underflow. For example, the client device 50 also includes a buffer 56 for buffering the inbound time segments obtained by the requester 54 before forwarding the buffered time segments to the video decoder 58 for decoding. The video decoder 58 can be part of the client device 50 or can be external to it. Figure 1F Video decoder 58 is shown as being external to client device 50 .

[0079] Thus, device 50 receives stream 52 from server 20 by requesting temporal segments of different streams in which the video is encoded at different bit rates, and outputs or forwards stream 52 to video decoder 58 to retrieve video at varying spatial resolutions.

[0080] To illustrate the latter case and how the apparatus 50 can encode video into the stream 52 at the second spatial resolution using an open GOP structure without losing random access dependent pictures, refer to Figure 1G .like Figure 1G As shown, the client device 50 retrieves the video 24 at a first spatial resolution during the first temporal portions 60 and 64 and retrieves the video 24 at an increased second spatial resolution during the third temporal portion 62 . Figure 1GThis situation is illustrated by depicting pictures of video 24 at different sizes. That is, during time portions 60 and 64, client device 50 retrieves time segments of a first stream provided or presented as available for output at the server, and during time portion 62, client device 50 retrieves time segments of another stream provided or presented as available for download at the server. However, within the second time portion 64, some pictures are needed as references for pictures within portion 62. Therefore, the decoder of client device 50 (a decoder inside it, not shown separately) upsamples the low-resolution pictures to the increased spatial resolution by upsampling from the first spatial resolution to the second spatial resolution, as shown by arrow 66. By this measure, the pictures of the second spatial resolution can obtain their correct temporal predictors. That is, the representation downloaded by client device 50 during time stage 62 can be encoded using an open GOP structure and random access dependent picture loss can be prevented in any case.

[0081] The embodiments described subsequently implement tools for correctly preparing / configuring the decoder to be ready to handle the maximum picture size, even if the client, for example, decides to first download the video at a lower resolution before switching to a higher resolution representation.

[0082] Figure 1H An embodiment of the present invention is shown. It shows a file generator 200 receiving at least two input video bitstreams 211, 212 from an encoder 210. The two input video bitstreams 211, 212 may contain video content 202, which may be differently encoded in the at least two input video bitstreams 211, 212 (e.g. at different spatial resolutions, such as 1080p, 4k, 8k, etc.). Each input video bitstream 211, 212 includes one or more coded video sequences (CVS) 203 with random access points (e.g. CRA), as described above. It can be seen that the pictures 10 contained in the two input video bitstreams 211, 212 may be encoded in an open GOP structure.

[0083] The two input video bitstreams 211, 212 may be fed from the encoder 210 to the file generator 200. The file generator 200 may be configured to generate a video file 201 from the at least two input video bitstreams 211, 212. It can be seen that the at least two input video bitstreams 211, 212 may be embedded in Figure 1H In one or more tracks 220, 221, 222 labeled as Track 0, Track 1 and Track 2.

[0084] More specifically, each track 220, 221, 222 may be subdivided into samples 231. The samples 231 may include video content 202. In other words, the video content 202 (i.e., pictures 10 encoded in at least two input video bitstreams 211, 212) may be embedded (and distributed) in the samples 231 of the tracks 220, 221, 222.

[0085] As described above, the video content 202 may include one or more coded video sequences (CVS) 203. Each coded video sequence 203 may include one or more groups of pictures (GOPs) 241, 242. Each coded video sequence (CVS) 203 may be associated with one or more initialization parameter sets 204 carrying decoding parameter information.

[0086] These initialization parameter sets 204 may include information (e.g., decoding parameter information) intended for the decoder to initialize its decoding behavior. The initialization parameter set 204 may be a sequence-related parameter set, including decoding parameter information valid for a coded video sequence (CVS) 203. An example would be an SPS (Sequence Parameter Set) defined in VVC. Additionally or alternatively, the initialization parameter set 204 may be a picture-related parameter set, including decoding parameter information valid for each picture of a coded video sequence (CVS) 203. An example would be a PPS (Picture Parameter Set) defined in VVC. Additionally or alternatively, the initialization parameter set 204 may be a video-related parameter set, including decoding parameter information valid for the entire video content 202. An example would be a VPS (Video Parameter Set) defined in VVC. Additionally or alternatively, the initialization parameter set 204 may be a global bitstream-related parameter set, including decoding parameter information valid for the entire bitstream 211, 212.

[0087] One or more initialization parameter sets 204 may be inserted into the reconstructed bitstream and may be referenced by samples 231 of one or more tracks 220 , 221 , 222 .

[0088] As described above, the one or more initialization parameter sets (e.g., PPS, SPS, VPS) 204 may include information intended for a decoder to initialize its decoding behavior. For example, the decoding parameter information included in the initialization parameter set 204 may indicate a predetermined (e.g., maximum) decoding capability required to decode the corresponding picture and / or coded video sequence (CVS) and / or video content and / or bitstream.

[0089] For example, if the video content 202 contained in the input video bitstream 211, 212 can be encoded at a predetermined spatial resolution (e.g., at 4k), the initialization parameter set 204 may include decoding parameter information to tell the decoder to initialize its decoding behavior so that it is ready and able to decode 4k encoded video content.

[0090] For example, in case of a sequence-dependent initialization parameter set 204 (e.g., SPS), the decoder initializes its decoding behavior before the start of a coded video sequence (CVS) 203. Each coded video sequence (CVS) 203 may be associated with its own sequence-dependent initialization parameter set 204, wherein the pictures 10 contained in the coded video sequence 203 refer to the initialization parameter set 204. Thus, in case of a closed GOP switch, a new coded video sequence 203 starts at the switching point (e.g., at the (I)RAP NAL unit), wherein the pictures 10 contained in the new (switched to) coded video sequence 203 refer to the new initialization parameter set 204 associated with the new (switched to) coded video sequence (CVS) 203. In other words, if a new initialization parameter set is signaled, a new coded video sequence 203 starts. Thus, a change from a first initialization parameter set to a different second initialization parameter set may trigger the start of a new coded video sequence (CVS) 203.

[0091] However, at an open GOP switch, the situation may be different. A switch at a predetermined switching point (e.g., at an (I)RAP NAL unit) may not start a new coded video sequence 203. In other words, at an open GOP switch, the coded video sequence 203 may continue even when a switch between two bitstreams 211, 212 occurs. Therefore, the initialization parameter set 204 does not necessarily change at the switching point. Therefore, the initialization parameter set 204 referenced by the picture 10 of the first (switched from) stream or track must conform to the initialization parameters referenced by the picture 10 of the second (switched to) stream or track.

[0092] In other words, the initialization parameter set 204 used to initialize the decoding behavior of the decoder for correctly decoding the picture 10 of the first (switched from) bitstream or track must also be valid for initializing the decoding behavior of the decoder for correctly decoding the picture 10 of the second (switched to) bitstream or track. In other words again, the initialization parameter set 204 used by the decoder for initial initialization (which in most cases will be the initialization parameter set 204 associated with the first (i.e. initial) bitstream or track) must enable the decoder to decode the first (switched from) bitstream or track and the second (switched to) bitstream or track. If this requirement is not met, the decoder may not be able to correctly decode the second (switched to) bitstream or track. Again, this situation may occur in particular at open GOP switching, because the switch from one bitstream or track to another may not start a new coded video sequence (CVS) 203.

[0093] This may lead to some problems which are solved by the concept of the present invention and will be referred to below. Figures 2 to 10 However, for further introductory purposes, reference will be made again to Figure 1H The embodiments described in .

[0094] like Figure 1H As shown, the file generator 200 may be configured to embed at least two input video bitstreams 211, 212 into different tracks 220, 221, 222 of the video file 201. The tracks 220, 221, 222 may switch between each other at random access points (e.g., (I)RAP NAL units) of the respectively embedded input video bitstreams 211, 212.

[0095] The file generator 200 may be configured to provide, preferably for each track 220, 221, 222, switching information 207 in the video file 201, the switching information 207 indicating whether different tracks 220, 221, 222 include one or more switching to candidates of the corresponding tracks 220, 221, 222, i.e., information whether it is possible to switch from the current track (=switch from track) to another track (=switch to track).

[0096] The switching information 207 may also include information about which of the different tracks 220, 221, 222 is such a potential switching-to candidate for the respective track. In other words, the switching information 207 may indicate whether switching is possible / allowed, and if so, from which track (referred to as "switching-from track") to which track (referred to as "switching-to track") the switching may be made.

[0097] The file generator 200 may also be configured to provide, for each track, in the video file 201 one or more initialization parameter sets (eg, SPS and / or PPS and / or VPS) 204 to be inserted into the reconstructed bitstream and referenced by its samples 231 .

[0098] The reconstructed bit stream (as will be referred to below) Fig. 1I In more detail) (e.g. in case no switching occurs between tracks) may include the input video bitstream embedded in the corresponding track. Additionally or alternatively, in case of track switching, the reconstructed bitstream may include a hybrid video bitstream resulting from switching between the input video bitstream embedded in the corresponding (switched from) track and the input video bitstream embedded in one or more switching candidates (i.e., switching from tracks) of the corresponding (switched to) track.

[0099] The video file 201 thus constructed can be transmitted to the file parser 300, which will now be referred to as Fig. 1I The file parser 300 is described. The file parser 300 is configured to receive the video file 201 generated by the file generator 200, as described above. The file parser 300 is also configured to parse the tracks 220, 221, 222 contained in the video file 201 and reconstruct the complete bitstream therefrom. Therefore, the file parser 300 can transmit the reconstructed bitstream 310 to the decoder 320 for decoding the video content 202 contained in the reconstructed bitstream 310 and creating a decoded output video bitstream 330 that is fed to the player device.

[0100] The reconstructed bitstream 310 may include a predetermined initialization parameter set 304 that enables the decoder 320 to correctly decode the entire video content 202. According to the present invention, the file parser 300 may be configured to select the predetermined initialization parameter set 304 based on one or more selection criteria (e.g., the highest decoding performance required), and feed the selected predetermined initialization parameter set 304 to the decoder 320.

[0101] Therefore, according to an embodiment, a file parser 310 for reconstructing a video bitstream 310 from a video file 201 is proposed, wherein the video file 201 comprises different tracks 220, 221, 222 in which at least two input video bitstreams 211, 212 are embedded, wherein the video content 202 is encoded differently (e.g. at different spatial resolutions) in the at least two input video bitstreams 211, 212, and wherein each input video bitstream 211, 212 comprises one or more coded video sequences (CVS) 203 with random access points.

[0102] The file parser 300 may be configured to retrieve switching information 207 relating to one or more potential switch-from tracks 220, 221, 222 (or to a group of tracks - see Sections 2.1 and 2.2 below) from the video file 201, wherein the switching information 207 indicates whether the different tracks 220, 221, 222 comprise one or more switch-to candidates for a corresponding track of the tracks 220, 221, 222 and / or which track of the different tracks 220, 221, 222 is one or more switch-to candidates for a corresponding track of the tracks 220, 221, 222. For example, if a switch from track 220 (as a switch-from track) to another track is envisaged, the switching information 207 may indicate whether one of the remaining tracks 221, 222 will be a switch-to candidate track to which it may be switched by reusing the predetermined initialization parameter set 304.

[0103] The file parser 300 can be configured to retrieve one or more initialization parameter sets (e.g., SPS and / or PPS and / or VPS) 204 from the video file 201 to be inserted into the reconstructed bitstream 310 and referenced by its samples 231. In order to determine the predetermined initialization parameter set 304, the file parser 300 can be configured to select one of the initialization parameter sets 204 based on one or more selection criteria (e.g., the maximum picture size in the CVS), as will be described in more detail below. The selected initialization parameter set 204 can be selected as the predetermined initialization parameter set 304 to be inserted into the reconstructed bitstream and can be used to initialize the entire CVS 203.

[0104] The reconstructed bitstream 310 (e.g. in the absence of track switching) may include the input video bitstreams 211, 212 embedded in the corresponding tracks 220, 221, 222. Additionally or alternatively, the reconstructed bitstream 310 (e.g. in the case of track switching) may include a hybrid video bitstream resulting from switching between the input video bitstream 211, 212 embedded in the corresponding (switched from) track and the input video bitstream 211, 212 embedded in one or more switching-to candidates (e.g. 221, 222) of the corresponding (switched from) track (e.g. 220).

[0105] The file parser 300 may also be configured to provide the decoder 320 with the above-mentioned predetermined initialization parameter set 304 retrieved from the video file 201 of the track of interest. The track of interest may be any track that should be processed by the file parser 300, i.e., in case of track switching, it may be a track switched from (e.g., 220), but may also be switched to a track (e.g., 221, 222). In other words, the track of interest may be any track that is to be processed in some way by the file parser 300 and is intended to be included in the reconstructed video bitstream 310 fed to the decoder 320. For example, if track switching is not performed and the reconstructed video bitstream 310 includes only one track (assuming track 220), the above-mentioned predetermined initialization parameter set 304 may be an initialization parameter set capable of processing track 220. Conversely, if a track switch (assuming between tracks 220 and 222) is performed, the above-mentioned predetermined initialization parameter set 304 may be a predetermined initialization parameter set capable of processing both tracks 220 and 222 by continuously reusing the predetermined initialization parameter set 304 (i.e., without reading any other initialization parameter set after switching).

[0106] The file parser 300 may also be configured to feed the decoder 320 (e.g. in the absence of track switching) the input video bitstreams 211, 212 embedded in the corresponding tracks 220, 221, 222. Thus, the reconstructed video bitstream 310 may include the input video bitstreams 211, 212. Additionally or alternatively, the file parser 300 may be configured to feed the decoder 320 (e.g. in the case of track switching) a mixed video bitstream resulting from switching between the input video bitstreams 211, 212 embedded in the corresponding (switched from) track and the input video bitstreams 211, 212 embedded in one or more switching-to candidates (e.g. 221 and 222) of the corresponding (switched from) track (e.g. 222). Thus, the reconstructed video bitstream 310 may include the mixed input video bitstream.

[0107] For example, a switch should be made from track 2 (222) to track 1 (221). In this non-limiting example, track 2 (222) may be referred to as the corresponding (switched from) track described above, while track 1 (221) may be referred to as one of the switched to candidates described above. For example, the second bitstream 212 may be embedded in track 2 (222), while the first bitstream 211 may be embedded in track 1 (221). Then, the track switch from track 2 (222) to track 1 (221) will result in a mixed video bitstream including a portion of the first input video bitstream 211 and another portion of the second input video bitstream 212. Therefore, the reconstructed video bitstream 310 may then include the mixed video bitstream and may be fed to the decoder 320 by the file parser 300 for decoding. As described above, the file parser 300 can be configured to select a predetermined initialization parameter set 304 to be transmitted to the decoder 320 together with the reconstructed bitstream 310 for initializing the decoder 320 and enabling the decoder 320 to correctly decode the reconstructed bitstream 310, which in this non-limiting example includes the mixed bitstream generated from track switching.

[0108] As referenced above Figure 1H As illustrated, the predetermined initialization parameter set 304 triggers the start of the coded video sequence 203. However, at an open GOP switch, switching from one track to another does not start a new CVS 203. Therefore, the decoder must be initialized at the beginning with initialization parameters that allow decoding of the entire reconstructed video bitstream 310 that may contain the above-mentioned mixed video bitstream (in case of a track switch). Therefore, the initialization parameters contained in the predetermined initialization parameter set 304 initially enable the decoder to decode the entire mixed video bitstream (with a track switch), so that the decoder does not have to read any other initialization parameter set at the switching point (i.e., after the switch).

[0109] Thus, the predetermined initialization parameter set 304 comprises one or more initialization parameters valid for the entire starting encoded video sequence 203 embedded in the aforementioned tracks of interest 220, 221, 222, ie any one or more tracks to be processed by the file parser 300 and fed to the decoder 320.

[0110] However, as mentioned above, especially for track switching or representation switching in DASH, some problems may occur if not handled properly. Specifically, VVC allows open GOP resolution switching, which is advantageous because higher efficiency can be achieved compared to closed GOP encoding, but there are several issues that need to be considered in the adaptive HTTP streaming environment.

[0111] Reference Figures 2 to 7 These problems that may occur in the prior art are described. Figures 8 to 10 The solutions of the present invention to these problems are described. Figures 2 to 8 Two different representations 401, 402 are exemplarily shown. The two different representations 401, 402 may correspond to two different tracks within the file 201 or to two different versions of the same content provided for DASH. The term "representation" may be specifically used for a streaming system such as DASH within the present disclosure, while the term "track" may be specifically used for a file format herein.

[0112] For example, in Figure 2 , two different representations 401, 402 of the same content are shown, one representation corresponding to the high resolution content 401 and the other representation corresponding to the low resolution content 402. The two different representations 401, 402 may correspond to two different tracks within the file 201 or to two different versions of the same content provided for DASH. In any case, the goal is to be able to switch from one content to another, e.g. Figure 3 shown.

[0113] Figure 3 Shown from Figure 2 The problem with this approach may arise with the integration of parameter sets. Different resolutions can be used for rate adaptation as an alternative to changing the encoded QP (quantization parameter), e.g. Figure 2 However, another purpose for providing different resolutions is that there may be different client capabilities that need to be covered, for example in terms of which levels a particular terminal device may support.

[0114] For example, it can be assumed that Figure 2 The high resolution representation 401 in may correspond to 4K, and the low resolution representation 402 may correspond to 1080p. The initialization parameter set (e.g., SPS) for the low resolution representation 402 will typically indicate level 4.0 or 4.1 (or anything corresponding to 1080p at a specific frame rate, etc.), so that decoders conforming to such profile and level know that they can decode such a bitstream, while the initialization parameter set (e.g., SPS) for the high resolution content 401 will typically indicate level 5.0 or 5.1 (or anything corresponding to 4K at a specific frame rate, etc., etc.).

[0115] Open GOP switching with resolution change relies on reference picture resampling (RPR). The idea is that a sequence-dependent initialization parameter set (e.g. SPS) signals the maximum picture size within the CVS, and the corresponding picture-dependent initialization parameter set (e.g. PPS) will indicate the real picture size for each picture of each access unit (AU). However, as for Figure 2 As discussed in the specific example of , the sequence-dependent initialization parameter sets (e.g. SPS) will need to be different for the level definition and the maximum resolution, so for a client that can only decode 1080p, the sequence-dependent initialization parameter sets (e.g. SPS) need to signal the level of 4.0 or 4.1 and the maximum picture size of 1080p.

[0116] Therefore, if a sequence-dependent initialization parameter set (e.g., SPS) is stored in the sample entry, then Figure 4 The situation depicted.

[0117] Figure 4 Shown for Figure 2 The most advanced sample entries of the SPS and in-band PPS of both bitstreams are stored. Figure 4 A first bitstream 211 is shown comprising a first encoded video sequence 2031 including a first group of pictures (GOP) 2411 and a second GOP 2421 . Figure 4 Also shown is a second bitstream 212 comprising a second encoded video sequence 2032 comprising a first group of pictures (GOP) 2412 and a second GOP 2422. The first GOPs 2411, 2412 may each include a random access point, in this specific non-limiting example an IDR picture 2791, 2792 (IDR: Instantaneous Decoding Refresh). The second GOPs 2421, 2422 may each include a random access point, in this specific non-limiting example a CRA picture 2801, 2802 (CRA: Clean Random Access).

[0118] In case of open GOP switching, GOPs may switch from one bitstream 211 to another bitstream 212 at their respective random access points. In this example, at their respective CRA pictures 2801, 2802. As previously mentioned, open GOP switching does not start a new coded video sequence. Thus, if (in an open GOP structure) a switch is made from one bitstream 211 to another bitstream 212 at their respective random access points, the coded video sequence 2031, 2032 continues even after the switch.

[0119] Therefore, if the target terminal device does not intend to perform switching between these two different resolutions, the corresponding sample entries with the correct sequence parameter set (SPS) are provided. Otherwise, if switching is expected, the correct parameter set can be downloaded. That is, the correct sample entries from different track headers or different movie boxes "moov" are used for streaming. See Figure 5 Example shown.

[0120] Figure 5 Two bitstream variants generated when switching is performed are shown. It can be seen that each representation 401, 402 is associated with one or more initialization parameter sets 204a, ..., 204f. For example, a picture-dependent initialization parameter set (e.g., PPS) 204a, 204b, 204c, 204d can be associated with each GOP 2411, 2421, 2412, 2422. Additionally or alternatively, a sequence-dependent initialization parameter set (e.g., SPS) 204e, 204f can be associated with each coded video sequence (CVS) 2031, 2032.

[0121] As mentioned above, ISOBMFF allows parameter sets 204a, ..., 204f to be stored in two different ways. The first way includes the so-called out-of-band parameter set integration, which means that the parameter sets 204a, ..., 204f are not stored together with other non-VCL and VCL NAL units (samples for ISOBMFF) of the AU, but are stored in the sample entry of the track. Basically, the sample entry gives detailed information about the type of encoding used and any initialization information required for the encoding. This information includes the parameter set 204.

[0122] A specific track may have several sample entries applicable to different parts of the bitstream. For example, if the bitstream includes two coded video sequences (CVSs), each CVS referring to a different SPS (with different content) with the same ID, then two sample entries are needed, in which the SPS with the same ID is stored. The samples within the track point to the correct sample entry used, whether using SampleToChunkBox "stcs" or when the samples appear in fragmented tracks, e.g. when using MPEG-DASH fragments, the track fragment header of the track contained in, e.g., a DASH fragment ("tfhd") contains a sample_description_entry pointing to the sample entry used. Thus, the correct out-of-band parameters can be used to decode the sample identified by the correct sample description index.

[0123] However, there are use cases where it is not possible to store all parameter sets in the sample entries. Note that when a file is generated, the sample entries are stored in a "moov" box, which is typically located at the beginning of the file. Specifically, for live streaming, only the first used SPS 204e, 204f is known and can be stored at "moov", and another different SPS may appear later during playback when the service is running. Obviously, one option might be to perform some kind of constrained encoding, where the flexibility of parameter changes in the SPS is limited, and all possible SPSs to be used are generated in advance and stored in the "moov" box, so that the corresponding sample entries can be referenced dynamically as needed. But in other scenarios, this less flexible approach may not be desired, and the SPS may appear in these samples, as shown below Figure 6 shown.

[0124] Figure 6 A second alternative of storing the initialization parameter sets 204a, ..., 204h, so-called in-band storage, is shown. In this case, the initialization parameter sets 204a, ..., 204h are transmitted together with the samples as so-called in-band parameter sets, so they are already present in the bitstream within the AU as required.

[0125] However, this leads to the fact that if no measures are taken, switching following the principle of the previous example will result in an erroneous bit stream, such as Figure 7 shown.

[0126] Figure 7 A first representation 401 is shown of a scenario where a switch is made from high resolution (4k) content to low resolution (1080p) content. The sequence-dependent initialization parameter set 204e initially contains 4K parameters. As described above, an open GOP switch does not start a new coded video sequence (CVS). Therefore, the in-band transmitted sequence-dependent initialization parameter set 204g containing low resolution parameters for 1080p is invalid for this CVS. However, at least the in-band transmitted picture-dependent initialization parameter set 204b will be valid because it includes low resolution parameters, i.e., since the decoder is initially initialized with the initialization parameter set 204a, 204e of the high resolution (4K) content, it is possible to decode pictures with low resolution (1080p). However, the two bitstreams are inconsistent, so the resulting mixed bitstream after the switch is erroneous.

[0127] The open GOP switch in the second representation 402 described above (i.e., when switching from low-resolution content (1080p) to high-resolution content (4K)) is even worse. The sequence-dependent initialization parameter set 204f initially contains 1080p parameters. As described above, an open GOP switch does not start a new coded video sequence (CVS). Therefore, the in-band transmitted sequence-dependent initialization parameter set 204h containing high-resolution parameters is invalid for this CVS. In addition, the in-band transmitted picture-dependent initialization parameter set 204d containing high-resolution parameters is also invalid for this CVS.

[0128] Likewise, an open GOP switch with CRA does not start a new CVS, thus requiring the SPS to be exactly the same in a conforming bitstream. This means Figure 7 The SPS 204g, 204h forked out in the CVS will be incorrect, and the PPS 204d in the above example will be invalid when switching from low resolution to high resolution because it will signal a larger picture size than the picture size signaled in the initial SPS at the beginning of the CVS.

[0129] 2. Inventive concept

[0130] To overcome the above problems, the client must export Figure 8 The correct bitstream with the appropriate parameter set is shown in the example of .

[0131] Figure 8 A consistent bitstream 211, 212 after switching with in-band SPS storage is shown. In particular, it can be seen in the second representation 402 above that a decoder for decoding the CVS 2032 in the bitstream 212 is initialized with an initialization parameter set 204f containing decoding parameter information for high resolution content (4K), even though the bitstream 112 actually starts with low resolution content (1080p).

[0132] In other words, one or more initialization parameter sets 204a, ..., 204h will be inserted into the reconstructed bitstream 310 and referenced by its samples 231. The one or more initialization parameter sets 204a, ..., 204h inserted into the reconstructed bitstream 310 are also referred to as predetermined initialization parameter sets 304 (see Fig. 1IAs mentioned above, the predetermined initialization parameter set 304 triggers the start of a coded video sequence 203 and it comprises one or more initialization parameters for the entire started coded video sequence (CVS) 203 embedded in the tracks of interest 220, 221, 222. Also, since an open GOP switch does not start a new CVS 203, the predetermined initialization parameter set 304 must be valid for the entire started CVS 203 even if the CVS 203 may comprise a mixed bitstream resulting from a (track or representation) switch (see Figure 8 bit streams 211 and 212 in FIG.

[0133] As described above, the (predetermined) initialization parameter set 204, 304 contains decoding parameter information for decoding the entire CVS 203 (hereinafter, both the initialization parameter set 204 and the predetermined initialization parameter set 304 will be referred to as the initialization parameter set). Specifically, the initialization parameter set 204, 304 is related to one or more video content attributes of the video content 202 contained in the encoded video sequence 203.

[0134] For example, the video content attribute includes at least one of the following:

[0135] • Maximum Codec Profile,

[0136] • Maximum codec level,

[0137] • the maximum picture width of pictures appearing in said starting coded video sequence 203,

[0138] • the maximum picture height of the pictures appearing in said starting coded video sequence 203,

[0139] • the maximum color format of the pictures present in said starting coded video sequence 203,

[0140] • the maximum bit depth of pictures present in said starting coded video sequence 203,

[0141] • The maximum number of picture slots in the decoded picture buffer DBP.

[0142] Thus, one or more of the above-mentioned video content attributes may be derived from the initialization parameter set 204, 304. As described above, the file parser may select a predetermined initialization parameter set 304 to be included in the reconstructed bitstream 310.

[0143] The file parser 300 of the present invention may be configured to select the predetermined initialization parameter set 304 based on:

[0144] • one or more of the signaled video content attributes,

[0145] • one or more maximum decoder capabilities,

[0146] • one or more desired operating points, and

[0147] • One or more desired target display capabilities.

[0148] For example, the predetermined initialization parameter set 304 may be selected based on the highest required global decoding capability at which the decoder can decode the entire coded video sequence (CVS) 2031 , 2032 without reinitialization, even if track switching is enabled.

[0149] Continue to refer Figure 8 Given the above scenario, the present invention may provide a corresponding file format to the client, which signals that it is allowed to export a consistent bitstream 211, 212. Two variants of the present invention are summarized as follows:

[0150] Signaled to identify the track group that can be switched from / switched to, and if so, which sample entry from which track needs to be used

[0151] If a switch (of a track or DASH fragment) is desired or a specific target picture size is envisaged, an alternative parameter set to be used is signaled.

[0152] 2.1 Hierarchical Track Grouping for Sample Item Selection

[0153] In this aspect of the invention, signaling is described that allows a client to identify tracks in which a switch can be made when sample entries for a particular track have been used for decoder initialization. That is, in client operation, fragments of tracks from such a track set (starting from sample positions with a specific SAP (stream access point) type (usually an open GOP CRA) that allow random access) can be concatenated and form a valid bitstream to be played back / fed to the decoder using appropriate sample entries for decoder initialization. Note that the example description box ("stsd") associated with a track in a movie box ("moov") describes one or more configuration parameter sets for the bitstream within the track. For example, when two different codecs are used for the bitstream within the track, there are as many sample entries as configuration parameter sets.

[0154] In other words, each of the different tracks 220, 221, 222 may be associated with its own sample description box. For each track 220, 221, 222, its respective initialization parameter set 204 may be transmitted out-of-band by associating a sample entry within its respective own sample description box with the respective track 220, 221, 222.

[0155] Thus, different tracks may have different sample description boxes associated with them. The signaling of the invention is characterized by expressing a switching hierarchy, i.e. the tracks are marked with information about which further tracks can be switched to in playback without reading further sample description boxes. This information corresponds to the information described above with reference to Figure 1H and Fig. 1I The switching information 207 in question. The switching information 207 also allows identifying those (switched to) tracks to which a switch does not require a decoder reinitialization, i.e., by appropriate sample entries in the sample description box, no further sample description boxes are needed. For example, the switching information 207 may indicate that tracks may be switched without using a sample description box of the switched to track at each switch, and that for decoding samples of one track, a sample description box of another track may be used.

[0156] In some embodiments, the switching information 207 may include an indication for identifying a track group ("aaaa", "bbbb") including one or more tracks 220, 221, 222 from which it can be switched, so-called switch-from tracks. A track group may also include one or more tracks 220, 221, 222 that can be switched to, so-called switch-to tracks or switch-to candidates. The switching information 207 may also identify which initialization parameter set 204 of which track 220, 221, 222 will be used as the predetermined initialization parameter set 304 for reconstructing the bitstream 310 at the beginning of the encoded video sequence 203. In other words, the switching information may provide an indication of whether the predetermined initialization parameter set may still be used after the switch.

[0157] The following syntax shows a possible implementation of a track group, in this example the track group is "aaaa":

[0158] aligned(8) class TrackGroupBox extends Box('trgr') {

[0159] }

[0160] aligned(8) class TrackGroupTypeBox(unsigned int(32) track_group_type)extends FullBox(track_group_type, version = 0, flags = 0)

[0161] {

[0162] unsigned int(32) track_group_id

[0163] if track_group_type == 'aaaa' {

[0164] unsigned int(32) track_switch_hierarchy_id

[0165] }

[0166] }

[0167] track_group_type indicates the grouping type and should be set to one of the following values, or a registered value, or a value from an exported specification or registry:

[0168] […]

[0169] "aaaa" indicates that the track belongs to a group of tracks with the same value of track_group_id, and that a player playing a track with a specific value A of track_switch_hierarchy_id can switch to another track in the group playing a track with a value B of track_switch_hierarchy_id (at a specific sample marked as a switching point (e.g., a Random Access Point (RAP) or Stream Access Point (SAP)) without reading new sample entries or forcing reinitialization of the decoder, where B is less than or equal to A.

[0170] track_switch_hierarchy_id indicates a track switch identifier.

[0171] Along Figure 8 For example, imagine two tracks 220, 221 hosting 1080p video that are marked by the track group of the present invention described above using a track_switch_id value of 0, and as Fig. 9 As shown, one track 222 hosting 4K video will be marked with a track_switch_id value of 1.

[0172] Fig. 9An exemplary use of track groups with a switching hierarchy is shown. The "aaaa" track group would indicate that the player can "down" switch to 1080p video tracks 220, 221 ("track 0" and "track 1") only when the sample entries of 4K video track 222 ("track 2") are used for initialization, without reinitializing the decoder with the corresponding 1080p video track sample entries. Otherwise, the player would only be allowed to switch between 1080p tracks 220, 221 ("track 0" and "track 1").

[0173] Thus, the file parser 300 of the present invention may be configured to group different tracks 220, 221, 222 into a track group "aaaa" of a specific track group type (track_group_type). The file parser 300 may be configured to retrieve, for each track 220, 221, 222 of the track group "aaaa", a track switch hierarchy ID (track_switch_hierarchy_id) indicating the switch hierarchy level of the respective track 220, 221, 222. The track switch hierarchy ID allows identifying whether switching to a potential switch candidate 220, 221, 222 is allowed in case of continuous reuse of a predetermined initialization parameter set 304 initially used to reconstruct the bitstream (310) at the beginning (203) of the encoded video sequence.

[0174] Continue to refer Fig. 9 , the track group may also indicate that samples of 1080p tracks 220, 221 ("track 0" and "track 1") are decodable when the sample description box of the 4K track 222 ("track 2") is used, but not vice versa. That is, only sample description boxes from tracks with higher track_switch_hierarchy_id may be used to decode samples of a particular track belonging to the group. In other words, if an initialization parameter set from a sample description box of a track with a first track_switch_hierarchy_id (e.g., track 222 with track_switch_hierarchy_id=1) is used to start CVS 203, it may switch to a track with a smaller second track_switch_hierarchy_id (e.g., tracks 220, 221 with track_switch_hierarchy_id=0).

[0175] In other words, if the bitstream 310 is reconstructed at the beginning of the encoded video sequence 203 by using a predetermined initialization parameter set 304 of a predetermined track (e.g., the 4K track 222) belonging to the track group “aaaa”, and if the predetermined track 222 includes a predetermined track switch hierarchy ID (e.g., track_switch_hierarchy_ID=1) indicating a predetermined switching hierarchy level, then a switch from any switch from the track group “aaaa” (e.g., from the 4K track 222) to any potential switch candidate (e.g., one of the 1080p tracks 220, 221) of the track group “aaaa” is allowed,

[0176] A) If the corresponding switch-to candidate (e.g., one of the 1080p tracks 220, 221) includes a track switch hierarchy ID indicating a switch hierarchy level that is smaller than the predetermined switch hierarchy level of the predetermined track for initialization (e.g., the 4K track 222). In this example, the track_switch_hierarchy_ID=1 of the 1080p track 220, 221 is smaller than the hierarchy level of the 4K track 222 of track_switch_hierarchy_ID=0. Therefore, if the CVS 203 is initialized with the initialization parameter set of the 4K track 222, it can switch to any of the “smaller” 1080p tracks 220, 221.

[0177] B) If the corresponding switch to candidate includes a track switch hierarchy ID indicating a switch hierarchy level that is the same as the predetermined switch hierarchy level of the predetermined track used for initialization. For example, if the initialization parameter set of one of the 1080p tracks 220, 221 is used to initialize the CVS 203, it can switch between the 1080p tracks 220, 221 (because they have the same track_switch_hierarchy_ID=0). Conversely, if the initialization parameter set of the 4K track 222 is used to initialize the CVS 203, it can switch from the 1080p track 220, 221 to the 4K track 222 (because the 4K track 222 has the same track_switch_hierarchy_ID=1 used for initialization) or switch between different 4K tracks (if available).

[0178] However, any switch from track group "aaaa" to any potential switch candidate track of said track group "aaaa" is not allowed.

[0179] If the corresponding switch to candidate includes a track switch hierarchy level ID indicating a switch hierarchy level greater than the predetermined switch hierarchy level of the predetermined track used for initialization. For example, if the initialization parameter set of one of the 1080p tracks 220, 221 (with track_switch_hierarchy_ID=0) is used to initialize the CVS 203, then switching to a "higher" 4K track 222 (with track_switch_hierarchy_ID=1) is not allowed.

[0180] In other words, with the help of the provided signaling, the player can select a sample description box for decoder initialization that corresponds to its maximum decoder capabilities and / or the expected maximum operating point due to various factors, such as provided and available operating points, network bandwidth, power budget, computational processing budget, display capabilities, etc. Thus, as long as the signaled hierarchy is followed, it will find the right set of parameters that can be used to decode the bitstream, regardless of whether a switch occurs.

[0181] Therefore, the file parser 300 may be configured to select the predetermined initialization parameter set 304 based on:

[0182] • one or more of the signaled video content attributes,

[0183] • one or more maximum decoder capabilities,

[0184] • one or more desired operating points, and

[0185] • One or more desired target display capabilities.

[0186] 2.2 Hierarchical entity grouping for sample item selection

[0187] Alternatively, the handover signaling messages are performed in entity groups and they point directly to the tracks where the handover can be done.

[0188] aligned(8) class SampleDescriptionSharingBox extends EntityToGroupBox('xxxx',0,0)

[0189] {for (i = 0; i < num_entities_in_group; i++)

[0190] unsigned int track_switch_hierarchy_id[i];

[0191] }

[0192] This box will identify the track ID values ​​of num_entities_in_group belonging to the group completed by any EntityToGroupBox, and for each of these entries, an ID value indicating the track containing the sample description box hierarchically will be given, i.e., as in the previous case with track groups, the sample description box with a higher ID can be used to decode samples of the track given a specific hierarchical ID. And a switch can be performed to any track with a lower ID than the one that has been used to obtain the sample description box.

[0193] Therefore, according to an embodiment, the file parser 300 can be configured to retrieve an entity group (sample description sharing box) comprising a reference (track_ID) to one or more tracks 220, 221, 222, which can be switched between each other while continuing to reuse the predetermined initialization parameter set 304 initially used to reconstruct the bitstream 310 at the beginning of the encoded video sequence 203, i.e., without changing / using a different SPS that triggers the start of a new CVS. Each track 220, 221, 222 belonging to the entity group may include a track switch hierarchy ID (track_switch_hierarchy_id) indicating the switching hierarchy level of the corresponding track 220, 221, 222.

[0194] If the bitstream 310 is reconstructed at the beginning of the coded video sequence 203 by using a predetermined initialization parameter set 304 of a predetermined track 222 belonging to an entity group, the predetermined track 222 comprising a predetermined track switch hierarchy ID (track_switch_hierarchy_id=1) indicating a predetermined switching hierarchy level, then any switch from the entity group to any potential switch candidate 220, 221, 222 of the track 220, 221, 222 of the entity group is allowed,

[0195] If the corresponding switch-to candidate 220 , 221 , 222 includes a track switch hierarchy ID (track_switch_hierarchy_id=0) indicating a switch hierarchy level that is equal to or less than the predetermined switch hierarchy level of the predetermined track 222 for initialization.

[0196] However, any switch from an entity group from a track 220, 221 to any potential switch candidate 220, 221, 222 belonging to said entity group is not allowed.

[0197] If the corresponding switch-to candidate 222 includes a track switch hierarchy ID (track_switch_hierarchy_id=1) indicating a switch hierarchy level greater than the predetermined switch hierarchy level (track_switch_hierarchy_id=0) of the predetermined switch-from track 220 , 221 .

[0198] It should be noted that track_switch_hierarchy_id[i] may also take the form of a simple boolean value in two levels of track hierarchy: a first set of tracks, when used for decoder initialization (using the corresponding sample description box), switching is allowed; and a second set of tracks, switching is not allowed.

[0199] Therefore, according to some embodiments, the track switch hierarchy ID (track_switch_hierarchy_id) may take the form of a Boolean value indicating a two-level track switch hierarchy, where

[0200] The first track set of the entity group includes a first track switch hierarchy ID (track_switch_hierarchy_id=0) indicating a first level of the two-level track switch hierarchy, and

[0201] The second track set of the entity group includes a second track switch hierarchy ID (track_switch_hierarchy_id=1) indicating a different second level of the track switch hierarchy.

[0202] If the bitstream 310 is reconstructed at the beginning of the coded video sequence 203 by using a predetermined set of initialization parameters 304 for tracks from a first set of tracks, switching to any other track of the entity group is allowed.

[0203] If the bitstream 310 is reconstructed at the beginning of the coded video sequence 203 by using a predetermined set of initialization parameters for tracks from the second set of tracks, switching to any other track of the entity group is not allowed.

[0204] Another alternative is to implicitly derive the hierarchy in the order in which the track IDs appear in the entity group. For example, the first listed track ID has the lowest hierarchy, and the last listed track ID has the highest hierarchy.

[0205] Therefore, in addition to or instead of explicitly signaling the predetermined track switch hierarchy level through the above-mentioned track switch hierarchy ID (track_switch_hierarchy_id), it is suggested to derive it implicitly.

[0206] For example, if a switch from the first track 222 to the second track 220 is performed, the file parser 300 may implicitly derive the track switch hierarchy level (track_switch_hierarchy_id) from parameters included in the switch-from track 222 and / or the switch-to candidate 220 , respectively.

[0207] The parameters may include at least one of the following:

[0208] • Predefined codec profiles,

[0209] • the predetermined codec level,

[0210] • a predetermined picture width of pictures appearing in said starting coded video sequence,

[0211] • a predetermined picture height of pictures appearing in said starting coded video sequence,

[0212] • a predetermined color format of pictures appearing in said starting coded video sequence,

[0213] • a predetermined bit depth of pictures appearing in said starting coded video sequence,

[0214] • A predetermined number of picture slots in the decoded picture buffer (DBP).

[0215] For example, if the first track 222 includes a parameter (e.g., picture width) having a higher value than a corresponding parameter (e.g., picture width) included in the second track 220, the first track 222 may have a higher switching hierarchy level than the second track 222. For example, a higher picture size, a superset profile, a higher level, a larger DPB, a higher chroma format, a higher bit depth, etc. will result in a higher hierarchy level.

[0216] In some embodiments, the references (track_id) of the tracks 220, 221, 222 belonging to the entity group are sequentially sorted in a predetermined order, such as Fig. 9 Here, the lowest 1080p track 220 has a reference of "0", i.e., track 220 is labeled "track 0", and the highest 4K track 222 has a highest reference of "2", i.e., track 222 is labeled "track 2". In such an embodiment, the switch hierarchy level (track_switch_hierarchy_id) of each track 220, 221, 222 may depend on the position of its reference (track_id) within the predetermined order.

[0217] For example, Fig. 9As exemplarily depicted, a reference (track_id) to the track 220 ("track 0") that appears first in the predetermined order may be associated with a predetermined track switch hierarchy ID (track_switch_hierarchy_id=0) that indicates the lowest switching hierarchy level. Conversely, a reference (track_id) to the track 222 ("track 2") that appears last in the predetermined order may be associated with a different predetermined track switch hierarchy ID (track_switch_hierarchy_id=1) that indicates the highest switching hierarchy level.

[0218] Of course, the reverse is also possible. That is, a reference (track_id) to a track that appears first in the predetermined order may be associated with a predetermined track switch hierarchy ID (track_switch_hierarchy_id=1) indicating the highest switch hierarchy level, and a reference (track_id) to a track that appears last in the predetermined order may be associated with a different predetermined track switch hierarchy ID (track_switch_hierarchy_id=0) indicating the lowest switch hierarchy level.

[0219] Although the present invention has been described so far at the track level, the present invention also relates to other system hierarchy levels, such as streaming video content in a streaming system. Therefore, further embodiments of the present invention relate to the above referenced Figure 1D and Figure 1F The described client network device 50 and server network device 20 are intended to be used for transmitting media content using a streaming protocol in a streaming system.

[0220] In streaming systems such as DASH, signaling or constraints need to be set. For DASH, there are representations rather than tracks. For example, in Figures 2 to 8 In FIG. 4 , two different representations 401, 402 are shown, each including one or more samples. Figure 4 and Figure 5 As exemplarily depicted, each representation 401, 402 may be associated with an initialization fragment 205 including one or more initialization parameter sets 204. For example, the initialization fragment 205 of each representation 401, 402 may include (as an initialization parameter set) a sample description box of tracks within each representation 401, 402.

[0221] Each initialization parameter set 204 (eg, each sample description box) may contain decoding parameter information to be inserted into the reconstructed bitstream 310 at the beginning of the encoded video sequence 203 and referenced by samples thereof.

[0222] Similar to the above-mentioned track switching, representations 401, 402 can be switched between each other, which (for example, in the case of a streaming system such as DASH) may be referred to herein as representation switching. Similar to the above-mentioned initialization parameter sets 204, 304, the initialization fragment 205 may be used for representation switching. Therefore, the client network device 50 of the present invention according to an embodiment may be configured to derive initialization fragment information indicating that a predetermined initialization fragment 205 of a predetermined representation 401 may be used for another representation 401. For example, the initialization fragment information may indicate that the initialization fragment 205 (e.g., a sample description box) of the first representation 401 may also be used for the second representation 402.

[0223] This means that, when the representation is switched, the predetermined initialization segment 205 of the first representation 401 may be selected and associated with the second representation 402. Therefore, the decoding parameter information contained in the predetermined initialization segment 205 will be inserted into the reconstructed bitstream 310 at the beginning of the encoded video sequence 203, and the reconstructed bitstream 310 includes the samples contained in the first representation 401 and the samples contained in the second representation 402. The above decoding parameter information will be referenced by all samples, that is, by the samples of the first representation 401 and the samples of the second representation 402.

[0224] In order to select the correct initialization fragment 205 that can be used for both representations 401 and 402, the client network device 50 of the present invention can base its selection on different criteria. For example, the client network device 50 can be configured to select the predetermined initialization fragment set 205 based on at least one of the following:

[0225] • Maximum decoder capability,

[0226] • the desired maximum operating point,

[0227] • Desired target display capability,

[0228] • Maximum Codec Profile,

[0229] • Maximum codec level,

[0230] • the maximum picture width of pictures appearing in said coded video sequence (203),

[0231] • the maximum picture height of pictures appearing in said coded video sequence (203),

[0232] • the maximum color format of pictures appearing in said coded video sequence (203),

[0233] • the maximum bit depth of pictures present in said starting coded video sequence (203), and

[0234] • Maximum number of picture slots in the decoded picture buffer (DBP).

[0235] As described above, the initialization segment information may indicate that the predetermined initialization segment 205 of the first representation 401 may also be used for the second representation 402. In some embodiments, a parameter may be added indicating that it is possible to use the initialization segment 205 (e.g., a sample description box) of another representation (e.g., the first representation 41), that is, the initialization segment 205 of the representation 401 may be used for the other representation 402. Therefore, the client network device 50 of the present invention may be configured to derive the initialization segment information from the additional parameter.

[0236] In some cases, it is advantageous for the network client 50 to already have the initialization segment information before downloading the media stream. Therefore, the initialization segment information may be transmitted via a manifest file, wherein the manifest description may point to a corresponding initialization segment 205, such as the initialization segment 205 of the first representation 401.

[0237] Or when a representation is indicated as switchable, e.g. for a particular profile belonging to the same adaptation set, it implicitly signals that an initialization fragment of another representation can be used. Similar properties can be indicated by cross-adaptation set switching when this is allowed.

[0238] Therefore, according to some embodiments, the client network device 50 may be configured to determine whether the representations 401, 402 belong to a predetermined set of adaptation sets and whether cross-adaptation set switching between adaptation sets within the set of adaptation sets is allowed. This allows the client network device 50 to implicitly derive the initialization fragment information, for example, implicitly deriving that the initialization fragment 205 of the first representation 401 can be used in the adaptation set of the second representation 402.

[0239] As described above, representation switching is possible, wherein different representations 401, 402 can be switched between each other. Therefore, the sample description boxes in each of the different initialization fragments 205 can be constructed in a way that they are aligned with the above-mentioned hierarchical system, i.e., it is known that as long as a certain hierarchy is met, the initialization fragments 205 of other representations 401, 402 can be used, as described above for the track.

[0240] For example, if representations 401, 402 are indicated as switchable, the initialization fragment information may include switching information for identifying one or more representations (e.g., the first representation 401) that can be switched from (i.e., the so-called switch-to-representation) and one or more representations (e.g., the second representation 402) that can be switched to (i.e., the so-called switch-to-representation or switch-to-candidate). The switching information can also identify which initialization fragment 205 from which representation 401, 402 to use when switching from the switch-to-representation 401 to the switch-to-representation 402, that is, it can be determined whether the selected predetermined initialization fragment 205 can still be used when switching or whether a new initialization fragment is required.

[0241] For example, let us assume that a switch from a first representation 401 to a second representation 402 is to be performed, i.e. the first representation will be the switched-to representation and the second representation 402 will be the switched-to representation. Furthermore, let us assume that the sample description box 205 of the first representation 401 is used, i.e. the first representation 401 is also referred to as a predetermined representation whose initialization fragment 205 (e.g. sample description box) is used for initialization. In the same manner as discussed above for tracks 220, 221, 222, each representation 401, 402 can be associated with a switching hierarchy level. If the predetermined initialization fragment 205 of the switched-to representation 401 is initially used, i.e. while continuing to reuse the initially selected predetermined initialization fragment 205, the switching hierarchy can identify whether a switch from the switched-to representation 401 to the potential switch-to representation 402 is allowed.

[0242] For example, if the switching hierarchy level of the switching to the representation 402 is equal to or less than the switching hierarchy level of the predetermined representation 401, then switching from any switching from the representation 401 to any potential switching to the representation 402 is allowed. Again, the predetermined representation is one whose initialization fragment 205 (e.g., a sample description box) is used for initialization. In this non-limiting example, we assume that a switching from the first representation 401 to the second representation 402 is performed.

[0243] Conversely, if the switch hierarchy level of the switch to representation 402 is greater than the switch hierarchy level of the predetermined representation 401, then any switch from representation 401 to any potential switch to representation 402 is not allowed.

[0244] Another option would be that the tiers follow the tiers indicated for a specific video, e.g. a higher tier indicated as a mimeType codec parameter indicates a higher tier. Thus, the switching tier level may follow the tier indicated for a specific video content 202, wherein a higher tier of the video content 202 indicated as a mimeType codec parameter may indicate a higher switching tier level.

[0245] Another option would be: also considering the profile value, a profile that is a subset of another profile would have a lower tier, and a profile that is a superset of another profile would have a higher tier. Thus, the switching tier level may follow the codec profile value, where a profile that is a subset of another profile has a lower tier level, and where a profile that is a superset of another profile has a higher tier level.

[0246] Similar to Section 2.1, with the help of the provided signaling, the player will select a sample description box for decoder initialization that corresponds to its maximum decoder capabilities and / or the expected maximum operating point due to various factors (e.g., provided and available operating points, network bandwidth, power budget, computational processing budget, display capabilities, etc.).

[0247] 2.3 Additional sample entry track grouping

[0248] Alternatively, additional sample entries to sample entries in a particular track are referenced by the track group, and when those additional sample entries are used for initialization of the decoder, switching between tracks of the track group does not require decoder reinitialization.

[0249] aligned(8) class TrackGroupBox extends Box('trgr') {

[0250] }

[0251] aligned(8) class TrackGroupTypeBox(unsigned int(32) track_group_type)extends FullBox(track_group_type, version = 0, flags = 0)

[0252] {

[0253] unsigned int(32) track_group_id

[0254] if track_group_type == 'bbbb' {

[0255] boolean current_sample_entry_flag;

[0256] unsigned int(8) num_sample_entry_track_ids;

[0257] for (i = 0; i < num_track_group_IDs; i++){

[0258] unsigned int(32) sample_entry_track_id[i]

[0259] }

[0260] }

[0261] }

[0262] track_group_type indicates the grouping type and should be set to one of the following values, or a registered value, or a value from an exported specification or registry:

[0263] […]

[0264] • "bbbb" indicates that the track belongs to a group of tracks with the same value of track_group_id, and when initialized with the sample entry of a track with an identifier equal to the value in sample_entry_track_id[], or with the sample entry of the current track when current_sample_entry_flag is equal to 1, the player can switch to playing any track in this track group without reinitializing the decoder.

[0265] current_sampe_entry_flag equal to 1 indicates that the sample entry of the current track is suitable for switching to any track of the current track group without reinitialization.

[0266] num_sample_entry_track_ids indicates the number of track identifiers in the track group box.

[0267] sample_entry_track_id[i] indicates the i-th track identifier of the sample entry to be used for decoder initialization, to allow the player to switch between tracks of a track group without reinitialization.

[0268] Fig. 9An example of indicating alternative sample entries for no initialization switching with one 1080p track 220 and two 4K tracks 221, 222 (where all tracks 220, 221, 222 are marked with a "bbbb" sample group) and the corresponding settings of current_sample_entry_flag and sample_entry_track_id[i]. That is, the 1080p track 220 is marked with current_sample_entry_flag=0 (1080p SPS is not sufficient for 4K playback) and sample_entry_track_id[0]=1, sample_entry_track_id[1]=2 (pointing to the sample entry of track 221 or track 222). The 4K tracks 221, 222 are marked with current_sample_entry_flag=1 (4K SPS also allows 1080 playback) and references to the other 4K tracks 221, 222, respectively. In other words, when current_sample_entry_flag = 1, selecting the sample entry of the current playing track only allows track switching. It should be noted that for the case of current_sample_entry_flag = 1, in order to save bits, signaling an additional sample_entry_track_id[] entry may be omitted.

[0269] Thus, some embodiments may provide a file parser 300 configured to retrieve, for each track 220, 221, 222 of track group "bbbb", an indication (current_sample_entry_flag) for indicating whether a track switch from a particular track 220 to any other potential switch to a candidate track 221, 222 within the track group "bbbb" is allowed while continuing to reuse the initialization parameters 304 initially used to reconstruct the bitstream 310 at the beginning of the encoded video sequence 203.

[0270] For example, if a specific track (e.g., track 222) has "current_sample_entry_flag=1", if the reconstructed bitstream 304 is initialized with the initialization parameter set 304 of the specific track 222, it can be signaled that track switching between any of the tracks 220, 221, 222 within the track group "bbbb" is allowed. Conversely, if a specific track (e.g., track 220) has "current_sample_entry_flag=0", if the reconstructed bitstream 304 is initialized with the initialization parameter set 304 of the specific track 222, it can be signaled that track switching between any of the tracks 220, 221, 220 within the track group "bbbb" is not allowed.

[0271] In other words, if the bitstream 310 is reconstructed at the beginning of the bitstream 310 by using the initialization parameter set 304 belonging to the corresponding switched-from track 222, the indication (current_sample_entry_flag) of the corresponding switched-from track (e.g., track 222) indicates whether switching from the corresponding switched-from track 222 to any other potential switching candidate tracks 220, 221 within the track group 'bbbb' is allowed (while continuing to reuse the initialization parameters 304 used to reconstruct the bitstream 310 at the beginning of the encoded video sequence 203).

[0272] Additionally or alternatively, the file parser 300 may be configured to retrieve an initialization parameter set list (sample_entry_track_id[]) for each track 220, 221, 222 of the track group "bbbb". For example, track 220 has an initialization parameter set list (sample_entry_track_id[]) including a first entry named "sample_entry_track_id[0]=1" and a second entry named "sample_entry_track_id[1]=2". The initialization parameter set list (sample_entry_track_id[]) explicitly points to those one or more switch-to tracks associated with the initialization parameter set 304 (e.g., "sample_entry_track_id[0]=1" points to track 221, and "sample_entry_track_id[1]=2" points to track 222), which initialization parameter set 304 allows track switching from the corresponding switch-to track (e.g., track 220) to any other potential switch-to candidate within the track group "bbbb" (e.g., to switch-to tracks 221, 222) when the bitstream 310 is initially reconstructed at the beginning of the coded video sequence 203 by using one of the one or more switch-to tracks (221, 222) contained in the initialization parameter set list (sample_entry_track_id[]).

[0273] In other words, referring to track 220, a track switch from track 220 to any other track within track group "bbbb" is only allowed if the initialization parameter set 304 of "track 1" 221 (sample_entry_track_id[0]=1) or if the initialization parameter set 304 of "track 2" 222 (sample_entry_track_id[1]=2) is used for initial reconstruction of the bitstream 310 at the beginning of CVS 203.

[0274] Similar to Section 2.1, with the help of the provided signaling, the player will select a sample description box for decoder initialization that corresponds to its maximum decoder capabilities and / or the expected maximum operating point due to various factors (e.g., provided and available operating points, network bandwidth, power budget, computational processing budget, display capabilities, etc.).

[0275] All aspects described from Section 2.1 to Section 2.3 rely on the indication of the hierarchy between tracks. In further embodiments, these hierarchies may be simply derived from parameters included in the tracks. For example, higher picture size, superset profile, higher level, larger DPB, higher chroma format, higher bit depth will result in higher hierarchy. Therefore, a group of switchable tracks may be signaled simply by using a track group mechanism or entity group, indicating that reuse of sample description boxes of different tracks may be done between tracks belonging to such a group as described above and that the hierarchy is implied by the functionality described in this paragraph and is not explicitly signaled.

[0276] 2.4 Example grouping for alternative in-band parameter sets

[0277] The above solution works for parameter sets obtained from the sample entry in the Sample Description box, so-called out-of-band parameter sets. However, there are parameter sets that need to be included with the sample in the track itself (see above for reference). Figure 7 described in the problem) as the case of so-called in-band parameter sets.

[0278] Therefore, in another alternative, in order to overcome the above-mentioned problem regarding the decoder initialization and the need to restart the CVS also in case of switching of in-band parameter sets at an open GOP position, the sample grouping mechanism is extended to include sample groups of additional parameter sets in the samples of a track, which additional parameter sets are alternatives to the regular in-band parameter sets for the track.

[0279] The player selects the correct parameter set based on the picture size, color format, bit depth and number of picture slots in the DPB (Decoded Picture Buffer) as indicated in the sample group entry. When the player initializes the decoder with parameters corresponding to the alternative parameters and wants to avoid decoder reinitialization, the parameter set (default values ​​for a particular track) within this sample needs to be swapped with the selected alternative parameter set.

[0280] For example, refer to Figure 7 and Figure 8 , a first representation (ie track) 401 comprises a first GOP 2411 and a second GOP 2421 , wherein parameter sets 204a , 204b , 204e , 204h are contained in the track together with samples.

[0281] Figure 7 An example is shown where the first representation (ie track) 401 switches from high resolution (4k) content to low resolution (1080p) content. The sequence-dependent initialization parameter set 204e initially contains 4K parameters. That is, the player initializes the decoder with the 4k parameters corresponding to the 4k initialization parameter set 204e.

[0282] However, the low-resolution content by default includes an in-band transmitted initialization parameter set 204g, which includes 1080p parameters and is the default initialization parameter set. As described above, an open GOP switch does not start a new coded video sequence (CVS). Therefore, the in-band transmitted sequence-dependent default initialization parameter set 204g containing low-resolution parameters for 1080p is invalid for this CVS.

[0283] As described above, the player initializes the decoder with the 4k initialization parameter set 204e. Therefore, if the decoder is to be prevented from being reinitialized after switching, the default 1080p initialization parameter set 204g must be replaced by an additional or alternative initialization parameter set. Specifically, the default 1080p initialization parameter set 204g must be replaced by an additional or alternative initialization parameter set that is compatible with the initialization parameter set 204e used by the player to initialize the decoder.

[0284] Figure 8 Such a scenario is shown, where the default initialization parameter set 204g ( Figure 7 ) is replaced by an alternative initialization parameter set 204h. The alternative initialization parameter set 204h is compatible with the initialization parameter set 204h used by the player to initialize the decoder. In this non-limiting example, the default 1080p initialization parameter set 204g ( Figure 7 ) Alternative 4k initialization parameter set 204h ( Figure 8 ) is replaced. This alternative 4k initialization parameter set 204g is compatible with the initialization parameter set 204e (which also contains 4k parameters) used by the player to initialize the decoder. Figure 7 and Figure 8 As can be seen in FIG. 2 , the default initialization parameter set 204g and the alternative initialization parameter set 204h are transmitted in-band within the sample.

[0285] According to some embodiments of the present invention, as described above, samples for which a default in-band initialization parameter set may have to be replaced by an alternative initialization parameter set may be grouped in sample groups. Each sample group may be associated with one or more initialization parameter sets to be inserted into the reconstructed bitstream and referenced by its samples. In other words, each sample group may include one or more alternative initialization parameter sets that may replace any default initialization parameter set.

[0286] The candidate initialization parameter set may be associated with a certain predetermined or desired operating point and / or maximum decoding capability of a decoder for decoding the entire coded video sequence (CVS). Figure 8 In the example shown, the 4k content may represent a maximum operating point, such that the 4k parameters included in the 4k initialization parameter set 204h may enable the decoder to decode the entire CVS without reinitialization.

[0287] The desired operating point and / or the maximum decoding behavior of the decoder depends on at least one of the following properties, wherein at least one of the properties is indicated in the corresponding sample group:

[0288] • Maximum Codec Profile,

[0289] • Maximum codec level,

[0290] • a maximum picture width of pictures contained in the one or more coded video sequences 203 to be decoded,

[0291] • the maximum picture height of the pictures contained in the one or more coded video sequences 203 to be decoded,

[0292] • a maximum picture bit depth of pictures contained in the one or more coded video sequences 203 to be decoded,

[0293] • the maximum picture color format of the pictures contained in the one or more coded video sequences 203 to be decoded, and

[0294] • A maximum size of a decoded picture buffer (DPB) for pictures contained in the one or more coded video sequences 203 to be decoded.

[0295] As described above, a sample group may contain multiple samples, and the sample group may be associated with an alternative initialization parameter set (e.g., initialization parameter set 204h). Additionally or alternatively, the sample group may be associated with more than one alternative initialization parameter set (e.g., any of initialization parameter sets 204a, ..., 204h). If the sample group is associated with more than one alternative initialization parameter set, then

[0296] A) each of the more than one initialization parameter sets 204a, ..., 204h may be associated with the same operating point, or

[0297] B) Each of the more than one initialization parameter sets 204a, ..., 204h may be associated with at least one operating point of the plurality of operating points.

[0298] According to option A), each of the initialization parameter sets 204a, ..., 204h may be associated with the same operating point. For example, each initialization parameter set 204a, ..., 204h may include parameters for decoding a specific resolution (eg, 4k).

[0299] According to option B), each of the initialization parameter sets 204a, ..., 204h may be associated with one or more operating points. For example, each initialization parameter set 204a, ..., 204h may include parameters for decoding a specific resolution (e.g., 1080p, 4k, 8k).

[0300] In any case, the file parser 301 of the present invention may be configured to initialize a predetermined set of parameters for a predetermined sample group (e.g., Figure 8 204h) is included in the reconstructed bitstream 310. In other words, the file parser 301 of the present invention can be configured to initialize the parameter set 204h ( Figure 8 ) replaces the default in-band initialization parameter set 204g ( Figure 7 ).

[0301] For example, a "pase" sample group and description box could be stored in a VisualSampleGroupEntry().

[0302] class ParameterSetSamplegroup () extends VisualSampleGroupEntry ('pase')

[0303] {

[0304] unsigned int(16) max_width;

[0305] unsigned int(16) max_height;

[0306] unsigned int(16) max_bitdepth;

[0307] unsigned int(16) max_color_format;

[0308] unsigned int(16) max_dec_pic_buffering_minus1;

[0309] unsigned int(16) paramSetNalUnitLength;

[0310] bit(8* paramSetNalUnitLength) paramSetNalUnit;

[0311] }

[0312] The above parameters may also be stored in SampleToGroupBox() instead of in VisualSampleGroupEntry() as exemplarily mentioned.

[0313] Therefore, when the "pase" sample description box is used to target the maximum decoding capability with at least one of max_widht, max_height, max_bitdepth, max_color_format and max_dec_pic_buffering_minus1+1 pictures, the parameter Set in the "pase" sample group needs to be inserted into the bitstream and the original bitstream needs to be removed.

[0314] This applies in particular to SPS, but also to VPS. Although a single box is shown above, two different boxes may be used, one for SPS and one for VPS, or the corresponding properties may be inherited from one parameter set to another (eg from VPS to SPS).

[0315] According to an embodiment, a "pase" sample group may also indicate one or more operation points, for example by indicating an output layer set index and / or a highest temporal layer identifier.

[0316] In addition to the signaled characteristics, further characteristics related to, for example, an operating point may also be signaled, such as a profile or a level.

[0317] As described above, samples within a track may reference a default in-band initialization parameter set 204g. Multiple samples may be grouped together in a sample group. In this regard, a sample may be assigned to more than one sample group. Samples within a sample group may reference one or more alternative initialization parameter sets 204h that may replace the default in-band initialization parameter set 204g.

[0318] Thus, if one or more samples within a track reference a default in-band initialization parameter set 204g associated with the track by default, and if the one or more samples are marked as belonging to at least one sample group, the file parser 300 may be configured to derive an alternative initialization parameter set (e.g., 4k initialization parameter set 204h) associated with the corresponding sample group. The file parser 300 may also be configured to replace the initialization parameter set (e.g., 1080p initialization parameter set 204g) that would otherwise be sent in-band with the alternative initialization parameter set 204h. This alternative initialization parameter set 204h for the sample group may be provided to the decoder as a predetermined (selected) initialization parameter set in-band (i.e., within the reconstructed bitstream 310).

[0319] If the track does not include any default in-band initialization parameter set, an alternative initialization parameter set (eg, initialization parameter set 204 h ) may be selected from one of the sample groups to be inserted into the reconstructed bitstream 310 .

[0320] Therefore, if a track is not associated with any default in-band initialization parameter set by default, and if one or more samples contained in the track are marked as belonging to at least one sample group of the one or more sample groups, the file parser 300 of the present invention can be configured to: insert a predetermined initialization parameter set (e.g., 204h) from the one or more initialization parameter sets associated with the at least one sample group into the reconstructed bitstream 310, and provide the predetermined initialization parameter set 204h for the at least one sample group within the reconstructed bitstream 310 to the decoder.

[0321] It may be signaled for an entire track whether the samples of the track may be associated with any default in-band initialization parameter set. Thus, an explicit signal may be provided indicating that the track is not associated with any default in-band initialization parameter set by default. Additionally or alternatively, an explicit signal may be provided indicating that an initialization parameter set for the track (e.g., 4k initialization parameter set 204h) must be selected from at least one of the available one or more sample groups and that the selected initialization parameter set 204h must be provided to the decoder as a predetermined initialization parameter set.

[0322] Additionally or alternatively, it may be signaled for individual samples of a sample group whether the sample may be associated with any default in-band initialization parameter set. Thus, an explicit signal may be provided indicating that those samples of a track marked as belonging to one or more sample groups do not refer to any default in-band initialization parameter set (e.g. 1080p initialization parameter set 204g) by default. Additionally or alternatively, an explicit signal may be provided indicating that for those samples, an initialization parameter set (e.g. 4k initialization parameter set 204h) from at least one of the available one or more sample groups to which those samples belong must be inserted into the reconstructed bitstream (310) and provided to the decoder as the predetermined initialization parameter set.

[0323] 2.5 Additional decoding request signaling

[0324] As mentioned above, the sample description box may contain multiple sample entries. These can also reflect changes in the underlying decoding process and may cause decoder reinitialization on track switching, which is desirable to avoid.

[0325] Note that VVC already contains NAL units called Decoding Capability Information (DCI) which indicate the conformance and capability requirements of the entire bitstream in addition to the individual coded video sequences included. This means: different CVSs within the bitstream and their individual conformance levels (lower or equal) and profiles (subset or equal) are decodable without reinitialization if the decoder is able to decode the signaled levels and profiles (which describe the complexity of the bitstream). However, in addition to the profiles and levels controlled by the used SPS, there are other aspects (such as bit depth and color format) that may cause problems and trigger decoder reinitialization when, for example, the DPB buffer management is not properly initialized to cope with different picture formats in the bitstream.

[0326] For example, assume that the DPB is initialized with 9 picture slots at 8 bits and 4:2:0 at a certain resolution, and at some time instance, the format changes, which is indicated in a new SPS (while still complying with the profile and level indicated in the DCI), such as the format changes to 10 bits and 4:4:4 at the same resolution and the same number of picture slots. Since the initial memory allocation is not sufficient for the new format of the decoded pictures, the decoder will need to be reinitialized (at least the DPB), and in some cases this will result in delays, glitches, frame grabbing and discontinuous playback and decoding.

[0327] Therefore, additional information needs to be signaled about how to initialize the decoder in this case and for the player to know if the decoder needs to be reinitialized.In a first embodiment, additional information about bit depth, color format, resolution and number of picture slots is transmitted by DCI extension as follows.

[0328]

[0329]

[0330] The above-mentioned DCI extension (dci_extension()) is global bitstream-related decoding parameter information (i.e., the parameters contained therein are globally valid for the entire bitstream 310) rather than sequence-related decoding parameter information in the case of SPS or picture-related decoding parameter information in the case of PPS. The DCI extension can be signaled by a flag (dci_extension-flag).

[0331] Therefore, according to some embodiments, a method for processing an input video bitstream 310 ( Fig. 1I). The pictures 10 are arranged in one or more consecutively arranged coded video sequences 203, each coded video sequence 203 being associated with its own sequence-dependent initialization parameter set (SPS) carrying sequence-dependent decoding parameter information.

[0332] The decoder 320 may be configured to initialize its decoding behavior by using predetermined decoding parameter information before starting to decode the encoded video sequence 203 for decoding the picture 10 of the video content 201. Therefore, the decoder 320 may be configured to derive a signal (dci_extension_flag) indicating to the decoder 320 that global bitstream-related decoding parameter information (dci_extension()) is used for its initialization instead of sequence-related decoding parameter information (SPS), the global bitstream-related decoding parameter information indicating a predetermined global decoding capability for all encoded video sequences 203 contained in the entire input video bitstream 310.

[0333] A corresponding encoder 210 ( Figure 1H ), the encoder 210 is configured to provide predetermined decoding parameter information to be used by the above-mentioned decoder 320 to initialize its decoding behavior at the beginning of the encoded video sequence 203 for decoding the picture 10 contained in the encoded video sequence 203. As mentioned above, each encoded video sequence 203 can be associated with its own sequence-related initialization parameter set (SPS) carrying sequence-related decoding parameter information. The encoder 210 can be configured to: encode a signal (dci_extension_flag) into the bitstream 310, which indicates to the decoder 320 to use global bitstream-related decoding parameter information (dci_extension()) instead of any sequence-related decoding parameter information for its initialization, and the global bitstream-related decoding parameter information indicates a predetermined global decoding capability for all encoded video sequences 203 contained in the entire input video bitstream 310.

[0334] The global bitstream-dependent decoding parameter information (dci_extension()) indicates the highest required global decoding capability 203 of the decoder to be able to decode all coded video sequences within the entire input video bitstream 310 without reinitialization.

[0335] Apart from the highlighted constraints, the following semantics are similar to those of the original SPS syntax elements.

[0336] dci_pic_width_max_in_luma_samples specifies the maximum width in units of luma samples of each decoded picture 10 that references the SPS. dci_pic_width_max_in_luma_samples shall not be equal to 0 and shall be an integer multiple of Max(8, MinCbSizeY).

[0337] Therefore, if the decoder 320 is initially initialized with the global bitstream-related decoding parameter information (dci_extension()), the global bitstream-related decoding parameter information (dci_extension) includes information about the maximum picture width (dci_pic_width_max_in_luma_samples) of picture 10 in units of luma samples that the decoder 320 can decode without reinitialization.

[0338] The sequence-dependent decoding parameter information (SPS) includes the maximum picture width in units of luma samples (sps_pic_width_max_in_luma_samples) for those pictures 10 contained in one specific coded video sequence 203 .

[0339] In contrast, the global bitstream-dependent decoding parameter information (dci_extension()) includes information about a global bitstream-dependent maximum picture width in units of luma samples (dci_pic_width_max_in_luma_samples) for all pictures 10 contained in all coded video sequences 203 in the entire input video bitstream 310,

[0340] A requirement for bitstream conformance is that the SPS referenced by picture 10 in the bitstream 310 does not have a value of sps_pic_width_max_in_luma_samples that is greater than dci_pic_width_max_in_luma_samples.

[0341] That is, the global bitstream-dependent maximum picture width (dci_pic_width_max_in_luma_samples) is greater than or equal to any occurring maximum sequence-dependent maximum picture width (sps_pic_width_max_in_luma_samples).

[0342] dci_pic_height_max_in_luma_samples specifies the maximum height in units of luma samples of each decoded picture that references the SPS. dci_pic_height_max_in_luma_samples shall not be equal to 0 and shall be an integer multiple of Max(8, MinCbSizeY).

[0343] Therefore, if the decoder 320 is initially initialized with the global bitstream-related decoding parameter information (dci_extension()), the global bitstream-related decoding parameter information (dci_extension()) includes information about the maximum picture height of picture 10 in units of luma samples that the decoder 320 can decode without reinitialization.

[0344] The sequence-dependent decoding parameter information (SPS) includes a maximum picture height in units of luma samples (sps_pic_height_max_in_luma_samples) for those pictures (10) contained in a particular coded video sequence (203).

[0345] Instead, the global bitstream-dependent decoding parameter information (dci_extension()) comprises information about a global bitstream-dependent maximum picture height in units of luma samples (dci_pic_height_max_in_luma_samples) for all pictures 10 contained in all coded video sequences 203 in said entire input video bitstream 310,

[0346] A requirement for bitstream conformance is that the SPS referenced by picture 10 in the bitstream 310 does not have a value of sps_pic_height_max_in_luma_samples that is greater than dci_pic_height_max_in_luma_samples.

[0347] Therefore, the global bitstream-dependent maximum picture height (dci_pic_height_max_in_luma_samples) is greater than or equal to any occurring maximum sequence-dependent maximum picture height (sps_pic_height_max_in_luma_samples).

[0348] dci_chroma_format_idc specifies the chroma samples relative to the luma samples, as specified in subsection 6.2 in the VVC specification.

[0349] Therefore, if the decoder 320 is initially initialized with the global bitstream-related decoding parameter information (dci_extension()), the global bitstream-related decoding parameter information (dci_extension()) includes information on a maximum chroma format (dci_pic_chroma_format_idc) of picture 10 that the decoder 320 can decode without reinitialization.

[0350] The sequence-dependent decoding parameter information (SPS) includes sequence-dependent maximum chroma format (sps_chroma_format_idc) about those pictures 10 contained in a particular coded video sequence 203 .

[0351] In contrast, the global bitstream-dependent decoding parameter information (dci_extension()) includes information about a global bitstream-dependent maximum chroma format (dci_chroma_format_idc) for all pictures 10 contained in all coded video sequences 203 in the entire input video bitstream 310 .

[0352] A requirement for bitstream conformance is that the SPS referenced by picture 10 in the bitstream 310 does not have a value of sps_chroma_format_idc greater than dci_chroma_format_idc.

[0353] That is, the global bitstream-dependent maximum chroma format (dci_chroma_format_idc) is greater than or equal to any occurring maximum sequence-dependent maximum chroma format (sps_chroma_format_idc).

[0354] Furthermore, the picture 10 contained in the one or more coded video sequences 203 refers to video-dependent decoding parameter information (VPS), the VPS comprising information about a video-dependent maximum chroma format of the picture 10 contained in the input video bitstream 310,

[0355] As described above, the global bitstream-related decoding parameter information (dci_extension()) includes information on the global bitstream-related maximum chroma format (dci_chroma_format_idc) of all pictures 10 included in the entire input video bitstream 310.

[0356] A requirement for bitstream conformance is that the VPS referenced by picture 10 in the bitstream 310 does not have a value of vps_ols_dpb_chroma_format[i] greater than dci_chroma_format_idc, where i is in the range of 0 to NumMultiLayerOlss-1.

[0357] That is, for any included set of output layers, the global bitstream-dependent maximum chroma format (dci_chroma_format_idc) is not greater than or equal to any occurring maximum video-dependent maximum chroma format (vps_ols_dpb_chroma_format).

[0358] dci_bitdepth_minus8 specifies the bit depth of samples of the luma and chroma arrays in the VVC specification and the value of the luma and chroma quantization parameter range offset in the VVC specification. dci_bitdepth_minus8 should be in the range of 0 to 2, inclusive.

[0359] Therefore, if the decoder (320) is initially initialized with the global bitstream-related decoding parameter information (dci_extension()), the global bitstream-related decoding parameter information (dci_extension()) includes information about the maximum bit depth (dci_bitdepth_minus8) of the picture (10) that the decoder (320) can decode without reinitialization.

[0360] The sequence-dependent decoding parameter information (SPS) includes information about the maximum bit depth (dci_bitdepth_minus8) of the pictures 10 contained in a particular coded video sequence 203 .

[0361] In contrast, the global bitstream-dependent decoding parameter information (dci_extension()) includes information about a global bitstream-dependent maximum bit depth (dci_bitdepth_minus8) of all pictures 10 contained in all coded video sequences 203 in the entire input video bitstream 310 .

[0362] A requirement for bitstream conformance is that the SPS referenced by picture 10 in the bitstream 310 does not have a value of sps_bitdepth_minus8 greater than dci_bitdepth_minus8.

[0363] That is, the global bitstream-dependent maximum bit depth (dci_bitdepth_minus8) is greater than or equal to any occurring maximum sequence-dependent maximum bit depth (sps_bitdepth_minus8).

[0364] dci_dpb_max_dec_pic_buffering_minus1 plus 1 specifies the maximum size of the DPB required to decode the bitstream in units of picture storage buffers. The value of dci_dpb_max_dec_pic_buffering_minus1 shall be in the range of 0 to MaxDpbSize-1, inclusive, where MaxDpbSize is as specified in section A.4.2 in the VVC specification.

[0365] Therefore, if the decoder 320 is initially initialized with the global bitstream-related decoding parameter information (dci_extension()), the global bitstream-related decoding parameter information (dci_extension()) includes information about a size (dci_dpb_max_dec_pic_buffering_minus1) of a decoded picture buffer (DPB) in units of a picture storage buffer that the decoder 320 needs to be able to decode the input video bitstream 310 without reinitialization.

[0366] The sequence-dependent decoding parameter information (SPS) includes information about a sequence-dependent maximum size (dpb_max_dec_pic_buffering_minus1) in units of picture storage buffers of a decoded picture buffer (DPB) for those pictures 10 contained in one specific coded video sequence 203 .

[0367] In contrast, the global bitstream-dependent decoding parameter information (dci_extension()) includes information about a global bitstream-dependent maximum size (dci_dpb_max_dec_pic_buffering_minus1) of a decoded picture buffer (DPB) for all pictures 10 contained in all coded video sequences 203 in the entire input video bitstream 310 .

[0368] A requirement for bitstream conformance is that the SPS referenced by picture 10 in the bitstream 310 does not have a value of dpb_max_dec_pic_buffering_minus1[i] greater than dci_dpb_max_dec_pic_buffering_minus1, where any value of i is in the range from 0 to MaxSubLayersMinus1.

[0369] That is, the global bitstream-dependent maximum size (dci_dpb_max_dec_pic_buffering_minus1) of the decoded picture buffer (DPB) is greater than or equal to the maximum sequence-dependent maximum size (dpb_max_dec_pic_buffering_minus1) of any occurrence of the decoded picture buffer (DPB).

[0370] In one embodiment, the global bitstream related decoding parameter information (dci_extension()) may be signaled in a decoding capability information (DCI) NAL unit or in a supplemental enhancement information (SEI) message.

[0371] Alternatively, these parameters may be indicated separately in a new Bitstream DPB Information (BDI) SEI message as follows with similar semantic constraints as above.

[0372]

[0373] Therefore, the decoder 320 may be configured to derive a signal (dci_extension_flag) for signaling the use of global bitstream related decoding parameter information (dci_extension()) from the bitstream (310).

[0374] In another embodiment, instead of signaling the corresponding parameter values, the client / player is suggested or required to allocate the maximum value of some or all items allowed in the specification of the maximum capabilities of the decoder (in particular the number of picture slots, but also not excluding picture size, bit depth, color format) to avoid reinitialization.

[0375] Therefore, it is proposed to provide a player comprising a decoder 320 for decoding a picture 10 of a video content 202 contained in an input video bitstream 310, wherein the picture 10 is arranged in one or more consecutively arranged coded video sequences (203), each coded video sequence (203) being associated with its own sequence-dependent initialization parameter set (SPS) carrying sequence-dependent decoding parameter information. As described above, the decoder 310 can be configured to initialize its decoding behavior by using at least one initialization parameter from a predetermined initialization parameter set (e.g., codec level, codec profile, picture width, picture height, bit depth, color format, number of picture slots in a DPB, and number of pictures) before starting to decode a specific coded video sequence, instead of using the signaled sequence-dependent decoding parameter information (SPS) 203, for decoding the picture 10 of the video content 202.

[0376] According to this embodiment, the player can be configured (e.g., recommended or required) to assign a maximum value for at least one parameter in the initialization parameter set (e.g., max_width, max_height, max_bitdepth, max_color_format, max_dec_pic_buffering_minus1, and numPics) to define the maximum decoding requirement that the decoder 320 can decode all encoded video sequences 203 in the bitstream 310 that meet these maximum decoding requirements without reinitialization.

[0377] The initialization parameter set defining the minimum decoding requirements of the decoder 320 includes at least one of the following parameters that can be assigned directly by the player rather than derived from any signal in the file:

[0378] • Maximum Codec Profile,

[0379] • Maximum codec level,

[0380] • Maximum decoder capability,

[0381] • Desired operating point,

[0382] • Desired target display capability,

[0383] • Maximum picture width of the picture to be decoded 10,

[0384] • Maximum picture height of picture 10 to be decoded,

[0385] • the maximum color format of the picture to be decoded 10,

[0386] • the maximum bit depth of picture 10 to be decoded, and

[0387] • Maximum number of picture slots in the decoded picture buffer (DBP).

[0388] An exemplary means by which the client / player initializes the decoder in the described manner is by adapting / rewriting existing in-band or out-of-band parameter sets before using them for the reconstructed bitstream 310. Thus, the player of the present invention may be configured to initialize the decoder 320 in the described manner by adapting / rewriting existing in-band or out-of-band parameter sets before using them for the reconstructed bitstream 310. For example, the player of the present invention may be configured to initialize the decoder 320 in the described manner by adjusting one or more of the parameters listed above.

[0389] The client / player may adapt / overwrite an existing in-band or out-of-band parameter set and thereby initialize the decoder 320, for example, by adjusting the maximum picture width and height indicated in the SPS according to the maximum decoder capability of the device and / or by adjusting the number of time slots in the decoded picture buffer to the maximum value of the corresponding specification, etc.

[0390] More specifically, for example, if no syntax elements are signaled, then the highest value for a particular level is inferred as described above. However, if some other values ​​are signaled (e.g., the maximum width and maximum height to be encountered in the bitstream 310), then the other missing syntax elements described above may be inferred, which implement values ​​higher than the default values, for example, if the picture width and picture height are less than the maximum values ​​in the level.

[0391] As an alternative, such signaling may be carried within the file format, for example, as a sample group. Thus, the above decoder 320 may be configured to derive a signal (dci_extension_flag) for signaling the use of global bitstream related decoding parameter information (dci_extension()) from the file parser 300 within the file format, via the sample group (DpbSamplegroup) to which one or more samples of the track (220, 221, 222) of the picture 10 of the input video bitstream 310 are embedded.

[0392] class DpbSamplegroup () extends VisualSampleGroupEntry ('dpbs')

[0393] {

[0394] unsigned int(16) max_width;

[0395] unsigned int(16) max_height;

[0396] unsigned int(16) max_bitdepth;

[0397] unsigned int(16) max_color_format;

[0398] unsigned int(16) max_dec_pic_buffering_minus1;

[0399] unsigned int(16) numPics;

[0400] }

[0401] It can be seen that the sample group DpbSamplegroup includes global bitstream-related decoding parameter information, which includes a set of one or more global bitstream-related decoding parameters. The global bitstream-related decoding parameter information indicates a predetermined (and in particular maximum) global decoding capability for all coded video sequences 203 contained in the entire input video bitstream 310, such as max_width, max_height, max_bitdepth, max_color_format, max_dec_pic_buffering_minus1, and numPics.

[0402] Specifically, the global bitstream-related decoding parameter information includes decoding capability information about at least one of the following:

[0403] • Maximum Codec Profile,

[0404] • Maximum codec level,

[0405] • Maximum picture width of the picture to be decoded 10,

[0406] • Maximum picture height of picture 10 to be decoded,

[0407] • the maximum color format of the picture to be decoded 10,

[0408] • the maximum bit depth of picture 10 to be decoded, and

[0409] • The maximum number of picture slots in the decoded picture buffer (DBP) of all coded video sequences (203) contained in the entire reconstructed video bitstream (310).

[0410] The entire reconstructed video bitstream 310 contains all the coded video sequences 203

[0411] For example, if there is a single sample group for the track indicating the maximum picture size (width and / or height) and the required number of slots (maxslots), the decoder 320 may be initialized with such information.

[0412] The file parser 300 may be configured to select the decoding capability information based on the following information:

[0413] • one or more maximum decoder capabilities,

[0414] • one or more desired operating points, and

[0415] • One or more desired target display capabilities.

[0416] The file parser 300 may be configured to inform the decoder 320 to use the above-mentioned global bitstream-dependent decoding parameter information instead of any sequence-dependent decoding parameter information for its initialization.

[0417] In summary, the file parser 300 of the present invention may be configured to reconstruct a video bitstream 310 from a video file 201, wherein the video file 201 includes one or more tracks 220, 221, 222, each track having one or more input video bitstreams 211, 212 embedded therein, wherein each of the one or more input video bitstreams 211, 212 includes one or more encoded video sequences 203. The file parser 300 may also be configured to retrieve a sample group (DpbSamplegroup) to which one or more samples of the one or more tracks belong from the video file 201, wherein the sample group (DpbSamplegroup) contains global bitstream-related decoding parameter information, which includes one or more global bitstream-related decoding parameters (e.g., codec level, codec profile, picture width, picture height, bit depth, color format, number of picture slots in DPB, and number of pictures). The file parser 300 may also be configured (e.g., suggested or required) to assign a maximum value for at least one decoding parameter in the decoding parameter set (e.g., in a sample group) (e.g., max_width, max_height, max_bitdepth, max_color_format, max_dec_pic_buffering_minus1, and numPics) to define a predetermined decoding capability at which the decoder 320 is able to decode all coded video sequences 203 in the bitstream 310 that conform to these predetermined decoding capabilities without reinitialization, rather than initializing the predetermined decoding capability using any sequence-related decoding parameter information.

[0418] The embodiment of the present invention also provides a file generator 200 for generating a video file 201 from one or more input video bitstreams 211, 212 (see Figure 1H), the video content 202 is encoded into the one or more input video bitstreams 211, 212, wherein the video file 201 comprises one or more tracks 20, 221, 222 in which the one or more input video bitstreams 211, 212 are embedded, wherein each of the one or more input video bitstreams 211, 212 comprises one or more encoded video sequences 203. The file generator 200 may be configured to provide, in the video file 201, a sample group (DpbSamplegroup) to which one or more samples of the one or more tracks belong, wherein the sample group (DpbSamplegroup) contains global bitstream-related decoding parameter information, the global bitstream-related decoding parameter information comprising one or more global bitstream-related decoding parameters (e.g., codec level, codec profile, picture width, picture height, bit depth, color format, number of picture slots in DPB, and number of pictures). The sample group (DpbSamplegroup) may also include a maximum value of at least one decoding parameter in the decoding parameter set (in the sample group) (for example, max_width, max_height, max_bitdepth, max_color_format, max_dec_pic_buffering_minus1 and numPics), which is used to define the predetermined decoding capability of the decoder 320 to decode all encoded video sequences 203 in the bitstream 310 that conform to these predetermined decoding capabilities without reinitialization, rather than using any sequence-related decoding parameter information to initialize the predetermined decoding capability.

[0419] The above signaling may indicate that, for a particular track, regardless of whether there is more than one sample entry with different codec parameters, given any sample in that track, the indicated configuration (worst case) applies.

[0420] A further embodiment of this aspect relates to a method for decoding a picture 10 of a video content 202 contained in an input video bitstream 310, the method comprising the steps of initializing a decoding behavior of a decoder 320 for decoding a picture 10 of the video content 202 by using predetermined decoding parameter information before starting to decode a coded video sequence 203. The pictures 10 are arranged in one or more consecutively arranged coded video sequences 203, each coded video sequence 203 being associated with its own sequence-dependent initialization parameter set (SPS) carrying sequence-dependent decoding parameter information. The method further comprises the step of deriving a signal (dci_extension_flag) indicating that the decoding behavior of the decoder 320 is initialized using global bitstream-dependent decoding parameter information (dci_extension) instead of sequence-dependent decoding parameter information, the global bitstream-dependent decoding parameter information indicating a predetermined global decoding capability for all coded video sequences (203) contained in the entire input video bitstream 310.

[0421] A further embodiment of this aspect relates to an encoder 210 for encoding pictures 10 of video content 202 into a video bitstream 310, wherein the pictures 10 are arranged in one or more coded video sequences 203, the method comprising the steps of providing predetermined decoding parameter information to be used by a decoder 320 to initialize its decoding behavior at the beginning of the coded video sequence 203 for decoding the pictures 10 contained in the coded video sequence 203, each coded video sequence 203 being associated with its own sequence-dependent initialization parameter set (SPS) carrying sequence-dependent decoding parameter information. The method further comprises the steps of encoding a signal (dci_extension_flag) into the bitstream 310, the signal indicating to the decoder 320 to use global bitstream-dependent decoding parameter information (dci_extension) for its initialization instead of sequence-dependent decoding parameter information, the global bitstream-dependent decoding parameter information indicating a predetermined global decoding capability for all coded video sequences 203 contained in the entire input video bitstream 310.

[0422] Although some aspects have been described in the context of an apparatus, it will be clear that these aspects also represent a description of a corresponding method, wherein a block or apparatus corresponds to a method step or a feature of a method step. Similarly, aspects described in the context of a method step also indicate a description of a feature of a corresponding block or item or a corresponding apparatus.

[0423] Some or all of the method steps may be performed by (or using) a hardware device (such as a microprocessor, a programmable computer or an electronic circuit). In some embodiments, one or more of the most important method steps may be performed by such a device.

[0424] According to certain implementation requirements, embodiments of the present invention may be implemented in hardware or software, or at least partially in hardware, or at least partially in software. The implementation may be performed using a digital storage medium (e.g., a floppy disk, DVD, Blu-ray, CD, ROM, PROM, EPROM, EEPROM, or flash memory) on which electronically readable control signals are stored, which cooperate (or are capable of cooperating) with a programmable computer system to perform the corresponding method. Therefore, the digital storage medium may be computer readable.

[0425] Some embodiments according to the invention comprise a data carrier having electronically readable control signals, which are capable of cooperating with a programmable computer system, such that one of the methods described herein is performed.

[0426] Generally, embodiments of the present invention can be implemented as a computer program product with a program code, the program code being operable to perform one of the methods when the computer program product runs on a computer. The program code may, for example, be stored on a machine readable carrier.

[0427] Other embodiments comprise the computer program for performing one of the methods described herein, stored on a machine readable carrier.

[0428] In other words, an embodiment of the inventive method is, therefore, a computer program having a program code for performing one of the methods described herein, when the computer program runs on a computer.

[0429] A further embodiment of the inventive method is, therefore, a data carrier or a digital storage medium or a computer-readable medium having recorded thereon the computer program for performing one of the methods described herein. The data carrier, the digital storage medium or the recorded medium are typically tangible and / or non-transitory.

[0430] Therefore, a further embodiment of the inventive method is a data stream or a sequence of signals representing the computer program for performing one of the methods described herein. The data stream or the sequence of signals may, for example, be configured to be transmitted via a data communication connection, for example via the Internet.

[0431] A further embodiment comprises a processing means, for example a computer or a programmable logic device, configured to or adapted to perform one of the methods described herein.

[0432] A further embodiment comprises a computer having installed thereon the computer program for performing one of the methods described herein.

[0433] Another embodiment according to the invention comprises an apparatus or system configured to transmit a computer program to a receiver (e.g. electronically or optically), the computer program being used to perform one of the methods described herein. The receiver may be, for example, a computer, a mobile device, a storage device, etc. The apparatus or system may, for example, comprise a file server for transmitting the computer program to the receiver.

[0434] In some embodiments, a programmable logic device (e.g., a field programmable gate array) can be used to perform some or all of the functions of the methods described herein. In some embodiments, a field programmable gate array can collaborate with a microprocessor to perform one of the methods described herein. Typically, these methods are preferably performed by any hardware device.

[0435] The devices described herein may be implemented using hardware devices, or using computers, or using a combination of hardware devices and computers.

[0436] The methods described herein may be performed using a hardware device, or using a computer, or using a combination of a hardware device and a computer.

[0437] Although the present disclosure has been described with reference to illustrative embodiments, this description is not intended to be interpreted as limiting. After reference to this description, various modifications and combinations of this illustrative embodiment and other embodiments of the present disclosure will be apparent to those skilled in the art. Therefore, it is intended that the attached claims cover any such modifications or embodiments.

[0438] 1. A file parser (300) for reconstructing a video bitstream (310) from a video file (201), wherein the video file (201) comprises different tracks (220, 221, 222) in which at least two input video bitstreams (211, 212) are embedded, wherein video content (202) is differently encoded in the at least two input video bitstreams (211, 212), and wherein each input video bitstream (211, 212) comprises one or more coded video sequences (CVS) (203) having random access points (2801, 2802),

[0439] Wherein, the file parser (300) is configured as follows:

[0440] Retrieve from the video file (201):

[0441] switching information (207), the switching information (207) indicating whether the different tracks (220, 221, 222) include one or more switching candidates for the corresponding track (220), and / or which track of the different tracks (220, 221, 222) is one or more switching candidates for the corresponding track (220), and

[0442] One or more initialization parameter sets (204, 304) to be inserted into a reconstructed bitstream (310) and to be referenced by its samples (231), wherein the reconstructed bitstream (310) comprises one of the following:

[0443] an input video bitstream (211) embedded in said corresponding track (220), and

[0444] a hybrid video bitstream resulting from switching between an input video bitstream (211) embedded in the corresponding track (220) and an input video bitstream (212) embedded in one or more switching candidates (221, 222) belonging to the corresponding track (220),

[0445] And wherein the file parser (300) is further configured to:

[0446] A predetermined set of initialization parameters (304) of tracks of interest (220, 221, 222) retrieved from the video file (210) is provided to a decoder (320), and the decoder (320) is fed with:

[0447] an input video bitstream (211) embedded in said corresponding track (220), and / or

[0448] A hybrid video bitstream is generated from switching between an input video bitstream (211) embedded in the corresponding track (220) and an input video bitstream (212) embedded in one or more switching candidates (221, 222) belonging to the corresponding track (220).

[0449] 2. The file parser (300) according to embodiment 1,

[0450] The switching information (207) includes an implicit or explicit indication for indicating, for switchable tracks (220, 221, 222), the track switching levels of those tracks (220, 221, 220) that can be switched between each other.

[0451] 3. The file parser (300) according to embodiment 1,

[0452] wherein each of the at least two input video bitstreams (211, 212) is encoded using an open group of pictures (GOP) structure comprising a random access point (2801, 2802), at which random access point a switch between the at least two input video bitstreams (211, 212) can be performed, and

[0453] Therein, the encoded video sequence (203) of the video content (202) continues after the switch without starting a new encoded video sequence (203).

[0454] 4. The file parser (300) according to embodiment 1,

[0455] The predetermined initialization parameter set (304) triggers the start of the coded video sequence (203) and comprises one or more initialization parameters for the entire started coded video sequence (203) embedded in the track of interest (220, 221, 222).

[0456] 5. The file parser (300) according to embodiment 1,

[0457] The initialization parameter set (204) signals one or more video content attributes of the video content (202) contained in the encoded video sequence (203).

[0458] 6. The file parser (300) according to embodiment 5,

[0459] The video content attribute includes at least one of the following:

[0460] • Predefined codec profiles,

[0461] • the predetermined codec level,

[0462] • a predetermined picture width of pictures present in said starting coded video sequence (203),

[0463] • a predetermined picture height of pictures appearing in said starting coded video sequence (203),

[0464] • a predetermined color format of pictures present in said starting coded video sequence (203),

[0465] • a predetermined bit depth of pictures present in said starting coded video sequence (203), and

[0466] • A predetermined number of picture slots in the decoded picture buffer DBP.

[0467] 7. The file parser (300) according to embodiment 5,

[0468] The video content attribute includes at least one of the following:

[0469] • Maximum Codec Profile,

[0470] • Maximum codec level,

[0471] • the maximum picture width of pictures appearing in said starting coded video sequence (203),

[0472] • the maximum picture height of pictures appearing in said starting coded video sequence (203),

[0473] • the maximum color format of pictures present in said starting coded video sequence (203),

[0474] • the maximum bit depth of pictures present in said starting coded video sequence (203), and

[0475] • The maximum number of picture slots in the decoded picture buffer DBP.

[0476] 8. The file parser (300) according to embodiment 5,

[0477] The method is configured to select the predetermined initialization parameter set (304) based on:

[0478] • one or more of the signaled video content attributes,

[0479] • one or more maximum decoder capabilities,

[0480] • one or more desired operating points, and

[0481] • One or more desired target display capabilities.

[0482] 9. The file parser (300) according to embodiment 1,

[0483] wherein the predetermined initialization parameter set (304) triggers the start of the coded video sequence (203) and comprises one or more initialization parameters for encoding each picture (10) in the coded video sequence (203) embedded in the track of interest (220, 221, 222), and

[0484] Wherein, for each picture (10) encoded in the encoded video sequence (203), the initialization parameter set (304) signals one or more of its real picture properties.

[0485] 10. The file parser (300) according to embodiment 1,

[0486] Each of the different tracks (220, 221, 222) is associated with its own sample description box, and

[0487] Therein, for each track (220, 221, 222), its corresponding initialization parameter set (204) is transmitted out-of-band by associating a sample entry within its corresponding own sample description box with the corresponding track (220, 221, 222).

[0488] 11. The file parser (300) according to embodiment 1,

[0489] The switching information (207) includes an indication of a group ("aaaa", "bbbb") for identifying:

[0490] one or more tracks (220, 221, 222) from which switching is possible, the so-called switching-from tracks, and

[0491] one or more tracks (220, 221, 222) that can be switched to, so-called switch-to tracks or switch-to candidates, and

[0492] The switching information (207) further identifies which initialization parameter set (204) from which track (220, 221, 222) is to be used to reconstruct the bitstream (310) at the beginning of the coded video sequence (203).

[0493] 12. The file parser (300) according to embodiment 1,

[0494] configured to group the different tracks (220, 221, 222) into track groups ("aaaa") of a specific track group type (track_group_type), and

[0495] For each track (220, 221, 222) in the track group ("aaaa"), a track switch hierarchy ID (track_switch_hierarchy_id) indicating the switching hierarchy of the corresponding track (220, 221, 222) is retrieved for identifying whether switching to a potential switch candidate (220, 221, 222) is allowed while continuing to reuse a predetermined initialization parameter set (304) initially used to reconstruct the bitstream (310) at the beginning of the coded video sequence (203).

[0496] 13. The file parser (300) according to embodiment 12, wherein

[0497] if the bitstream (310) is reconstructed at the beginning of the coded video sequence (203) by using a predetermined initialization parameter set (304) of a predetermined track (222) belonging to the track group ("aaaa"), the predetermined track (222) comprising a predetermined track switch hierarchy ID (track_switch_hierarchy_ID=1) indicating a predetermined switch hierarchy level,

[0498] If the corresponding switch-to candidate (220, 221, 222) includes a track switch level ID indicating a switch level level equal to or less than the predetermined switch level level of the predetermined track (222),

[0499] Track Track Track Track then allows switching from any switch of the track group ("aaaa") from track (220, 221, 222) to any potential switching of the track group ("aaaa") to candidate (220, 221, 222).

[0500] 14. The file parser (300) according to embodiment 12, wherein

[0501] if the bitstream (310) is reconstructed at the beginning of the coded video sequence (203) by using a predetermined initialization parameter set (304) of a predetermined track (220) belonging to the track group ("aaaa"), the predetermined track (220) comprising a predetermined track switch hierarchy ID (track_switch_hierarchy_ID=0) indicating a predetermined switch hierarchy level,

[0502] If the corresponding switch candidate (222) includes a track switch level ID indicating a switch level level greater than the predetermined switch level level of the predetermined track (220), the track track track track

[0503] Any switch from the track group ("aaaa") to any potential switch from track (220, 221, 222) to the track group ("aaaa") to candidate (220, 221, 222) is not allowed.

[0504] 15. The file parser (300) according to embodiment 1,

[0505] configured to retrieve a group of entities, namely a sample description sharing box, comprising references (track_ID) to one or more tracks (220, 221, 222) that can be switched between each other while continuing to reuse a predetermined set of initialization parameters (304) initially used to reconstruct the bitstream (310) at the beginning of the coded video sequence (203),

[0506] Therein, each track (220, 221, 222) belonging to the entity group includes a track switch hierarchy ID (track_switch_hierarchy_id) indicating a switch hierarchy level of the corresponding track (220, 221, 222).

[0507] 16. The file parser (300) according to embodiment 15, wherein

[0508] if the bitstream (310) is reconstructed at the beginning of the coded video sequence (203) by using a predetermined initialization parameter set (304) of a predetermined track (222) belonging to the entity group, the predetermined track (222) comprising a predetermined track switch hierarchy ID (track_switch_hierarchy_ID=1) indicating a predetermined switch hierarchy level,

[0509] If the corresponding switch-to candidate (220, 221, 222) includes a track switch level ID indicating a switch level level equal to or less than the predetermined switch level level of the predetermined track (222) (for initialization),

[0510] Track Track Track then allows switching from any switch from the entity group to any potential switch candidate (220, 221, 222) of the entity group.

[0511] 17. The file parser (300) according to embodiment 15,

[0512] if the bitstream (310) is reconstructed at the beginning of the coded video sequence (203) by using a predetermined initialization parameter set (304) of a predetermined track (220) belonging to the entity group, the predetermined track (220) comprising a predetermined track switch hierarchy ID (track_switch_hierarchy_ID=0) indicating a predetermined switch hierarchy level,

[0513] If the corresponding switch-to candidate (222) includes a track switch hierarchy ID indicating a switch hierarchy level (track_switch_hierarchy_id=1) greater than the predetermined switch hierarchy level of the predetermined switch-from track (220, 221),

[0514] Track Track Track then does not allow switching from any switch from track (220, 221) of the entity group to any potential switch candidate (222) belonging to the entity group.

[0515] 18. The file parser (300) according to embodiment 1, wherein

[0516] The file parser (300) is configured to implicitly derive a track switch hierarchy level (track_switch_hierarchy_id) from parameters included in the switch-from track (222) and / or the switch-to candidate (220, 221), respectively.

[0517] 19. The file parser (300) according to embodiment 18,

[0518] The parameters include at least one of the following:

[0519] • Predefined codec profiles,

[0520] • the predetermined codec level,

[0521] • a predetermined picture width of pictures appearing in said starting coded video sequence,

[0522] • a predetermined picture height of pictures appearing in said starting coded video sequence,

[0523] • a predetermined color format of pictures appearing in said starting coded video sequence,

[0524] • the predetermined bit depth of the pictures appearing in said starting coded video sequence, and

[0525] • A predetermined number of picture slots in the decoded picture buffer DBP.

[0526] 20. The file parser (300) according to embodiment 18,

[0527] is configured to implicitly derive the implicit track switch hierarchy level (track_switch_hierarchy_id) as follows:

[0528] If the first track (222) includes a parameter (eg, picture width) having a higher value than the corresponding parameter (eg, picture width) included in the second track (220),

[0529] Then the first track (222) has a higher switch hierarchy level (track_switch_hierarchy_id) than the second track (220).

[0530] 21. The file parser (300) according to embodiment 12,

[0531] The track switch hierarchy ID (track_switch_hierarchy_id) is in the form of a Boolean value indicating a two-level track switch hierarchy, where:

[0532] The first track set of the entity group includes a first track switch hierarchy ID (track_switch_hierarchy_id) indicating a first level of the two-level track switch hierarchy, and

[0533] The second track set of the entity group includes a second track switch hierarchy ID (track_switch_hierarchy_id) indicating a different second level of the track switch hierarchy, where:

[0534] If the bitstream (310) is reconstructed at the beginning of the coded video sequence (203) by using a predetermined set of initialization parameters (304) for tracks from the first set of tracks,

[0535] then switching to any other track of the entity group is allowed, and

[0536] If the bitstream (310) is reconstructed at the beginning of the coded video sequence (203) by using a predetermined set of initialization parameters (304) for tracks from the second set of tracks,

[0537] Switching to any other track of the entity group is not allowed.

[0538] 22. The file parser (300) according to embodiment 12, wherein

[0539] The references of the tracks (220, 221, 222) belonging to the entity group are sequentially ordered in a predetermined order,

[0540] Therein, the switch hierarchy level (track_switch_hierarchy_id) of each track (220, 221, 222) depends on the position of its reference within the predetermined order.

[0541] 23. The file parser (300) according to embodiment 22,

[0542] wherein the reference (track_id) of the track (220) that appears first in said predetermined order is associated with a predetermined track switch hierarchy ID (track_switch_hierarchy_ID=0) indicating a lowest switch hierarchy level, and

[0543] Therein, a reference (track_id) of the track (222) that appears last in said predetermined order is associated with a different predetermined track switch hierarchy ID (track_switch_hierarchy_id=1) indicating a highest switch hierarchy level.

[0544] 24. The file parser (300) according to embodiment 22,

[0545] wherein a reference (track_id) of the track that appears first in said predetermined order is associated with a predetermined track switch hierarchy ID (track_switch_hierarchy_id) indicating a highest switch hierarchy level, and

[0546] Therein, a reference (track_id) of a track that occurs last in said predetermined order is associated with a different predetermined track switch hierarchy ID (track_switch_hierarchy_id) indicating a lowest switch hierarchy level.

[0547] 25. The file parser (300) according to embodiment 1,

[0548] is configured as:

[0549] grouping the different tracks (220, 221, 222) into track groups ("bbbb") of a specific track group type (track_group_type), and

[0550] For each track (220, 221, 222) of the track group ("bbbb") retrieve:

[0551] an indication (current_sample_entry_flag) for indicating whether a track switch from the corresponding track (220) to any other potential switch candidate (221, 222) within the track group ("bbbb") is allowed while continuing to reuse the initialization parameter set (304) initially used to reconstruct the bitstream (310) at the beginning of the coded video sequence (203), and / or

[0552] An initialization parameter set list (sample_entry_track_id[]) explicitly pointing to those one or more tracks (221, 222) associated with the initialization parameter set (304), allowing a track switch from the corresponding switch-from track (220) to any other potential switch-to candidate (221, 222) within the track group ("bbbb"), upon initial reconstruction of the bitstream (310) at the beginning of the coded video sequence (203) by using one of those one or more initialization parameter sets (304) for switching to the track (221, 222) contained in the initialization parameter set list (sample_entry_track_id[]).

[0553] 26. The file parser (300) according to embodiment 25,

[0554] wherein if the bitstream (310) is initially reconstructed at the beginning of the coded video sequence (203) by using an initialization parameter set (304) belonging to the corresponding switched-from track (222),

[0555] The indication (current_sample_entry_flag) of the corresponding switch-from track (222) indicates whether switching from the corresponding switch-from track (222) to any other potential switch-to candidate (220, 221) within the track group ("bbbb") is allowed.

[0556] 27. A file generator (200) for generating a video file (201) from at least two input video bitstreams (211, 212), wherein video content (202) is encoded differently in the at least two input video bitstreams (211, 212), and wherein each input video bitstream (211, 212) comprises one or more encoded video sequences (203) having random access points (2801, 2802),

[0557] Wherein, the file generator (200) is configured as follows:

[0558] The at least two input video bitstreams (211, 212) are embedded in different tracks (220, 221, 222) of the video file (201), the tracks (220, 221, 222) being switchable between each other at predetermined switching points, and

[0559] For the tracks (220, 221, 222), the video file (201) provides:

[0560] switching information (207), the switching information (207) indicating whether the different tracks (220, 221, 222) include one or more switching candidates for the corresponding track (220), and / or which track of the different tracks (220, 221, 222) is one or more switching candidates for the corresponding track (220), and

[0561] One or more initialization parameter sets (204, 304) to be inserted into a reconstructed bitstream (310) and to be referenced by samples thereof, wherein the reconstructed bitstream (310) comprises one of:

[0562] an input video bitstream (211) embedded in said corresponding track (220), and

[0563] A hybrid video bitstream is generated from switching between an input video bitstream (211) embedded in the corresponding track (220) and an input video bitstream (212) embedded in one or more switching candidates (221, 222) belonging to the corresponding track (220).

[0564] 28. The file generator (200) according to embodiment 27,

[0565] The switching information (207) includes an implicit or explicit indication for indicating, for switchable tracks (220, 221, 222), the track switching levels of those tracks (220, 221, 220) that can be switched between each other.

[0566] 29. The file generator (200) according to embodiment 27,

[0567] wherein each of the at least two input video bitstreams (211, 212) is encoded using an open group of pictures (GOP) structure comprising a random access point (2801, 2802), at which random access point a switch between the at least two input video bitstreams (211, 212) can be performed, and

[0568] Therein, the encoded video sequence (203) of the video content (202) continues after the switch without starting a new encoded video sequence (203).

[0569] 30. The file generator (200) according to embodiment 27,

[0570] The predetermined initialization parameter set (304) triggers the start of the coded video sequence (203) and comprises one or more initialization parameters for the entire started coded video sequence (203) embedded in the track of interest (220, 221, 222).

[0571] 31. The file generator (200) according to embodiment 27,

[0572] The initialization parameter set (204) signals one or more video content attributes of the video content (202) contained in the encoded video sequence (203).

[0573] 32. The file generator (200) according to embodiment 31,

[0574] The video content attribute includes at least one of the following:

[0575] • Predefined codec profiles,

[0576] • the intended codec level,

[0577] • a predetermined picture width of pictures present in said starting coded video sequence (203),

[0578] • a predetermined picture height of pictures appearing in said starting coded video sequence (203),

[0579] • a predetermined color format of pictures present in said starting coded video sequence (203),

[0580] • a predetermined bit depth of pictures present in said starting coded video sequence (203), and

[0581] • A predetermined number of picture slots in the decoded picture buffer DBP.

[0582] 33. The file generator (200) according to embodiment 31,

[0583] The video content attribute includes at least one of the following:

[0584] • Maximum Codec Profile,

[0585] • Maximum codec level,

[0586] • the maximum picture width of pictures appearing in said starting coded video sequence (203),

[0587] • the maximum picture height of pictures appearing in said starting coded video sequence (203),

[0588] • the maximum color format of pictures present in said starting coded video sequence (203),

[0589] • the maximum bit depth of pictures present in said starting coded video sequence (203), and

[0590] • The maximum number of picture slots in the decoded picture buffer DBP.

[0591] 34. The file generator (200) according to embodiment 31,

[0592] The method is configured to select the predetermined initialization parameter set (304) based on:

[0593] • one or more of the signaled video content attributes,

[0594] • one or more maximum decoder capabilities,

[0595] • one or more desired operating points, and

[0596] • One or more desired target display capabilities.

[0597] 35. The file generator (200) according to embodiment 27,

[0598] wherein the predetermined initialization parameter set (304) triggers the start of the coded video sequence (203) and comprises one or more initialization parameters for encoding each picture (10) in the coded video sequence (10) embedded in the track of interest (220, 221, 222), and

[0599] Wherein, for each picture (10) encoded in the encoded video sequence (203), the initialization parameter set (304) signals one or more of its real picture properties.

[0600] 36. The file generator (200) according to embodiment 27,

[0601] Each of the different tracks (220, 221, 222) is associated with its own sample description box, and

[0602] Therein, for each track (220, 221, 222), its corresponding initialization parameter set (204) is transmitted out-of-band by associating a sample entry within its corresponding own sample description box with the corresponding track (220, 221, 222).

[0603] 37. The file generator (200) according to embodiment 27,

[0604] The switching information (207) includes an indication of a group ("aaaa", "bbbb") for identifying:

[0605] one or more tracks (220, 221, 222) from which switching is possible, the so-called switching-from tracks, and

[0606] one or more tracks (220, 221, 222) that can be switched to, so-called switch-to tracks or switch-to candidates, and

[0607] The switching information (207) further identifies which initialization parameter set (304) from which track (220, 221, 222) is to be used to reconstruct the bitstream (310) at the beginning of the coded video sequence (203).

[0608] 38. The file generator (200) according to embodiment 27,

[0609] configured to group the different tracks (220, 221, 222) into track groups ("aaaa") of a specific track group type (track_group_type), and

[0610] A track switch hierarchy ID (track_switch_hierarchy_id) indicating a switch hierarchy of the corresponding track (220, 221, 222) is retrieved for each track (220, 221, 222) in the track group ("aaaa") for identifying whether a switch to a potential switch candidate (220, 221, 222) is allowed while continuing to reuse a predetermined initialization parameter set (304) initially used to reconstruct the bitstream (310) at the beginning of the coded video sequence (203).

[0611] 39. The file generator (200) according to embodiment 38, wherein

[0612] if the bitstream (310) is reconstructed at the beginning of the coded video sequence (203) by using a predetermined initialization parameter set (304) of a predetermined track (222) belonging to the track group ("aaaa"), the predetermined track (222) comprising a predetermined track switch hierarchy ID (track_switch_hierarchy_ID=1) indicating a predetermined switch hierarchy level,

[0613] If the corresponding switch-to candidate (220, 221, 222) includes a track switch level ID indicating a switch level level equal to or less than the predetermined switch level level of the predetermined track (222),

[0614] Track Track Track Track then allows switching from any switch of the track group ("aaaa") from track (220, 221, 222) to any potential switching of the track group ("aaaa") to candidate (220, 221, 222).

[0615] 40. The file generator (200) according to embodiment 38, wherein

[0616] if the bitstream (310) is reconstructed at the beginning of the coded video sequence (203) by using a predetermined initialization parameter set (304) of a predetermined track (220) belonging to the track group ("aaaa"), the predetermined track (220) comprising a predetermined track switch hierarchy ID (track_switch_hierarchy_ID=1) indicating a predetermined switch hierarchy level,

[0617] If the corresponding switch-to candidate (222) includes a track switch hierarchy ID (track_switch_hierarchy_ID=1) indicating a switch hierarchy level greater than the predetermined switch hierarchy level (track_switch_hierarchy_ID=0) of the predetermined track (220),

[0618] Track Track Track Track then does not allow any switch from the track group ("aaaa") to any potential switch from track (220, 221, 222) to candidate (220, 221, 222) of the track group ("aaaa").

[0619] 41. The file generator (200) according to embodiment 27,

[0620] is configured to provide an entity group, namely a sample description sharing box, comprising references (track_ID) of one or more tracks (220, 221, 222) that can be switched between each other,

[0621] Therein, each track ( 220 , 221 , 222 ) belonging to the entity group includes a track switch hierarchy ID (track_switch_hierarchy_id) indicating a switch hierarchy level of the corresponding track.

[0622] 42. The file generator (200) according to embodiment 41, wherein the track track

[0623] if the bitstream is reconstructed at the beginning of the coded video sequence (203) by using a predetermined initialization parameter set (304) of a predetermined track (222) belonging to the entity group, the predetermined track (222) comprising a predetermined track switch hierarchy ID (track_switch_hierarchy_ID=1) indicating a predetermined switch hierarchy level,

[0624] If the corresponding switch-to candidate (220, 221, 222) includes a track switch level ID indicating a switch level level equal to or less than the predetermined switch level level of the predetermined track (222),

[0625] Then a switch from any switch of the entity group from track (220, 221, 222) to any potential switch of the entity group to candidate (220, 221, 222) is allowed.

[0626] 43. The file generator (200) according to embodiment 41,

[0627] if the bitstream (310) is reconstructed at the beginning of the coded video sequence (203) by using a predetermined initialization parameter set (304) of a predetermined track (220) belonging to the entity group, the predetermined track (220) comprising a predetermined track switch hierarchy ID (track_switch_hierarchy_ID=0) indicating a predetermined switch hierarchy level,

[0628] If the corresponding switch-to candidate (222) includes a track switch hierarchy ID (track_switch_hierarchy_ID=1) indicating a switch hierarchy level greater than the predetermined switch hierarchy level of the predetermined switch-from track (220, 221),

[0629] Track Track Track then does not allow switching from any switch from track (220, 221) of the entity group to any potential switch candidate (222) belonging to the entity group.

[0630] 44. The file generator (200) according to embodiment 27, wherein:

[0631] The file generator (200) is configured to implicitly derive a track switch hierarchy level (track_switch_hierarchy_id) from parameters included in the switch-from track (222) and / or the switch-to candidate (220, 221), respectively.

[0632] 45. The file generator (200) according to embodiment 44,

[0633] The parameters include at least one of the following:

[0634] • Predefined codec profiles,

[0635] • the predetermined codec level,

[0636] • a predetermined picture width of pictures appearing in said starting coded video sequence,

[0637] • a predetermined picture height of pictures appearing in said starting coded video sequence,

[0638] • a predetermined color format of pictures appearing in said starting coded video sequence,

[0639] • the predetermined bit depth of the pictures appearing in said starting coded video sequence, and

[0640] • A predetermined number of picture slots in the decoded picture buffer DBP.

[0641] 46. ​​The file generator (200) according to embodiment 44,

[0642] is configured to implicitly derive the implicit track switch hierarchy level (track_switch_hierarchy_id) as follows:

[0643] If the first track (222) includes a parameter (eg, picture width) having a higher value than the corresponding parameter (eg, picture width) included in the second track (220),

[0644] Then the first track (222) has a higher switch hierarchy level (track_switch_hierarchy_id) than the second track (220).

[0645] 47. The file generator (200) according to embodiment 38,

[0646] The track switch hierarchy ID (track_switch_hierarchy_id) is in the form of a Boolean value indicating a two-level track switch hierarchy, where:

[0647] The first track set of the entity group includes a first track switch hierarchy ID (track_switch_hierarchy_id) indicating a first level of the two-level track switch hierarchy, and

[0648] The second track set of the entity group includes a second track switch hierarchy ID (track_switch_hierarchy_id) indicating a different second level of the track switch hierarchy, where:

[0649] If the bitstream (310) is reconstructed at the beginning of the coded video sequence (203) by using a predetermined set of initialization parameters (304) for tracks from the first set of tracks,

[0650] then switching to any other track of the entity group is allowed, and

[0651] If the bitstream (310) is reconstructed at the beginning of the coded video sequence (203) by using a predetermined set of initialization parameters (304) for tracks from the second set of tracks,

[0652] Switching to any other track of the entity group is not allowed.

[0653] 48. The file generator (200) according to embodiment 41, wherein

[0654] The references of the tracks (220, 221, 222) belonging to the entity group are sequentially ordered in a predetermined order,

[0655] Therein, the switch hierarchy level (track_switch_hierarchy_id) of each track (220, 221, 222) depends on the position of its reference within the predetermined order.

[0656] 49. The file generator (200) according to embodiment 48,

[0657] wherein the reference (track_id) of the track (220) that appears first in said predetermined order is associated with a predetermined track switch hierarchy ID (track_switch_hierarchy_ID=0) indicating a lowest switch hierarchy level, and

[0658] Therein, a reference (track_id) of the track (222) that appears last in said predetermined order is associated with a different predetermined track switch hierarchy ID (track_switch_hierarchy_id=1) indicating a highest switch hierarchy level.

[0659] 50. The file generator (200) according to embodiment 48,

[0660] wherein a reference (track_id) of the track that appears first in said predetermined order is associated with a predetermined track switch hierarchy ID (track_switch_hierarchy_id) indicating a highest switch hierarchy level, and

[0661] Therein, a reference (track_id) of a track that occurs last in said predetermined order is associated with a different predetermined track switch hierarchy ID (track_switch_hierarchy_id) indicating a lowest switch hierarchy level.

[0662] 51. The file generator (200) according to embodiment 27,

[0663] is configured as:

[0664] grouping the different tracks (220, 221, 222) into track groups ("bbbb") of a specific track group type (track_group_type), and

[0665] For each track (220, 221, 222) of the track group ("bbbb"):

[0666] an indication (current_sample_entry_flag) for indicating whether a track switch from the corresponding track (220) to any other potential switch candidate (221, 222) within the track group ("bbbb") is allowed while continuing to reuse the initialization parameter set (304) initially used to reconstruct the bitstream (310) at the beginning of the coded video sequence (203), and / or

[0667] An initialization parameter set list (sample_entry_track_id[]) explicitly pointing to those one or more tracks (221, 222) associated with the initialization parameter set (304), allowing a track switch from the corresponding switch-from track (220) to any other potential switch-to candidate (221, 222) within the track group ("bbbb") when the bitstream (310) is initially reconstructed at the beginning of the coded video sequence (203) by using one of those one or more initialization parameter sets (304) for switching to the track (221, 222) contained in the initialization parameter set list (sample_entry_track_id[]).

[0668] 52. The file generator (200) according to embodiment 51,

[0669] wherein if the bitstream (310) is initially reconstructed at the beginning of the coded video sequence (203) by using an initialization parameter set (304) belonging to the corresponding switched-from track (222),

[0670] The indication (current_sample_entry_flag) of the corresponding switch-from track (222) indicates whether switching from the corresponding switch-from track (222) to any other potential switch-to candidate (220, 221) within the track group ("bbbb") is allowed.

[0671] 53. A client network device for retrieving media content from a server using a streaming protocol,

[0672] The client network device includes the file parser (300) according to embodiment 1.

[0673] 54. A server network device for providing media content to a client using a streaming protocol,

[0674] Wherein, the server network device includes the file generator according to Example 27.

[0675] 55. A system comprising:

[0676] A client network device according to embodiment 53, and

[0677] A server network device according to embodiment 54.

[0678] 56. A method for reconstructing a video bitstream (310) from a video file (201), wherein the video file (201) comprises different tracks (220, 221, 222) in which at least two input video bitstreams (211, 212) are embedded, wherein video content (202) is differently encoded in the at least two input video bitstreams (211, 212), and wherein each input video bitstream (211, 212) comprises one or more encoded video sequences (203) having random access points (2801, 2802),

[0679] Wherein, the method comprises the following steps:

[0680] Retrieve from the video file (201):

[0681] switching information (207), the switching information (207) indicating whether the different tracks (220, 221, 222) include one or more switching candidates for the corresponding track (220), and / or which track of the different tracks (220, 221, 222) is one or more switching candidates for the corresponding track (220), and

[0682] One or more initialization parameter sets (204, 304) to be inserted into a reconstructed bitstream (310) and to be referenced by samples thereof, wherein the reconstructed bitstream (310) comprises one of:

[0683] an input video bitstream (211) embedded in said corresponding track (220), and

[0684] a hybrid video bitstream resulting from switching between an input video bitstream (211) embedded in the corresponding track (220) and an input video bitstream (212) embedded in one or more switching candidates (221, 222) belonging to the corresponding track (220),

[0685] And wherein the method further comprises the following steps:

[0686] A predetermined set of initialization parameters (304) of tracks of interest (220, 221, 222) retrieved from the video file (201) is provided to a decoder (320), and the decoder (320) is fed with:

[0687] an input video bitstream (211) embedded in said corresponding track (220), and / or

[0688] A hybrid video bitstream is generated from switching between an input video bitstream (211) embedded in the corresponding track (220) and an input video bitstream (212) embedded in one or more switching candidates (221, 222) belonging to the corresponding track (220).

[0689] 57. A method for generating a video file (201) from at least two input video bitstreams (211, 212), wherein video content (202) is encoded differently in the at least two input video bitstreams (211, 212), and wherein each input video bitstream (211, 212) comprises one or more encoded video sequences (203) having random access points (2801, 2802),

[0690] Wherein, the method comprises the following steps:

[0691] embedding the at least two input video bitstreams (211, 212) into different tracks (220, 221, 222) of the video file (201), the tracks (220, 221, 222) being capable of switching between each other at random access points (2801, 2802) of the embedded input video bitstreams (211, 212), and

[0692] For the tracks (220, 221, 222), the video file (201) provides:

[0693] switching information (207), the switching information (207) indicating whether the different tracks (220, 221, 222) include one or more switching candidates for the corresponding track (220), and / or which track of the different tracks (220, 221, 222) is one or more switching candidates for the corresponding track (220), and

[0694] One or more initialization parameter sets (204, 304) to be inserted into a reconstructed bitstream (310) and to be referenced by samples thereof, wherein the reconstructed bitstream (310) comprises one of:

[0695] an input video bitstream (211) embedded in said corresponding track (220), and

[0696] A hybrid video bitstream is generated from switching between an input video bitstream (211) embedded in the corresponding track (220) and an input video bitstream (212) embedded in one or more switching candidates (221, 222) belonging to the corresponding track (220).

[0697] 58. A computer program for implementing the method according to embodiment 56 or 57 when executed on a computer or a signal processor.

[0698] 59. A client network device (50) for retrieving media content from a server (20) using a streaming protocol,

[0699] The media content is contained in one or more media streams, each of the one or more media streams contains one or more representations (401, 402), wherein each representation (401, 402) includes one or more tracks (220, 221, 222), and the one or more tracks (220, 221, 222) contain samples that reference a picture (10) of the video content (201), wherein the picture (10) is encoded in one or more coded video sequences (203) of the one or more bitstreams (211, 212),

[0700] wherein each of the representations (401, 402) is associated with an initialization fragment (205), the initialization fragment (205) comprising one or more initialization parameter sets (204) for those tracks (220, 221, 222) contained in the respective representation (401, 402),

[0701] wherein each initialization parameter set (204, 304) contains decoding parameter information to be inserted into a reconstructed bitstream (310) at the beginning of a coded video sequence (203) and referenced by its samples, and

[0702] The client network device (50) is configured to derive initialization fragment information, wherein the initialization fragment information indicates that a predetermined initialization fragment (205) of a predetermined representation (401) can be used for another representation (402).

[0703] 60. The client network device (50) according to embodiment 59,

[0704] is configured to select the predetermined initialization fragment (205) and to associate the predetermined initialization fragment (205) of the predetermined representation (401) with the other representation (402), wherein:

[0705] The decoding parameter information contained in the predetermined initialization segment (205) of the predetermined representation (401) will be inserted at the beginning of the encoded video sequence (203) into a reconstructed bitstream (310) having:

[0706] samples of tracks (220, 221, 222) contained in the predetermined representation (401), and

[0707] the samples of the tracks (220, 221, 222) contained in the other representation (402),

[0708] And wherein the decoding parameter information is to be referenced by samples of the predetermined representation (401) and samples of the other representation (402).

[0709] 61. The client network device (50) according to embodiment 59,

[0710] is configured to select a predetermined set of initialization fragments (205) based on at least one of the following:

[0711] • Maximum decoder capability,

[0712] • the desired maximum operating point,

[0713] • Desired target display capability,

[0714] • Maximum Codec Profile,

[0715] • Maximum codec level,

[0716] • the maximum picture width of pictures appearing in said coded video sequence (203),

[0717] • the maximum picture height of pictures appearing in said coded video sequence (203),

[0718] • the maximum color format of pictures appearing in said coded video sequence (203),

[0719] • the maximum bit depth of pictures present in said starting coded video sequence (203), and

[0720] • The maximum number of picture slots in the decoded picture buffer DBP.

[0721] 62. The client network device (50) according to embodiment 59,

[0722] The method is configured to derive the initialization segment information from additional parameters.

[0723] 63. The client network device (50) according to embodiment 59,

[0724] The method comprises: deriving the initialization segment information from a manifest file of a manifest, wherein the manifest description points to a corresponding initialization segment (205), so that the initialization segment information is available to the client network device (50) before the client network device (50) downloads the corresponding one or more media streams.

[0725] 64. The client network device (50) according to embodiment 59,

[0726] is configured to determine:

[0727] whether the representation (401, 402) belongs to a predetermined set of adaptation sets, and

[0728] whether to allow crossover adaptation set switching between adaptation sets within the predetermined set of adaptation sets,

[0729] Thereby, the initialization fragment information is implicitly derived.

[0730] 65. The client network device (50) according to embodiment 59,

[0731] If the representation (401, 402) is indicated as switchable, the initialization fragment information includes switching information for identifying the following items:

[0732] one or more representations (401) from which it is possible to switch, the so-called switched-from representations (401), and

[0733] one or more representations (402) that can be switched to, so-called switch-to representations or switch-to candidates, and

[0734] The switching information also identifies which initialization fragment (205) from which representation (401, 402) will be used when switching from the self-representation (401) to the switch-to representation (402) (i.e., whether the predetermined initialization fragment can still be used when switching or whether a new initialization fragment is required).

[0735] 66. The client network device (50) according to embodiment 65,

[0736] is configured to associate each representation (401, 402) with a switching hierarchy level to identify:

[0737] If the predetermined initialization segment (205) of the predetermined representation (401) is initially used, whether to allow switching from the switching from the representation (401) to the potential switching to the representation (402).

[0738] 67. The client network device (50) according to embodiment 66,

[0739] Wherein, if the switching hierarchy level of the switching to the representation (402) is equal to or lower than the switching hierarchy level of the predetermined representation (401),

[0740] Then switching from any switch-from representation (401) to any potential switch-to representation (402) is allowed.

[0741] 68. The client network device (50) according to embodiment 66,

[0742] Wherein, if the switching level level of the switching to the representation (402) is greater than the switching level level of the predetermined representation (401),

[0743] Switching from any switch-from representation (401) to any potential switch-to representation (402) is not allowed.

[0744] 69. The client network device (50) according to embodiment 65, wherein

[0745] The switching hierarchy level follows the level indicated for the specific video content (202),

[0746] Therein, a higher level of the video content (202) indicated as a mimeType codec parameter indicates a higher switching hierarchy level.

[0747] 70. The client network device (50) according to embodiment 65, wherein

[0748] The switching hierarchy level follows the codec profile value,

[0749] wherein a profile that is a subset of another profile has a lower hierarchy level, and

[0750] Among them, a profile that is a superset of another profile has a higher hierarchy level.

[0751] 71. The client network device (50) according to embodiment 59, wherein

[0752] The client network device (50) includes at least one file parser (300) as claimed in embodiment 1, 78 or 123.

[0753] 72. A server network device (20) for providing media content to a client (50) using a streaming protocol,

[0754] wherein the media content is transmitted in one or more media streams, each of the one or more media streams comprising one or more representations (401, 402), wherein each representation (401, 402) comprises one or more tracks, wherein the one or more tracks comprise samples referencing a picture (10) of the video content (202), wherein the picture (10) is encoded in one or more coded video sequences (203) of the one or more bitstreams (211, 212),

[0755] wherein each of the representations (401, 402) is associated with an initialization fragment (205), the initialization fragment (205) comprising one or more initialization parameter sets (204) for those tracks contained in the respective representation (401, 402),

[0756] wherein each initialization parameter set (204) contains decoding parameter information to be inserted into the reconstructed bitstream (310) at the beginning of the coded video sequence (203) and referenced by its samples, and

[0757] The server network device (20) is configured to transmit initialization fragment information, wherein the initialization fragment information indicates that a predetermined initialization fragment (205) of a predetermined representation (401) can be used for another representation (402).

[0758] 73. The server network device (20) according to embodiment 72, wherein

[0759] The server network device (20) includes at least one file generator (200) as claimed in embodiments 27, 89 or 145.

[0760] 74. A system comprising:

[0761] The client network device (50) according to embodiment 59, and

[0762] A server network device (20) according to embodiment 72.

[0763] 75. A method for retrieving media content from a server (20) using a streaming protocol,

[0764] wherein the media content is contained in one or more media streams, each of the one or more media streams comprising one or more representations (401, 402), wherein each representation (401, 402) comprises one or more tracks, wherein the one or more tracks comprise samples of pictures (10) of reference video content (202), wherein the pictures (10) are encoded in one or more coded video sequences (203) of the one or more bitstreams (211, 212),

[0765] wherein each of the representations (401, 402) is associated with an initialization fragment (205), the initialization fragment (205) comprising one or more initialization parameter sets (204) for those tracks contained in the respective representation (401, 402),

[0766] wherein each initialization parameter set (204) contains decoding parameter information to be inserted into the reconstructed bitstream (310) at the beginning of the coded video sequence (203) and referenced by its samples, and

[0767] The method comprises the step of deriving initialization fragment information, wherein the initialization fragment information indicates that a predetermined initialization fragment (205) of a predetermined representation (401) can be used for another representation (402).

[0768] 76. A method for providing media content to a client (50) using a streaming protocol,

[0769] wherein the media content is transmitted in one or more media streams, each of the one or more media streams comprising one or more representations (401, 402), wherein each representation (401, 402) comprises one or more tracks, wherein the one or more tracks comprise samples referencing a picture (10) of the video content (202), wherein the picture (10) is encoded in one or more coded video sequences (203) of the one or more bitstreams (211, 212),

[0770] wherein each of the representations (401, 402) is associated with an initialization fragment (205), the initialization fragment (205) comprising one or more initialization parameter sets (204) for those tracks contained in the respective representation (401, 402),

[0771] wherein each initialization parameter set (204) contains decoding parameter information to be inserted into the reconstructed bitstream (310) at the beginning of the coded video sequence (203) and referenced by its samples, and

[0772] The method further comprises the step of transmitting initialization fragment information, wherein the initialization fragment information indicates that a predetermined initialization fragment (205) of a predetermined representation (401) can be used for another representation (402).

[0773] 77. A computer program for implementing the method according to embodiment 75 or 76 when executed on a computer or a signal processor.

[0774] 78. A file parser (300) for reconstructing a video bitstream (310) from a video file (201), wherein the video file (201) comprises one or more tracks (220, 221, 222), each track having one or more input video bitstreams (211, 212) embedded therein, wherein each of the one or more input video bitstreams (211, 212) comprises one or more coded video sequences (203),

[0775] Wherein, the file parser (300) is configured as follows:

[0776] Retrieve from the video file (201):

[0777] one or more sample groups (ParameterSetSamplegroup) to which one or more samples of the one or more tracks (220, 221, 222) belong, and

[0778] an indication of which samples belong to which sample group (ParameterSetSamplegroup),

[0779] Each sample group (ParameterSetSamplegroup) is associated with one or more initialization parameter sets (204a, ..., 204h) to be inserted into the reconstructed bitstream (310) and referenced by its samples, wherein if a sample group (ParameterSetSamplegroup) is associated with more than one initialization parameter set (204a, ..., 204h), then:

[0780] A) each of the more than one initialization parameter sets (204a, ..., 204h) is associated with the same operating point, or

[0781] B) each of the more than one initialization parameter sets (204a, ..., 204h) is associated with at least one operating point of the plurality of operating points,

[0782] Wherein, the file parser (301) is further configured to:

[0783] A predetermined initialization parameter set (204h) of a predetermined sample group (ParameterSetSamplegroup) is included in the reconstructed bitstream (310).

[0784] 79. The file parser (300) according to embodiment 78,

[0785] The method is configured to select a predetermined set of initialization parameters (204h) of the predetermined group of samples based on a desired operating point and / or a desired decoding behavior of a decoder for decoding the one or more encoded video sequences (203).

[0786] 80. The file parser (300) according to embodiment 79,

[0787] wherein the expected operating point and / or decoding behavior of the decoder depends on at least one of the following properties, and wherein at least one of the properties is contained in a video file (201) and associated with a corresponding sample group:

[0788] • Predefined codec profiles,

[0789] • the intended codec level,

[0790] • a predetermined picture width of a picture contained in the one or more coded video sequences (203) to be decoded,

[0791] • a predetermined picture height of a picture contained in the one or more coded video sequences (203) to be decoded,

[0792] • a predetermined picture bit depth of pictures contained in the one or more coded video sequences (203) to be decoded,

[0793] • a predetermined picture color format of pictures contained in the one or more coded video sequences (203) to be decoded, and

[0794] • a predetermined size of a decoded picture buffer DPB for pictures contained in the one or more coded video sequences (203) to be decoded.

[0795] 81. The file parser (300) according to embodiment 78,

[0796] The method is configured to select a predetermined set of initialization parameters (204h) of the predetermined group of samples based on a desired operating point and / or a maximum decoding capability of a decoder for decoding the one or more encoded video sequences (203).

[0797] 82. The file parser (300) according to embodiment 81,

[0798] wherein the desired operating point and / or the maximum decoding behavior of the decoder depends on at least one of the following properties, and wherein at least one of the properties is contained in a video file (201) and is associated with a corresponding sample group:

[0799] • Maximum Codec Profile,

[0800] • Maximum codec level,

[0801] • a maximum picture width of pictures contained in the one or more coded video sequences (203) to be decoded,

[0802] • a maximum picture height of pictures contained in the one or more coded video sequences (203) to be decoded,

[0803] • a maximum picture bit depth of pictures contained in the one or more coded video sequences (203) to be decoded,

[0804] • the maximum picture color format of the pictures contained in the one or more coded video sequences (203) to be decoded, and

[0805] • a maximum size of a decoded picture buffer DPB for pictures contained in the one or more coded video sequences (203) to be decoded.

[0806] 83. The file parser (300) according to embodiment 78, wherein

[0807] If one or more samples within a track reference a default in-band initialization parameter set (204g) associated with the track by default, and

[0808] If the one or more samples are marked as belonging to at least one sample group of the one or more sample groups,

[0809] Then the file parser (300) is configured as follows:

[0810] deriving predetermined initialization parameters (204h) of one or more initialization parameter sets (204a, ..., 204h) associated with the respective sample group,

[0811] and replacing an otherwise in-band sent initialization parameter set (204g) with said initialization parameter set (204h) of said corresponding sample group, and

[0812] The initialization parameter set (204h) for the sample group is provided to the decoder as a predetermined initialization parameter set within the reconstructed bitstream (310), the initialization parameter set (204h) replacing the initialization parameter set (204g) otherwise sent in-band.

[0813] 84. The file parser (300) according to embodiment 78, wherein

[0814] If the track is not associated by default with any default in-band initialization parameter set (204a, ..., 204g), and

[0815] If one or more samples contained in the track are marked as belonging to at least one sample group of the one or more sample groups,

[0816] Then the file parser (300) is configured as follows:

[0817] inserting a predetermined initialization parameter set (204h) of the one or more initialization parameter sets associated with the at least one sample group into the reconstructed bitstream (310), and

[0818] The initialization parameter set (204h) of the at least one sample group is provided to the decoder as a predetermined initialization parameter set within the reconstructed bitstream (310).

[0819] 85. The file parser (300) according to embodiment 84,

[0820] is configured to derive a signal for a track, the signal indicating:

[0821] The track is not associated by default with any default in-band initialization parameter set (204a, ..., 204g), and / or

[0822] An initialization parameter set (204h) for the track must be obtained from at least one sample group of the available one or more sample groups, and the obtained initialization parameter set (204h) must be provided to the decoder as the predetermined initialization parameter set.

[0823] 86. The file parser (300) according to embodiment 84,

[0824] is configured to derive a signal for a track, the signal indicating:

[0825] Those samples of the track that are marked as belonging to one or more sample groups do not by default refer to any default in-band initialization parameter set (204a, ..., 204g), and / or

[0826] For those samples, an initialization parameter set (204h) from at least one of the available one or more sample groups to which those samples belong has to be inserted into the reconstructed bitstream (310) and provided to the decoder as the predetermined initialization parameter set.

[0827] 87. The file parser (300) according to embodiment 78,

[0828] Among them, a sample can be assigned to more than one sample group.

[0829] 88. The file parser (300) according to embodiment 78,

[0830] The predetermined initialization parameter set (204a, ..., 204h) is at least one of a picture parameter set PPS, a sequence parameter set SPS and a video parameter set VPS.

[0831] 89. A file generator (200) for generating a video file (201) from one or more input video bitstreams (211, 212), video content (202) being encoded into the one or more input video bitstreams (211, 212), wherein the video file (201) comprises one or more tracks in which the one or more input video bitstreams (211, 212) are embedded, wherein each of the one or more input video bitstreams (211, 212) comprises one or more encoded video sequences (203),

[0832] Wherein, the file generator (200) is configured as follows:

[0833] grouping one or more samples of the track into at least one sample group (ParameterSetSamplegroup), and marking the one or more samples as belonging to the at least one sample group (ParameterSetSamplegroup),

[0834] wherein each sample group (ParameterSetSamplegroup) is associated with one or more initialization parameter sets (204a, ..., 204h) to be used at the beginning of the one or more coded video sequences (203), wherein if a sample group (ParameterSetSamplegroup) is associated with more than one initialization parameter set (204a, ..., 204h), then:

[0835] A) each of the more than one initialization parameter sets (204a, ..., 204h) is associated with the same operating point, or

[0836] B) Each of the more than one initialization parameter sets (204a, ..., 204h) is associated with at least one operating point of the plurality of operating points.

[0837] 90. The file generator (200) according to embodiment 89,

[0838] The method is configured to group the one or more samples into at least one sample group based on an expected operation point and / or an expected decoding behavior of a decoder for decoding the one or more encoded video sequences (203).

[0839] 91. The file generator (200) according to embodiment 90,

[0840] wherein the expected operating point and / or decoding behavior of the decoder depends on at least one of the following properties, and wherein at least one of the properties is contained in a video file (201) and associated with a corresponding sample group:

[0841] • Predefined codec profiles,

[0842] • the predetermined codec level,

[0843] • a predetermined picture width of a picture contained in the one or more coded video sequences (203) to be decoded,

[0844] • a predetermined picture height of a picture contained in the one or more coded video sequences (203) to be decoded,

[0845] • a predetermined picture bit depth of pictures contained in the one or more coded video sequences (203) to be decoded,

[0846] • a predetermined picture color format of pictures contained in the one or more coded video sequences (203) to be decoded, and

[0847] • a predetermined size of a decoded picture buffer DPB for pictures contained in the one or more coded video sequences (203) to be decoded.

[0848] 92. The file generator (200) according to embodiment 89,

[0849] The method is configured to group the one or more samples into at least one sample group based on an expected operating point and / or a maximum decoding capability of a decoder for decoding the one or more encoded video sequences.

[0850] 93. The file generator (200) according to embodiment 92,

[0851] wherein the desired operating point and / or the maximum decoding behavior of the decoder depends on at least one of the following properties, and wherein at least one of the properties is contained in a video file (201) and is associated with a corresponding sample group:

[0852] • Maximum Codec Profile,

[0853] • Maximum codec level,

[0854] • a maximum picture width of pictures contained in the one or more coded video sequences (203) to be decoded,

[0855] • a maximum picture height of pictures contained in the one or more coded video sequences (203) to be decoded,

[0856] • a maximum picture bit depth of pictures contained in the one or more coded video sequences (203) to be decoded,

[0857] • the maximum picture color format of the pictures contained in the one or more coded video sequences (203) to be decoded, and

[0858] • a maximum size of a decoded picture buffer DPB for pictures contained in the one or more coded video sequences (203) to be decoded.

[0859] 94. The file generator (200) according to embodiment 89, wherein

[0860] If one or more samples within a track reference a default in-band initialization parameter set (204g) associated by default with the track,

[0861] Then the file generator (200) is configured as follows:

[0862] marking the one or more samples as belonging to at least one sample group among the one or more sample groups,

[0863] The default in-band initialization parameter set (204g) is replaced with a predetermined initialization parameter set (204h) of one or more initialization parameter sets (204a, ..., 204h) associated with a corresponding sample group.

[0864] 95. The file generator (200) according to embodiment 89, wherein

[0865] If the track is not associated with any default in-band initialization parameter set (204g) by default,

[0866] The file generator (200) is then configured to signal an indication for the track to indicate:

[0867] The track is not associated by default with any default in-band initialization parameter set (204g), and / or

[0868] The initialization parameter set (204h) for the track must be obtained from at least one of the available one or more sample groups into which the samples of the track are grouped.

[0869] 96. The file generator (200) according to embodiment 89,

[0870] Among them, a sample can be assigned to more than one sample group.

[0871] 97. The file generator (200) according to embodiment 89,

[0872] The predetermined initialization parameter set (204a, ..., 204h) is at least one of a picture parameter set PPS, a sequence parameter set SPS and a video parameter set VPS.

[0873] 98. A system comprising:

[0874] A file parser (300) according to embodiment 78, and

[0875] A file generator (200) according to embodiment 89.

[0876] 99. The system according to embodiment 98,

[0877] Also included is a video decoder (320), the video decoder (320) being configured to:

[0878] receiving the reconstructed video bitstream (310) from the file parser (300), and

[0879] Video content (201) from the reconstructed video bitstream (310) is decoded.

[0880] 100. The system according to embodiment 98,

[0881] Also included is a video encoder, the video encoder being configured to:

[0882] encoding the video content (201) into the one or more input video bitstreams (211, 212), and

[0883] The one or more input video bitstreams (211, 212) are fed into the file generator (200).

[0884] 101. A client network device (50),

[0885] comprising a file parser (300) according to embodiment 78,

[0886] The client network device (50) is configured to download a video file (201) from the server network device (20), and

[0887] The file parser (300) is configured to generate a reconstructed video bit stream (310) according to a video file (201) downloaded from the server network device (20).

[0888] 102. A server network device (20),

[0889] comprising a file generator (200) according to embodiment 89,

[0890] The file generator (200) generates a video file (201), and

[0891] The server network device (20) is configured to send the video file (201) to the client network device (50).

[0892] 103. A system comprising:

[0893] The server network device (20) according to embodiment 102, and

[0894] According to the client network device (50) of embodiment 101,

[0895] The server network device (20) is configured to generate a video file (201), and

[0896] The client network device (50) downloads the video file (201) from the server network device (20).

[0897] 104. A method for reconstructing a video bitstream (310) from a video file (201), wherein the video file (201) comprises one or more tracks, each track having one or more input video bitstreams (211, 212) embedded therein, wherein each of the one or more input video bitstreams (211, 212) comprises one or more coded video sequences (203),

[0898] Wherein, the method comprises the following steps:

[0899] Retrieve from the video file (201):

[0900] the one or more sample groups (ParameterSetSamplegroup) to which the one or more samples of the one or more tracks belong, and

[0901] an indication of which samples belong to which sample group (ParameterSetSamplegroup),

[0902] Each sample group (ParameterSetSamplegroup) is associated with at least one initialization parameter set (204a, ..., 204h) to be inserted into the reconstructed bitstream (310) and referenced by its samples,

[0903] And wherein, the method further comprises the following steps:

[0904] A predetermined initialization parameter set (204h) of a predetermined sample group (ParameterSetSamplegroup) is provided to the decoder.

[0905] 105. A method for generating a video file (201) from one or more input video bitstreams (211, 212), into which video content (202) is encoded, wherein the video file (201) comprises one or more tracks in which the one or more input video bitstreams (211, 212) are embedded, wherein each of the one or more input video bitstreams (211, 212) comprises one or more encoded video sequences (203),

[0906] Wherein, the method comprises the following steps:

[0907] grouping one or more samples of the track into at least one sample group (ParameterSetSamplegroup), and marking the one or more samples as belonging to the at least one sample group (ParameterSetSamplegroup),

[0908] Therein, each sample group (ParameterSetSamplegroup) is associated with at least one initialization parameter set (204a, ..., 204h) to be used at the beginning of the one or more encoded video sequences (203).

[0909] 106. A computer program for implementing the method according to embodiment 104 or 105 when executed on a computer or a signal processor.

[0910] 107. A decoder (320) for decoding a picture (10) of video content (202) contained in an input video bitstream (310),

[0911] The picture (10) is arranged in one or more consecutively arranged coded video sequences (203), each coded video sequence (203) is associated with its own sequence-dependent initialization parameter set SPS carrying sequence-dependent decoding parameter information,

[0912] The decoder (320) is configured to initialize its decoding behavior by using predetermined decoding parameter information before starting to decode the encoded video sequence (203) for decoding the picture (10) of the video content (201),

[0913] The decoder (320) is configured to derive a signal (dci_extension_flag) indicating to the decoder (320):

[0914] Its initialization is performed using global bitstream-dependent decoding parameter information (dci_extension()) instead of sequence-dependent decoding parameter information SPS, which indicates a predetermined global decoding capability for all coded video sequences (203) contained in the entire input video bitstream (310).

[0915] 108. The decoder (320) according to embodiment 107,

[0916] The global bitstream-related decoding parameter information (dci_extension()) indicates the highest required global decoding capability of the decoder (320) to decode all coded video sequences (203) within the entire input video bitstream (310) without reinitialization.

[0917] 109. The decoder (320) according to any one of embodiments 107,

[0918] Wherein, if the decoder (320) is initially initialized with the global bitstream-related decoding parameter information (dci_extension()), the global bitstream-related decoding parameter information (dci_extension()) includes information about a maximum picture width (dci_pic_width_max_in_luma_samples) of a picture (10) that the decoder (320) can decode without reinitialization.

[0919] 110. The decoder (320) according to embodiment 107,

[0920] wherein the sequence-dependent decoding parameter information (SPS) comprises information about a sequence-dependent maximum picture width in units of luma samples (sps_pic_width_max_in_luma_samples) of those pictures (10) contained in a particular coded video sequence (203), and

[0921] The global bitstream-related decoding parameter information (dci_extension()) includes information about a global bitstream-related maximum picture width (dci_pic_width_max_in_luma_samples) in units of luma samples for all pictures (10) contained in all coded video sequences (203) in the entire input video bitstream (310),

[0922] The global bitstream-dependent maximum picture width (dci_pic_width_max_in_luma_samples) is greater than or equal to any occurring maximum sequence-dependent maximum picture width (sps_pic_width_max_in_luma_samples).

[0923] 111. The decoder (320) according to embodiment 107,

[0924] Wherein, if the decoder (320) is initially initialized with the global bitstream-related decoding parameter information (dci_extension()), the global bitstream-related decoding parameter information (dci_extension()) includes information about a maximum picture height in units of luma samples of a picture (10) that the decoder (320) can decode without reinitialization.

[0925] 112. The decoder (320) according to embodiment 107,

[0926] wherein the sequence-dependent decoding parameter information (SPS) comprises information about a sequence-dependent maximum picture height in units of luma samples (sps_pic_height_max_in_luma_samples) of those pictures (10) contained in a particular coded video sequence (203), and

[0927] The global bitstream-related decoding parameter information (dci_extension()) includes information about a global bitstream-related maximum picture height (dci_pic_height_max_in_luma_samples) in units of luma samples for all pictures (10) contained in all coded video sequences (203) in the entire input video bitstream (310),

[0928] The global bitstream-related maximum picture height (dci_pic_height_max_in_luma_samples) is greater than or equal to any occurring maximum sequence-related maximum picture height (sps_pic_height_max_in_luma_samples).

[0929] 113. The decoder (320) according to embodiment 107,

[0930] Wherein, if the decoder (320) is initially initialized with the global bitstream-related decoding parameter information (dci_extension()), the global bitstream-related decoding parameter information (dci_extension()) includes information about a maximum chroma format (dci_pic_chroma_format_idc) of a picture (10) that the decoder (320) can decode without reinitialization.

[0931] 114. The decoder (320) according to embodiment 107,

[0932] wherein the sequence-dependent decoding parameter information (SPS) comprises information about the sequence-dependent maximum chroma format (sps_chroma_format_idc) of those pictures (10) contained in a particular coded video sequence (203), and

[0933] The global bitstream-dependent decoding parameter information (dci_extension()) includes information about a global bitstream-dependent maximum chroma format (dci_chroma_format_idc) of all pictures (10) contained in all coded video sequences (203) in the entire input video bitstream (310),

[0934] The global bitstream-dependent maximum chroma format (dci_chroma_format_idc) is greater than or equal to any occurring maximum sequence-dependent maximum chroma format (sps_chroma_format_idc).

[0935] 115. The decoder (320) according to embodiment 107,

[0936] wherein the pictures (10) contained in one or more coded video sequences (203) refer to video-related decoding parameter information (VPS), wherein the video-related decoding parameter information includes information about a video-related maximum chroma format of the pictures (10) contained in the input video bitstream (310),

[0937] The global bitstream-related decoding parameter information (dci_extension()) includes information about a global bitstream-related maximum chroma format (dci_chroma_format_idc) of all pictures (10) contained in the entire input video bitstream (310),

[0938] Wherein, for any of the included output layer sets, the global bitstream-dependent maximum chroma format (dci_chroma_format_idc) is not greater than or equal to any occurring maximum video-dependent maximum chroma format (vps_ols_dpb_chroma_format).

[0939] 116. The decoder (320) according to embodiment 107,

[0940] Wherein, if the decoder (320) is initially initialized with the global bitstream-related decoding parameter information (dci_extension()), the global bitstream-related decoding parameter information (dci_extension()) includes information about a maximum bit depth (dci_bitdepth_minus8) of a picture (10) that the decoder (320) can decode without reinitialization.

[0941] 117. The decoder (320) according to embodiment 107,

[0942] wherein the sequence-dependent decoding parameter information (SPS) comprises information about the maximum bit depth (dci_bitdepth_minus8) of those pictures (10) contained in a particular coded video sequence (203), and

[0943] The global bitstream-related decoding parameter information (dci_extension()) includes information about the global bitstream-related maximum bit depth (dci_bitdepth_minus8) of all pictures (10) contained in all coded video sequences (203) in the entire input video bitstream (310),

[0944] The global bitstream-dependent maximum bit depth (dci_bitdepth_minus8) is greater than or equal to any occurring maximum sequence-dependent maximum bit depth (sps_bitdepth_minus8).

[0945] 118. The decoder (320) according to embodiment 107,

[0946] Wherein, if the decoder (320) is initially initialized with the global bitstream-related decoding parameter information (dci_extension()), the global bitstream-related decoding parameter information (dci_extension()) includes information about the size (dci_dpb_max_dec_pic_buffering_minus1) of the decoding picture buffer DPB in units of picture storage buffers required for the decoder (320) to be able to decode the input video bitstream (310) without reinitialization.

[0947] 119. The decoder (320) according to embodiment 107,

[0948] The sequence-dependent decoding parameter information (SPS) comprises information about a sequence-dependent maximum size (dpb_max_dec_pic_buffering_minus1) in units of picture storage buffers of a decoded picture buffer DPB for those pictures (10) contained in a specific coded video sequence (203), and

[0949] wherein the global bitstream-dependent decoding parameter information (dci_extension()) comprises information on a global bitstream-dependent maximum size (dci_dpb_max_dec_pic_buffering_minus1) of a decoded picture buffer DPB for all pictures (10) contained in all coded video sequences (203) in the entire input video bitstream (310),

[0950] Therein, the global bitstream-dependent maximum size (dci_dpb_max_dec_pic_buffering_minus1) of the decoded picture buffer DPB is greater than or equal to any occurring maximum sequence-dependent maximum size (dpb_max_dec_pic_buffering_minus1) of the decoded picture buffer DPB.

[0951] 120. The decoder (320) according to embodiment 107,

[0952] The global bitstream-related decoding parameter information (dci_extension()) is signaled in a decoding capability information (DCI) NAL unit or in a supplemental enhancement information SEI message.

[0953] 121. The decoder (320) according to embodiment 107,

[0954] Therein, the global bitstream related decoding parameter information (dci_extension()) is signaled in a new bitstream DPB information (BDI) SEI message.

[0955] 122. The decoder (320) according to embodiment 107,

[0956] Therein, the decoder (320) is configured to derive a signal (dci_extension_flag) for signaling the use of the global bitstream related decoding parameter information (dci_extension()) as follows:

[0957] from the bitstream (310), or

[0958] From a file parser (300) within a file format, a sample group (DpbSamplegroup) to which one or more samples of a track (220, 221, 222) of a picture (10) of the input video bitstream (310) are embedded is utilized.

[0959] 123. A file parser (300) for reconstructing a video bitstream (310) from a video file (201), wherein the video file (201) comprises one or more tracks (220, 221, 222), each track having one or more input video bitstreams (211, 212) embedded therein, wherein each of the one or more input video bitstreams (211, 212) comprises one or more coded video sequences (203),

[0960] Wherein, the file parser (300) is configured as follows:

[0961] Retrieve from the video file (201):

[0962] a sample group (DpbSamplegroup) to which one or more samples of the one or more tracks belong, wherein the sample group (DpbSamplegroup) includes global bitstream-related decoding parameter information, and the global bitstream-related decoding parameter information includes a set of one or more global bitstream-related decoding parameters,

[0963] Wherein, the file parser (300) is configured as follows:

[0964] allocating a maximum value of at least one decoding parameter in the decoding parameter set for defining a predetermined decoding capability of the decoder (320) capable of decoding all coded video sequences (203) in the bitstream (310) that comply with these predetermined decoding capabilities without reinitialization,

[0965] Rather than using any sequence-related decoding parameter information to initialize the predetermined decoding capability.

[0966] 124. The file parser (300) according to embodiment 123,

[0967] The method is further configured to inform the decoder (320) to use the global bitstream-dependent decoding parameter information, rather than any sequence-dependent decoding parameter information, for its initialization.

[0968] 125. The file parser (300) according to embodiment 123,

[0969] The global bitstream-related decoding parameter information indicates a predetermined global decoding capability for all encoded video sequences (203) contained in the entire input video bitstream (310).

[0970] 126. The file parser (300) according to embodiment 123,

[0971] The global bitstream-related decoding parameter information indicates the maximum global decoding capability for all encoded video sequences (203) contained in the entire input video bitstream (310).

[0972] 127. The file parser (300) according to embodiment 123,

[0973] The global bitstream-related decoding parameter information includes decoding capability information about at least one of the following items for all coded video sequences (203) included in the entire reconstructed video bitstream (310):

[0974] • Maximum Codec Profile,

[0975] • Maximum codec level,

[0976] • the maximum picture width of the pictures to be decoded (10),

[0977] • the maximum picture height of the pictures to be decoded (10),

[0978] • the maximum color format of the pictures to decode (10),

[0979] • the maximum bit depth of the pictures to be decoded (10), and

[0980] • Maximum number of picture slots in the decoded picture buffer (DBP).

[0981] 128. The file parser (300) according to embodiment 123,

[0982] is configured to select the decoding capability information based on:

[0983] • one or more maximum decoder capabilities,

[0984] • one or more desired operating points, and

[0985] • One or more desired target display capabilities.

[0986] 129. An encoder (210) for encoding pictures (10) of video content (202) into a video bitstream (310), wherein the pictures (10) are arranged in one or more coded video sequences (203),

[0987] The encoder (210) is configured to provide predetermined decoding parameter information to be used by the decoder (320) to initialize its decoding behavior at the beginning of the encoded video sequence (203) for decoding a picture (10) contained in the encoded video sequence (203),

[0988] Each coded video sequence (203) is associated with its own sequence-dependent initialization parameter set (SPS) carrying sequence-dependent decoding parameter information.

[0989] The encoder (210) is configured to encode a signal (dci_extension_flag) into the bitstream (310), the signal indicating to the decoder (320):

[0990] The global bitstream-dependent decoding parameter information (dci_extension) is used for its initialization instead of the sequence-dependent decoding parameter information, and the global bitstream-dependent decoding parameter information indicates the predetermined global decoding capability for all the coded video sequences (203) contained in the whole input video bitstream (310).

[0991] 130. The encoder (210) according to embodiment 129,

[0992] The global bitstream-related decoding parameter information (dci_extension) indicates the highest required global decoding capability (203) of the decoder (320) to decode all coded video sequences within the entire input video bitstream (310) without reinitialization.

[0993] 131. The encoder (210) according to embodiment 129,

[0994] Wherein, if the decoder (320) is initially initialized with the global bitstream-related decoding parameter information (dci_extension()), the global bitstream-related decoding parameter information (dci_extension) includes information about a maximum picture width (dci_pic_width_max_in_luma_samples) of a picture (10) that the decoder (320) can decode without reinitialization.

[0995] 132. The encoder (210) according to embodiment 129,

[0996] wherein the sequence-dependent decoding parameter information (SPS) comprises information about a sequence-dependent maximum picture width in units of luma samples (sps_pic_width_max_in_luma_samples) of those pictures (10) contained in a particular coded video sequence (203), and

[0997] The global bitstream-related decoding parameter information (dci_extension()) includes information about a global bitstream-related maximum picture width (dci_pic_width_max_in_luma_samples) in units of luma samples for all pictures (10) contained in all coded video sequences (203) in the entire input video bitstream (310),

[0998] The global bitstream-dependent maximum picture width (dci_pic_width_max_in_luma_samples) is greater than or equal to any occurring maximum sequence-dependent maximum picture width (sps_pic_width_max_in_luma_samples).

[0999] 133. The encoder (210) according to embodiment 129,

[1000] Wherein, if the decoder (320) is initially initialized with the global bitstream-related decoding parameter information (dci_extension()), the global bitstream-related decoding parameter information (dci_extension()) includes information about a maximum picture height in units of luma samples of a picture (10) that the decoder (320) can decode without reinitialization.

[1001] 134. The encoder (210) according to embodiment 129,

[1002] wherein the sequence-dependent decoding parameter information (SPS) comprises information about a sequence-dependent maximum picture height in units of luma samples (sps_pic_height_max_in_luma_samples) of those pictures (10) contained in a particular coded video sequence (203), and

[1003] The global bitstream-related decoding parameter information (dci_extension()) includes information about a global bitstream-related maximum picture height (dci_pic_height_max_in_luma_samples) in units of luma samples for all pictures (10) contained in all coded video sequences (203) in the entire input video bitstream (310),

[1004] The global bitstream-related maximum picture height (dci_pic_height_max_in_luma_samples) is greater than or equal to any occurring maximum sequence-related maximum picture height (sps_pic_height_max_in_luma_samples).

[1005] 135. The encoder (210) according to embodiment 129,

[1006] Wherein, if the decoder (320) is initially initialized with the global bitstream-related decoding parameter information (dci_extension()), the global bitstream-related decoding parameter information (dci_extension()) includes information about a maximum chroma format (dci_pic_chroma_format_idc) of a picture (10) that the decoder (320) can decode without reinitialization.

[1007] 136. The encoder (210) according to embodiment 129,

[1008] wherein the sequence-dependent decoding parameter information (SPS) comprises information about the sequence-dependent maximum chroma format (sps_chroma_format_idc) of those pictures (10) contained in a particular coded video sequence (203), and

[1009] The global bitstream-dependent decoding parameter information (dci_extension()) includes information about a global bitstream-dependent maximum chroma format (dci_chroma_format_idc) of all pictures (10) contained in all coded video sequences (203) in the entire input video bitstream (310),

[1010] The global bitstream-dependent maximum chroma format (dci_chroma_format_idc) is greater than or equal to any occurring maximum sequence-dependent maximum chroma format (sps_chroma_format_idc).

[1011] 137. The encoder (210) according to embodiment 129,

[1012] wherein the pictures (10) contained in one or more coded video sequences (203) refer to video-related decoding parameter information (VPS), wherein the video-related decoding parameter information includes information about a video-related maximum chroma format of the pictures (10) contained in the input video bitstream (310),

[1013] The global bitstream-related decoding parameter information (dci_extension()) includes information about a global bitstream-related maximum chroma format (dci_chroma_format_idc) of all pictures (10) contained in the entire input video bitstream (310),

[1014] Wherein, for any of the included output layer sets, the global bitstream-dependent maximum chroma format (dci_chroma_format_idc) is not greater than or equal to any occurring maximum video-dependent maximum chroma format (vps_ols_dpb_chroma_format).

[1015] 138. The encoder (210) according to embodiment 129,

[1016] Wherein, if the decoder (320) is initially initialized with the global bitstream-related decoding parameter information (dci_extension()), the global bitstream-related decoding parameter information (dci_extension()) includes information about a maximum bit depth (dci_bitdepth_minus8) of a picture (10) that the decoder (320) can decode without reinitialization.

[1017] 139. The encoder (210) according to embodiment 129,

[1018] wherein the sequence-dependent decoding parameter information (SPS) comprises information about the maximum bit depth (dci_bitdepth_minus8) of those pictures (10) contained in a particular coded video sequence (203), and

[1019] The global bitstream-related decoding parameter information (dci_extension()) includes information about the global bitstream-related maximum bit depth (dci_bitdepth_minus8) of all pictures (10) contained in all coded video sequences (203) in the entire input video bitstream (310),

[1020] The global bitstream-dependent maximum bit depth (dci_bitdepth_minus8) is greater than or equal to any occurring maximum sequence-dependent maximum bit depth (sps_bitdepth_minus8).

[1021] 140. The encoder (210) according to embodiment 129,

[1022] Wherein, if the decoder (320) is initially initialized with the global bitstream-related decoding parameter information (dci_extension()), the global bitstream-related decoding parameter information (dci_extension()) includes information about the size (dci_dpb_max_dec_pic_buffering_minus1) of the decoding picture buffer DPB in units of picture storage buffers required for the decoder (320) to decode the input video bitstream (310) without reinitialization.

[1023] 141. The encoder (210) according to embodiment 129,

[1024] The sequence-dependent decoding parameter information (SPS) comprises information about a sequence-dependent maximum size (dpb_max_dec_pic_buffering_minus1) in units of picture storage buffers of a decoded picture buffer DPB for those pictures (10) contained in a specific coded video sequence (203), and

[1025] wherein the global bitstream-dependent decoding parameter information (dci_extension()) comprises information on a global bitstream-dependent maximum size (dci_dpb_max_dec_pic_buffering_minus1) of a decoded picture buffer DPB for all pictures (10) contained in all coded video sequences (203) in the entire input video bitstream (310),

[1026] Therein, the global bitstream-dependent maximum size (dci_dpb_max_dec_pic_buffering_minus1) of the decoded picture buffer DPB is greater than or equal to any occurring maximum sequence-dependent maximum size (dpb_max_dec_pic_buffering_minus1) of the decoded picture buffer DPB.

[1027] 142. The encoder (210) according to embodiment 129,

[1028] The global bitstream-related decoding parameter information (dci_extension()) is signaled in a decoding capability information (DCI) NAL unit or in a supplemental enhancement information SEI message.

[1029] 143. The encoder (210) according to embodiment 129,

[1030] Therein, the global bitstream related decoding parameter information (dci_extension()) is signaled in a new bitstream DPB information (BDI) SEI message.

[1031] 144. The encoder (210) according to embodiment 129,

[1032] Wherein, the encoder is configured as:

[1033] encoding a signal (dci_extension_flag) for signaling the use of said global bitstream-related decoding parameter information (dci_extension()) into said bitstream (310), or

[1034] A signal (dci_extension_flag) is provided to a file generator (200) within the file format via a sample group (DpbSamplegroup) to which one or more samples of a track (220, 221, 222) of a picture (10) of the input video bitstream (310) is embedded.

[1035] 145. A file generator (200) for generating a video file (201) from one or more input video bitstreams (211, 212), video content (202) being encoded into the one or more input video bitstreams (211, 212), wherein the video file (201) comprises one or more tracks (220, 221, 222) in which the one or more input video bitstreams (211, 212) are embedded, wherein each of the one or more input video bitstreams (211, 212) comprises one or more encoded video sequences (203),

[1036] Wherein, the file generator (200) is configured as follows:

[1037] In the video file (201) it is provided:

[1038] a sample group (DpbSamplegroup) to which one or more samples of the one or more tracks belong, wherein the sample group (DpbSamplegroup) includes global bitstream-related decoding parameter information, and the global bitstream-related decoding parameter information includes a set of one or more global bitstream-related decoding parameters,

[1039] information about a maximum value of at least one decoding parameter in a decoding parameter set for defining predetermined decoding capabilities of a decoder (320) capable of decoding all coded video sequences (203) in a bitstream (310) that comply with these predetermined decoding capabilities without reinitialization,

[1040] Rather than using any sequence-related decoding parameter information to initialize the predetermined decoding capability.

[1041] 146. The file generator (200) according to embodiment 145,

[1042] The global bitstream-related decoding parameter information indicates the maximum global decoding capability for all encoded video sequences (203) contained in the entire input video bitstream (310).

[1043] 147. The file generator (200) according to embodiment 145,

[1044] The global bitstream-related decoding parameter information includes decoding capability information of all coded video sequences (203) included in the entire reconstructed video bitstream (310) regarding at least one of the following:

[1045] • Maximum Codec Profile,

[1046] • Maximum codec level,

[1047] • the maximum picture width of the pictures to be decoded (10),

[1048] • the maximum picture height of the pictures to be decoded (10),

[1049] • the maximum color format of the pictures to decode (10),

[1050] • the maximum bit depth of the pictures to be decoded (10), and

[1051] • Maximum number of picture slots in the decoded picture buffer (DBP).

[1052] 148. A method for decoding a picture (10) of video content (202) contained in an input video bitstream (310), the method comprising:

[1053] Initializing a decoding behavior of a decoder (320) for decoding a picture (10) of the video content (202) by using predetermined decoding parameter information before starting to decode the encoded video sequence (203),

[1054] The pictures (10) are arranged in one or more consecutively arranged coded video sequences (203), each coded video sequence (203) being associated with its own sequence-dependent initialization parameter set (SPS) carrying sequence-dependent decoding parameter information,

[1055] A derived signal (dci_extension_flag) indicating:

[1056] The decoding behavior of the decoder (320) is initialized using global bitstream-dependent decoding parameter information (dci_extension) instead of sequence-dependent decoding parameter information, the global bitstream-dependent decoding parameter information indicating a predetermined global decoding capability for all coded video sequences (203) contained in the entire input video bitstream (310).

[1057] 149. A method for encoding pictures (10) of video content (202) into a video bitstream (310), wherein the pictures (10) are arranged in one or more coded video sequences (203), the method comprising:

[1058] providing predetermined decoding parameter information to be used by a decoder (320) to initialize its decoding behavior at the beginning of a coded video sequence (203) for decoding a picture (10) contained in the coded video sequence (203),

[1059] wherein each coded video sequence (203) is associated with its own sequence-dependent initialization parameter set (SPS) carrying sequence-dependent decoding parameter information, and

[1060] A signal (dci_extension_flag) is encoded into the bitstream (310) which indicates to the decoder (320):

[1061] The global bitstream-dependent decoding parameter information (dci_extension) is used for its initialization instead of the sequence-dependent decoding parameter information, and the global bitstream-dependent decoding parameter information indicates the predetermined global decoding capability for all the coded video sequences (203) contained in the whole input video bitstream (310).

[1062] 150. A bitstream (310) obtained by the method according to embodiment 148 or 149.

[1063] 151. A computer program for implementing the method according to embodiment 148 or 149 when executed on a computer or a signal processor.

[1064] 152. A player comprising a decoder (320) for decoding a picture (10) of video content (202) contained in an input video bitstream (310),

[1065] The pictures (10) are arranged in one or more consecutively arranged coded video sequences (203), each coded video sequence (203) being associated with its own sequence-dependent initialization parameter set (SPS) carrying sequence-dependent decoding parameter information,

[1066] The decoder (320) is configured to initialize its decoding behavior for decoding a picture (10) of the video content (201) by using at least one initialization parameter from a predetermined initialization parameter set before starting decoding of a specific coded video sequence (203) instead of using signaled sequence dependent decoding parameter information (SPS),

[1067] Wherein, the player is configured to distribute:

[1068] The maximum value of at least one parameter in the initialization parameter set is used to define the minimum decoding requirement for the decoder (320) to decode all encoded video sequences (203) in the bitstream (310) that meet the minimum decoding requirement without reinitialization.

[1069] 153. The player according to embodiment 152,

[1070] The initialization parameter set defining the minimum decoding requirements of the decoder (320) includes at least one of the following parameters that can be directly assigned by the player:

[1071] • Maximum Codec Profile,

[1072] • Maximum codec level,

[1073] • Maximum decoder capability,

[1074] • Desired operating point,

[1075] • Desired target display capability,

[1076] • the maximum picture width of the pictures to be decoded (10),

[1077] • the maximum picture height of the pictures to be decoded (10),

[1078] • the maximum color format of the pictures to decode (10),

[1079] • the maximum bit depth of the pictures to be decoded (10), and

[1080] • The maximum number of picture slots in the decoded picture buffer DBP.

[1081] 154. The player according to embodiment 152,

[1082] The method is configured to initialize the decoder (320) in said manner by adapting / rewriting existing in-band or out-of-band parameter sets before using them in said reconstructed bitstream (310).

[1083] 155. The player according to embodiment 154,

[1084] Configured to initialize the decoder in the manner described by adjusting one or more of the parameters described in embodiment 153.

[1085] 156. The player according to embodiment 154,

[1086] The decoder is configured to be initialized in the manner described by adjusting the maximum picture width and height indicated in an initialization parameter set (e.g., SPS) according to the maximum decoder capability and / or by adjusting the number of time slots in the decoded picture buffer to a maximum value of the corresponding specification.

Claims

1. A file parser (300) for reconstructing a video bitstream (310) from a video file (201), wherein: The video file (201) comprises one or more tracks (220, 221, 222), each track having one or more input video bitstreams (211, 212) embedded therein, wherein each of the one or more input video bitstreams (211, 212) comprises one or more coded video sequences (203), Wherein, the file parser (300) is configured as follows: Retrieve from the video file (201): one or more sample groups (ParameterSetSamplegroup) to which one or more samples of the one or more tracks (220, 221, 222) belong, and an indication of which samples belong to which sample group (ParameterSetSamplegroup), Each sample group (ParameterSetSamplegroup) is associated with one or more initialization parameter sets (204a, ..., 204h) to be inserted into the reconstructed bitstream (310) and referenced by its samples, Wherein, the file parser (301) is further configured to: A predetermined initialization parameter set (204h) of a predetermined sample group (ParameterSetSamplegroup) is included in the reconstructed bitstream (310).

2. The file parser (300) according to claim 1, in, If a sample group (ParameterSetSamplegroup) is associated with more than one initialization parameter set (204a, ..., 204h), then: A) each of the more than one initialization parameter sets (204a, ..., 204h) is associated with the same operating point, or B) Each of the more than one initialization parameter sets (204a, ..., 204h) is associated with at least one operating point of the plurality of operating points.

3. The file parser (300) according to claim 1, The method is configured to select a predetermined set of initialization parameters (204h) of the predetermined group of samples based on a desired operating point and / or a desired decoding behavior of a decoder for decoding the one or more encoded video sequences (203).

4. The file parser (300) according to claim 3, in, The expected operating point and / or decoding behavior of the decoder depends on at least one of the following properties, and wherein at least one of the properties is contained in a video file (201) and associated with a corresponding sample group: • Predefined codec profiles, • the intended codec level, • a predetermined picture width of a picture contained in the one or more coded video sequences (203) to be decoded, • a predetermined picture height of a picture contained in the one or more coded video sequences (203) to be decoded, • a predetermined picture bit depth of pictures contained in the one or more coded video sequences (203) to be decoded, • a predetermined picture color format of pictures contained in the one or more coded video sequences (203) to be decoded, and • a predetermined size of a decoded picture buffer DPB for pictures contained in the one or more coded video sequences (203) to be decoded.

5. The file parser (300) according to claim 1, The method is configured to select a predetermined set of initialization parameters (204h) of the predetermined group of samples based on a desired operating point and / or a maximum decoding capability of a decoder for decoding the one or more encoded video sequences (203).

6. The file parser (300) according to claim 5, in, The desired operating point and / or maximum decoding behavior of the decoder depends on at least one of the following properties, and wherein at least one of the properties is contained in a video file (201) and associated with a corresponding sample group: • Maximum Codec Profile, • Maximum codec level, • a maximum picture width of pictures contained in the one or more coded video sequences (203) to be decoded, • a maximum picture height of pictures contained in the one or more coded video sequences (203) to be decoded, • a maximum picture bit depth of pictures contained in the one or more coded video sequences (203) to be decoded, • the maximum picture color format of the pictures contained in the one or more coded video sequences (203) to be decoded, and • a maximum size of a decoded picture buffer DPB for pictures contained in the one or more coded video sequences (203) to be decoded.

7. The file parser (300) according to claim 1, wherein If one or more samples within a track reference a default in-band initialization parameter set (204g) associated with the track by default, and If the one or more samples are marked as belonging to at least one sample group of the one or more sample groups, Then the file parser (300) is configured as follows: deriving predetermined initialization parameters (204h) of one or more initialization parameter sets (204a, ..., 204h) associated with the respective sample group, and replacing an otherwise in-band sent initialization parameter set (204g) with said initialization parameter set (204h) of said corresponding sample group, and The initialization parameter set (204h) for the sample group is provided to the decoder as a predetermined initialization parameter set within the reconstructed bitstream (310), the initialization parameter set (204h) replacing the initialization parameter set (204g) otherwise sent in-band.

8. The file parser (300) according to claim 1, wherein If the track is not associated by default with any default in-band initialization parameter set (204a, ..., 204g), and If one or more samples contained in the track are marked as belonging to at least one sample group of the one or more sample groups, Then the file parser (300) is configured as follows: inserting a predetermined initialization parameter set (204h) of the one or more initialization parameter sets associated with the at least one sample group into the reconstructed bitstream (310), and The initialization parameter set (204h) of the at least one sample group is provided to the decoder as a predetermined initialization parameter set within the reconstructed bitstream (310).

9. The file parser (300) according to claim 8, is configured to derive a signal for a track, the signal indicating: The track is not associated by default with any default in-band initialization parameter set (204a, ..., 204g), and / or An initialization parameter set (204h) for the track must be obtained from at least one sample group of the available one or more sample groups, and the obtained initialization parameter set (204h) must be provided to the decoder as the predetermined initialization parameter set.

10. The file parser (300) according to claim 8, is configured to derive a signal for a track, the signal indicating: Those samples of the track that are marked as belonging to one or more sample groups do not by default refer to any default in-band initialization parameter set (204a, ..., 204g), and / or For those samples, an initialization parameter set (204h) from at least one of the available one or more sample groups to which those samples belong has to be inserted into the reconstructed bitstream (310) and provided to the decoder as the predetermined initialization parameter set.

11. The file parser (300) according to claim 1, in, A sample can be assigned to more than one sample group.

12. The file parser (300) according to claim 1, in, The predetermined initialization parameter set (204a, ..., 204h) is at least one of a picture parameter set PPS, a sequence parameter set SPS, and a video parameter set VPS.

13. A method for reconstructing a video bitstream (310) from a video file (201), wherein: The video file (201) comprises one or more tracks, each track having one or more input video bitstreams (211, 212) embedded therein, wherein each of the one or more input video bitstreams (211, 212) comprises one or more coded video sequences (203), Wherein, the method comprises the following steps: Retrieve from said video file (201): the one or more sample groups (ParameterSetSamplegroup) to which the one or more samples of the one or more tracks belong, and an indication of which samples belong to which sample group (ParameterSetSamplegroup), Each sample group (ParameterSetSamplegroup) is associated with at least one initialization parameter set (204a, ..., 204h) to be inserted into the reconstructed bitstream (310) and referenced by its samples, And wherein, the method further comprises the following steps: A predetermined initialization parameter set (204h) of a predetermined sample group (ParameterSetSamplegroup) is provided to the decoder.

14. A computer program for implementing the method according to claim 13 when executed on a computer or a signal processor.

Images