Method, device and system for ISOBMFF file enhanced track grouping

Abstract

A method of processing a media stream is described herein. The method comprises: receiving one or more media streams and auxiliary information for the one or more media streams, the one or more media streams comprising media content comprising one or more tracks; generating an ISO basic media file format (ISOBMFF) file based on the one or more media streams and the auxiliary information; and outputting the generated ISOBMFF file for further processing. A corresponding method, a corresponding apparatus and a computer program product for processing an ISOBMFF file are further described.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to U.S. Provisional Application No. 63 / 394,027 filed on August 1, 2022 and U.S. Provisional Application No. 63 / 417,426 filed on October 19, 2022, both of which are incorporated herein by reference in their entireties. Technical Field

[0003] The present disclosure generally relates to a method for processing a media stream. Specifically, the method includes generating an ISO base media file format (ISOBMFF) file based on one or more media streams and side information. The present disclosure also relates to a method for processing an ISOBMFF file, a corresponding apparatus, and a computer program product.

[0004] Although some embodiments will be described herein with particular reference to this disclosure, it will be appreciated that the disclosure is not limited to this field of use and may be applicable in a wider context. Background Art

[0005] Any discussion of the background art throughout the disclosure should in no way be considered as an admission that such art is widely known or forms part of the common general knowledge in the art.

[0006] For various reasons, such as saving transmission bandwidth, modern video compression schemes often spread the overall available media data across several streams.

[0007] For streaming applications, one media asset can be encoded as streams using different bitrates. This set of streams - called a "Representation" in Dynamic Adaptive Streaming over HTTP (DASH) as defined by the Moving Picture Experts Group (MPEG) MPEG DASH in ISO / IEC 23009, and a "CMAF track" in Common Media Application Format for Segmented Media (CMAF) as defined by MPEG in ISO / IEC 23000-19 - is grouped to allow dynamic switching between these streams, thereby reacting to changing network conditions. This grouping is called an "AdaptationSet" in DASH and a "Switching Set" in CMAF.

[0008] The International Organization for Standardization (ISO) includes and specifies a base media file format commonly referred to as ISOBMFF. In particular, the ISOBMFF is specified by ISO through ISO / IEC 14496-12 MPEG-4 Part 12. In general, ISOBMFF is a container file format that defines a common structure for time-based multimedia files (such as video and / or audio).

[0009] In a typical setup, a group of one or more encoders will encode the input signal(s) into tracks at different bitrates. Additionally, if the input signal has multiple variations (e.g. variations in spoken language or variations by adding spoken subtitles), several of these groups of encoders will run in parallel to produce all the required bitrate variations. In a typical current implementation, these encoders will each output an ISOBMFF with one ISOBMFF track.

[0010] ISOBMFF tracks may be grouped using derivatives of TrackGroupTypeBox as defined in ISO / IEC 14496-12 MPEG-4 Part 12, Section 8.3.4. The grouping type is specified by the track_group_type identifier. By choosing different track_group_type identifiers, different grouping types with different semantics may be defined.

[0011] To define a group of tracks, instead of listing all grouped tracks in a list in TrackGroupTypeBox, grouping is implemented by implicitly building equivalence classes. All tracks belonging to a particular group carry a TrackGroupTypeBox with the same type and id identifier. In other words, two tracks belong to the same group if and only if they contain a derivative of this TrackGroupTypeBox with the same type and id identifier.

[0012] MPEG is currently developing a first mechanism to organize the properties of a track group: TrackGroupDescriptionBox acts as a container for TrackGroupEntryBox and its derivatives.

[0013] The second mechanism for grouping tracks in ISOBMFF is to use a derivative of EntityToGroupBox as defined in ISO / IEC 14496-12 MPEG-4 Part 12, Section 8.18.3. The grouping type is specified by the grouping_type identifier. By choosing different grouping_type identifiers, different grouping types with different semantics can be defined.

[0014] A group of tracks (e.g., tracks using the entity grouping mechanism) can use an instance of EntityToGroupBox to reference all tracks by their corresponding track_id.

[0015] The third mechanism for grouping tracks in ISOBMFF is to use the alternative_group field in the track header box as defined in ISO / IEC 14496-12 MPEG-4 Part 12, Section 8.3.2. This type of grouping has no grouping type. Grouping is achieved by assigning the same identifier to the alternative_group field of all tracks in a group, where identifier "0" means that the corresponding track does not belong to any group.

[0016] Furthermore, to prepare the ISOBMFF tracks for segmented transmission, each of these tracks is processed by a segmenter that breaks them into short-duration segment streams, each typically representing a few seconds. Finally, the DASH manifest generator describes the configuration of the segmented streams in a manifest file. The manifest file then contains a high-level description of the various streams (i.e., language, accessibility attributes, bitrate, etc.) and their relationships to each other.

[0017] However, further derivatives are still needed to allow signaling of dependencies between groups of tracks. Summary of the invention

[0018] According to a first aspect of the present disclosure, a method for processing a media stream is provided. The method may include receiving one or more media streams and auxiliary information for the one or more media streams, wherein the one or more media streams include media content including (e.g., divided into) one or more tracks. The method may further include generating an ISO base media file format ISOBMFF file based on the one or more media streams and the auxiliary information. And the method may include outputting the generated ISOBMFF file for further processing. It is worth noting that the auxiliary information used to generate the ISOBMFF file is not output for further processing.

[0019] Through the above configuration, the method allows all media stream tracks (e.g., with different bit rates originating from respective encoding processes) to be saved in one ISOBMFF file, thereby signaling not only dependencies between tracks but also dependencies between track groups. On the other hand, the auxiliary information in this framework is not required by downstream devices.

[0020] In some embodiments, the auxiliary information may indicate interrelationships between some or all of the one or more tracks.

[0021] In some embodiments, generating the ISOBMFF file may include grouping some or all of the one or more tracks into one or more track groups.

[0022] In some embodiments, the one or more track groups may correspond to one or more DASH AdaptationSets, and one or more tracks within each track group may correspond to one or more DASH representations within the DASH AdaptationSet.

[0023] In some embodiments, the one or more track groups may correspond to one or more DASH AdaptationSets having the possibility of switching playback from an identified source DASH AdaptationSet to a set of target DASH AdaptationSets.

[0024] In some embodiments, the one or more track groups may correspond to one or more CMAF switch sets, and one or more tracks in each track group may correspond to one or more CMAF tracks in the CMAF switch set.

[0025] In some embodiments, for each of one or more tracks within a track group, the ISOBMFF file may include a track group type specific box indicating a predefined and specific track group type and a track group identifier, wherein tracks with the same track group type and track group identifier may belong to the same track group, and the track group type specific box is included in the ISOBMFF track group box.

[0026] In some embodiments, for each of one or more tracks within a track group, the ISOBMFF file may include a packet type specific box indicating a predefined and specific packet type and a packet identifier, wherein the packet type specific box may be included in an ISOBMFF packet list box.

[0027] In some embodiments, for each of the switching possibilities, the ISOBMFF file may further include a track group entry box of a predefined type whose track group identifier is equal to the track group identifier identifying the track group of the source DASH AdaptationSet, and whose track group identifier list is equal to those track group identifiers identifying the track group of the target DASH AdaptationSet.

[0028] In some embodiments, generating the ISOBMFF file may further include grouping the one or more track groups into one or more track group groups based on the auxiliary information.

[0029] In some embodiments, each track grouping may indicate a relationship between one or more track groups within the track grouping.

[0030] In some embodiments, the one or more track groupings may correspond to one or more CMAF selection sets.

[0031] In some embodiments, the one or more track groupings may correspond to one or more CMAF-aligned switching sets.

[0032] In some embodiments, for each of the one or more track group groups, the ISOBMFF file may further include a track group group specific box including a list of references to track groups belonging to the track group group.

[0033] In some embodiments, each of the one or more track group specific boxes may be instantiated by a predefined track group entry type of a track group entry box included in an ISOBMFF track group description box.

[0034] In some embodiments, each of the one or more track group specific boxes may indicate their respective track group identifier, which enables reference to the track group as a track group.

[0035] In some embodiments, the auxiliary information may not be output together with the ISOBMFF file for further processing.

[0036] According to a second aspect of the present disclosure, a method for processing an ISO base media file format ISOBMFF file is provided. The method may include receiving an ISOBMFF file, the file including one or more tracks of media content and one or more boxes characterizing the interrelationships between some or all of the one or more tracks. The method may further include extracting information about each interrelationship between some or all of the one or more tracks from the one or more boxes. The method may further include segmenting the ISOBMFF file based on the extracted information to obtain a plurality of fragments. And the method may include outputting the plurality of fragments.

[0037] In some embodiments, the interrelationships may be indicated by one or more track group boxes, thereby grouping one or more tracks into track groups corresponding to the respective interrelationships.

[0038] In some embodiments, the track group may correspond to a DASH adaptation set (Adaptation Set), and one or more tracks in the track group may correspond to one or more DASH representations (Representation) in the DASH AdaptationSet.

[0039] In some embodiments, the track group may correspond to a DASH AdaptationSet with the possibility of switching playback from an identified source DASH AdaptationSet to a set of target DASH AdaptationSets.

[0040] In some embodiments, the track group may correspond to a CMAF switch set, and one or more tracks in the track group may correspond to one or more CMAF tracks in the CMAF switch set.

[0041] In some embodiments, for each of one or more tracks within a track group, the corresponding mutual relationship may be indicated by a track group type specific box indicating a predefined and specific track group type and a track group identifier, wherein tracks having the same track group type and track group identifier may belong to the same track group, and the track group type specific box is included in the ISOBMFF track group box.

[0042] In some embodiments, for each of one or more tracks within a track group, the corresponding interrelationship may be indicated by a packet type specific box indicating a predefined and specific packet type and packet identifier. The packet type specific box may be included in an ISOBMFF packet list box.

[0043] In some embodiments, for each of said switching possibilities, the corresponding mutual relationship may be further indicated by a track group entry box of a predefined type, whose track group identifier is equal to the track group identifier identifying the track group of the source DASH AdaptationSet, and whose track group identifier list is equal to those track group identifiers identifying the track group of the target DASH AdaptationSet.

[0044] In some embodiments, the interrelationship may be further indicated by grouping the one or more track groups into one or more track group groups.

[0045] In some embodiments, each track grouping may indicate a relationship between one or more track groups within the track grouping.

[0046] In some embodiments, the track grouping may correspond to a CMAF selection set.

[0047] In some embodiments, the track grouping may correspond to a CMAF-aligned switch set.

[0048] In some embodiments, the mutual relationship may be indicated by one or more track group specific boxes, each track group specific box comprising a list of references to track groups belonging to the track group.

[0049] In some embodiments, each of the one or more track group specific boxes may be instantiated by a predefined track group entry type of a track group entry box included in an ISOBMFF track group description box.

[0050] In some embodiments, each of the one or more track group specific boxes may indicate its respective track group identifier, which enables reference to the track group as a track group.

[0051] In some embodiments, the method may further include generating a manifest file based on the ISOBMFF file and outputting the manifest file together with the plurality of fragments.

[0052] In some embodiments, the manifest file may include information extracted from the one or more boxes regarding each interrelationship between some or all of the one or more tracks.

[0053] In some embodiments, the manifest file may be a DASH Media Presentation Description MPD file.

[0054] In some embodiments, segmenting the ISOBMFF file may include generating one or more DASH initialization segments including information extracted from the one or more boxes about each interrelationship between some or all of the one or more tracks.

[0055] In some embodiments, the DASH MPD file may include information included in the DASH initialization segment.

[0056] In some embodiments, the DASH MPD file may include information about each interrelationship between some or all of the one or more tracks based on the ISOBMFF file, but the DASH initialization segment may not include the interrelationship information.

[0057] According to a third aspect of the present disclosure, a device for processing a media stream is provided. The device may include one or more processors, and the processors are configured to perform a method including the following operations: receiving one or more media streams and auxiliary information for the one or more media streams, wherein the one or more media streams include media content divided into one or more tracks; generating an ISO base media file format ISOBMFF file based on the one or more media streams and the auxiliary information; and outputting the generated ISOBMFF file for further processing.

[0058] In some embodiments, the apparatus may include a multiplexer.

[0059] According to a fourth aspect of the present disclosure, a device for processing an ISO base media file format ISOBMFF file is provided. The device may include one or more processors, and the processors are configured to perform a method including the following operations: receiving an ISOBMFF file, the file including one or more tracks of media content and one or more boxes characterizing the mutual relationship between some or all of the one or more tracks; extracting information about each mutual relationship between some or all of the one or more tracks from the one or more boxes; segmenting the ISOBMFF file based on the extracted information to obtain a plurality of fragments; and outputting the plurality of fragments.

[0060] In some embodiments, the apparatus may include a segmenter.

[0061] In some embodiments, the one or more processors may be further configured to generate a manifest file based on the ISOBMFF file and output the manifest file together with the plurality of fragments.

[0062] In some embodiments, the apparatus may further comprise a manifest generator (eg, a so-called "dasher").

[0063] According to a fifth aspect of the present disclosure, a system of an apparatus for processing a media stream as described herein and an apparatus for processing an ISOBMFF file as described herein is provided.

[0064] According to a sixth aspect of the present disclosure, a program is provided, the program comprising instructions, which, when executed by a processor, causes the processor to execute the method for processing a media stream and / or the method for processing an ISOBMFF file as described herein.

[0065] According to a seventh aspect of the present disclosure, a computer-readable storage medium storing the program is provided. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] Example embodiments of the present disclosure will now be described, by way of example only, with reference to the accompanying drawings, in which:

[0067] Figure 1 A first example of a system of an apparatus for processing a media stream and an apparatus for processing an ISOBMFF file is illustrated.

[0068] Figure 2 An example of a method of processing a media stream according to an embodiment of the present disclosure is illustrated.

[0069] Figure 3 An example of the structure of an ISOBMFF file including enhanced track grouping according to an embodiment of the present disclosure is illustrated.

[0070] Figure 4 Another example of the structure of an ISOBMFF file including enhanced track grouping according to an embodiment of the present disclosure is illustrated.

[0071] Figure 5 Another example of the structure of an ISOBMFF file including enhanced track grouping according to an embodiment of the present disclosure is illustrated.

[0072] Figure 6 Another example of the structure of an ISOBMFF file including enhanced track grouping according to an embodiment of the present disclosure is illustrated. Figure 7 An example of a method of processing an ISOBMFF file according to an embodiment of the present disclosure is illustrated.

[0073] Figure 8 An example of the structure of an initialization segment based on an ISOBMFF file according to an embodiment of the present disclosure is illustrated.

[0074] Fig. 9 A second example of a system of an apparatus for processing a media stream and an apparatus for processing an ISOBMFF file according to an embodiment of the present disclosure is illustrated.

[0075] Fig.10 An apparatus including one or more processors and a memory according to an embodiment of the present disclosure is illustrated. DETAILED DESCRIPTION

[0076] Overview

[0077] When streams of different bitrates are produced according to the respective encoding processes, it is beneficial to keep all tracks in one ISOBMFF file (or multi-track fragment stream). If such a mechanism is defined, it can be used to represent DASH AdaptationSets and the representations they contain, for example. However, so far, there is no defined signaling for grouping tracks into the above sets so that subsequent streaming packagers know how to separate them into single-track fragment streams for Internet streaming. CMAF-aligned switch sets and selection sets are a concept that makes it necessary not only to group tracks, but also to construct groupings of track groups. While ISOBMFF can already signal dependencies between tracks, it does not provide a means to signal dependencies between track groups (such as those defined in CMAF).

[0078] In a typical setup, the encoder, segmenter, and manifest generator must be set to the same configuration using a global control instance. These can be implemented in a variety of software environments running on a variety of hardware platforms. This global configuration places certain restrictions on the deployment of these components, which the present disclosure addresses by making all components independent by creating a data path for this configuration data in the same ISOBMFF container that also contains the encoded media.

[0079] First example of handling media streams and ISOBMFF files

[0080] refer to Figure 1 The present invention provides an example of a system of an apparatus for processing a media stream and an apparatus for processing an ISO base media file format ISOBMFF file.

[0081] In this example, at the encoding side 100, one or more media streams are generated by encoding the same video content at three different bit rates: encoded at a high bit rate in encoder 1 101, encoded at a medium bit rate in encoder 2 102, and encoded at a low bit rate in encoder 3 103. Accompanying audio content is encoded in encoder 4 104 together with the video content.

[0082] Subsequently, the media streams thus generated are processed separately in the corresponding multiplexers. Mux 1 106 processes the media stream including one or more tracks of video content encoded at a high bit rate, Mux 2 107 processes the media stream including one or more tracks of video content encoded at a medium bit rate, Mux 3 108 processes the media stream including one or more tracks of video content encoded at a low bit rate, and Mux 4 109 processes the media stream including one or more tracks of audio content. Each multiplexer then outputs a corresponding separate ISOBMFF file (MP4) 110, 111, 112, and 113 for each media stream. Additional information (auxiliary information) 105 indicating the mutual relationship between some or all of one or more tracks in one or more media streams may be output together with the respective ISOBMFF files for further processing.

[0083] In a stream packager (streaming packager) 120, each ISOBMFF file 110, 111, 112, 113 is then segmented by a corresponding segmenter (Seg 1 121; Seg 2 122; Seg 3 123 and Seg 4 124) to generate a corresponding initialization segment 127 and media segments 128 based on each individual ISOBMFF file for streaming. The stream packager 120 can be implemented alone or in combination with other components described herein. The initialization segment 127 can contain metadata for presenting a media stream encapsulated in a media segment 128 that conforms to a media format. These segments 127, 128 can be output for transmission (e.g., via the Internet) and / or streaming 129. For example, the segments can be uploaded to a server, such as an origin server of a content delivery network (CDN).

[0084] Referring to MPEG-DASH (ISO / IEC 23009-1), the initialization segment and media segment can be described as follows:

[0085] Initialization segment: A fragment containing metadata necessary for presenting the media stream encapsulated in the media segment;

[0086] Media segment: A fragment that conforms to the media format being used, combined with zero or more preceding fragments and the initialization segment (if any) to enable playback.

[0087] In the manifest generator 125, a manifest file 126 is additionally generated based on the received additional information 105 and the initialization segment 127. The manifest file references the metadata for streaming. The manifest file (specified by MPEG-DASH and HLS) describes the media presentation; resource identifiers for all segments are provided, as well as media asset descriptions. The plurality of segments 127, 128 and the manifest file 126 are then output for streaming (over the Internet) 129. For example, the plurality of segments 127, 128 and the manifest file 126 may be output to a streaming server or provided to a multicast server or broadcast transmitter.

[0088] Figure 1 A first example of processing of media streams and ISOBMFF files is described. Figure 1 Each module in can be executed separately or in combination in one or more processors. In this case, a separate ISOBMFF file will be generated for each of the one or more media streams and output together with the corresponding additional information. Therefore, these separate ISOBMFF files do not contain the information about the relationship between the tracks included in the auxiliary information.

[0089] Second example of processing media streams and ISOBMFF files

[0090] Reference now Figure 2 , which illustrates a flowchart of an improved method 200 for processing a media stream according to an embodiment of the present disclosure.

[0091] In step S201, one or more media streams and auxiliary information for the one or more media streams are received, for example, by a receiver. For example, the receiver may be part of a multiplexer. The one or more media streams include media content that includes (eg, is divided into) one or more tracks.

[0092] In step S202, a (single) ISO base media file format ISOBMFF file is generated based on one or more media streams and auxiliary information. Figure 1 In the example in , the auxiliary information may not be output together with the ISOBMFF file for further processing. Therefore, in step S203, the generated (single) ISOBMFF file is output for further processing. Further processing may include processing by a streaming media packager (stream packager), such as separating the ISOBMFF file into segment streams for Internet streaming. For example, the ISOBMFF file may be output for archiving and / or segmented for transmission (e.g., via the Internet) and / or streaming.

[0093] In an embodiment, the auxiliary information may indicate interrelationships between some or all of the one or more tracks. As described herein, when an ISOBMFF file is generated based on one or more media streams and the auxiliary information, the single ISOBMFF file thus includes information about interrelationships between some or all of the one or more tracks.

[0094] These interrelationships between some or all of the one or more tracks may be represented in the ISOBMFF file 301 by one or more boxes, such as Figures 3 to 6 The example structure is shown in the figure.

[0095] It is noteworthy that, as used herein, in some possible cases, the term "box" can be generally used to refer to an object-oriented building block defined by a unique (box) type identifier (and possibly, a corresponding length), such as described in ISO14496-12. Of course, the term "box" used throughout this disclosure should not be understood to be limited to such specifications. On the contrary, the term "box" should generally be understood to be any suitable data structure or data container that can be used as a data placeholder. In addition, as the technician will also understand and recognize, such a "box" can be referred to by using any other suitable term. An example can be: in some possible specifications (including the first definition of MP4), in some cases a "box" can be alternatively referred to as an "atom". Further, depending on various embodiments and / or requirements, one or more boxes can have the same or different levels (or positions), can be nested (child boxes and parent boxes), etc., as the technician will understand and recognize.

[0096] In an embodiment, generating the ISOBMFF file 301 may include grouping some or all of the one or more tracks into one or more track groups. Depending on the use case, these one or more track groups may correspond to one or more DASH AdaptationSets 315, and one or more tracks within each track group may then correspond to one or more DASH representations 316 within the DASH AdaptationSet 315. In an embodiment, these one or more track groups may correspond to one or more DASH AdaptationSets 514 with the possibility of switching playback from an identified source DASH AdaptationSet to a set of target DASH AdaptationSets. Alternatively, these one or more track groups may correspond to one or more CMAF switch sets 320, and one or more tracks within each track group may then correspond to one or more CMAF tracks within the CMAF switch set 320.

[0097] In an embodiment, for each of one or more tracks within a track group, the ISOBMFF file 301, 401 may include a track group type specific box 309, 413 indicating a predefined and specific track group type and a track group identifier 414. Tracks with the same track group type and track group identifier may belong to the same track group, and the track group type specific box 309, 413 may be included in the ISOBMFF track group box 306, 412.

[0098] Different grouping mechanisms may also be applied. In an embodiment, for each of one or more tracks within a track group, the ISOBMFF file may include a grouping type specific box indicating a predefined and specific grouping type and a grouping identifier, wherein the grouping type specific box may be included in an ISOBMFF grouping list box.

[0099] In the case of one or more DASH AdaptationSets 514 with switching possibilities, for each of these switching possibilities, the ISOBMFF file 501, 601 may further include a track group entry box 510, 603 of a predefined type whose track group identifier 604 is equal to the track group identifiers identifying the track groups of the source DASH AdaptationSet and whose track group identifier list 605 is equal to those track group identifiers identifying the track groups of the target DASH AdaptationSet.

[0100] In other words, for example, a new type of track group 309, 509 for a DASH AdaptationSet may be created. In order to collect several tracks into an AdaptationSet 315, 514 as a Representation 316, 516, a new track_group_type 309, 509 may be defined, which is referred to as AdaptationSetGroup 309, 509 in this document in a non-binding manner and only for explanation purposes, such as Figure 3 and Figure 5 The example structure is shown in the figure.

[0101] The AdaptationSetGroup 309, 509 may be derived from the TrackGroupTypeBox 307, 507 and identified by the 4CC code. Requirements and semantics may further describe the purpose of the grouping.

[0102] A streaming packager may be able to read the AdaptationSetGroup 309 , 509 information, split the tracks in the file into separate fragments based on that information, and write a summary of all of these separate fragments into a streaming manifest 311 , 511 .

[0103] For the case where a fully multiplexed file arrives at a player (e.g., a client), a mechanism may be needed to prevent the player from playing all included tracks simultaneously. In order to prohibit the media player from playing all tracks within a group and instead play only one track, the track_in_movie or track_enabled flag in TrackHeaderBox 305, 411, 505 may be cleared as needed to select a default track for playback.

[0104] Further use cases might require not only building track groups, but also being able to group those track groups themselves.

[0105] In an embodiment, generating the ISOBMFF file 301, 501 may therefore further comprise grouping one or more track groups into one or more track group groups based on the auxiliary information. Each track group group may indicate a mutual relationship between one or more track groups within the track group group.

[0106] In CMAF 317, "switching sets" 320 may be grouped into "aligned switching sets" 318 or "CMAF selection sets" 319. In order to represent aligned switching sets on the ISOBMFF level, it may be necessary to construct groupings of groups 310. In an embodiment, one or more track groupings may thus correspond to one or more CMAF selection sets 319. Alternatively, one or more track groupings may correspond to one or more CMAF aligned switching sets 318.

[0107] In an embodiment, for each of the one or more track groupings, the ISOBMFF file 301, 401 may further include a track grouping specific box 310, 404 including a list of references to track groups belonging to the track grouping 405. Each of the one or more track grouping specific boxes 310, 404 may be instantiated by a predefined track group entry type of a track group entry box 308 included in an ISOBMFF track group description box 304, 402. Each of the one or more track grouping specific boxes 310, 404 may indicate their respective track group identifier 405, which enables reference to the track grouping in the form of a track group.

[0108] Using the AdaptationSetGroup concept as described above can be achieved by defining a new derivative of TrackGroupEntryBox308, 404, referred to in this document in an illustrative and non-binding manner as GroupOfGroups310, 404, such as Figure 3 and Figure 4A new GroupOfGroups box 310, 404 may carry a track_group_ids list 406. Each AdaptationSetGroup with an ID that appears in the track_group_ids list may be part of a "group of groups" 310, 404. Such a grouping of groups may signal a switch set or a selection set of alignments.

[0109] refer to Figure 4 and Figure 5 For example, Figure 4 The diagram shows an example with two AdaptationSets, each of which consists of two Representations. Figure 4 In the example, the ISOBMFF file 401 generated as described herein contains 4 tracks:

[0110] Track 1 407 and track 2 408 are grouped by track_group_type=NEW1 413 , with track_group_id=11 414 .

[0111] Track 3 409 and track 4 410 are grouped by track_group_type=NEW1, with track_group_id=12.

[0112] This new track_group_type="NEW1" indicates that the packet should be interpreted by the streaming packager to provide these tracks as different representations in the AdaptationSet / SwitchSet. The DASH manifest generated from the above example has the following structure (here: for ease of explanation, id is set to the same as Figure 4 ), their respective counterparts in:

[0113] Table 1 shows the Figure 4 The DASH manifest (Media Presentation Description MPD) is generated from the information contained / extracted from the ISOBMFF file shown in the example.

[0114]

[0115] Table 1

[0116] The new TrackGroupEntryType - "NEW2" 403 - in TrackGroupDescriptionbox 402 enhances the TrackGroup with additional attributes. For example, all attributes and sub-elements of DASH AdaptationSet may appear here as simple fields or sub-boxes and will not be described in detail here.

[0117] Further, in this example, a new track_group_entry_type="NEW3" 404 is shown together with track_group_id=21 405. This entry type is used to define a "grouping of groups"; in this example, it groups track_group_id=11 and track_group_id=12 into a new grouping of track_group_id=21.

[0118] As mentioned above, such a grouping of groups can be either a CMAF "selection set" or a CMAF "aligned switch set", depending on the assigned track_group_entry_type.

[0119] Figure 5 An example of DASH AdaptationSetSwitching as described herein is shown. Table 2 shows the Figure 5 The corresponding DASH manifest (Media Presentation Description MPD) is generated from the information contained / extracted in the ISOBMFF file shown in the example.

[0120]

[0121] Table 2

[0122] Switching across AdaptationSets

[0123] Another application could be to use this mechanism for MPEG DASH "switching across Adaptation Sets" as per section 5.3.3.5 in ISO / IEC 23009-1:2019. MPEG DASH defines special SupplementalPropertyDescriptors (schemeIdUri="urn:mpeg:dash:adaptation-set-switching:2016") to signal the possibility of seamlessly switching from one AdaptationSet (source) to another AdaptationSet (target). In MPEG DASH, this is modeled as a directed graph, i.e., each AdaptationSet can have a unique and different list of AdaptationSets that can be switched to.

[0124] refer to Figure 5 For example, for this use case, a derivative of TrackGroupEntryBox 508 (referred to as SwitchingDG in this document in a non-binding manner and only for the purpose of explanation) may be defined as follows (similar to Figure 3 The AdaptationSetSwitchbox 510 may be derived from the TrackGroupEntryBox 308 and may carry a single track_group_id to identify the AdaptationSet it applies to, and a list of track_group_ids. Each entry in this list may represent an AdaptationSet that may be seamlessly switched. This structure provides all the information needed to write a special SupplementalPropertyDescriptor 515 in the DASH manifest 511.

[0125] Switching across representations

[0126] A similar dependency mechanism is defined in MPEG DASH through the @dependencyId and @associationId attributes, which define dependencies between Representations (according to Section 5.3.5 of ISO / IEC 23009-1:2019).

[0127] MPEG DASH defines dependent Representations (i.e., Representations that depend on other Representations) as "regular Representations, but they depend on a set of complementary Representations for decoding and / or rendering". Signaling is defined as "@dependencyId contains the value of the @id attribute of all complementary Representations (i.e., the Representations that are required for rendering and / or decoding the media content component contained in the dependent Representation)". This can be modeled as a directed acyclic graph, i.e., each Representation can have a unique and different list of Representations that it can depend on.

[0128] For this use case, another derivative of TrackGroupEntryBox 308 (referred to as DependencyDAG in this document in a non-binding manner and only for explanation purposes) is defined as follows (similar to GroupOfGroups box 310): DependencyDAG box can be derived from TrackGroupEntryBox 308 and can carry a single track_group_id to identify the track whose dependencies are listed, and a list of track_group_ids that are tracks. This structure provides all the information needed to write the @dependencyId attribute in the DASH manifest.

[0129] MPEG DASH defines associated Representations as “Representations that provide information about their relationship to other Representations.” This concept is very similar to dependency representations and can therefore be modeled with a similar derivative of TrackGroupEntryBox 308 .

[0130] Generalization of grouping of groups

[0131] This grouping can be defined only for a specific track_group_type (i.e. SwitchingSetGroup), or generalized to any type of track_groups. The type of the "group of groups" can be explicitly specified by a type identifier (i.e. track_group_entry_type), or implicitly given by the type of the member group, or indicated by a to-be-defined element within a new GroupOfGroup Box. If this new "GroupOfGroup" box carries its own track_group_id, further cascading can be done.

[0132] Reference now Figure 7, which illustrates an example of a method 700 for processing an ISO base media file format ISOBMFF file.

[0133] In step S701, a (single) ISOBMFF file is received (e.g. by a receiver, which may be part of a segmenter), the ISOBMFF file comprising one or more tracks of media content and one or more boxes characterizing interrelationships between some or all of the one or more tracks. In step S702, information about each interrelationship between some or all of the one or more tracks is extracted from the one or more boxes.

[0134] In step S703, the ISOBMFF file is segmented based on the extracted information to obtain a plurality of fragments.

[0135] The plurality of segments are then output in step S704. The ISOBMFF segments may be output for transmission (e.g., via the Internet) and / or streaming. For example, the ISOBMFF segments may be uploaded to a server (e.g., an origin server of a content delivery network (CDN)), and / or provided to a multicast server or broadcast transmitter.

[0136] In an embodiment, the interrelationships may be indicated by one or more track group boxes, thereby grouping one or more tracks into track groups corresponding to the respective interrelationships. As described above, a track group may correspond to a DASH AdaptationSet, and one or more tracks within a track group correspond to one or more DASH representations within a DASH adaptation set. In the case of a DASH AdaptationSet, a track group may correspond to a DASH AdaptationSet with the possibility of switching playback from an identified source DASH AdaptationSet to a set of target DASH AdaptationSets. Alternatively, a track group may correspond to a CMAF switch set, and one or more tracks within a track group correspond to one or more CMAF tracks within a CMAF switch set.

[0137] In an embodiment, for each of one or more tracks within a track group, the corresponding mutual relationship may be indicated by a track group type specific box indicating a predefined and specific track group type and track group identifier. Tracks with the same track group type and track group identifier may belong to the same track group, and the track group type specific box is included in the ISOBMFF track group box.

[0138] In an embodiment, for each of one or more tracks within a track group, the corresponding relationship may be indicated by a grouping type specific box indicating a predefined and specific grouping type and grouping identifier, wherein the grouping type specific box may be included in an ISOBMFF grouping list box.

[0139] In the case of DASH adaptation sets with switching possibilities, for each of said switching possibilities the corresponding mutual relationship may be further indicated by a track group entry box of a predefined type whose track group identifier is equal to the track group identifier identifying the track group of the source DASH AdaptationSet and whose track group identifier list is equal to those track group identifiers identifying the track group of the target DASH AdaptationSet.

[0140] In an embodiment, the interrelationships may be further indicated by grouping one or more track groups into one or more track groupings. Each track grouping may indicate interrelationships between one or more track groups within the track grouping. In an embodiment, the track groupings may correspond to CMAF selection sets. Alternatively, the track groupings may correspond to CMAF aligned switch sets.

[0141] In an embodiment, the mutual relationship may be indicated by one or more track group grouping specific boxes, each track group grouping specific box comprising a list of references to track groups belonging to the track group grouping. Each of the one or more track group grouping specific boxes may be instantiated by a predefined track group entry type of a track group entry box included in an ISOBMFF track group description box. Each of the one or more track group grouping specific boxes may indicate its respective track group identifier, which enables reference to the track group grouping in the form of a track group.

[0142] In an embodiment, the method may further include generating a manifest file based on the ISOBMFF file and outputting the manifest file together with the plurality of fragments. The manifest file may include information extracted from one or more boxes about each interrelationship between some or all of the one or more tracks. As described above and Figure 5 As shown in the example of , by generating an enhanced ISOBMFF file, it is no longer necessary to signal additional auxiliary information for generating a manifest file. The manifest file may be a DASH media presentation description (MPD) file.

[0143] refer to Figure 8As an example, in an embodiment, segmenting an ISOBMFF file may include generating one or more DASH initialization segments 801, 811, 812, 813, which include information extracted from one or more boxes 802-810 about each relationship between some or all of one or more tracks.

[0144] The DASH MPD file may include information included in the DASH initialization segment. Alternatively, the DASH MPD file may include information about each mutual relationship between some or all of one or more tracks based on the ISOBMFF file, but the DASH initialization segment does not include the mutual relationship information.

[0145] refer to Fig. 9 , illustrating a system of an apparatus 900 for processing a media stream and an apparatus 920 for processing an ISO base media file format ISOBMFF file.

[0146] The apparatus for processing a media stream may be implemented, for example, as an encoder configuration 900 including four encoders 901 , 902 , 903 , and 904 and a multiplexer 906 .

[0147] The apparatus for processing an ISO base media file format ISOBMFF file may be implemented as, for example, a stream packager (streaming media packager) 920 , which includes a segmenter 921 and a manifest generator 922 .

[0148] Multiplexer 906 receives one or more (in this case four) media streams from encoders 901, 902, 903, and 904. The four media streams include (contain) media content (i.e., video and audio) that includes (is divided into) one or more tracks. The video content can be encoded at different bit rates (high, medium, low).

[0149] The multiplexer 906 further receives auxiliary information 905 for one or more media streams, the auxiliary information characterizing the relationship between some or all of the one or more tracks, and generates an ISO base media file format ISOBMFF file (MP4) 907 based on the one or more media streams and the auxiliary information. The multiplexer 906 outputs the generated ISOBMFF file for further processing by the stream packager 920.

[0150] The segmenter 921 receives an ISOBMFF file 907, which includes one or more tracks of media content and one or more boxes characterizing the relationship between some or all of the one or more tracks. The segmenter 921 extracts information about each relationship between some or all of the one or more tracks from the one or more boxes, and segments the ISOBMFF file 907 based on the extracted information to obtain multiple segments 924, 925. The multiple segments may include an initialization segment 924 and a media segment 925. The multiple segments are output for streaming. For example, the multiple segments 924, 925 may be output to a streaming media server or provided to a multicast server or a broadcast transmitter.

[0151] exist Fig. 9 In the example of , the manifest generator 922 further generates a manifest file 923 based on the initialization segment 924, and outputs the manifest file together with the plurality of fragments 924, 925 for streaming transmission.

[0152] In the case of a DASH MPD file, the DASH MPD file may include information included in the DASH initialization segment. Alternatively, the DASH MPD file may include information about each mutual relationship between some or all of one or more tracks based on the ISOBMFF file, but the DASH initialization segment does not include the mutual relationship information.

[0153] It is worth noting that the methods described herein may also be implemented as a program comprising instructions which, when executed by a processor, cause the processor to perform the methods. Fig.10 A corresponding example of a device 1000 is illustrated, the device comprising a memory 1001 and a processor 1002. The program may be stored on a computer-readable storage medium. In other embodiments, the device may have multiple processors.

[0154] A computing device implementing the techniques described above may have the following example architecture. Other architectures are also possible, including architectures with more or fewer components. In some embodiments, the example architecture includes one or more processors (e.g., a dual-core The components may include a processor, one or more output devices (e.g., an LCD), one or more network interfaces, one or more input devices (e.g., a mouse, a keyboard, a touch-sensitive display), and one or more computer-readable media (e.g., RAM, ROM, SDRAM, a hard disk, an optical disk, flash memory, etc.). These components may exchange communications and data via one or more communication channels (e.g., buses), which may utilize various hardware and software to facilitate the transfer of data and control signals between the components.

[0155] The term "computer-readable medium" refers to media that participate in providing instructions to a processor for execution, including but not limited to non-volatile media (e.g., optical or magnetic disks), volatile media (e.g., memory), and transmission media. Transmission media include but are not limited to coaxial cables, copper wire, and fiber optics.

[0156] The computer readable medium may further include an operating system (e.g., The operating system may include an operating system, an audio interface manager, an audio processing manager, and a real-time content distributor. The operating system may be multi-user, multi-processing, multi-tasking, multi-threaded, real-time, etc. The operating system performs basic tasks including, but not limited to: identifying input from and providing output to network interfaces and / or devices; tracking and managing files and directories on computer-readable media (e.g., memory or storage devices); controlling peripheral devices; and managing services on one or more communication channels. The network communication module includes various components for establishing and maintaining network connections (e.g., software for implementing communication protocols such as TCP / IP, HTTP, etc.).

[0157] The architecture may be implemented in a parallel processing or peer-to-peer infrastructure, or on a single device having one or more processors.The software may include multiple software components or may be a single body of code.

[0158] The described features can be advantageously implemented in one or more computer programs that can be executed on a programmable system including at least one programmable processor coupled to receive data and instructions from and transmit data and instructions to a data storage system, at least one input device, and at least one output device. A computer program is a set of instructions that can be used directly or indirectly in a computer to perform a certain activity or bring about a certain result. A computer program can be written in any form of programming language (e.g., Objective-C, Java), including compiled or interpreted languages, and it can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, browser-based web application, or other unit suitable for use in a computing environment.

[0159] Suitable processors for executing a program of instructions include, by way of example, both general-purpose and special-purpose microprocessors, and a single processor or one of multiple processors or cores of any kind of computer. Typically, the processor will receive instructions and data from a read-only memory or a random access memory or both. The essential elements of a computer are a processor for executing instructions and one or more memories for storing instructions and data. Typically, a computer will also include, or be operably coupled to communicate with, one or more mass storage devices for storing data files; such devices include magnetic disks, such as internal hard disks and removable disks; magneto-optical disks; and optical disks. Storage devices suitable for tangibly embodying computer program instructions and data include all forms of non-volatile memory, which include, by way of example, semiconductor memory devices, such as EPROM, EEPROM, and flash memory devices; magnetic disks, such as internal hard disks and removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks. The processor and memory may be supplemented by or incorporated into an ASIC (Application Specific Integrated Circuit).

[0160] To provide interaction with a user, features may be implemented on a computer having a display device for displaying information to the user, such as a CRT (cathode ray tube) or LCD (liquid crystal display) monitor or retina display device. The computer may have a touch surface input device (e.g., a touch screen) or a keyboard and a pointing device such as a mouse or trackball, through which the user can provide input to the computer. The computer may have a voice input device for receiving voice commands from the user.

[0161] Features may be implemented in a computer system that includes back-end components, such as a data server, or includes middleware components, such as an application server or an Internet server, or includes front-end components, such as a client computer with a graphical user interface or an Internet browser, or any combination thereof. The components of the system may be connected by any form or medium of digital data communication, such as a communication network. Examples of communication networks include, for example, LANs, WANs, and the computers and networks that form the Internet.

[0162] A computing system may include a client and a server. The client and the server are usually remote from each other and typically interact through a communication network. The relationship between the client and the server is due to the computer program running on the respective computers and having a client-server relationship relative to each other. In some embodiments, the server transmits data (e.g., an HTML page) to a client device (e.g., in order to display data to a user interacting with the client device and receive user input from the user). Data generated at the client device (e.g., the result of a user interaction) can be received from the client device at the server.

[0163] A system of one or more computers may be configured to perform specific actions by virtue of having software, firmware, hardware, or a combination thereof installed on the system that causes the system to perform the actions in operation. One or more computer programs may be configured to perform specific actions by virtue of including instructions that, when executed by a data processing device, cause the device to perform the actions.

[0164] Although this specification contains many specific implementation details, these details should not be interpreted as limitations on the scope of any invention or what may be claimed, but rather, are regarded as descriptions of features that are peculiar to a particular embodiment of a particular invention. Specific features described in the context of a separate embodiment in this specification may also be implemented in combination in a single embodiment. Conversely, the various features described in the context of a single embodiment may also be implemented in multiple embodiments individually or in the form of any suitable sub-combination. In addition, although features may be described above as acting in a particular combination and even initially claimed, one or more features from the claimed combination may be deleted from the combination in some cases, and the claimed combination may be directed to a sub-combination or a variant of a sub-combination.

[0165] Similarly, although operations are depicted in a particular order in the accompanying drawings, this should not be understood as requiring such operations to be performed in the particular order shown or in a sequential order, or performing all illustrated operations to achieve desired results. In some cases, multitasking and parallel processing can be advantageous. In addition, the separation of various system components in the embodiments described above should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems can be generally integrated together in a single software product or encapsulated in multiple software products.

[0166] Unless otherwise specifically stated, it will be apparent from the following discussion that it should be understood that throughout the disclosed discussion, terms such as "processing," "computing," "calculating," "determining," "analyzing," etc. are utilized to refer to the actions and / or processes of a computer or computing system or similar electronic computing device to manipulate and / or transform data represented as physical (e.g., electronic) quantities into other data similarly represented as physical quantities.

[0167] References throughout the disclosure to "an example embodiment," "some example embodiments," or "example embodiments" mean that a particular feature, structure, or characteristic described in connection with the example embodiment is included in at least one example embodiment of the present disclosure. Thus, various appearances of the phrases "in an example embodiment," "in some example embodiments," or "in an example embodiment" throughout the disclosure are not necessarily all referring to the same example embodiment. Furthermore, in one or more example embodiments, the particular features, structures, or characteristics may be combined in any suitable manner, as will be apparent to one of ordinary skill in the art from this disclosure.

[0168] As used herein, unless otherwise specified, the use of ordinal adjectives "first," "second," "third," etc. to describe common objects merely indicates that different instances of similar objects are being referred to and is not intended to imply that the objects described must be in a given order in time, space, rank, or in any other manner.

[0169] Likewise, it should be understood that the phraseology and terminology used herein is for descriptive purposes and should not be regarded as limiting. The use of "including," "comprising," or "having" and variations thereof are meant to encompass the items listed thereafter and equivalents thereof as well as additional items. Unless otherwise specified or limited, the terms "mounted," "connected," "supported," and "coupled" and variations thereof are used broadly and encompass direct and indirect mountings, connections, supports, and couplings.

[0170] In the claims below and the description herein, any of the terms comprising, including, or which comprises is an open term, which means at least including the elements / features that follow, but not excluding other elements / features. Therefore, when the term comprising is used in a claim, the term should not be interpreted as being limited to the devices or elements or steps listed thereafter. For example, the scope of the expression "a device comprising A and B" should not be limited to a device comprising only elements A and B. As used herein, the term including, or any of which includes, or that includes, is also an open term, which also means at least including the elements / features that follow the term, but not excluding other elements / features. Therefore, including is synonymous with comprising and means comprising.

[0171] It should be understood that in the above description of the example embodiments of the present disclosure, various features of the present disclosure are sometimes combined together in a single example embodiment, figure, or description thereof to simplify the present disclosure and to help understand one or more of the various creative aspects. However, the method of the present disclosure should not be interpreted as reflecting the intention that the claims require more features than the features explicitly recited in each claim. On the contrary, as reflected in the following claims, each creative aspect lies in less than all the features of a single previously disclosed example embodiment. Therefore, the claims following the specification are hereby expressly incorporated into this specification, wherein each claim independently serves as a separate example embodiment of the present disclosure.

[0172] In addition, although some example embodiments described herein include some features included in other example embodiments and do not include other features included in other example embodiments, as will be understood by those skilled in the art, the combination of features of different example embodiments is intended to be within the scope of the present disclosure and form different example embodiments. For example, in the appended claims, any of the example embodiments claimed for protection may be used in any combination.

[0173] In the description provided herein, many specific details are set forth. However, it should be understood that the exemplary embodiments of the present disclosure can be practiced without these specific details. In other instances, well-known methods, structures, and techniques are not shown in detail in order to avoid obscuring the understanding of this specification.

[0174] Therefore, although the mode that is considered to be the best mode of the present disclosure has been described, it will be recognized by those skilled in the art that other and further modifications may be made thereto without departing from the spirit of the present disclosure, and it is intended to claim all such changes and modifications that fall within the scope of the present disclosure. For example, any formula given above merely represents a program that may be used. Functions may be added or deleted from the block diagram, and operations may be interchanged among functional blocks. Steps may be added or deleted to the methods described within the scope of the present disclosure.

Claims

1. A method for processing a media stream, the method comprising: Receiving one or more media streams and auxiliary information for the one or more media streams, the one or more media streams comprising media content, the media content comprising one or more tracks; Generate an ISO base media file format ISOBMFF file based on the one or more media streams and the auxiliary information; as well as Output the generated ISOBMFF file for further processing.

2. The method of claim 1, wherein: The auxiliary information indicates interrelationships between some or all of the one or more tracks.

3. The method according to claim 1 or 2, wherein: Generating the ISOBMFF file includes grouping some or all of the one or more tracks into one or more track groups.

4. The method of claim 3, wherein: The one or more track groups correspond to one or more DASH AdaptationSets, and the one or more tracks in each track group correspond to one or more DASH representations in the DASH AdaptationSet.

5. The method of claim 4, wherein: The one or more track groups correspond to one or more DASH AdaptationSets having a possibility of switching playback from an identified source DASH AdaptationSet to a set of target DASH AdaptationSets.

6. The method of claim 3, wherein: The one or more track groups correspond to one or more CMAF switch sets, and the one or more tracks in each track group correspond to one or more CMAF tracks in the CMAF switch set.

7. The method according to any one of claims 3 to 6, wherein: The ISOBMFF file comprises: for each of the one or more tracks within the track group, a track group type specific box indicating a predefined and specific track group type and a track group identifier, wherein tracks characterized as having the same track group type and track group identifier belong to the same track group, and the track group type specific box is included in the ISOBMFF track group box.

8. The method according to any one of claims 3 to 6, wherein: The ISOBMFF file comprises: for each track of the one or more tracks within the track group, a packet type specific box indicating a predefined and specific packet type and a packet identifier, wherein the packet type specific box is included in an ISOBMFF packet list box.

9. A method as claimed in claim 7 or 8 when dependent on claim 5, wherein The ISOBMFF file further comprises: a track group entry box of a predefined type for each of the switching possibilities, and its track group identifier and a track group identifier list, wherein the track group identifier of the track group entry box is equal to the track group identifier identifying the track group of the source DASH AdaptationSet, and the track group identifier list is equal to those track group identifiers identifying the track group of the target DASH AdaptationSet.

10. The method according to any one of claims 3 to 9, wherein: Generating the ISOBMFF file further includes grouping the one or more track groups into one or more track group groups based on the auxiliary information.

11. The method of claim 10, wherein: Each track grouping indicates a relationship between the one or more track groups within the track grouping.

12. A method as claimed in claim 10 or 11 when dependent on claim 6, wherein The one or more track groupings correspond to one or more CMAF selection sets.

13. A method as claimed in claim 10 or 11 when dependent on claim 6, wherein The one or more track group groups correspond to one or more CMAF-aligned switching sets.

14. The method according to any one of claims 10 to 13, wherein: For each track group group of the one or more track group groups, the ISOBMFF file further comprises a track group group specific box comprising a list of references to track groups belonging to the track group group.

15. The method of claim 14, wherein: Each of the one or more track group group specific boxes is instantiated by a predefined track group entry type of a track group entry box included in an ISOBMFF track group description box.

16. The method according to claim 14 or 15, wherein: Each of the one or more track group specific boxes indicates their respective track group identifier, which enables reference to the track group as a track group.

17. The method according to any one of claims 1 to 16, wherein: The auxiliary information is not output with the ISOBMFF file for further processing.

18. A method for processing an ISO base media file format (ISOBMFF) file, the method comprising: receiving an ISOBMFF file comprising one or more tracks of media content and one or more boxes characterizing interrelationships between some or all of the one or more tracks; extracting from the one or more boxes information about each interrelationship between the some or all of the one or more tracks; Segmenting the ISOBMFF file based on the extracted information to obtain a plurality of fragments; as well as The plurality of fragments are output.

19. The method of claim 18, wherein: The interrelationships are indicated by one or more track group boxes, thereby grouping one or more tracks into track groups corresponding to the respective interrelationships.

20. The method of claim 19, wherein: The track group corresponds to a DASH AdaptationSet, and the one or more tracks in the track group correspond to one or more DASH representations in the DASH AdaptationSet.

21. The method of claim 20, wherein: The track groups correspond to DASH AdaptationSets with the possibility to switch playback from an identified source DASH AdaptationSet to a set of target DASH AdaptationSets.

22. The method of claim 19, wherein: The track group corresponds to a CMAF switching set, and the one or more tracks in the track group correspond to one or more CMAF tracks in the CMAF switching set.

23. The method of any one of claims 19 to 22, wherein: For each of the one or more tracks within a track group, the respective mutual relationships are indicated by a track group type specific box indicating the predefined and specific track group type and track group identifier, wherein tracks characterized as having the same track group type and track group identifier belong to the same track group, and the track group type specific box is included in the ISOBMFF track group box.

24. The method of any one of claims 19 to 22, wherein: For each of one or more tracks within a track group, the corresponding relationship is indicated by a grouping type specific box, which indicates the predefined and specific grouping type and grouping identifier, wherein the grouping type specific box is included in the ISOBMFF grouping list box.

25. A method as claimed in claim 23 or 24 when appended to claim 21, wherein For each of the switching possibilities, the respective interrelationship is further indicated by a track group entry box of a predefined type using its track group identifier and a track group identifier list, wherein the track group identifier of the track group entry box is equal to the track group identifier identifying the track group of the source DASH AdaptationSet, and the track group identifier list is equal to those track group identifiers identifying the track group of the target DASH AdaptationSet.

26. The method of any one of claims 19 to 25, wherein: The interrelationship is further indicated by grouping the one or more track groups into one or more track group groups.

27. The method of claim 26, wherein: Each track grouping indicates a relationship between the one or more track groups within the track grouping.

28. A method as claimed in claim 26 or 27 when dependent on claim 22, wherein The track grouping corresponds to a CMAF selection set.

29. A method as claimed in claim 26 or 27 when dependent on claim 22, wherein The track groupings correspond to CMAF-aligned switch sets.

30. The method of any one of claims 26 to 29, wherein: The mutual relationships are indicated by one or more track group specific boxes, each track group specific box comprising a list of references to track groups belonging to the track group.

31. The method of claim 30, wherein: Each of the one or more track group specific boxes is instantiated by a predefined track group entry type of a track group entry box included in an ISOBMFF track group description box.

32. The method of claim 30 or 31, wherein: Each of the one or more track group specific boxes indicates its respective track group identifier, which enables reference to the track group as a track group.

33. The method of any one of claims 18 to 32, wherein: The method further includes generating a manifest file based on the ISOBMFF file and outputting the manifest file together with the plurality of fragments.

34. The method of claim 33, wherein: The manifest file includes information extracted from the one or more boxes regarding each interrelationship between the some or all of the one or more tracks.

35. The method of claim 33 or 34, wherein: The manifest file is a DASH media presentation description MPD file.

36. The method of any one of claims 18 to 35, wherein: The segmenting of the ISOBMFF file includes generating one or more DASH initialization segments, the DASH initialization segments including information extracted from the one or more boxes about each interrelationship between some or all of the one or more tracks.

37. A method as claimed in claim 36 when appended to claim 35, wherein The DASH MPD file includes the information included in the DASH initialization segment.

38. A method as claimed in claim 36 when appended to claim 35, wherein: The DASH MPD file includes information about each mutual relationship between some or all of the one or more tracks based on the ISOBMFF file, but the DASH initialization segment does not include the mutual relationship information.

39. An apparatus for processing a media stream, the apparatus comprising one or more processors, the one or more processors configured to perform a method comprising: Receiving one or more media streams and auxiliary information for the one or more media streams, the one or more media streams comprising media content, the media content comprising one or more tracks; Generate an ISO base media file format ISOBMFF file based on the one or more media streams and the auxiliary information; as well as Output the generated ISOBMFF file for further processing.

40. The apparatus of claim 39, wherein: The apparatus includes a multiplexer.

41. An apparatus for processing an ISO base media file format (ISOBMFF) file, the apparatus comprising one or more processors, the one or more processors being configured to perform a method comprising the following operations: Receiving an ISOBMFF file, the ISOBMFF file comprising one or more tracks of media content and one or more boxes characterizing interrelationships between some or all of the one or more tracks; extracting from the one or more boxes information about each interrelationship between the some or all of the one or more tracks; Segmenting the ISOBMFF file based on the extracted information to obtain a plurality of fragments; as well as The plurality of fragments are output.

42. The apparatus of claim 41, wherein: The apparatus includes a segmenter.

43. The device of claim 41 or 42, wherein: The one or more processors are further configured to generate a manifest file based on the ISOBMFF file and output the manifest file together with the plurality of fragments.

44. The apparatus of claim 43, wherein: The apparatus further includes a manifest generator.

45. A system comprising an apparatus for processing a media stream according to claim 39 or 40 and an apparatus for processing an ISO base media file format (ISOBMFF) file according to any one of claims 41 to 44.

46. ​​A program comprising instructions which, when executed by a processor, cause the processor to perform the method according to any one of claims 1 to 38.

47. A computer-readable storage medium storing the program according to claim 46.

Images