Method, apparatus, and system for improved grouping of tracks in an ISOBMFF file

Abstract

This specification describes a method for processing media streams. The method includes receiving one or more media streams and side information about the one or more media streams, where the one or more media streams include media content including one or more tracks, generating an ISO Base Media File Format (ISOBMFF) file based on the one or more media streams and the side information, and outputting the generated ISOBMFF file for further processing. Further described are respective methods, respective apparatuses, and computer program products for processing the ISOBMFF file.

Description

Technical Field

[0001] [Cross - Reference to Related Applications] This application claims priority based on U.S. Provisional Patent Application No. 63 / 394,027, filed on August 01, 2022, and U.S. Provisional Patent Application No. 63 / 417,426, filed on October 19, 2022, both of which are hereby incorporated by reference in their entirety.

[0002] [Technical Field] The present disclosure generally relates to a method for processing media streams. The method particularly includes generating an ISO Base Media File Format (ISOBMFF) file based on one or more media streams and side information. The present disclosure further relates to a method for processing an ISOBMFF file, respective apparatuses, and computer program products.

[0003] In this specification, some embodiments are described with particular reference to its disclosure, but it will be understood that the present disclosure is not limited to such fields of use and is applicable in a broader context.

Background Art

[0004] Any discussion of background art throughout this disclosure should not be construed as an admission that such technology is widely known or forms part of the common general knowledge in the art.

[0005] Modern video compression schemes can generally take advantage of spreading the overall available media data across several streams for various reasons, including the potential to save transmission bandwidth.

[0006] In the case of streaming applications, one media asset can be encoded into a stream using different bitrates. This set of streams is grouped to enable dynamic switching between these streams to react to changing network conditions and is referred to as "Representations" in HTTP-based Dynamic Adaptive Streaming over HTTP (DASH) as defined in ISO / IEC 23009 by the Motion Pictures Expert Group (MPEG), and as "CMAF tracks" in the Common Media Application Format for Segmented Media (CMAF) as specified by the MPEG in ISO / IEC 23000-19. The grouping is referred to as "AdaptationSet" in DASH and "Switching Set" in CMAF.

[0007] The International Organization for Standardization (ISO) defines specifications including the Base Media File format, generally known as ISOBMFF. In particular, this ISOBMFF is specified by ISO / IEC 14496-12 MPEG-4 Part 12 by the ISO. Generally, ISOBMFF is a container file format that defines a common structure for time-based multimedia files such as video and / or audio.

[0008] In a typical setup, a group of one or more encoders encodes an input signal onto a track at different bitrates. Further, if there are several variations of the input signal (for example, varying in the spoken language or through the addition of spoken subtitles), some groups of these encoder groups will operate in parallel to create variations at all the required bitrates. In current typical implementations, the encoders will each output an ISOBMFF with one ISOBMFF track.

[0009] ISOBMFF tracks can be grouped using a derivative of TrackGroupTypeBoxes as defined in ISO / IEC 14496-12 MPEG-4 Part 12 section 8.3.4. The type of grouping is specified by the track_group_type identifier. By selecting different track_group_type identifiers, different types of grouping with different semantics can be defined.

[0010] Rather than listing all the grouped tracks in a list within the TrackGroupTypeBox to define a group of tracks, the grouping is achieved by the implicit construction of equivalence classes. All tracks that are part of a particular group have a TrackGroupTypeBox with the same type and id identifiers. In other words, and only if, two tracks contain such a derivative of a TrackGroupTypeBox with the same type and id identifiers, then both tracks belong to the same group.

[0011] A first mechanism for collating the properties of track groups is currently under development in MPEG: The TrackGroupDescriptionBox serves as a container for TrackGroupEntryBox and its derivatives.

[0012] The second mechanism in ISOBMFF for grouping tracks uses a derivative of EntityToGroupBoxes as defined in ISO / IEC 14496-12 MPEG-4 Part 12 section 8.18.3. The type of grouping is specified by the grouping_type identifier. By selecting different grouping_type identifiers, different types of grouping with different semantics can be defined.

[0013] A group of tracks (e.g., tracks using the entity grouping mechanism) may use an instance of EntityToGroupBox to reference all tracks through their respective track_ids.

[0014] The third mechanism in ISOBMFF for grouping tracks uses the alternate_group field in the track header box as defined in ISO / IEC 14496-12 MPEG-4 Part 12 section 8.3.2. There is no available grouping type for this type of grouping. Grouping is achieved by assigning the same identifier to the alternate_group field of all tracks in the group, where an identifier of "0" means that each track does not belong to any group.

[0015] Furthermore, to prepare ISOBMFF tracks for segmented transmission, each of these tracks is subject to the action of a segmenter that divides the track into a stream of short-duration segments where each segment typically represents a few seconds. Finally, the DASH manifest generator describes the composition of the segmented stream in a manifest file. The manifest file contains high-level descriptions of the different streams (i.e., language, accessibility properties, bitrate, etc.) and their interrelationships.

[0016] However, in order to be able to signal dependencies between groups of tracks, further derived forms are still needed. SUMMARY OF THE INVENTION MEANS FOR SOLVING THE PROBLEM

[0017] According to a first aspect of the present disclosure, a method for processing a media stream is provided. The method comprises receiving one or more media streams and side information for the one or more media streams, wherein the one or more media streams comprise media content including one or more tracks (e.g., divided into one or more tracks). The method may further comprise generating an ISO Base Media File Format (ISOBMFF) file based on the one or more media streams and the side information. The method may further comprise outputting the generated ISOBMFF file for further processing. In particular, the side information used to generate the ISOBMFF file is not output for further processing.

[0018] Configured as described above, the method enables, for example, holding all tracks of media streams having different bitrates resulting from respective encoding processes in a single ISOBMFF file, and thus signaling dependencies not only between tracks but also between groups of tracks. On the other hand, the side information in this framework is not required by downstream devices.

[0019] In some embodiments, the side information may indicate the interrelationship between some or all of the one or more tracks.

[0020] In some embodiments, generating the ISOBMFF file may include grouping some or all of the one or more tracks into one or more track groups.

[0021] In some embodiments, one or more track groups may correspond to one or more DASH AdaptationSets, and one or more tracks within each track group may correspond to one or more DASH Representations within the DASH AdaptationSet.

[0022] In some embodiments, one or more track groups may correspond to one or more DASH AdaptationSets having the potential to switch playback from an identified source DASH AdaptationSet to a set of destination DASH AdaptationSets.

[0023] In some embodiments, one or more track groups may correspond to one or more CMAF switching sets, and one or more tracks within each track group may correspond to one or more CMAF tracks within the CMAF switching set.

[0024] In some embodiments, the ISOBMFF file may include, for each of one or more tracks within a track group, a track group type specific box indicating a predefined specific track group type and a track group identifier, tracks characterized by the same track group type and track group identifier may belong to the same track group, and the track group type specific box is included within the ISOBMFF track group box.

[0025] In some embodiments, the ISOBMFF file may include, for each of one or more tracks within a track group, a grouping type specific box indicating a predefined specific grouping type and a group identifier, and the grouping type specific box may be included within the ISOBMFF group list box.

[0026] In some embodiments, the ISOBMFF file may further include a track group entry box of a predefined type for each of the switching capabilities, wherein the track group identifier is equal to the track group identifier of the track group that identifies the source DASH AdaptationSet, and the list of track group identifiers is equal to those track group identifiers of the track groups that identify the destination DASH AdaptationSet.

[0027] In some embodiments, generating the ISOBMFF file may further include grouping one or more track groups into one or more groups of track groups based on side information.

[0028] In some embodiments, each group of track groups may indicate the interrelationship between one or more track groups within the group of track groups.

[0029] In some embodiments, one or more groups of track groups may correspond to one or more CMAF selection sets.

[0030] In some embodiments, one or more groups of track groups may correspond to one or more CMAF aligned switching sets.

[0031] In some embodiments, the ISOBMFF file may further include a track group group-specific box that includes a list of references to the track groups belonging to the group of track groups for each of the one or more groups of track groups.

[0032] In some embodiments, each of one or more groups of track group identification boxes may be instantiated by a predefined track group entry type of a track group entry box included in an ISOBMFF track group description box.

[0033] In some embodiments, each of one or more groups of track group identification boxes may indicate their respective track group identifiers, which enable a group of track groups to be referenced as track groups.

[0034] In some embodiments, side information may not be output in parallel with the ISOBMFF file for further processing.

[0035] According to a second aspect of the present disclosure, a method for processing an ISO Base Media File Format (ISOBMFF) file is provided. The method may include receiving an ISOBMFF file including one or more tracks of media content and one or more boxes characterizing interrelationships between some or all of the one or more tracks. The method may further include extracting information regarding each interrelationship between some or all of the one or more tracks from the one or more boxes. The method may further include segmenting the ISOBMFF file based on the extracted information to obtain a plurality of segments. The method may also include outputting the plurality of segments.

[0036] In some embodiments, the interrelationship may be indicated by one or more track group boxes that group one or more tracks into track groups corresponding to their respective interrelationships.

[0037] In some embodiments, a track group may correspond to a DASH AdaptationSet, and one or more tracks within the track group may correspond to one or more DASH Representations within the DASH AdaptationSet.

[0038] In some embodiments, a track group may correspond to a DASH AdaptationSet that has the potential ability to switch playback from an identified source DASH AdaptationSet to a set of destination DASH AdaptationSets.

[0039] In some embodiments, a track group may correspond to a CMAF switching set, and one or more tracks within the track group may correspond to one or more CMAF tracks within the CMAF switching set.

[0040] For each of one or more tracks within a track group in some embodiments, each respective relationship may be indicated by a track group type specific box that indicates a predefined specific track group type and a track group identifier, and tracks characterized using the same track group type and track group identifier may belong to the same track group, and the track group type specific box is included in the ISOBMFF track group box.

[0041] For each of one or more tracks within a track group in some embodiments, each respective relationship may be indicated by a grouping type specific box that indicates a predefined specific grouping type and a group identifier. The grouping type specific box may be included in the ISOBMFF group list box.

[0042] In some embodiments, for each switching potential, each of the respective relationships may be further indicated by a track group entry box of a predefined type, the track group identifier of which is equal to the track group identifier of the track group that identifies the source DASH AdaptationSet, and the list of track group identifiers is equal to the track group identifier of the track group that identifies the destination DASH AdaptationSet.

[0043] In some embodiments, the relationships may be further indicated by grouping one or more track groups into one or more groups of track groups.

[0044] In some embodiments, each group of track groups may indicate the relationship between one or more track groups within the group of track groups.

[0045] In some embodiments, the group of track groups may correspond to a CMAF selection set.

[0046] In some embodiments, the group of track groups may correspond to a CMAF aligned switching set.

[0047] In some embodiments, the relationships may be indicated by a specific box for one or more groups of track groups, each containing a list of references to the track groups belonging to the group of track groups.

[0048] In some embodiments, each of one or more groups of track group specific boxes may be instantiated by a predefined track group entry type of the track group entry box contained in the ISOBMFF track group description box.

[0049] In some embodiments, each of one or more groups of track group - specific boxes may indicate its respective track group identifier, and the track group identifier enables referring to the group of track groups as track groups.

[0050] In some embodiments, the method may further include generating a manifest file based on an ISOBMFF file and outputting the manifest file in parallel with a plurality of segments.

[0051] In some embodiments, the manifest file may include information regarding each inter - relationship among some or all of one or more tracks extracted from one or more boxes.

[0052] In some embodiments, the manifest file may be a DASH Media Presentation Description (MPD) file.

[0053] In some embodiments, segmenting the ISOBMFF file may include generating one or more DASH initialization segments, and the DASH initialization segments include information regarding each inter - relationship among some or all of one or more tracks extracted from one or more boxes.

[0054] In some embodiments, the DASH MPD file may include the information included in the DASH initialization segments.

[0055] In one embodiment, the DASH MPD file includes information regarding each inter - relationship among some or all of one or more tracks based on the ISOBMFF file, but the DASH initialization segments may exclude the inter - relationship information.

[0056] According to a third aspect of the present disclosure, an apparatus for processing media streams is provided. The apparatus is configured to cause one or more processors to perform a method including receiving one or more media streams and side information about the one or more media streams, where the one or more media streams include media content divided into one or more tracks, generating an ISO Base Media File Format (ISOBMFF) file based on the one or more media streams and the side information, and outputting the generated ISOBMFF file for further processing.

[0057] In some embodiments, the apparatus may comprise a Multiplexer.

[0058] According to a fourth aspect of the present disclosure, an apparatus for processing an ISO Base Media File Format (ISOBMFF) file is provided. The apparatus is configured to cause one or more processors to perform a method including receiving an ISOBMFF file including one or more tracks of media content and one or more boxes characterizing interrelationships between some or all of the one or more tracks, extracting information regarding each interrelationship between some or all of the one or more tracks from the one or more boxes, segmenting the ISOBMFF file based on the extracted information to obtain a plurality of segments, and outputting the plurality of segments.

[0059] In some embodiments, the apparatus may comprise a Segmenter.

[0060] In some embodiments, the one or more processors may be further configured to generate a manifest file based on the ISOBMFF file and output the manifest file in parallel with the plurality of segments.

[0061] In some embodiments, the apparatus may further include a manifest generator (e.g., a so-called "dASher").

[0062] According to a fifth aspect of the present disclosure, a system of an apparatus for processing a media stream described herein and an apparatus for processing an ISOBMFF file described herein is provided.

[0063] According to a sixth aspect of the present disclosure, there is provided a program including instructions that, when executed by a processor, cause the processor to perform a method of processing a processing media stream described herein and / or a method of processing an ISOBMFF file.

[0064] According to a seventh aspect of the present disclosure, a computer-readable storage medium storing the program is provided.

Brief Description of the Drawings

[0065]

Fig. 1

Fig. 2

Fig. 3

Fig. 4

Fig. 5

Fig. 6

Fig. 7

Fig. 8-1

Fig. 8-2

Fig. 9

Fig. 10

[0066] Exemplary embodiments of the present disclosure will be described below by way of example only with reference to the accompanying drawings.

[0067] SUMMARY When streams having different bitrates result from their respective encoding processes, it can be beneficial to hold all the tracks within a single ISOBMFF file (i.e., a multi-track segment stream). If such a mechanism were defined, it could be used, for example, to represent DASH AdaptationSets and the Representations contained therein. However, to date, no signaling has been defined to group tracks into the above sets and show how a subsequent streaming packager would divide them into single-track segment streams for Internet streaming. CMAF aligned switching set and selection set are concepts that require not only grouping tracks but also constructing groups of groups of tracks. ISOBMFF can already signal dependencies between tracks, but does not provide a means to signal dependencies between groups of tracks as defined by CMAF.

[0068] In a typical setup, the encoder, segmenter, and manifest generator must be set up in the same configuration using a global control instance. These can be implemented in various software environments operating on various hardware platforms. This global configuration creates a data path for this configuration data, including the encoded media, in the same ISOBMFF container and makes all components stand-alone, imposing some constraints on the deployment of these components addressed by the present disclosure.

[0069] [First Example of Processing Media Streams and ISOBMFF Files] Referring to the example of FIG. 1, a system of an apparatus for processing a media stream and an apparatus for processing an ISO Base Media File Format (ISOBMFF) file is shown.

[0070] In this example, at the encoding side 100, one or more media streams are generated by encoding the same video content at three different bitrates, i.e., at a high bitrate in encoder 1, 101, at a medium bitrate in encoder 2, 102, and at a low bitrate in encoder 3, 103. The accompanying audio content is encoded in parallel with the video content in encoder 4, 104.

[0071] The media streams thus generated are then processed individually in their respective multiplexers. Mux 1, 106 processes a media stream containing one or more tracks of video content encoded at a high bitrate, Mux 2 (107) processes a media stream containing one or more tracks of video content encoded at a medium bitrate, Mux 3 (108) processes a media stream containing one or more tracks of video content encoded at a low bitrate, and Mux 4 (109) processes a media stream containing one or more tracks of audio content. Each multiplexer then outputs respective individual ISOBMFF files (MP4) 110, 111, 112, and 113 for each of the media streams. Additional information (side information) 105 indicating the interrelationship between some or all of one or more tracks in one or more media streams can be output in parallel with the individual ISOBMFF files for further processing.

[0072] In the Stream Packager (streaming packager) 120, the individual ISOBMFF files 110, 111, 112, 113 are then individually segmented by their respective Segmenters, namely, Seg 1, 121, Seg 2, 122, Seg 3, 123, and Seg 4, 124 to generate respective initialization segments 127 and media segments 128 for streaming based on each of the individual ISOBMFF files. The Stream Packager 120 can be implemented separately from or in combination with other components described herein. The initialization segment 127 may include metadata for presenting a media stream encapsulated in a media segment 128 that conforms to the media format. These segments 127, 128 can be output for transmission and / or streaming 129 (e.g., via the Internet). For example, the segments can be uploaded to a server such as an origin server of a content delivery network (CDN).

[0073] Referring to MPEG-DASH (ISO / IEC 23009-1), the initialization segment and the media segment can be described as follows. Initialization Segment: A segment that contains the metadata necessary to present a media stream encapsulated in a media segment. Media Segment: A segment that conforms to the media format in use and enables playback when combined with zero or more preceding segments and an initialization segment (if any).

[0074] In Manifest Generator 125, manifest file 126 is additionally generated based on the received additional information 105 and initialization segment 127. The manifest file references metadata for streaming. The manifest file (also specified by MPEG-DASH and HLS) describes the media presentation and provides resource identifiers for all segments along with media asset descriptions. The multiple segments 127, 128, and the manifest file 126 are then output for streaming 129 (over the Internet). For example, the multiple segments 127, 128, and the manifest file 126 may be output to a streaming server or provided to a multicast server or a broadcast transmitter.

[0075] Figure 1 illustrates a first example of the processing of a media stream and an ISOBMFF file. Each of the modules in Figure 1 may be performed individually or in combination on one or more processors. In this case, individual ISOBMFF files are generated for each of one or more media streams and output in parallel with their respective additional information. These individual ISOBMFF files do not, therefore, contain information regarding the inter-track relationships such as that included in side information.

[0076] [Second Example of Processing Media Streams and ISOBMFF Files] Referring now to the example of Figure 2, an improved method 200 for processing a media stream according to an embodiment of the present disclosure is shown in a flow process chart.

[0077] In step S201, one or more media streams and side information about the one or more media streams are received, for example, by a receiver. The receiver can be, for example, part of a Multiplexer. The one or more media streams include media content that includes one or more tracks (e.g., is divided into one or more tracks).

[0078] In step S202, based on the one or more media streams and the side information, a (single) ISO Base Media File Format (ISOBMFF) file is generated. That is, the side information may not be output in parallel with the ISOBMFF file for further processing, as in the example of FIG. 1. Thus, in step S203, then, for further processing, the generated (single) ISOBMFF file is output. The further processing can include, for example, processing by a streaming packager (stream packager) to divide the ISOBMFF file into segment streams for Internet streaming. For example, the ISOBMFF file can be output for archival and / or transmission segmentation) and / or streaming (e.g., via the Internet).

[0079] In one embodiment, the side information can indicate the interrelationships among some or all of the one or more tracks. When generating the ISOBMFF file based on the one or more media streams and the side information, this single ISOBMFF file thus includes information about the interrelationships among some or all of the one or more tracks described herein.

[0080] These interrelationships among some or all of the one or more tracks can be characterized by one or more boxes in the ISOBMFF file 301, as shown in the exemplary structures of FIGS. 3-6.

[0081] In particular, as used herein, the term "box" is generally used to refer to an object-oriented building block, which, in some possible cases, is defined by a unique (box) type identifier (and possibly also its respective length), such as described in ISO 14496-12. Of course, the term "box" as used throughout this disclosure should not be understood to be limited to only such specifications. Rather, the term "box" is generally understood to be any suitable data structure or data container that can function as a placeholder for data. Further, as will be understood and recognized by those of ordinary skill in the art, such a "box" can be referred to using any other suitable term. As an example, in some possible specifications (including the first definition of MP4), the "box" may, in some cases, or may be referred to as an "atom". Further, as will be understood and recognized by those of ordinary skill in the art, one or more boxes can be at the same or different levels (or positions), nested (child / sub-box vs. parent box), etc., depending on the various implementation forms and / or requirements.

[0082] In one embodiment, generating the ISOBMFF file (301) may include grouping some or all of one or more tracks into one or more track groups. Depending on the use case, these one or more track groups may correspond to one or more DASH AdaptationSets 315, and one or more tracks within each track group may then correspond to one or more DASH Representations 316 within the DASH AdaptationSet 315. In one embodiment, these one or more track groups may correspond to one or more DASH AdaptationSets 514 having the potential to switch playback from an identified source DASH AdaptationSet to a set of destination DASH AdaptationSets. Alternatively, these one or more track groups may correspond to one or more CMAF switching sets 320, and one or more tracks within each track group may then correspond to one or more CMAF tracks within the CMAF switching set 320.

[0083] In one embodiment, the ISOBMFF files 301, 401 may include a track group type specific box 309, 413 indicating a predefined specific track group type and a track group identifier 414 for each of one or more tracks within the track group. Tracks characterized by the same track group type and track group identifier may belong to the same track group, and the track group type specific box 309, 413 may be included within the ISOBMFF track group box 306, 412.

[0084] Different grouping mechanisms may also be applied. In one embodiment, the ISOBMFF file may include a grouping type specific box that indicates a specific grouping type and a group identifier for each of one or more tracks within a track group, and the grouping type specific box may be included in the ISOBMFF group list box.

[0085] In the case of one or more DASH AdaptationSets 514 having the potential to switch, the ISOBMFF files 501, 601 may further include track group entry boxes 510, 603 of a predefined type for each of the switching potentials, the track group identifier 604 of which is equal to the track group identifier of the track group identifying the source DASH AdaptationSet, and the list 605 of track group identifiers is equal to those track group identifiers of the track groups identifying the destination DASH AdaptationSet.

[0086] In other words, for example, new types of track groups for DASH AdaptationSets 309, 509 may be created. For the purpose of gathering some tracks as Representations 316, 516 in AdaptationSets 315, 514, as shown in the exemplary structures of FIGS. 3 and 5, for the purposes of this document, new track_group_types 309, 509, simply for illustrative purposes without limitation, may be defined and referred to as AdaptationSetGroups 309, 509.

[0087] This AdaptationSetGroup 309, 509 is derived from the TrackGroupTypeBox 307, 507 and may be identified by a 4CC code. Requirements and semantics may further describe the purpose of this group.

[0088] The streaming packager may be able to read the AdaptationSetGroup 309, 509 information, split the tracks from the file into separate segments according to this information, and write a summary of all these individual segments to the streaming manifest 311, 511.

[0089] In the case where a fully multiplexed file reaches a player (e.g., of a client), a mechanism may be required to prevent the player from simultaneously playing all the included tracks. To disable the media player from playing all tracks rather than just one group, either the track_in_movie flag or the track_enabled flag within the TrackHeaderBox 305, 411, 505 may be cleared as needed to select one default track to play.

[0090] A further use case may require that it be possible not only to build groups of tracks, but also to group these groups themselves.

[0091] In one embodiment, generating the ISOBMFF file 301, 501 may thus further include grouping one or more track groups into one or more groups of track groups based on the side information. Each group of track groups may indicate the interrelationship between one or more track groups within the group of track groups.

[0092] In CMAF 317, the "Switching Set" 320 can be combined into an "aligned switching set" 318 or a "CMAF selection set" 319. To represent an aligned switching set at the ISOBMFF level, it may be necessary to construct a group of groups 310. In one embodiment, one or more groups of track groups may thus correspond to one or more CMAF selection sets 319. Alternatively, one or more groups of track groups may correspond to one or more CMAF aligned switching sets 318.

[0093] In one embodiment, the ISOBMFF files 301, 401 may further comprise group - specific boxes 310, 404 of track groups, each for one or more groups of track groups, containing a list of references to track groups belonging to a group of track groups 405. Each of one or more groups of the group - specific boxes 310, 404 may be instantiated by a predefined track - group entry type of a track - group entry box 308 included in an ISOBMFF track - group description box 304, 402. Each of one or more groups of the group - specific boxes 310, 404 can indicate their respective track - group identifiers 405, which enable referring to the group of track groups as a track group.

[0094] Using the concept of AdaptationSetGroup as described above can be achieved, for the purposes of this document as shown in FIGS. 3 and 4, by defining new derived forms of the TrackGroupEntryBoxes 308, 404, referred to as GroupOfGroups 310, 404, in a merely illustrative and non-binding manner. The new GroupOfGroups boxes 310, 404 may have a list 406 of track_group_id. Each AdaptationSetGroup having an identifier appearing in the list of track_group_id may be part of the "group of groups" 310, 404. This group of groups may signal an aligned switching set or a selection set.

[0095] Referring to the examples of FIGS. 4 and 5, FIG. 4 shows an example where there are two AdaptationSets each consisting of two Representations. In the example of FIG. 4, the ISOBMFF file 401 generated as described herein contains four tracks.

[0096] Track 1, 407, and Track 2, 408, are grouped by track_group_type = NEW1, 413 with track_group_id = 11, 414.

[0097] Track 3, 409, and Track 4, 410, are grouped by track_group_type = NEW1 with track_group_id = 12.

[0098] This new track_group_type = "NEW1" indicates that this group is interpreted by the streaming packager to provide these tracks as different Representations within an AdaptationSet / Switching Set. The DASH manifest generated from the above example has the following structure (here, for the sake of explanation, the id is set to match each corresponding one in Figure 4).

[0099] Table 1 shows a DASH manifest (media presentation description, MPD) generated based on the information included in / extracted from the ISOBMFF file shown in the example of Figure 4.

[0100] [Table 1]

[0101] The new TrackGroupEntryType 403, which is "NEW2" within the TrackGroupDescriptionbox 402, enhances this TrackGroup with additional characteristics. As an example, all attributes and sub-elements of a DASH AdaptationSet can exist here either as simple fields or as child boxes not further detailed here.

[0102] Furthermore, in this example, the new track_group_entry_type = "NEW3", 404 is shown together with track_group_id = 21, 405. This entry type is used to define a "group of groups" and in this example, groups track_group_id = 11 together with track_group_id = 12 into a new group with track_group_id = 21.

[0103] As described above, the group of this group can be a CMAF "selection set" or a CMAF "aligned switching set" depending on the assigned track_group_entry_type.

[0104] Figure 5 shows an example of DASH AdaptationSetSwitching described in this specification. Table 2 shows each DASH manifest (Media Presentation Description, MPD) generated based on the information included in / extracted from the ISOBMFF file shown in the example of Figure 5.

[0105] [Table 2]

[0106] [Switching between AdaptationSets] Another use can be to use this mechanism for MPEG DASH "switching between AdaptationSets" in accordance with Section 5.3.3.5 of ISO / IEC 23009-1:2019. MPEG DASH defines special SupplementalPropertyDescriptors (schemeIdUri = "urn:mpeg:dash:adaptation-set-switching:2016") to signal the possibility of seamless switching from one AdaptationSet (source) to another AdaptationSet (destination). In MPEG DASH, this is modeled as a directed graph, i.e., each AdaptationSet can have a separate different list of AdaptationSets that can be switched.

[0107] Referring to the example of Figure 5, in this use case, a derived form of TrackGroupEntryBox 508, which is simply called SwitchingDG for illustrative purposes only and without binding for the purposes of this document, can be defined as follows (similar to the GroupOfGroups box 310 in Figure 3). AdaptationSetSwitchbox 510 can be derived from TrackGroupEntryBox 308 and can have a single track_group_id for identifying the AdaptationSet to which it applies, along with a list of track_group_ids. Each entry in that list can represent an AdaptationSet for which seamless switching is possible. This structure supplies all the information necessary for writing a special SupplementalPropertyDescriptor 515 to the DASH manifest 511.

[0108] [Switching between Representations] A similar dependency mechanism is defined in MPEG DASH via the @dependencyId and @associationId attributes that define dependencies between Representations (in accordance with section 5.3.5 of ISO / IEC 23009-1:2019).

[0109] MPEG DASH defines a dependent Representation (i.e., a Representation that depends on other Representations) as a "normal Representation, except that it depends on a set of complementary Representations for decoding and / or presentation." Signaling is defined as "@dependencyId contains the values of the @id attributes of all complementary Representations, i.e., the Representations necessary to present and / or decode the media content components included in this dependent Representation." This can be modeled as a directed acyclic graph, i.e., each Representation can have a distinct list of Representations on which it can depend.

[0110] For this use case, other derived forms of the TrackGroupEntryBox 308 are, for the purposes of this document, merely illustrative and non-bindingly referred to as DependencyDAG and are defined as follows (similar to the GroupOfGroups box 310). The DependencyDAG box can be derived from the TrackGroupEntryBox 308 and can have a single track_group_id for identifying the track whose dependencies it lists, along with a list of track_group_ids that are tracks. This structure supplies all the information necessary for writing the @dependencyId attribute to the DASH manifest.

[0111] MPEG DASH defines an associated Representation as a "Representation that provides information about the relationship to other Representations." This concept is very similar to the dependent Representation and can thus be modeled using a similar derived form of the TrackGroupEntryBox 308.

[0112] [Generalization of Group of Groups] This grouping may be defined only for a specific track_group_type (i.e., SwitchingSetGroup), or may be generalized to group any type of track_group. The type of "group of groups" can be specified explicitly by a type identifier (i.e., track_group_entry_type), or implicitly given by the type of the member groups, or indicated by undefined elements within a new GroupOfGroup box. If this new "GroupOfGroup" box has its own track_group_id, further cascading connections may be possible.

[0113] Referring now to the example of FIG. 7, an example of a method 700 for processing an ISO Base Media File Format (ISOBMFF) file is shown.

[0114] In step S701, a (single) ISOBMFF file is received (e.g., by a receiver, which may be part of a segmenter), and the ISOBMFF file includes one or more tracks of media content and one or more boxes characterizing the interrelationships among some or all of the one or more tracks. In step S702, information regarding each interrelationship among some or all of the one or more tracks is extracted from the one or more boxes.

[0115] In step S703, the ISOBMFF file is segmented based on the extracted information to obtain a plurality of segments.

[0116] Then, in step S704, a plurality of segments are output. The ISOBMFF segments can be output for transmission and / or streaming (e.g., via the Internet). For example, the ISOBMFF segments can be uploaded to a server such as an origin server of a content delivery network (CDN) and / or provided to a multicast server or a broadcast transmitter.

[0117] In one embodiment, the correlation can be indicated by one or more track group boxes that group one or more tracks into track groups corresponding to each correlation. As described above, a track group may correspond to a DASH AdaptationSet, and one or more tracks within a track group may correspond to one or more DASH Representations within the DASH AdaptationSet. In the case of a DASH AdaptationSet, the track group may correspond to a DASH AdaptationSet having the potential to switch playback from an identified source DASH AdaptationSet to a set of destination DASH AdaptationSets. Alternatively, the track group may correspond to a CMAF switching set, and one or more tracks within the track group may correspond to one or more CMAF tracks within the CMAF switching set.

[0118] In one embodiment, for each of one or more tracks within a track group, each correlation can be indicated by a track group type specific box that indicates a predefined specific track group type and a track group identifier. Tracks characterized by the same track group type and track group identifier can belong to the same track group, and the track group type specific box is included in the ISOBMFF track group box.

[0119] In one embodiment, for each of one or more tracks within a track group, the respective relationships may be indicated by a grouping type specific box indicating a predefined specific grouping type and a group identifier, and the grouping type specific box may be included in the ISOBMFF group list box.

[0120] In the case of a DASH AdaptationSet having switching potential, for each of the switching potentials, the respective relationships may be further indicated by a track group entry box of a predefined type, the track group identifier of which is equal to the track group identifier of the track group identifying the source DASH AdaptationSet, and the list of track group identifiers is equal to the track group identifier of the track group identifying the destination DASH AdaptationSet.

[0121] In one embodiment, the relationships may be further indicated by grouping one or more track groups into one or more groups of track groups. Each group of track groups may indicate the relationships between one or more track groups within the group of track groups. In one embodiment, the group of track groups may correspond to a CMAF selection set. Alternatively, the group of track groups may correspond to a CMAF aligned switching set.

[0122] In one embodiment, the interrelationship may be indicated by a particular box to one or more groups of track groups, each including a list of references to track groups belonging to the group of track groups. Each of one or more groups of track group identification boxes may be instantiated by a predefined track group entry type of a track group entry box included in the ISOBMFF track group description box. Each of one or more groups of track group identification boxes may indicate its respective track group identifier, which enables referring to the group of track groups as a track group.

[0123] In one embodiment, the method may further include generating a manifest file based on the ISOBMFF file and outputting the manifest file in parallel with a plurality of segments. The manifest file may include information regarding each interrelationship among some or all of one or more tracks extracted from one or more boxes. As described above and also shown in the example of FIG. 5, by generating an improved ISOBMFF file, it is no longer necessary to signal additional side information for generating the manifest file. The manifest file may be a DASH Media Presentation Description (MPD) file.

[0124] Referring to the example of FIG. 8, in one embodiment, segmenting the ISOBMFF file may include generating one or more DASH initialization segments 801, 811, 812, 813, which include information regarding each interrelationship among some or all of one or more tracks extracted from one or more boxes 802 to 810.

[0125] A DASH MPD file may contain information included in the DASH initialization segment. Alternatively, a DASH MPD file may contain information regarding each interrelationship among some or all of one or more tracks based on an ISOBMFF file, but the DASH initialization segment excludes the interrelationship information.

[0126] Referring to the example of FIG. 9, a system of an apparatus 900 for processing a media stream and an apparatus 920 for processing an ISO Base Media File Format (ISOBMFF) file is shown.

[0127] The apparatus for processing a media stream may be implemented as an encoder configuration 900 including, for example, four encoders 901, 902, 903, and 904, and a Multiplexer 906.

[0128] The apparatus for processing an ISO Base Media File Format (ISOBMFF) file may be implemented as a Stream Packager 920 including, for example, a Segmenter 921 and a Manifest Generator 922.

[0129] Multiplexer 906 receives one or more, in this case four, media streams from encoders 901, 902, 903, and 904. The four media streams comprise media content (i.e., including) video and audio, which is (split into) including one or more tracks. The video content may be encoded at different bitrates: high, medium, and low.

[0130] Multiplexer 906 further receives side information 905 of one or more media streams, where the side information characterizes the interrelationships between some or all of one or more tracks, and based on the one or more media streams and the side information, generates an ISO Base Media File Format (ISOBMFF) file (MP4) 907. Multiplexer 906 outputs the generated ISOBMFF file for further processing by Stream Packager 920.

[0131] Segmenter 921 receives an ISOBMFF file 907 that includes one or more tracks of media content and one or more boxes that characterize the interrelationships between some or all of the one or more tracks. Segmenter 921 extracts information regarding each interrelationship between some or all of the one or more tracks from the one or more boxes, and segments the ISOBMFF file 907 based on the extracted information to obtain a plurality of segments 924, 925. The plurality of segments may include an initialization segment 924 and media segments 925. The plurality of segments are output for streaming. For example, the plurality of segments 924, 925 may be output to a streaming server, or provided to a multicast server or a broadcast transmitter.

[0132] In the example of FIG. 9, the manifest file 923 is further generated by Manifest Generator 922 based on the initialization segment 924 and is output in parallel with the plurality of segments 924, 925 for streaming.

[0133] In the case of a DASH MPD file, the DASH MPD file may contain information included in the DASH initialization segment. Alternatively, the DASH MPD file may contain information regarding each interrelationship among some or all of one or more tracks based on an ISOBMFF file, but the DASH initialization segment excludes the interrelationship information.

[0134] Note that the methods described herein can also be implemented as a program including instructions that cause a processor to perform the methods when executed by the processor. FIG. 10 shows an example of each of a device 1000 including a memory 1001 and a processor 1002. The program can be stored in a computer-readable storage medium. In other implementations, the device may have two or more processors.

[0135] A computing device implementing the above technology may have the following exemplary architecture. Other architectures including architectures having more or fewer components are also possible. In some implementations, the exemplary architecture includes one or more processors (e.g., a dual-core Intel® processor), one or more output devices (e.g., an LCD), one or more network interfaces, one or more input devices (e.g., a mouse, a keyboard, a touch-sensitive display), and one or more computer-readable media (e.g., RAM, ROM, SDRAM, a hard disk, an optical disk, flash memory, etc.). These components can communicate and exchange data via one or more communication channels (e.g., a bus), and the communication channel can utilize various hardware and software to facilitate the transfer of data and control signals between the components.

[0136] The term "computer-readable medium" refers to a medium involved in providing instructions to a processor for execution, including but not limited to non-volatile media (such as optical or magnetic disks), volatile media (such as memory), and transmission media. Transmission media includes, but is not limited to, coaxial cables, copper wire, and optical fibers.

[0137] The computer-readable medium may further include an operating system (such as the Linux® operating system), a network communication module, an audio interface manager, an audio processing manager, and a live content distributor. The operating system performs basic tasks including, but not limited to, recognizing input from network interfaces and / or devices and providing output thereto, tracking and managing files and directories on a computer-readable medium (such as memory or storage device), controlling peripheral devices, and managing traffic on one or more communication channels. The network communication module includes various components for establishing and maintaining network connections (such as software for implementing communication protocols such as TCP / IP, HTTP, etc.).

[0138] The architecture may be implemented in a parallel processing or peer-to-peer infrastructure or on a single device having one or more processors. The software may include multiple software components or may be a single piece of code.

[0139] The described features can be advantageously implemented by one or more computer programs executable on a programmable system including at least one programmable processor coupled to receive data and instructions from, and to transmit data and instructions to, a data storage system, at least one input device, and at least one output device. A computer program is a set of instructions that can be used, directly or indirectly, in a computer to perform an action or to cause a result. The computer program can be written in any form of programming language including compiled or interpreted languages (e.g., Objective-C, Java (registered trademark)), and can be deployed in any form, including as a stand-alone program, or as a module, component, subroutine, browser-based web application, or other unit suitable for use in a computing environment.

[0140] Processors suitable for the execution of a program of instructions include, by way of example, both general and special purpose microprocessors, and any one or more processors or cores of any kind of computer. Generally, a processor receives instructions and data from a read only memory or a random access memory or both. Essential elements of a computer are a processor for executing instructions and one or more memories for storing instructions and data. Generally, a computer also includes or is operatively coupled to communicate with one or more mass storage devices for storing data files, such devices including magnetic disks, such as internal hard disks and removable disks, magneto-optical disks, and optical disks. Storage devices suitable for tangibly embodying computer program instructions and data include, by way of example, semiconductor memory devices such as EPROM, EEPROM, and flash memory devices; magnetic disks such as internal hard disks and removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks, including all forms of nonvolatile memory. The processor and the memory may be supplemented or incorporated in an ASIC (application specific integrated circuit).

[0141] For purposes of interacting with a user, the features may be implemented on a computer having a display device such as a CRT (cathode ray tube), an LCD (liquid crystal display) monitor, or a retinal display device for displaying information to the user. The computer can have a touch surface input device (e.g., a touch screen), or a keyboard, and a pointing device such as a mouse or a trackball, whereby the user can provide input to the computer. The computer can have a voice input device for receiving voice commands from the user.

[0142] The features can be implemented in a computer system that includes backend components such as a data server, or middleware components such as an application server or an Internet server, or frontend components such as a client computer having a graphical user interface or an Internet browser, or any combination thereof. The components of the system can be connected by any form or medium of digital data communication such as a communication network. Examples of communication networks include, for example, LANs, WANs, and the computers and networks that form the Internet.

[0143] A computing system can include clients and servers. Clients and servers are generally remote from each other and typically interact via a communication network. The relationship of client and server arises by computer programs that run on respective computers and have a client-server relationship to each other. In some embodiments, the server transmits data (e.g., an HTML page) to a client device (e.g., for the purpose of displaying the data to a user interacting with the client device and receiving user input from the user). Data generated at the client device (e.g., as a result of user interaction) can be received at the server from the client device.

[0144] One or more computer systems can be configured to perform particular actions by having software, firmware, hardware, or any combination thereof installed in the system that, during operation, causes the system to perform actions. One or more computer programs can be configured to perform particular actions by including instructions that, when executed by a data processing apparatus, cause the apparatus to perform actions.

[0145] This specification includes many details of particular implementations, which should not be construed as limitations on the scope of any invention or what can be claimed, but rather as descriptions of features particular to specific embodiments of a particular invention. Some features described herein in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, the various features described in the context of a single embodiment can also be implemented separately, or in any suitable sub-combination, in multiple embodiments. Further, features may be described above as acting in a certain combination and may initially be claimed as such, but one or more features from the claimed combination can, in some cases, be excised from the combination, and the claimed combination can be directed to a sub-combination or a variation of a sub-combination.

[0146] Similarly, operations are shown in the drawings in a particular order, which should not be understood as requiring that such operations be performed in the particular order shown, or in a sequential order, or that all of the illustrated operations be performed, in order to achieve desirable results. In some situations, multitasking and parallel processing may be advantageous. Further, the separation of various system components in the embodiments described above should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product, or packaged into multiple software products.

[0147] Unless otherwise specified, as will be apparent from the following description, throughout this disclosure, descriptions using terms such as "processing," "computing," "calculating," "determining," "analyzing," etc., refer to the actions and / or processes of a computer or computing system, or similar electronic computing device, that manipulate and / or transform data represented as physical quantities, such as electronic quantities, into other data similarly represented as physical quantities.

[0148] Throughout this disclosure, references to "one exemplary embodiment," "some exemplary embodiments," or "exemplary embodiments" mean that a particular feature, structure, or characteristic described in connection with an exemplary embodiment is included in at least one exemplary embodiment of the disclosure. Thus, the appearances of the phrases "in one exemplary embodiment," "in some exemplary embodiments," or "in exemplary embodiments" in various places throughout this disclosure do not necessarily all refer to the same exemplary embodiment. Furthermore, particular features, structures, or characteristics may be combined in any suitable manner, as would be apparent to one of ordinary skill in the art from this disclosure, in one or more exemplary embodiments.

[0149] As used herein, unless otherwise specified, the use of ordinal adjectives such as "first," "second," "third," etc. to describe a common object merely indicates that different instances of a similar object are being referred to and is not intended to imply that the objects so described must be in a given order, temporally, spatially, in ranking, or in any other way.

[0150] It is also to be understood that the phraseology and terminology used herein is for purposes of description and should not be regarded as limiting. The use of "including," "comprising," or "having," and variations thereof, is meant to encompass the items listed thereafter and equivalents thereof, as well as additional items. Unless otherwise specified or limited, the terms "mounted," "connected," "supported," and "coupled," and variations thereof, are used broadly and encompass both direct and indirect mounting, connecting, supporting, and coupling.

[0151] In the following claims and in the description of this specification, the terms "comprising," "comprised of," or "which comprises" are all open terms meaning the inclusion of at least the element(s) / feature(s) that follow, but not the exclusion of others. Thus, when used in the claims, the term "comprising" should not be interpreted as being limited to the means or elements or steps listed thereafter. For example, the scope of the expression "device comprising A and B" should not be limited to a device consisting only of elements A and B. As used herein, the terms "including," "which includes," or "that includes" are all open terms meaning the inclusion of at least the element(s) / feature(s) that follow, but not the exclusion of others. Thus, "including" is synonymous with "comprising" and means "comprising."

[0152] In the above description of exemplary embodiments of the present disclosure, it should be appreciated that various features of the present disclosure may be grouped together in a single exemplary embodiment, figure, or description thereof for the purpose of streamlining the disclosure and aiding in understanding one or more of the various inventive aspects. This method of disclosure, however, should not be interpreted as reflecting an intention that the claims require more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive aspects lie in less than all features of a single, foregoing disclosed exemplary embodiment. Accordingly, the claims that follow this specification are expressly incorporated herein, with each claim standing on its own as a separate exemplary embodiment of the present disclosure.

[0153] Furthermore, some of the exemplary embodiments described herein include some features that are included in other exemplary embodiments, but do not include other features, and combinations of features of different exemplary embodiments are within the scope of the present disclosure and, as will be understood by those skilled in the art, form different exemplary embodiments. For example, in the following claims, any of the claimed exemplary embodiments may be used in any combination.

[0154] The description provided herein sets forth numerous specific details. However, it will be understood that the exemplary embodiments of the present disclosure may be practiced without these specific details. In other instances, well-known methods, structures, and techniques have not been shown in detail so as not to obscure the understanding of this description.

[0155] Accordingly, while the best mode contemplated of the present disclosure has been described, those skilled in the art will recognize that other and further modifications may be made thereto without departing from the spirit of the present disclosure, and it is intended to claim all such changes and modifications as fall within the scope of the present disclosure. For example, any of the formulas given above are merely representative examples of procedures that may be used. Functions may be added to or removed from the block diagrams, and operations may be interchanged between functional blocks. Steps may be added to or deleted from the described methods within the scope of the present disclosure.

Claims

Claim 1 A method for processing media streams, comprising: receiving one or more media streams and side information about the one or more media streams, wherein the one or more media streams include media content including one or more tracks; generating an ISO Base Media File Format (ISOBMFF) file based on the one or more media streams and the side information; outputting the generated ISOBMFF file for further processing. The method as described above. Claim 2 The method according to claim 1, wherein the side information indicates the interrelationship between some or all of the one or more tracks. Claim 3 The method according to claim 1, wherein generating the ISOBMFF file includes grouping some or all of the one or more tracks into one or more track groups. Claim 4 The method according to claim 3, wherein the one or more track groups correspond to one or more DASH AdaptationSets, and the one or more tracks within each track group correspond to one or more DASH Representations within the DASH AdaptationSet. Claim 5 The method according to claim 4, wherein the one or more track groups correspond to one or more DASH AdaptationSets having the potential to switch playback from an identified source DASH AdaptationSet to a set of destination DASH AdaptationSets. Claim 6 The method according to claim 3, wherein the one or more track groups correspond to one or more CMAF switching sets, and the one or more tracks within each track group correspond to one or more CMAF tracks within the CMAF switching set. Claim 7 The ISOBMFF file includes, for each of the one or more tracks within a track group, a track group type identification box indicating a predefined specific track group type and a track group identifier, tracks characterized by the same track group type and track group identifier belong to the same track group, and the track group type identification box is included in an ISOBMFF track group box, the method according to claim 5.

8. The ISOBMFF file includes, for each of the one or more tracks within a track group, a grouping type identification box indicating a predefined specific grouping type and a group identifier, and the grouping type identification box is included in an ISOBMFF group list box, the method according to claim 5.

9. The ISOBMFF file further includes, for each of the switching capabilities, a track group entry box of a predefined type, the track group identifier of which is equal to the track group identifier of the track group identifying the source DASH AdaptationSet, and the list of track group identifiers is equal to the track group identifier of the track group identifying the destination DASH AdaptationSet, the method according to claim 7 or 8 dependent on claim 5.

10. Generating the ISOBMFF file further includes grouping the one or more track groups into one or more groups of track groups based on the side information, the method according to claim 6.

11. Each group of track groups indicates the interrelationship between the one or more track groups within the group of track groups, the method according to claim 10.

12. The one or more groups of track groups correspond to one or more CMAF selection sets, the method according to claim 10 or 11 dependent on claim 6.

13. The one or more groups of track groups correspond to one or more CMAF aligned switching sets, the method according to claim 10 or 11 dependent on claim 6.

14. The method according to claim 10, wherein the ISOBMFF file further includes, for each of the one or more groups of track groups, a track group identification box including a list of references to track groups belonging to the group of track groups.

15. The method according to claim 14, wherein each of the one or more groups of track group identification boxes is instantiated by a predefined track group entry type of a track group entry box included in an ISOBMFF track group description box.

16. The method according to claim 14 or 15, wherein each of the groups of the one or more track group identification boxes indicates their respective track group identifiers, and the track group identifiers enable the group of track groups to be referenced as a track group.

17. The method according to claim 1, wherein the side information is not output in parallel with the ISOBMFF file for further processing.

18. A method for processing an ISO Base Media File Format (ISOBMFF) file, comprising: receiving an ISOBMFF file including one or more tracks of media content and one or more boxes characterizing interrelationships between some or all of the one or more tracks; extracting information regarding each interrelationship between some or all of the one or more tracks from the one or more boxes; segmenting the ISOBMFF file based on the extracted information to obtain a plurality of segments; outputting the plurality of segments.

19. The method according to claim 18, wherein the interrelationship is indicated by one or more track group boxes that group one or more tracks into track groups corresponding to their respective interrelationships.

20. The method according to claim 19, wherein the track group corresponds to a DASH AdaptationSet, and the one or more tracks in the track group correspond to one or more DASH Representations in the DASH AdaptationSet. **Claim 21** The method according to claim 20, wherein the track group corresponds to a DASH AdaptationSet having the potential to switch playback from an identified source DASH AdaptationSet to a set of destination DASH AdaptationSets. **Claim 22** The method according to claim 19, wherein the track group corresponds to a CMAF switching set, and the one or more tracks in the track group correspond to one or more CMAF tracks in the CMAF switching set. **Claim 23** For each of the one or more tracks in the track group, the respective relationships are indicated by a track group type specific box indicating a predefined specific track group type and a track group identifier, and tracks characterized using the same track group type and track group identifier belong to the same track group, and the track group type specific box is included in the ISOBMFF track group box. The method according to claim 21. **Claim 24** For each of the one or more tracks in the track group, the respective relationships are indicated by a grouping type specific box indicating a predefined specific grouping type and a group identifier, and the grouping type specific box is included in the ISOBMFF group list box. The method according to claim 21. **Claim 25** For each of the switching potential capabilities, each of the respective relationships is further indicated by a track group entry box of a predefined type, the track group identifier of which is equal to the track group identifier of the track group that identifies the source DASH AdaptationSet, and the list of track group identifiers is equal to the track group identifier of the track group that identifies the destination DASH AdaptationSet. The method according to claim 23 or 24, dependent on claim 21.

26. The method according to claim 22, wherein the relationship is further indicated by grouping the one or more track groups into one or more groups of track groups.

27. The method according to claim 26, wherein each group of track groups indicates the relationship between the one or more track groups within the group of track groups.

28. The method according to claim 26 or 27, dependent on claim 22, wherein the group of track groups corresponds to a CMAF selection set.

29. The method according to claim 26 or 27, dependent on claim 22, wherein the group of track groups corresponds to a CMAF aligned switching set.

30. The method according to claim 26, wherein the relationship is indicated by one or more groups of track group specific boxes, each of which includes a list of references to the track groups belonging to the group of track groups.

31. The method according to claim 30, wherein each of the one or more groups of track group specific boxes is instantiated by a predefined track group entry type of a track group entry box included in an ISOBMFF track group description box.

32. The method according to claim 30, wherein each of the groups of the one or more track group specific boxes indicates its individual track group identifier, and the track group identifier enables the group of track groups to be referenced as a track group.

33. The method according to claim 18, further comprising generating a manifest file based on the ISOBMFF file and outputting the manifest file in parallel with the plurality of segments.

34. The method according to claim 33, wherein the manifest file includes information regarding each mutual relationship among some or all of the one or more tracks extracted from the one or more boxes.

35. The method according to claim 33, wherein the manifest file is a DASH Media Presentation Description (MPD) file.

36. Segmenting the ISOBMFF file includes generating one or more DASH initialization segments, and the DASH initialization segments include information regarding each mutual relationship among some or all of the one or more tracks extracted from the one or more boxes. The method according to claim 35.

37. The method according to claim 36, which depends on claim 35, wherein the DASH MPD file includes the information included in the DASH initialization segments.

38. The method according to claim 36, which depends on claim 35, wherein the DASH MPD file includes information regarding each mutual relationship among some or all of the one or more tracks based on the ISOBMFF file, but the DASH initialization segments exclude the mutual relationship information.

39. An apparatus for processing a media stream, the apparatus comprising: receiving one or more media streams and side information about the one or more media streams, the one or more media streams including media content including one or more tracks; generating an ISO Base Media File Format (ISOBMFF) file based on the one or more media streams and the side information; outputting the generated ISOBMFF file for further processing; including one or more processors configured to perform a method including: apparatus.

40. The apparatus according to claim 39, comprising a Multiplexer.

41. An apparatus for processing ISO Base Media File Format (ISOBMFF) files, the apparatus comprising: Receiving an ISOBMFF file including one or more tracks of media content and one or more boxes characterizing the interrelationships among some or all of the one or more tracks; Extracting information regarding each interrelationship among some or all of the one or more tracks from the one or more boxes; Segmenting the ISOBMFF file based on the extracted information to obtain a plurality of segments; Outputting the plurality of segments; Including one or more processors configured to perform a method comprising: Apparatus. **Claim 42** The apparatus according to claim 41, further comprising a Segmenter. **Claim 43** The apparatus according to claim 41 or 42, wherein the one or more processors are further configured to generate a manifest file based on the ISOBMFF file and output the manifest file in parallel with the plurality of segments. **Claim 44** The apparatus according to claim 43, further comprising a Manifest Generator. **Claim 45** A system of an apparatus for processing a media stream according to claim 39 and an apparatus for processing an ISO Base Media File Format (ISOBMFF) file according to claim 41. **Claim 46** A program including instructions that, when executed by a processor, cause the processor to perform the method according to claim 1. **Claim 47** A computer-readable storage medium storing the program according to claim 46.

Images