Method, device, and computer program for improving encapsulation and parsing of multi-files content data

Abstract

At least one embodiment of a method for generating a first ISOBMFF file for describing a media presentation, in a processing device, the method comprising generating metadata referencing at least one entity in a second ISOBMFF file, the second ISOBMFF file comprising a description of the at least one entity and associated media data, the generated metadata comprising a first identifier for identifying the at least one entity in the first ISOBMFF file, the first identifier for identifying the at least one entity in the first ISOBMFF file being associated with location information for locating the second ISOBMFF file and with a second identifier for identifying the at least one entity in the second ISOBMFF file, and generating the first ISOBMFF file, the generated first ISOBMFF file comprising the generated metadata.

Description

[0001] METHOD, DEVICE, AND COMPUTER PROGRAM FOR IMPROVING ENCAPSULATION AND PARSING OF MULTI-FILES CONTENT DATA

[0002] FIELD OF THE DISCLOSURE

[0003] The present disclosure relates to a method, a device, and a computer program for improving encapsulation and parsing of multi-files media data, for example to improve flexibility.

[0004] BACKGROUND OF THE DISCLOSURE

[0005] The ISO base media file format (ISOBMFF, also called file format) is a general format forming the basis for a number of other more specific file formats. ISOBMFF is standardized by the International Standardization Organization as ISO / IEC 14496-12. This format contains characteristics of media data like timing, size, or media information for timed sequences of media data, such as audio-visual presentations and / or for non-timed media data such as a set (or a non-timed sequence) of still images. These characteristics are generally referred to as metadata (or structure-data), in opposition to media data (or data). An ISO Base media file (also referred to as media file or movie file or media presentation) may come as one file containing the whole presentation or as multiple segment files, each segment containing a temporal portion of the presentation. An ISO Base media file is structured into “boxes”, each of which being identified by a four-character code, also denoted FourCC or 4CC. In the file format, the overall presentation is called a movie. A movie, or a presentation, is logically divided into tracks, or consists of one or more tracks, wherein each track represents a timed sequence of media (for example a timed sequence of video pictures, a picture being either a frame or a field for interlaced video). Within each track, each timed unit is called a sample. Each track has one or more sample descriptions. Each sample in the track is tied to a description by reference. All the structure-data or metadata, including that defining the placement and timing of the media, is contained in structured boxes. The media data (frames of video or samples of audio, for example) is referred to by this structure-data or metadata. Media data are stored in media data boxes (e.g. ‘mdat, ‘imda’, etc.). A presentation may contain zero or more media data boxes. The overall duration of each track is defined in the structure-data.

[0006] An ISOBMFF file may comprise a reference to external sample data by using a DataReferenceBox (‘dref) box. However, this box does not make it possible to refer media data and associated metadata. Accordingly, it may be needed to duplicate the metadata associated with the referenced data, within the file referencing these data. As a consequence, any change in the file storing the referenced data must be reflected in the file referencing the data (due to file offset changes).

[0007] Since media presentations are getting more and more complex with preselection of tracks to play together, alternative encodings of a same media for dynamic switching, etc., and since each media stream may be prepared in one or several servers and packaged for streaming in other servers, there is a need for flexibility when encapsulating media data using ISOBMFF.

[0008] SUMMARY OF THE DISCLOSURE

[0009] The present disclosure has been devised to address one or more of the foregoing concerns.

[0010] According to a first aspect of the disclosure, there is provided a method for generating a first ISOBMFF file for describing a media presentation, in a processing device, the method comprising: generating metadata referencing at least one entity in a second ISOBMFF file, the second ISOBMFF file comprising a description of the at least one entity and associated media data, the generated metadata comprising a first identifier for identifying the at least one entity in the first ISOBMFF file, the first identifier for identifying the at least one entity in the first ISOBMFF file being associated with location information for locating the second ISOBMFF file and with a second identifier for identifying the at least one entity in the second ISOBMFF file; and generating the first ISOBMFF file, the generated first ISOBMFF file comprising the generated metadata.

[0011] Accordingly, the method of the disclosure makes it possible to reference, within a media file, entities defined and stored in another media file, for example tracks, items, or grouping data structures (e.g., track groups), providing flexibility when encapsulating media data according to a standard such as ISOBMFF. In addition, the method of the disclosure makes it possible to reuse existing encapsulated data and to avoid duplication of data structures.

[0012] Several use cases for adding tracks by reference in a file can be seen, such as: adding a new language track (audio or subtitles) to a file without having to rewrite the video part; simpler authoring of track groups and meta data without having to rewrite the entire file; building a presentation from already authored (encoder output, CMAF recording) track files; possibility to create generic files (templates) referring to media tracks; adding alternative tracks, DASH content preparation (a single ‘moov1can be constructed with all proper information (kind, language, dependencies, etc.) from external files at very low cost) (this last point relates to the on-going exploration on DASH / CMAF / FF alignment).

[0013] Among other advantages, having external tracks can therefore benefit authoring, content preparation and even content delivery.

[0014] According to some embodiments, the at least one entity is at least one track, the at least one track comprising a sequence of samples of the media presentation.

[0015] Still according to some embodiments, the at least one entity is at least one item.

[0016] Still according to some embodiments, the at least one entity is at least one grouping data structure grouping tracks, items, or tracks and items.

[0017] Still according to some embodiments, the generated metadata further comprise the location information for locating the second ISOBMFF file and the second identifier for identifying the entity in the second file.

[0018] Still according to some embodiments, association information associating the first identifier with the location information and the second identifier is comprised in a dedicated box different from the generated metadata.

[0019] Still according to some embodiments, the generated metadata further comprise a type of media data associated with the at least one entity.

[0020] Still according to some embodiments, the generated metadata further comprise at least a partial description of the at least one entity in the first ISOBMFF file.

[0021] Still according to some embodiments, the generated metadata further comprise flags indicating which of metadata describing the at least one entity in the first ISOBMFF file and in the second ISOBMFF file are to be used.

[0022] According to a second aspect of the disclosure, there is provided a method for processing a first ISOBMFF file describing a media presentation, in a processing device, the method comprising: obtaining the first ISOBMFF file, the first ISOBMFF file comprising metadata referencing at least one entity in a second ISOBMFF file, the second ISOBMFF file comprising a description of the at least one entity and associated media data, the metadata referencing the at least one entity comprising a first identifier for identifying the at least one entity in the first ISOBMFF file, the first identifier for identifying the at least one entity in the first ISOBMFF file being associated with location information for locating the second ISOBMFF file and with a second identifierfor identifying the at least one entity in the second ISOBMFF file; obtaining the at least one entity in the second ISOBMFF file; and processing the obtained at least one entity.

[0023] Accordingly, the method of the disclosure makes it possible to reference, within a media file, entities defined and stored in another media file, for example tracks, items, or grouping data structures (e.g., track groups), providing flexibility when encapsulating media data according to a standard such as ISOBMFF. In addition, the method of the disclosure makes it possible to reuse existing encapsulated data and to avoid duplication of data structures.

[0024] According to some embodiments, the method further comprises determining whether the first ISOBMFF file can be processed.

[0025] Still according to some embodiments, the method further comprises determining whether the at least one entity is to be processed.

[0026] Still according to some embodiments, the method further comprises determining whether the at least one entity can be processed.

[0027] Still according to some embodiments, the at least one entity is at least one track, the at least one track comprising a sequence of samples.

[0028] Still according to some embodiments, the at least one entity is at least one item.

[0029] Still according to some embodiments, the at least one entity is at least one grouping data structure grouping tracks, items, or tracks and items.

[0030] Still according to some embodiments, the at least one entity is requested on the basis of the location information and of the second identifier.

[0031] Still according to some embodiments, the metadata referencing the at least one entity further comprise a type of media data associated with the at least one entity, the method further comprising verifying a validity of the at least one entity on the basis of the type.

[0032] Still according to some embodiments, the metadata referencing the at least one entity further comprise at least a partial description of the at least one entity in the first ISOBMFF file. Still according to some embodiments, the method further comprises identifying metadata describing the at least one entity, wherein the metadata referencing the at least one entity further comprise flags indicating which of metadata describing the at least one entity in the first ISOBMFF file and in the second ISOBMFF file are to be used.

[0033] According to other aspects of the disclosure, there is provided a processing device comprising a processing unit configured for carrying out each step of the methods described above. The other aspects of the present disclosure have optional features and advantages similar to the first and second above-mentioned aspects. At least parts of the methods according to the disclosure may be computer implemented. Accordingly, the present disclosure may take the form of an entirely hardware embodiment, an entirely software embodiment (including firmware, resident software, micro-code, etc.) or an embodiment combining software and hardware aspects that may all generally be referred to herein as a "circuit", "module" or "system". Furthermore, the present disclosure may take the form of a computer program product embodied in any tangible medium of expression having computer usable program code embodied in the medium.

[0034] Since the present disclosure can be implemented in software, the present disclosure can be embodied as computer-readable code for provision to a programmable apparatus on any suitable carrier medium. A tangible carrier medium may comprise a storage medium such as a floppy disk, a CD-ROM, a hard disk drive, a magnetic tape device or a solid-state memory device and the like. A transient carrier medium may include a signal such as an electrical signal, an electronic signal, an optical signal, an acoustic signal, a magnetic signal or an electromagnetic signal, e.g. a microwave or RF signal.

[0035] BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Embodiments of the disclosure will now be described, by way of example only, and with reference to the following drawings in which:

[0037] Figure 1 illustrates an example of referencing a track, in an ISOBMFF file, that is defined and stored in another ISOBMFF file;

[0038] Figure 2 illustrates an overview of a method for encapsulating and parsing multimedia presentations according to some embodiments of the disclosure;

[0039] Figure 3 illustrates an example of steps of a method for referencing external tracks in an ISOBMFF referring file; Figure 4 illustrates an example of steps of a method for processing a referring file referencing external tracks, and processing the referenced external tracks;

[0040] Figure 5 illustrates an example of steps for determining whether an external track is valid or not, i.e. , whether it can be successfully obtained or not;

[0041] Figure 6 provides examples of ISOBMFF boxes for referencing external tracks in ISOBMFF files;

[0042] Figure 7 illustrates an example of steps of a method for referencing external entities in an ISOBMFF referring file;

[0043] Figure 8 provides examples of ISOBMFF boxes for referencing external entities in ISOBMFF files; and

[0044] Figure 9 schematically illustrates a processing device configured to implement at least one embodiment of the present disclosure.

[0045] DETAILED DESCRIPTION OF THE DISCLOSURE

[0046] According to some embodiments of the disclosure, one or several entities such as entities that may be defined in an ISOBMFF file, that are stored in an ISOBMFF file encapsulating media data, referenced or signaled in another ISOBMFF file also encapsulating media data. For the sake of illustration, such entities, that may be denoted external entities (from the point of view of the file referencing these entities), may be tracks, items (i.e., data which do not require timed processing, as opposed to sample data (i.e., data associated with a single time)), or grouping data structures (e.g., groups of tracks, groups of items, and / or groups of tracks and items).

[0047] To that end, the file referencing the entities, denoted the referring file (or master file or main file), comprises a location indication of the files storing the entities, denoted the referred files (or remote files), and an identifier of the entities in the referred files. In addition, the referring file comprises an identifier of the entities in the referring file, associated with the identifier of the entity in the referred file, to avoid any potential conflict of identifiers, in particular when multiple external entities are referenced in the referring file. These multiple external entities may come from a single referred file or from different referred files. The referring file may reference one, all or a subset of entities declared in a referred file.

[0048] According to some embodiments, track groups or entity groups are defined in the referring file for locating a file comprising an external entity (e.g. track or item), identifying the external entity, and identifying the external entity in the referring file. Figure 1 illustrates an example of a referring file encapsulating media data and referencing tracks, denoted external tracks, encapsulated in a referred file. For the sake of illustration, the referring file 100 and the referred file 150 comply with the ISO Base Media File Format.

[0049] As illustrated, media data encapsulated in referring file 100 starts with a FileTypeBox (‘ftyp’) box 105 providing a set of brands identifying the precise specifications to which the encapsulated media data conform, that are used by a reader to determine whether it can process the encapsulated media data. The ‘ftyp’ box is followed by a MovieBox (‘moov’) box referenced 110. The MovieBox box provides initialization information that is needed for a reader to initiate processing of the encapsulated media data. In particular, it provides a description of the presentation content, the number of tracks and their identifiers, and information regarding their respective timelines and characteristics. In particular, the ‘unif brand defined in ISOBMFF may also apply to referring files to indicate that items, track, entity groups, and track groups may share the same scope within the referring file. If one referred file has a ‘unif brand, it may not be reflected in the referring file with a ‘unif brand. In opposite, when all referred files have the ‘unif brand, this same brand is preferably set in the referring file.

[0050] For the sake of illustration, the MovieBox box may indicate that the presentation comprises three tracks. One of these tracks is a conventional track (i.e. , not an external one) described in the ‘trak’ box 115, that corresponding media data are stored in the MediaDataBox (‘mdat’) box 120 (or in an identified media data box ‘imda’) of the referring file. Media data for track 115 could also be stored in an external file and be referenced by the DataReferenceBox ( 'dref ' ) . The two other tracks are external tracks. They are referenced in ExternaiirackBox (‘ext / c’) boxes 125 and 130 that are not associated with any media data stored in referring file 100 but that refer to ‘trak’ boxes 165 and 170 of referred file 150.

[0051] All the media data stored in referring file 100, for example samples for the track 115, are stored in the MediaDataBox (‘mdat’) box 120. As explained here after, the data corresponding to external tracks 125 and 130 are stored in MediaDataBox in the referred files 175, or in another files, depending on the indications in the ‘dref’ of these external tracks.

[0052] It is noted that ISOBMFF and its extensions comprise several grouping mechanisms to group together tracks, static items, or samples and to associate a group description with a group. A group typically shares common semantic and / or characteristics. For example, MovieBox (‘moov”) box 110 may contain sample groups associating properties to groups of samples for a track. The sample groups characterized by a grouping type may be defined by two linked boxes, a SampleToGroupBox (‘sbgp’) box that represents the assignment of samples to sample groups and a SampleGroupDescriptionBox (‘sgpd’) box that contains a sample group entry for each sample group describing the properties of the group.

[0053] Likewise, media data encapsulated in referred file 150 starts with a FileTypeBox (‘ftyp’) box 155 providing a set of brands identifying the specifications to which the encapsulated media data conform, that are used by a reader to determine whether it can process the encapsulated media data. Again, the ‘ftyp’ box is followed by a MovieBox (‘moov’) box, referenced 160, which provides initialization information that is needed for a reader to initiate processing of the encapsulated media data.

[0054] According to the illustrated example and unlike referring file 100, referred file 150 only comprises conventional (or local) tracks, two conventional tracks in this example. One of these tracks is described in the ‘trak’ box 165 and the other one in the ‘trak’ box 170. The corresponding media data are stored in the MediaDataBox (‘mdaf) box 175, for example as sequences of samples 180 and 185. These two conventional tracks of referred file 150 are used as external tracks by referring file 100, as tracks 125 and 130 respectively.

[0055] It is observed that referred file 150 is a conventional file (a conventional ISOBMFF file in this example) that may be processed independently from referring file 100.

[0056] According to the example illustrated in Figure 1 , the referring ISOBMFF file contains both conventional (local) tracks and external (remote) tracks. However, it must be understood that a referring file may contain no conventional track and only reference external tracks. Likewise, while the external tracks referenced in the referring file belong to the same referred file in the example illustrated in Figure 1 , it must be understood that the external tracks may belong to different referred files. Still according to particular embodiments, a referring file may reference an external track defined and stored in a referred file comprising both conventional tracks and external tracks. A referring file provides a pre-declaration of tracks to be used in the referring file, their actual declaration being described in a referred file. It may be interpreted as an indirection to the track declaration from a referring file to a referred file.

[0057] General encapsulation and parsing process according to embodiments of the disclosure Figure 2 illustrates an overview of a method for encapsulating and parsing multimedia presentations according to some embodiments of the disclosure. The media data to be encapsulated or stored independently from each other, referenced 205-1 and 205-2, can be video, spatial subpart of a video, audio, volumetric visual media, subtitles, or a combination of such elementary streams. They are encapsulated into two media files 225-1 and 225-2, media file 225-1 being a referring media file, like referring file 100 in Figure 1 , referencing an entity, for example a track, stored in media file 225-2, that is a referred file like referred file 150 in Figure 1.

[0058] As illustrated, each of one or several servers 200-1 and 200-2 used for encapsulating the media data comprises an encapsulation module 220-1 or 220-2 (also called ISOBMFF writer or simply writer). The servers 200-1 and 200-2 may be connected, via a network interface (not represented), to a communication network 230 to which is also connected, via a network interface (not represented), a client 250. An example of client is a media player on a PC, tablet, TV or smartphone.

[0059] According to the illustrated example, server 200-2 processes media data 205-2 to prepare it for streaming or for storage, this is called encapsulation. It mainly consists in adding metadata describing the media data in terms of kinds of data, codec in use, size, data offsets, timing, etc. in encapsulation module 220-2. As mentioned above, media data 205-2 may correspond to audio presentation or video presentation or both, possibly with subtitles or timed text. Media data 205-2 may be the raw data captured by sensors or generated by a content creator or editing tools. Media data 205- 2 may be available as compressed or encoded media data 215-2, possibly according to different encoded versions. This means that the encoding or compression may be performed by the server itself with an encoder module like encoder module 210-2 (possibly one per media type) or remotely from the server.

[0060] Likewise, server 200-1 (that may be server 200-2 or that may be another server) processes media data 205-1 in encapsulation module 220-1 to prepare it for streaming or for storage. Again, media data 205-1 may be the raw data captured by sensors or generated by a content creator or editing tools. Media data 205-1 may be available as compressed or encoded media data 215-1 , possibly according to different encoded versions (the encoding or compression may be performed by the server itself with an encoder module like encoder module 210-1 (possibly one per media type) or remotely from the server). It is noted that media data 205-1 and 205-2 may be encapsulated at approximately the same time or completely out of sync, media data 205-2 being preferably encapsulated before media data 205-1 to simplify referencing.

[0061] As suggested in Figure 2 with the arrow between media file 225-2 and encapsulation module 220-1 , encapsulation module 220-1 adds references to entities of media file 225-2 in media file 225-1 to reference entities of media file 225-2 (external entities). More precisely, encapsulation module 220-1 adds, in media file 225-1 , location information of media file 225-2, an identifier of the entities in media file 225-2, and an identifier of the entities in media file 225-1 . For the sake of illustration and as mentioned above, such entities may be tracks.

[0062] Client 250 is used for processing data received from communication network 230, or read from a (local or remote) storage device, for example for processing media file 225-1. The received data are de-encapsulated (or parsed) in de-encapsulation module 260 (also known as a ISOBMFF parser, ISOBMFF reader, or simply parser or reader). Upon determining that media file 225-1 references external entities, i.e. , entities described and stored in a referred media file (225-2), the referred media file is obtained and parsed in de-encapsulation module 260 to obtain the referenced entities.

[0063] Next, the de-encapsulated data (or parsed data) may be decoded by a decoding module for storage, display or for output to an application or to user(s) of the decoded data 275-1 and / or 275-2. The decoder module, possibly one or more per media type, for example decoder modules 270-1 and 270-2, may be part of the client, may be an external module, or may be a dedicated hardware. The de-encapsulated data may correspond to encoded media data 265-1 and 265-2 (e.g. video bit-stream, audio bitstream, etc.). Preferably, only the entities of the referred file that are referenced as external entities in the referring file are parsed and decoded.

[0064] For the sake of illustration, media file 225-1 and 225-2 may encapsulate media data into boxes according to the ISO Base Media File Format (ISOBMFF, ISO / IEC 14496-12) and its derived specifications (e.g. Carriage of NAL unit structured video in the ISOBMFF, ISO / IEC 14496-15 or Common Media Application Format, CMAF ISO / IEC 23001-19 or carriage of volumetric media like MPEG-I Parts 10 or 18).

[0065] Referencing external tracks in an ISOBMFF referring file

[0066] Figure 3 illustrates an example of steps of a method for referencing external tracks in an ISOBMFF referring file, for example referring file 100 in Figure 1. As illustrated, a first step is directed to obtaining a first ISOBMFF file (step 300), corresponding to the referring file. This file may be a file newly created or an already existing file that is being edited. It may be selected automatically or upon a user’s request.

[0067] Next, the tracks to be referenced as external tracks, that are described and stored in one or several second ISOBMFF files, are identified (step 305). Identification of these external tracks may be carried out automatically or upon a user’s choice.

[0068] After having identified the one or several tracks to be referenced in the referring file as external tracks, the location of the second ISOBMFF files where the identified tracks are described and stored, i.e., the location of the one or several referred files, is determined. An indication of the location of the referred files is then added to the referring file obtained in step 300, along with an identifier of the external tracks within the corresponding referred files (step 310).

[0069] Such a location of the referred files and such an identifier of the external tracks in the referred files may be stored in ISOBMFF boxes such as the ones described by reference to Figure 6, where the location indication is defined as a string named “location” (references 605 and 650 in Figure 6) and the track identifier is stored as an unsigned integer on 32 bits “referenced_track_ID”. For the sake of illustration, the location may be the location of the file in the form of an URL (Uniform Resource Locator), URN (Uniform Resource Name), or a UUID (Universally Unique Identifier) that identifies a file in a given context, while the track identifier may be the track identifier “trackJD” indicated in the TrackHeaderBox of the track in the referred file. It is observed that the examples provided in Figure 6 are just examples, but there are many possible boxes for describing external tracks.

[0070] Next, an identifier of each of the external tracks in the referring file (i.e., the identifier that is to be used for processing the referring file), is defined and associated with the corresponding identifier of the track in the referred file. Such an identifier of an external track in the referring file may be the same as the identifier of this external track in the corresponding referred file or may be different. The association between the identifiers of the track in the referred file and in the referring file may result from the data structure used for storing these identifiers. The aim of using a different identifier in the referring file and in the referred file is to avoid conflict of track identifiers since a track identifier is unique within a file but two different files may use the same track identifier. This allows processing an external track as any other conventional track. The defined identifier is added in the referring file (step 315), for each external track. In particular, such a track identifier (track_iD) may be defined in a TrackHeaderBox (as described with reference to Figure 6, in view of references 600 and 650). Such a track identifier may then be used in the context of the referring file to define a track group (TrackGroupBox, ‘trg ) or a track reference (TrackReferenceBox, ‘tref) or in any box or structure using a trackJD, or more generally an entity identifier (for example the ‘stmi’ sample group that is listing itemJDs).

[0071] According to some embodiments, an indication of the handler type of each external track is also added in the referring file (not represented). Adding the handler type to the referring file makes it possible for a file reader to determine whether it may process the corresponding track, without having to access and load the referred file. In addition, this allows verifying that the indicated type in the referring file is the same as the indicated type in the referred file, when the external track is selected to be processed (as described in reference to Figures 4 and 5). For the sake of illustration and as illustrated in Figure 6 (references 605 and 650), this indication may be provided through a property “referenced_handler_type” (naturally, other names may be used for this kind of property). The handler type may also be useful to check that relationships between the tracks declared in the referring file, possibly indicated with 'trg or ‘tref, are actually allowed by ISOBMFF. This may be done when creating or editing the file but this may be done as well when parsing the file: if an inconsistency is detected, player may not process the file or at least a subset of tracks with invalid relationships.

[0072] Processing an ISOBMFF file referencing external tracks

[0073] Figure 4 illustrates an example of steps of a method for processing a referring file, for example referring file 100 in Figure 1 , referencing external tracks, and processing the external tracks in a file reader.

[0074] As illustrated, a first step is directed to obtaining a first ISOBMFF file (step 400) which is the referring file to process. Once obtained, the file reader determines whether it can process the obtained referring file, i.e. whether it can process indicated brands and track handler types (step 405). This follows the same process as for ISOBMFF files without any external track. In an embodiment, a dedicated brand, such as “extt”, may be defined in order to indicate that an ISO Base Media File may contain one or more external track(s).

[0075] Next, after having determined that the file reader may process the obtained file, the file reader determines whether one or several external tracks referenced in the obtained file should be taken into consideration (step 410). This may follow the same rules as for conventional tracks, e.g. looking at user preferences, groups, etc. The external tracks to be processed are selected. If none of the external tracks should be processed, the referring file is processed on a conventional way (not illustrated).

[0076] Next, if at least one external track is selected to be processed, the corresponding referred file (i.e., the file describing the external track and containing its media data) is loaded using the location indication stored in the referring file (step 415). It is then checked whether the selected external track can be successfully obtained (step 420), as described in more detail with reference to Figure 5. This is done for each selected external track.

[0077] According to some embodiments, if one or several external tracks are invalid (i.e., they cannot be successfully obtained), the file reader rejects the referring file (step 425) and the process ends. According to other embodiments, the file reader may choose to present only the valid external tracks, a subset of the valid external tracks, or only the conventional tracks, as it is done for files without any external track.

[0078] Otherwise, if the selected external tracks are valid (i.e., they have been successfully obtained), the external tracks are processed independently from the conventional tracks of the referring file, i.e., as if they were processed in a second file reader, but using track groups and references defined in the referring file. This means that global movie structures of the referred file, such as ‘trex’ or ‘pssh’ may be required to process the external tracks.

[0079] In some cases, a referred file may contain a ‘meta’ box at file level or at ‘moov’ level, and / or an ‘udta’ box at ‘moov’ level. Such boxes may be used to define tags or any other information (e.g., label boxes, copyright boxes, user-data information related boxes, or any other boxes) at a higher level than for a given track. In a particular embodiment, ‘meta’ at file level or at ‘moov1level and ‘udta’ at ‘moov1level in the referred files should be ignored. In this particular embodiment, only the ‘udta’ box present at the ‘moov’ level or the ‘meta’ box present at file or movie level of the referring file, if any, should apply. By doing so, there is no risk of conflict between the tags or information defined in such boxes in a referred file and the tags or information defined in the same or corresponding boxes in a referring file. Alternatively, specific means may be provided to enable considering these boxes, in both the referred file and the referring file, while preventing conflicts. For instance, a rule may be defined so that a tag or an item of information defined at file level or at ‘moov1level in the referred file may be preserved provided a tag with the same name or a same item of information is not defined at file level or at ‘moov1level in the referring file. In a particular embodiment, a dedicated box may be defined to copy the tags or any other items of information defined at file level or at ‘moov1level in the referred file directly into the referring file. This allows accessing these tags or items of information without having to read the referred file. For instance, such a box may be named ImportedllserDataBox or ImportedMetaBox and may be located inside the ExternalTrackBox. As another alternative, the tags defined in ‘meta’ at file level or at ‘moov1level, or the tags defined in ‘udta’ at ‘moov1level may be put in the ‘meta’ or respectively ‘udta’ box of the corresponding track provided these tags or other information are not already defined in this ‘meta’ or said ‘udta’ box.

[0080] In some cases, assigning a name to a track may be convenient. This may also be true for an external track. As for a legacy track, an external track may be assigned a name through the definition of a LabelBox in the ‘udta’ box associated to the considered track. Alternatively, a name may also be assigned to a track through a HandlerBox ‘hdlr1. This box is typically contained in a ‘trak box, for instance in the referred file of considered external track. The name of an external track may be defined to be the name of the corresponding track in the referred file if any, or it may be defined (or redefined) in the referring file. Optionally, a flag may be defined to indicate whether the name in the referred file should be considered or not. If not, the track in the referring file would have no name unless a name is defined in the referring file. In an embodiment, this name may be defined through the definition of an additional field contained in the hierarchy of the boxes related to the external track. As an example, if a syntax such as the one of reference 650 in Figure 6 is considered (i.e. ‘extk is a FullBox), a field such as new_name may be defined in said ‘extk’ box. The setting of this field may be conditioned by the setting of a given flag value, for instance a TRACK_NAME_DEFINE flag (if not set, the name of track in the referred file, if any, would typically apply).

[0081] In some cases, a main track in a given ISOBMFF file may refer to other (secondary) tracks in the same ISOBMFF file. This is for instance the case of a multitrack file, where a main track (or master or base track) refers to one or several secondary tracks (this may for instance be the case of an ISOBMFF file carrying G-PCC or V-PCC data encapsulated as multiple tracks or HEVC tile tracks or VVC subpicture tracks). In such cases, the main track may be embedded in another file as an external track. A file reader processing such an external track has to consider the references from the main track to the secondary tracks as references associated with the referred file (e.g. the identifier of a secondary track is defined in the scope of the referred file). In particular, if data associated with one such secondary track is needed, it should obtain it from the referred file. Alternatively, it may be defined that in such cases, not only the main track should be referred to in the referring file, but also the secondary tracks, i.e. the main track and secondary tracks are all defined as external tracks in the referring file, the relationships between the main and the secondary tracks also being redefined in the referring file. Redefining the main and secondary tracks offers a safer approach to avoid risk of track ID conflicts.

[0082] In some cases, a first track in a given ISOBMFF file may be referred to by other (secondary) tracks in the same ISOBMFF file. For instance, a first track may be referred to by a metadata track or by a derived track. This first track may be embedded in another file as an external track. A file reader processing such an external track has to consider the references from the secondary tracks to the first track as references associated with the referred file (e.g., the identifier of a secondary track is defined in the scope of the referred file). In particular, if data associated with one such secondary track is needed, it should obtain it from the referred file. A file reader may decide to use a reference to the external track in the referred file depending on the type of the reference. For example, if the reference corresponds to metadata describing the external track, the file reader may decide to use this metadata. If the reference corresponds to a derived track, the file reader may decide to ignore this derived track. Possibly, the types of the references to import from the referred file may be signaled in the description of the external track.

[0083] In some embodiments, at step 430, processing the referenced track may require processing a track-reference. This track-reference relies on the track identifier values for defining the track it refers to. When processing a track-reference, it is first determined inside which ISOBMFF file the track-reference is defined. Then, the trackreference is processed using track identifier values from the ISOBMFF file it is defined in. This processing enables to process correctly track-references without confusing the track identifiers defined in the referring ISOBMFF file with those defined in the referred to ISOBMFF file.

[0084] In some embodiments, a track group may be defined as an external track group by referencing a track group defined in another ISOBMFF file. In these embodiments, processing the referenced track at step 430 may require processing a track group. The processing of a track group may be realized similarly to what is described above for processing a track-reference.

[0085] More generally, when an external track is defined in a referring file, it may be checked whether the referred file contains boxes with references to the corresponding track referred by the external track, such as track groups, entity groups, or track references. For instance, this may be the case with switching groups, alternate groups, PreselectionGroupBox, or TrackGroupDescriptionBox. Different options may be considered for handling such cases. A first option may be to simply ignore such boxes and consider that only this referred track is integrated in the referring file as an external track. This may be the case, for example, when only one track declared as being part of a group of tracks in one or more referred files is included in a referring file. In this case, if the referred track originally declared an alternate or switching group (e.g., in ‘tkhd’ box and alternate_group field), its TrackHeaderBox in the ExternalTrackBox should reset this value (set it to undefined, for example using value 0). Likewise, if there was a TrackSelectionBox, this may be ignored.

[0086] A second option may be to consider all such boxes also in the context of the referring file. This may either be achieved at processing time (i.e. when obtaining the data from the referred file, not only the considered track, but also related boxes are obtained and considered in the context of the referring file), or by including these boxes in the referring file in a dedicated set of boxes. In both cases, this may involve changing the track identifiers of considered tracks by replacing the identifiers used in the referred file by the identifiers used in the referring file. This second option may be used when several (at least 2) or all tracks declared as being part of a group of tracks in one or more referred files are included in a referring file. In this case, if the referred tracks originally declared an alternate or switching group (e.g. in ‘tkhd’ box and alternate_group field), their TrackHeaderBox in the ExternalTrackBox should reuse this value (or reallocate a new one) in order to preserve the indication of the group. Likewise, if the referred tracks declared a TrackSelectionBox, this box may be considered either directly from the referred file or as TrackSelectionBox copied in the ExternalTrackBox. The copy may be decided if a flags value is set for the external track indicating that user data for the track should be ignored or should not be imported (as explained hereafter). For instance, this second option may be enabled by defining a dedicated flags value in ExternalTrackLocationBox, for instance IMPORT_CONTEXT, where the context represents the boxes such as track groups, entity groups or track-references comprising a reference to considered track. Alternatively, 2 bits of the flags value may be used to indicate at least one of the following modes: the context is ignored, the context is imported in the referring file, and the context is not imported in the referring file but should be obtained from referred file when the referring file is processed. Figure 5 illustrates an example of steps for determining whether an external track is valid or not, i.e. , whether it can be successfully obtained or not.

[0087] As illustrated, a first step is directed to determining whether the location of the referred file describing the external track and storing its media data is invalid (step 500). For the sake of illustration, this may be the case if the location indication for the considered external track does not correspond to an actual file. If so, the considered external track is marked as invalid and the “false” value is returned to indicate that the considered external track cannot be successfully obtained.

[0088] On the contrary, if the referred file location provided in the referring file is valid, another test is carried out to determine whether the file and / or track cannot be processed by the reader due to brand requirements in ‘ftyp’ or ‘ttyp’ (step 505). If the file and / or track cannot be processed by the reader due to brand requirements in ‘ftyp’ or ‘ttyp’, the “false” value is returned.

[0089] On the contrary, if there is no issue with brand requirements, another test is carried out to determine whether the TrackBox corresponding to the external track cannot be found in the referred file (step 510). For the sake of illustration, this would be the case if no TrackHeaderBox containing a track_iD identical to the indicated track identifier for the external track is present in the referred file. Additionally, this step may also comprise a step of verification of the validity of the track. For example, the track may be considered as not valid if the corresponding TrackBox contains some invalid data (e.g., a missing mandatory box). If the TrackBox corresponding to the external track cannot be found or if it is not valid, the “false” value is returned.

[0090] Optionally, if the TrackBox corresponding to the external track can be found in the referred file and is valid and if the handler type of the external track is indicated in the referring file, a further test may be carried out to determine whether the external track handler type does not match with the handler type indicated in the referring file (step 515), e.g. in ExternaiTrackLocationBox, as described in reference to reference 600 in Figure 6. If the handler types do not match, the “false” value is returned.

[0091] On the contrary, if the handler types match and / or if the TrackBox corresponding to the external track can be found in the referred file and is valid, another test is carried out to determine whether the description of the referred file in the referring file contains a TrackTypeBox with unsupported brands (step 520). If the description of the referred file in the referring file contains a TrackTypeBox with unsupported brands, the “false” value is returned. On the contrary, if the description of the referred file in the referring file contains a TrackTypeBox with supported brands, the “true” value is returned.

[0092] When an external track is detected as being invalid, this item of information may be recorded in the referring file, for example in the TrackHeaderBox of the ExternalTrackBox, by settings the flags value track_in_movie to false. This will avoid application to process this track.

[0093] Examples of referencing external tracks

[0094] Figure 6 provides examples of ISOBMFF boxes for referencing external tracks in ISOBMFF files. References 600 and 605 illustrate a first example wherein external tracks are referenced by using two boxes, whereas reference 650 illustrates a second example wherein external tracks are referenced by using a single box.

[0095] Reference 600 illustrates an example of a box denoted ExternalTrackBox, that defines the following fields:

[0096] - exti indicates the location of the file comprising the description of the external track and storing its media data. According to some embodiments, this box must be present and must be the first box occurring in ExternalTrackBox;

[0097] - t khd indicates the track header of the external track within the context of the referring file. Still according to some embodiments, this box must be present and must be the second box occurring in ExternalTrackBox; and

[0098] - other indicates any possible box allowed as a child of TrackBox, except MediaBox and TrackHeaderBox, or ExtendedLanguageBox. In this example, MediaBox (‘mdia’) is not allowed to be specified in the referring file as it contains data that describe the samples, hence data that should not be modified. However, according to some embodiments, it may be allowed to define or redefine some part of the ‘mdia’ data in the referring file. For example, this may be the case of ExtendedLanguageBox, which is a child of the ‘mdia’ box and may be redefined to fix an invalid or inaccurate ExtendedLanguageBox that may be defined in the referred file, for instance to indicate that a subtitle track is not just in English, but in American English. On the other hand, TrackHeaderBox is also not allowed in other boxes, but for a different reason compared to MediaBox. Indeed, a TrackHeaderBox has already been defined in the ExternalTrackBox, and only one TrackHeaderBox should be defined in a given ExternalTrackBox. In this example, further restrictions on the type of children allowed in ExternalTrackBox are given by the ExternalTrackLocationBox. As previously explained, an ExternaiirackBox may be used to reference a track from another ISO Base Media file, as defined by its irackBox and other track- related structures. The track being referred to is called an external track (it is external from the point of view of the file referencing such a track that is not described and stored in the file). As described above, the file containing the ExternaiirackBox is called the referring file, and the file containing the description of the external track and its media data is called the referred file. Referred files may be ISOBMFF compliant files.

[0099] In a variant, rather than referring to an external track by defining an ExternaiirackBox, the external track may be referred using a irackBox containing a DataReferenceBox comprising a dedicated entry providing the location of the referred file and the identifier of the external track in the referred file, or a dedicated box indicating where the track declaration can be found. It could even be the External? rackLocationBox.

[0100] The tracks of a referred file, that correspond to external tracks of a referring file, may be fragmented or not, independently of whether the referring file is fragmented or not. Derived specifications may further restrict possible combinations.

[0101] The timeline of a track corresponding to an external track may be modified by an edit list in the referring file.

[0102] The userDataBox and MetaBox of a track corresponding to an external track may be overridden or augmented.

[0103] According to some embodiments, track references and track groups of the referred files are ignored and only track references and groups (track groups, entity groups, preselection groups, possibly with their track group description boxes) defined in the referring files are valid.

[0104] The track_ID of the TrackHeaderBox present in ExternaiirackBox of the referring file gives the identifier of the track in the referring file, that may be used to describe track references, track groups and other track relationships relying on track identifiers within the referring file. This allows defining track relations or track groups independently from the identifiers used in the referred files.

[0105] Additionally, the TrackHeaderBox provides presentation information of the external track within the presentation of the referring file, such as track width / height, matrix, volumes and track flags.

[0106] Still according to some embodiments, the following restrictions are set on the TrackHeaderBox in the ExternaiirackBox of an external track: if the duration field is undefined (all 1s) and there is no edit list for this track, then the duration of the track is the duration of the referenced track.

[0107] The sample description, offsets, sizes, etc. for an external track are the ones declared in the referred file indicated by location in the ExternaiTrackLocationBox.

[0108] As described above, the ExternaiirackBox may contain any box that may be a child of TrackBox (i.e. , ‘udta’, ‘meta’, ‘edts’, ‘tref, ‘trgf, ‘mdia’, ‘tkhd, ‘ttyp’), or even any box that may be a child of TrackBox (even if not a direct child). This makes it possible to define external tracks at an equivalent level as TrackBox. However, for external tracks, there is a need to clearly define which description data, or metadata, applies. Indeed, track description data from the referred file may apply, as well as track description data from the referring file. In some cases, description data from the referred file may be ignored. In other cases, description data from the referred file may be used as default, but may be overridden by description data from the referring file. In even other cases, it may be relevant to combine both description data from the referred file and description data from the referring file.

[0109] References 600 and 605 illustrate an example of referencing external tracks, that define how to consider description data from the referred file and description data from the referring file. For example, it is proposed that track groups (‘trgr1) and track references (‘tref) from the referred file are ignored. However, different rules may be chosen. A generic solution may be contemplated by considering, for each possible child of TrackBox, a set of flags or bits that allows indicating selected rules. This may be done for all children (sub-boxes) of TrackBox or a subset of them. Alternatively, the number of settable rules may depend on the considered child, in which case different flags or bits may be used to enable selecting a given rule for a given child.

[0110] As an example, for each child (or sub-box) of TrackBox, the following rules may be defined:

[0111] - ignore’, in this case, the corresponding child, even if present in the referred file, is ignored;

[0112] - replace’, in this case, the corresponding child in the referred file is considered as valid for the external track, except if the same kind of child is defined in the referring file for this external track, in which case only the child in the referring file applies to the considered external track; and

[0113] - combine’, in this case, the corresponding child in the referred file is combined with the same child in the referring file (e.g., ‘edts’ child from the referred file for considered external track is combined with ‘edts’ child from the referring file for said external track). This rule has to be further specified for each type of child to clearly determine what it means to combine two such children data. In the case of edit lists, a combination may consist in applying to an external track the edits contained in the referred file and then in applying the edits specified in the ExternalTrackBox. Optionally, if different types of combinations are defined, each combination may correspond to a different rule that may be chosen for a given child of a given external track.

[0114] In order to set any of the considered rules, some flags may be defined for each considered child. For instance, for a given child denoted XXX of TrackBox (hence also a possible child Of ExternalTrackBox), flags EXTERNAL_TRACK_XXX_IGNORE, EXTERNAL_TRACK_XXX_REPLACE, or EXTERNAL_TRACK_XXX_COMBINE may be defined. Alternatively, and for the sake of illustration, two bits may be used to indicate the applicable rule if there exist three different possible rules. According to other embodiments, a given flag or a given set of bits may be used to describe the rules that apply to a given set of possible children. This is less flexible than having dedicated flag / bits for each type of possible child, but it requires fewer signaling bits.

[0115] Naturally, other rules may be defined. For example, a “forbidden” rule may be defined in order to indicate that a given child should not be redefined in the ExternalTrackBox. In addition, it may be possible to define specific flags that determine which rules should apply. For example, if a given flag is set, it may be decided that the “ignore" rule applies whereas, if not set, the “forbidden” rule would apply.

[0116] As an example, ExternalTrackBox may be associated with the box type ‘extk’ and contained by the container ‘moov1(MovieBox). This box is typically not mandatory (an ISOBMFF file does not necessarily contain an ExternalTrackBox), and a MovieBox may contain zero or more ExternaiTrackBoxes. According to ISOBMFF specification, a MovieBox should always contain a TrackBox. However, this may no more be required if a box such as ExternalTrackBox is added to ISOBMFF. Instead, it may be required that a MovieBox contains at least one track, this track being either a TrackBox or an ExternalTrackBox.

[0117] Reference 605 is an example of a box denoted ExternalTrackLocationBox, that is associated with the ExternalTrackBox illustrated with reference 600, wherein the following fields may be defined:

[0118] - referenced_track_iD indicates the identifier (track_iD) of the external track in the referred file. If its value is 0, this indicates that the referenced track is the first TrackBox present in the MovieBox of the referred file. The external track should be declared through a TrackBox, i.e., recursively referencing external tracks is preferably forbidden, although possible. The external track can use external data references or not. This can be constrained by derived specifications;

[0119] - ref erenced_handier_type indicates the handler (media) type of the track. It should be equal to the handler type of the external track in the referred file. Therefore, when a file writer adds an external file to an ISOBMFF file, it advantageously defines the ref erenced_handier_type based on the handler type indicated in the referred file for this track (which can be known from the ‘hdl box contained in the ‘mdia’ box of the track);

[0120] - media_timescaie indicates the timescale used to express the edit list, if any, for this external track. If not present, value is 0 and no edit list shall be present for this external track; (e.g., in the “other” boxes in 600) and

[0121] - location indicates the location of the referred file, for example as a URL or a URN, depending for example on the flags EXTERNAL_TRACK_URN defined hereafter. Alternatively, the location may be any indication that allows obtaining a corresponding referred file. For example, and according to particular embodiments, location may be a URI or another kind of identifier that allows a given application to determine how to obtain a corresponding document, e.g., by determining its path or by requesting it to a server. An example of such another kind of identifier may be a UUID (Universally Unique Identifier).

[0122] According to the illustrated example, the ExternaiTrackLocationBox makes it possible to identify an external track by its track iD in a referred file.

[0123] In addition, different flags may be considered. For example, the following values may be defined for the flags field Of the ExternaiTrackLocationBox:

[0124] - EXTERNAL_TRACK_EDTS_OVERWRITE (flag mask is 0x000001): if this flag is set, it indicates that one (and exactly one) edit list is present in the container for the edit lists in the ExternaiTrackBox for this track, changing (overriding) the default timing of the external track. Any edit list present in the external track should then be ignored. Otherwise, if it is not set, there should be no EditBox in the ExternaiTrackBox, and the edit lists of the external track apply. Alternatively, the presence of an EditBox may be determined through examination of the ExternaiTrackBox, in which case there is no need for defining a flag dedicated to indicating the presence of this box. In this case, the line if (flags & 1) of reference 605 may be replaced by if (edit_list_is_present), where edit_list_is_present is a Boolean value that indicates whether an EditBox is present in the ExternaiTrackBox based on the examination of this box. Optionally, a flag such as EXTERNAL_TRACK_EDTS_OVERWRITE and a Boolean value such as edit_list_is_present may be both defined, and the test of reference 605 may then be if ((flags & 1) or edit_list_is_present). As a remark, instead of

[0125] EXTERNAL_TRACK_EDTS_OVERWRITE, a flag EXTERNAL_TRACK_EDTS_SKIP may be defined, with a slightly different meaning: this flag should be set to 1 if any edit list present in the referred track should be ignored. In particular, if an edit list is present in the container for the edit lists in the ExternaiTrackBox for this track, flag should be set to 1 , possibly implicitly. Otherwise (flag is not set), there should be no EditBox in the ExternaiTrackBox, and the edit lists of the external track apply.

[0126] - EXTERNAL_TRACK_URN (flag mask is 0x000002): if this flag is set, it indicates that the location field is a URN string otherwise, if it not set, the location string is a URL. Alternatively, if the location indication may be a URL, a URN or a UUID, this flag could be complemented by another flag (e.g. 0x000004, in which case next flags are also shifted) such as EXTERNAL_TRACK_UUID that should be considered if and only if EXTERNAL_TRACK_URN is not set, and indicates, if set, that the location field is a UUID. Otherwise, if not set, the location string is a URL;

[0127] In a variant, a 2-bit flag denoted EXTERNAL_TRACK_LOCATION (flag mask is 0x000006, in which case next flags (EXTERNAL_TRACK_UDTA_IGNORE and so on) are shifted) may be defined, this 2-bit flag having for example the following possible values:

[0128] 0: the location field is a URN string,

[0129] 1 : the location field is a URL string, 2: the location field is a UUID and other values are reserved.

[0130] - EXTERNAL_TRACK_UDTA_IGNORE (flag mask is 0x000004): if this flag is set, it indicates that any userDataBox defined in the ‘trak’ box of the external track should be ignored. Otherwise, if it is not set, UserDataBox present in the referring track completes UserDataBox information of the external track. The resulting user data consists in the union of the user data declared in the different UserDataBox (i.e., if a given property is defined in both the referring file and the referred file, the property defined in the referring file is taken into consideration and the property defined in the referred file is ignored. Alternatively, the property from the referred file may be taken into account and the property from the referring file is ignored); and - EXTERNAL_TRACK_META_IGNORE (flag mask is 0x000008): if this flag is set, it indicates that any MetaBox defined in the ‘trak’ box of the external track should be ignored. Otherwise, if it is not set, MetaBox present in the referring track completes MetaBox information of the external track. The resulting meta data consists in the union of the meta data declared in the different MetaBoxes (i.e., if a given property is defined in both the referring file and the referred file, the property defined in the referring file is taken into consideration and the property defined in the referred file is ignored. Alternatively, the property from the referred file may be taken into consideration and the property from the referring file is ignored).

[0131] Alternatively, it may be decided to always include a media_timescaie field in order to define a timescale used to express an edit list that may be contained in this external track. An advantage of doing so is that it allows defining a timescale in all situations. In particular, a timescale is defined even if no edit list is present in the referred file for the considered track. In other words, in this case, there is no correlation between the presence of an edit list and the definition of a media timescale. A default value may be defined for the media timescale. For instance, a value of 0 may indicate that the media timescale is actually not set. This value may be used when there is no edit list defined in the referring file for the considered external track. If the value is different from 0, it may be used to express a preferred timescale in case of future insertion of an edit list. If an edit list is present for the external track in the referring file, value should not be 0. As another option, in case a media timescale definition is absent from a box defining the external track, the timescale defined at ‘moov1level in the referring file (or in the referred file) may be used. In an embodiment, the timescale used may be indicated using 2 values, one corresponding to the timescale defined at ‘moov1level in the referring file, the other one corresponding to the timescale defined at ‘moov1level in the referred file. In other words, in this alternative, when EXTERNAL_TRACK_EDTS_OVERWRITE (or in a variant, EXTERNAL_TRACK_EDTS_SKIP) is set and no edit list is present in the ExternalTrackBox, this implies that any edit present in the referred track is ignored and no edit is applied to the track. If edits from the referred track are used, file readers may need to remap the edit list durations from the timescale of the referred movie to the timescale of the referring movie, if these timescales differ.

[0132] It is observed that if the indicated location is a URL, it can be an absolute or a relative URL. According to some embodiments, the located resource should be a compliant ISOBMFF file. Still according to some embodiments, relative URLs are relative to the file that contains this location.

[0133] As an example, ExternaiTrackLocationBox may be associated with the box type ‘extf and contained by the container ‘extk’ (ExternaiirackBox). This box is preferably mandatory and its container ExternaiirackBox should contain exactly one ExternaiTrackLocationBox.

[0134] Reference 650 illustrates an example for referencing external tracks in ISOBMFF files. It is an alternative to boxes 600 and 605. Box 650 combines boxes 600 and 605. The same properties are considered, and various flags may also be defined in this case, as described with regards to reference 605.

[0135] Referencing external entities in an ISOBMFF referring file

[0136] Figure 7 illustrates an example of steps of a method for referencing external entities in an ISOBMFF referring file.

[0137] Such entities may be tracks, as described previously, or other entities such as items, track groups that would be defined at top level of the file, or “entities to group”.

[0138] As illustrated, a first step is directed to obtaining a first ISOBMFF file (step 700), corresponding to the referring file. This file may be a file newly created or an already existing file that is being edited. It may be selected automatically or upon a user’s request.

[0139] Next, the entities to be referenced as external entities, that are described and stored in one or several second ISOBMFF files, are identified (step 705). Identification of these external entities may be carried out automatically or upon a user’s choice.

[0140] After having identified the one or several entities to be referenced in the referring file as external entities, the location of the second ISOBMFF files where the identified entities are described and stored, i.e., the location of the one or several referred files, is determined. An indication of the location of the referred files is then added to the referring file obtained in step 700, along with an identifier of the external entities within the corresponding referred files (step 710).

[0141] Such a location of the referred files and such an identifier of the external entities in the referred files may be stored in ISOBMFF boxes such as the ones described by reference to Figure 8, where the location indication is defined as a string named “location” (e.g., references 800 and 805 in Figure 8) and the entity identifier is stored as an unsigned integer on 32 bits (e.g., “referencedJtemJD” or “referenced_group_ID”). For the sake of illustration, the location may be the location of the file in the form of an URL (Uniform Resource Locator) or URN (Uniform Resource Name). It is observed that the examples provided in Figure 8 are just examples, but there are many possible boxes for describing external items or, more generally, external entities.

[0142] Next, an identifier of each of the external entities in the referring file (i.e. , the identifier that is to be used for processing the referring file), is defined and associated with the corresponding identifier of the entity in the referred file (step 715). Such an identifier of an external entity in the referring file may be the same as the identifier of this external entity in the corresponding referred file or may be different. The association between the identifiers of the entity in the referred file and in the referring file may result from the data structure used for storing these identifiers. The aim of using a different identifier in the referring file and in the referred file is to avoid conflict of entity identifiers.

[0143] It is to be noted that the location indication of step 710, the external identifier of step 710, and the internal identifier of step 715, are not necessarily comprised in the same box. For instance, in the case of a track, a new version of T rackBox may be defined that enables defining whether a track is external or not. In all cases (i.e., whether the track is external or not), a track identifier for this track is defined. In addition, if the track is an external track (which may be known based on the version of the box, or based on a dedicated value of the flags field of the box, e.g., track_is_external with value 0x000010 or any non-conflicting value, or based on a Boolean property of the box or a specific entry in a DataReferenceBox ‘dref’ associated with the track), the location of the referred file and the external track identifier in the referred file may be described in a dedicated box that maps internal identifiers onto external identifiers (and external locations). Such a dedicated box may for instance be located in a MovieBox. More generally, this applies to any kind of external entity that may be referred to in a referring file. In another example, the location indications may be comprised in a dedicated table so that they may be referred to through an index, and another box may be used to define a mapping between internal identifiers (identifiers used in the referring file) and external entities (each external entity being characterized by an external location and an external identifier). An advantage of above embodiment is to define mapping information for all references to external entities in one single place to ease the editing of the referring file.

[0144] According to some embodiments, an indication of the type of each external entity is also added in the referring file (not represented). Adding the type of entity to the referring file makes it possible for a file reader to determine whether it may process the corresponding entity, without having to access and load the referred file. In addition, this allows verifying that the indicated type in the referring file is the same as the indicated type in the referred file, when the external entity is selected to be processed.

[0145] Figure 8 provides examples of ISOBMFF boxes for referencing external entities in ISOBMFF files. Reference 800 illustrates an example of an ISOBMFF box for referencing external items in ISOBMFF files and reference 805 illustrates an example of an ISOBMFF box for referencing external entities to group in ISOBMFF files.

[0146] More precisely, reference 800 illustrates an External item box making it possible to reference one or several external items. According to this example, the Externaiitem box comprises the number of items to reference (item_count) and, for each item to reference, it defines the following properties:

[0147] - item_iD, which provides an identifier of the considered item in the referring file;

[0148] - ref erenced_item_iD, which provides the identifier of the considered item in the referred file;

[0149] - ref erenced_item_type, which is optional and indicates the type of the considered item, which should correspond to the item_type property of this item in the referred file; and

[0150] - location, which indicates the location of the referred file comprising the considered item.

[0151] Optionally, the location property may be factorized for several external items belonging to the same referred file. For example, factorization may be achieved by first considering a iocation_count property, corresponding to the number of different referred files in the Externaiitem box. Then, for each considered referred file, its location is indicated, as well as the corresponding number of items (item_count), and for each item, item ID, referenced item ID, and referenced item type are specified. Alternatively, factorization may also be achieved by taking location property out of the loop, and by using a different Externaiitem box for each referred file.

[0152] It is also noted that different versions may be specified so as to support different number of bits for item identifiers. In the example illustrated with reference 800, identifiers are 16-bit unsigned integer, but it may for instance be decided that if version is equal to 1 , 32 bits should be used instead. As an example, the definition of an Externaiitem box may be the following: class Externaiitem extends FullBox ( ' iext ' , version , 0) { if (version == 0) { unsigned int(16) item count; } else { unsigned int(32) item count; } for (i=0; i < item count; i+ + ) { if (version == 0) { unsigned int(16) item ID; unsigned int(16) referenced item ID; } else { unsigned int(32) item ID; unsigned int(32) referenced item ID; } unsigned int(32) referenced item type; utf8string location ; } }

[0153] In some embodiments, the location field may be shared by several external items for obtaining a more compact representation. For instance, the definition of an Externalltem box may be the following: class Externalltem extends FullBox ( ' iext ' , version, 0) { unsigned int(16) reference count; for (j = 0; j < reference count; j+ + ) { utf8string location; if (version == 0) { unsigned int(16) item count; } else { unsigned int(32) item count;

[0154] } for (1=0; i < item count; i+ + ) { if (version == 0) { unsigned int(16) item ID; unsigned int(16) referenced item ID; } else { unsi gned int (32) i tem ID; unsigned int (32) referenced i tem ID;

[0155] } unsigned int (32) referenced i tem type; } } }

[0156] In this definition, reference_count is the number of ISOBMFF files comprising items that are referred to as external items.

[0157] In some embodiments, other fields of the ItemlnfoEntry box may be replicated in the Externalltem box to provide more information about the referred item.

[0158] For an external item, there should typically be no entry in the Hoc’ box (ItemLocation box), or in ‘iinf’ (Iteminfo box), as these data are already comprised in the referred file and have no reason to be redefined. Optionally, it is possible to enable redefinition of specific parts of these boxes.

[0159] Like for the tracks and their children boxes, means may be provided to define how to handle the item properties of each item by considering item properties possibly defined in the referred file as well as item properties possibly defined in the referring file. For example, it may be decided that item properties from referring file override those of the referred file, except for item properties marked as essential in the referred file. Possibly the Externalltem box may comprise some signaling to indicate how to combine the item properties defined in the referred file with those defined in the referring file.

[0160] In some embodiments, an ISOBMFF file may refer to item properties defined in another file as external item properties. For instance, a new ExternalltemPropertyBox box may be added to the ItemPropertiesBox to signal external item properties defined in another file: ali gned (8) class ItemPropertiesBox extends Box ( 'iprp ' ) {

[0161] ExternalltemPropertyBox external property; / / optional ItemPropertyContainerBox property container ; ItemPropertyAssociationBox associa ti on [ ] ;

[0162] }

[0163] This ExternalltemPropertyBox, ‘ipex’, may be defined as follows: ali gned (8) class ExternalltemPropertyBox ( ' ipex ' ) { unsi gned int (1 6) reference count ; for (j = 0; j < reference count; j + + ) { utf8string location ; unsi gned int (32) entry count ; for (i = 0; i < entry count; i + + ) { unsi gned int (1 6) property index;

[0164] }

[0165] }

[0166] }

[0167] In this definition, reference_count is the number of ISOBMFF files referred to. location is an identifier of an ISOBMFF file referred to. entry_count is the number of external item properties defined as a reference to an item property of the referred to ISOBMFF file. property_index is the 1 -based index of a referred to item property used as an external item property.

[0168] In these embodiments, external item properties may be prepended to the list of properties defined by the ItemPropertyContainerBox, ‘ipco’. Alternatively, these external item properties may be appended to the list of properties defined by the ItemPropertyContainerBox.

[0169] In some cases, a main item or a main entity in a given ISOBMFF file may refer to other (secondary) items or entities or be referred by other (secondary) items or entities. This is for example the case for a derived item that references one or more input items. This is also the case for a main item that is referenced by a metadata item describing it using a ‘cdsc’ item reference. In such cases, the main item may be embedded into another file as an external item. A file reader processing such an external item has to consider the references from the main item to secondary items or entities or from secondary items or entities to the main item as references associated with the referred file (e.g., the identifier of the main item or of a secondary item is defined in the scope of the referred file). The file reader has to handle correctly the references between items depending on where the reference is defined. For a reference defined in the referring file, the item identifiers (item_iDs) used by the reference are those defined in the referring file. For a reference defined in the referred file, the item identifiers (item_iDs) used are those defined in the referred file. Possibly, a file reader processing such an external item may use the references in the referred file from secondary items or entities to the main item depending on the type of the reference. For instance, a reference of type ‘cdsc’ from an item or an entity to a main item referred to as an external item from another file may be used. Indeed, such a reference defines metadata for the item and may be useful while processing the item. As another example, a reference of type ‘dimg’ from a secondary item or entity to a main item referred to as an external item from another file may be ignored. Indeed, such a reference defines another item from this item and may not be useful while processing the item.

[0170] Reference 805 illustrates an ExternaiEntityToGroup box which makes it possible to include an external “Entity to Group” in a given ISOBMFF file, using the similar means as the ones described above for including an external track or an external item.

[0171] For the sake of illustration, ExternaiEntityToGroup comprises the following properties:

[0172] - ref erenced_group_iD, which indicates an identifier for an Entity to Group in the referred file;

[0173] - ref erenced_grouping_type, which indicates a grouping type for the Entity to Group in the referred file; and

[0174] - location, which indicates the location of the referred file.

[0175] Processing an item or entity reference may be carried out similarly to processing a track-reference as described above. Processing an entity to group may be carried out similarly to processing a track group as described above.

[0176] An ISOBMFF file may be marked as only usable through a reference. For instance, no track in such a file would all have its flags value track_in_movie that is set (i.e. no track can be selected for presentation). Alternatively or complementarily, the box hierarchy of such files may be different from usual ISOBMFF files. In particular, it may be decided that for such files, no ‘ftyp’ box may be provided, or even no ‘moov’ box (or an empty one). In some cases, it may even be considered that such file may start directly by the entity they define. For instance, the main box would be the one describing the considered entity (e.g. a TrackBox for a track), instead of a MovieBox, as is usually the case for ISOBMFF files.

[0177] In the ExternalTrackBox, there may be a specific indication to override some properties of an external entity. For example, an item may be associated with an essential property or samples of a track may be associated with an essential sample group. If, in the referring file, these properties are changed to become non-essential, then the essential aspect of the property or of the sample group should be overridden. This may be done through a specific flags value, for example in the ExternalTrackLocationBox with the other flags to control overriding or ignoring the subbox of an external track. For example, a flags value is defined EXTERNAL_TRACK_ESSENTIAL_IGNORE (0x00020) that, when set, cancels all the essential sample group for a track. This means that any essential sample group in the referred file for this track should be considered by parsers as a regular sample group, i.e. no more essential. When not set, the essential sample groups declared in the referred file are not changed and apply in the presentation described in the main file. A flags value EXTERNAL_TRACK_ESSENTIAL_OVERWRITE (for example 0x00040) may also be defined. When set, it allows to redefine the essentiality of the sample groups present in the external track. When not set, the essential sample groups declared in the referred file are not changed and apply in the presentation described in the main file, unless the flags value EXTERNAL_TRACK_ESSENTIAL_IGNORE is set. In order to indicate which essential sample groups are overridden or possibly ignored, additional parameters may be defined in the ExternalTrackLocationBox. Their presence is controlled by the values of the flags value for essential sample groups. These parameters may indicate which essential sample group should be ignored or overridden. Otherwise, the essential sample group is indicated by its grouping type and possibly as the pair (grouping_type, grouping_type_parameter) when the grouping_type_parameter is defined for the essential sample group. When no further parameters are present when the EXTERNAL_TRACK_ESSENTIAL_IGNORE flags value is set, it should be interpreted as all essential samples groups are ignored. Then, the ‘esgh’ sample group should also be ignored and the sample entry for the external track should be the one in the OriginalFormatBox and not the restricted sample entry (‘resv1, ‘resa’, etc. depending on the media type). When essential sample groups are modified (EXTERNAL_TRACK_ESSENTIAL_OVERWRITE flags value is set), this may require the definition of a new ‘esgh’ sample group for the track in a dedicated box (either in ExternalTrackLocationBox or in a dedicated box). However, it may be too complex to map samples to the new ‘esgh’ sample group, so preferably a default sample grouping with static_mapping is preferable to remap samples onto a list of operations or transformations to perform by the reader. It is to be noted that the same mechanism may apply to items and their essential properties. Similar flags value may be defined in the ExternalltemBox to ignore or overwrite essential item properties. Further parameters to indicate which property should be ignored or overridden may rely on the implicit index of the property in the ‘ipco’ box of the referred file. Figure 9 is a schematic block diagram of a computing device 900 for implementation of one or more embodiments of the disclosure. The computing device 900 may be a device such as a micro-computer, a workstation, or a light portable device. The computing device 900 comprises a communication bus 902 connected to:

[0178] - a central processing unit (CPU) 904, such as a microprocessor;

[0179] - a random access memory (RAM) 908 for storing the executable code of the method of embodiments of the disclosure as well as the registers adapted to record variables and parameters necessary for implementing the method for encapsulating, indexing, de-encapsulating, and / or accessing data, the memory capacity thereof can be expanded by an optional RAM connected to an expansion port for example;

[0180] - a read only memory (ROM) 906 for storing computer programs for implementing embodiments of the disclosure;

[0181] - a network interface 912 that is, in turn, typically connected to a communication network 914 over which digital data to be processed are transmitted or received. The network interface 912 can be a single network interface, or composed of a set of different network interfaces (for instance wired and wireless interfaces, or different kinds of wired or wireless interfaces). Data are written to the network interface for transmission or are read from the network interface for reception under the control of the software application running in the CPU 904;

[0182] - a user interface (Ul) 916 for receiving inputs from a user or to display information to a user;

[0183] - a hard disk (HD) 910; and / or

[0184] - an I / O module 918 for receiving / sending data from / to external devices such as a video source or display.

[0185] The executable code may be stored either in read only memory 906, on the hard disk 910 or on a removable digital medium for example such as a disk. According to a variant, the executable code of the programs can be received by means of a communication network, via the network interface 912, in order to be stored in one of the storage means of the communication device 900, such as the hard disk 910, before being executed.

[0186] The central processing unit 904 is adapted to control and direct the execution of the instructions or portions of software code of the program or programs according to embodiments of the disclosure, which instructions are stored in one of the aforementioned storage means. After powering on, the CPU 904 is capable of executing instructions from main RAM memory 908 relating to a software application after those instructions have been loaded from the program ROM 906 or the hard-disc (HD) 910 for example. Such a software application, when executed by the CPU 904, causes the steps of the flowcharts shown in the previous figures to be performed.

[0187] In this embodiment, the apparatus is a programmable apparatus which uses software to implement the disclosure. However, alternatively, the present disclosure may be implemented in hardware (for example, in the form of an Application Specific Integrated Circuit or ASIC).

[0188] Although the present disclosure has been described hereinabove with reference to specific embodiments, the present disclosure is not limited to the specific embodiments, and modifications will be apparent to a person skilled in the art which lie within the scope of the present disclosure.

[0189] Many further modifications and variations will suggest themselves to those versed in the art upon making reference to the foregoing illustrative embodiments, which are given by way of example only and which are not intended to limit the scope of the disclosure, that being determined solely by the appended claims. In particular the different features from different embodiments may be interchanged, where appropriate.

[0190] In the claims, the word “comprising” does not exclude other elements or steps, and the indefinite article “a” or “an” does not exclude a plurality. The mere fact that different features are recited in mutually different dependent claims does not indicate that a combination of these features cannot be advantageously used.

Claims

CLAIMS1. A method for generating a first ISOBMFF file for describing a media presentation, in a processing device, the method comprising: generating metadata referencing at least one entity in a second ISOBMFF file, the second ISOBMFF file comprising a description of the at least one entity and associated media data, the generated metadata comprising a first identifier for identifying the at least one entity in the first ISOBMFF file, the first identifier for identifying the at least one entity in the first ISOBMFF file being associated with location information for locating the second ISOBMFF file and with a second identifier for identifying the at least one entity in the second ISOBMFF file; and generating the first ISOBMFF file, the generated first ISOBMFF file comprising the generated metadata.

2. The method of claim 1 , wherein the at least one entity is at least one track, the at least one track comprising a sequence of samples of the media presentation.

3. The method of claim 1 , wherein the at least one entity is at least one item.

4. The method of claim 1 , wherein the at least one entity is at least one grouping data structure grouping tracks, items, or tracks and items.

5. The method of any one of claims 1 to 4, wherein the generated metadata further comprise the location information for locating the second ISOBMFF file and the second identifier for identifying the entity in the second file.

6. The method of any one of claim 1 to 4, wherein association information associating the first identifier with the location information and the second identifier is comprised in a dedicated box different from the generated metadata.

7. The method of any one of claims 1 to 6, wherein the generated metadata further comprise a type of media data associated with the at least one entity.

8. The method of any one of claims 1 to 7, wherein the generated metadata further comprise at least a partial description of the at least one entity in the first ISOBMFF file.

9. The method of claim 8, wherein the generated metadata further comprise flags indicating which of metadata describing the at least one entity in the first ISOBMFF file and in the second ISOBMFF file are to be used.

10. A method for processing a first ISOBMFF file describing a media presentation, in a processing device, the method comprising: obtaining the first ISOBMFF file, the first ISOBMFF file comprising metadata referencing at least one entity in a second ISOBMFF file, the second ISOBMFF file comprising a description of the at least one entity and associated media data, the metadata referencing the at least one entity comprising a first identifier for identifying the at least one entity in the first ISOBMFF file, the first identifier for identifying the at least one entity in the first ISOBMFF file being associated with location information for locating the second ISOBMFF file and with a second identifierfor identifying the at least one entity in the second ISOBMFF file; obtaining the at least one entity in the second ISOBMFF file; and processing the obtained at least one entity.

11. The method of claim 10, further comprising determining whether the first ISOBMFF file can be processed.

12. The method of claim 10 or claim 11 , further comprising determining whether the at least one entity is to be processed.

13. The method of any one of claims 10 to 12, further comprising determining whether the at least one entity can be processed.

14. The method of any one of claims 10 to 13, wherein the at least one entity is at least one track, the at least one track comprising a sequence of samples.

15. The method of any one of claims 10 to 13, wherein the at least one entity is at least one item.

16. The method of any one of claims 10 to 13, wherein the at least one entity is at least one grouping data structure grouping tracks, items, or tracks and items.

17. The method of any one of claims 10 to 16, wherein the at least one entity is requested on the basis of the location information and of the second identifier.

18. The method of any one of claims 10 to 17, wherein the metadata referencing the at least one entity further comprise a type of media data associated with the at least one entity, the method further comprising verifying a validity of the at least one entity on the basis of the type.

19. The method of any one of claims 10 to 18, wherein the metadata referencing the at least one entity further comprise at least a partial description of the at least one entity in the first ISOBMFF file.

20. The method of claim 19, further comprising identifying metadata describing the at least one entity, wherein the metadata referencing the at least one entity further comprise flags indicating which of metadata describing the at least one entity in the first ISOBMFF file and in the second ISOBMFF file are to be used.

21. A computer program product for a programmable apparatus, the computer program product comprising instructions for carrying out each step of the method according to any one of claims 1 to 20 when the program is loaded and executed by a programmable apparatus.

22. A non-transitory computer-readable storage medium storing instructions of a computer program for implementing the method according to any one of claims 1 to 20.

23. A processing device comprising a processing unit configured for carrying out each step of the method according to any one of claims 1 to 20.

Images