Method, device, and computer program for improving the description of sample attributes of encapsulated media data

Abstract

At least an embodiment of a method of encapsulating media data in a media file, in a processing device, the method comprising generating a track comprising a sequence of samples based on the media data, generating descriptive metadata for reconstructing a list of references for each sample of the sequence of samples, each reference comprising a sample identifier code and a list of attributes, and generating the media file comprising the generated track and the descriptive metadata, wherein the descriptive metadata comprise a first indication indicating a relative position of a reference relative to a given sample and a second indication indicating a number of samples following the reference in the sequence of samples, the list of references comprising at least one repetition of a set of references comprising the reference referred to by the relative position and as many following samples as set in the number of samples.

Description

[0001] METHOD, DEVICE, AND COMPUTER PROGRAM FOR IMPROVING THE DESCRIPTION OF SAMPLE ATTRIBUTES OF ENCAPSULATED MEDIA DATA

[0002] FIELD OF THE DISCLOSURE

[0003] The present disclosure relates to a method, a device, and a computer program for improving the description of sample attributes, for example sample properties or characteristics and / or sample references or dependencies, of encapsulated media data in a media file.

[0004] BACKGROUND OF THE DISCLOSURE

[0005] The disclosure relates to encapsulating and processing media data, e.g. according to ISO Base Media File Format (ISOBMFF) as defined by the MPEG standardization organization, to provide a flexible and extensible format that facilitates interchange, management, editing, and presentation of one or more timed media data or bit-streams and to improve its delivery for example over an IP (Internet Protocol) network such as the Internet using adaptive HTTP (Hypertext Transfer Protocol) streaming protocol. The processing of encapsulated media data, e.g. according to ISO Base Media File Format (ISOBMFF), may comprise displaying, streaming, filtering or repairing the media data.

[0006] The International Standard Organization Base Media File Format (ISOBMFF, ISO / IEC 14496-12, MPEG-4 Part-12) is a well-known flexible and extensible format that describes the encapsulation of timed media data or bit-streams either for local storage or transmission via a network or via another bit-stream delivery mechanism. The timed media data may represent encoded media data. This file format has several extensions, e.g. MPEG-4 Part-15, ISO / IEC 14496-15 that describes encapsulation tools for various NAL (Network Abstraction Layer) unit-based video encoding formats. Examples of such encoding formats are AVC (Advanced Video Coding), SVC (Scalable Video Coding), HEVC (High Efficiency Video Coding), L-HEVC (Layered HEVC), MV-HEVC (Multiview HEVC), LCEVC (Low Complexity Enhancement Video Coding) or VVC (Versatile Video Coding). Other examples of file format extensions are ISO / IEC 23090-18 for carriage of Geometry-based Point Cloud Compression (G-PCC), ISO / IEC 23090-10 for carriage of Visual Volumetric Video-based Coding (V3C) Data, or Video-based Dynamic Mesh Coding (V-DMC). ISOBMFF is object-oriented. It is composed of building blocks called boxes (also denoted objects, atoms, structure-data, or data structures, each of which being identified by a four character code) that are sequentially or hierarchically organized and that define descriptive parameters of the timed media data or bit-stream such as timing, properties of the timed media data, offsets in the timed media data, and structure parameters. In the file format, the overall presentation over time is called a movie. The movie is described by a movie box (with the four character code ‘mooV) at the top level of the media or presentation file. This movie box represents an initialization information container containing a set of various boxes describing the presentation. It may be logically divided into tracks represented by track boxes (with the four character code ‘trai ). Each track (uniquely identified by a track identifier track_ID)) represents a timed sequence of media data pertaining to the presentation (for example frames of video, timed metadata, or sampled audio samples). Within each track, each timed unit of media data is called a sample, which may be a video frame, an audio sample, or a set of timed metadata. In other words, a sample of the track represents all the media data associated with a single time in the track. Samples are implicitly numbered in sequence. The actual sample data are in boxes called Media Data boxes (with the four character code ‘mdaf) or Identified Media Data boxes (with the four character code ‘imda’) at the same level as the movie box. It is noted that the description cost of samples may be partly reduced by mutualizing sample descriptions when possible as well as the sample to chunk mapping, i.e. by regrouping samples in fixed-size chunks of samples. The movie may also be fragmented, i.e. organized temporally as a movie box containing information for the whole presentation followed by a list of movie fragments, a movie fragment being a pair of Movie Fragment box (with the four-character code ‘moof) and Media Data box or a pair of Movie Fragment box and Identified Media Data box. Within a movie fragment, there is a set of track fragments described by Track Fragment boxes (with the four character code ‘traf) in the Movie Fragment box. There are zero or more track fragments per movie fragment. In turn, the Track Fragment box contains zero or more Track Run boxes (‘truri), each of which documenting a contiguous run of samples for that track fragment.

[0007] ISOBMFF and its extensions specify a grouping mechanism dedicated to groups of samples, generically called sample groups, that are based on the combination of two boxes, either a sample-to-group box (with the four character code ‘sbgp’) with a sample group description box (with the four character code ‘sgpd), or a compact sample- to-group box (with the four character code ‘csgp’) with the sample group description box ’sgpd’. The sample grouping mechanism can be used to indicate that certain properties associated with an indicated grouping type or semantic and defined in the sample group description box ‘sgpd’ apply, within a track, to a group of samples indicated by a sample- to-group box ‘sbgp’ or a compact sample-to-group box ‘csgp’.

[0008] While these file formats have proven to be efficient, there is a continuous need to improve the description of the properties associated with a sample, in particular to reduce the cost in bytes of these descriptions, and e.g. to improve the description of sample references or dependencies of samples within a track or across tracks.

[0009] SUMMARY OF THE DISCLOSURE

[0010] The present disclosure has been devised to address one or more of the foregoing concerns.

[0011] According to a first aspect of the disclosure, it is provided a method of encapsulating media data in a media file, in a processing device, the method comprising: generating a track comprising a sequence of samples based on the media data; generating descriptive metadata for reconstructing a list of references for samples of the sequence of samples, each reference comprising a sample identifier code and a list of attributes; and generating the media file comprising the generated track and the descriptive metadata; wherein the descriptive metadata comprise a first indication indicating a relative position of a reference relative to a given sample and a second indication indicating a number of samples following the reference in the sequence of samples, the list of references comprising at least one repetition of a set of references comprising the reference referred to by the relative position and as many following samples as set in the number of samples.

[0012] Accordingly, the method of the disclosure makes it possible to improve encapsulation efficiency, in particular by improving mutualization of metadata used to describe different sets of media data.

[0013] According to particular embodiments, the list of attributes is a list of dependencies, each of the dependencies being a dependency to another sample.

[0014] Still according to particular embodiments, the list of attributes consists in properties.

[0015] Still according to particular embodiments, the sample identifier code is determined as a function of a picture order count of the sample. Still according to particular embodiments, the first and the second indication define a first pattern, the first pattern being referenced by a first and a third set of samples, a second set of samples, located between the first and third set of samples, referencing a second pattern, different from the first pattern.

[0016] Still according to particular embodiments, the descriptive metadata further comprise a third indication indicating a number of samples to describe based on the reference defined by the first and the second indication.

[0017] Still according to particular embodiments, a single structure signals both a description of attributes and an association of the attributes with samples.

[0018] Still according to particular embodiments, a first structure signals descriptions of attributes and a second structure signals associations of these descriptions with samples.

[0019] Still according to particular embodiments, the descriptive metadata further comprise a reference flag to indicate whether a set of at least one sample is described as a function of the list of references comprising at least one repetition of the set of references or as an explicit reference.

[0020] Still according to particular embodiments, the descriptive metadata further comprise an identifier flag to indicate whether a sample identifier code is a value of the sample identifier or a difference between the sample identifier and an identifier of a preceding sample in the reconstructed list of references.

[0021] Still according to particular embodiments, the descriptive metadata further comprise an identifier flag to indicate whether both a sample identifier code is a value of the sample identifier, and the list of attributes is a list of sample identifier values.

[0022] Still according to particular embodiments, if the reference flag is set to a first predetermined value and the identifier flag is set to a second predetermined value, the descriptive metadata comprises a sample identifier value for each sample indicated by the third indication.

[0023] Still according to particular embodiments, each of the dependencies is a direct dependency to another sample.

[0024] According to a second aspect of the disclosure, it is provided a method for processing a media file encapsulating media data, in a processing device, the media data comprising a plurality of samples, the method comprising: obtaining, from the media file, a track comprising a sequence of samples based on the media data; obtaining, from the media file, descriptive metadata for reconstructing a list of references for samples of the sequence of samples, each reference comprising a sample identifier code and a list of attributes; and wherein the descriptive metadata comprise a first indication indicating a relative position of a reference relative to a given sample and a second indication indicating a number of samples following the reference in the sequence of samples, the list of references comprising at least one repetition of a set of references comprising the reference referred to by the relative position and as many following samples as set in the number of samples.

[0025] Accordingly, the method of the disclosure makes it possible to improve encapsulation efficiency, in particular by improving mutualization of metadata used to describe different sets of media data.

[0026] According to particular embodiments, the list of attributes is a list of dependencies, each of the dependencies being a dependency to another sample.

[0027] Still according to particular embodiments, the list of attributes consists in properties.

[0028] Still according to particular embodiments, the sample identifier code is determined as a function of a picture order count of the sample.

[0029] Still according to particular embodiments, the first and the second indication define a first pattern, the first pattern being referenced by a first and a third set of samples, a second set of samples, located between the first and third set of samples, referencing a second pattern, different from the first pattern.

[0030] Still according to particular embodiments, the descriptive metadata further comprise a third indication indicating a number of samples to describe based on the reference defined by the first and the second indication.

[0031] Still according to particular embodiments, a single structure signals both a description of attributes and an association of the attributes with samples.

[0032] Still according to particular embodiments, a first structure signals descriptions of attributes and a second structure signals associations of these descriptions with samples.

[0033] Still according to particular embodiments, the descriptive metadata further comprise a reference flag to indicate whether a set of at least one sample is described as a function of the list of references comprising at least one repetition of the set of references or as an explicit reference. Still according to particular embodiments, the descriptive metadata further comprise an identifier flag to indicate whether a sample identifier code is a value of the sample identifier or a difference between the sample identifier and an identifier of a preceding sample in the reconstructed list of references.

[0034] Still according to particular embodiments, the descriptive metadata further comprise an identifier flag to indicate whether both a sample identifier code is a value of the sample identifier, and the list of attributes is a list of sample identifier values.

[0035] Still according to particular embodiments, if the reference flag is set to a first predetermined value and the identifier flag is set to a second predetermined value, the descriptive metadata comprises a sample identifier value for each sample indicated by the third indication.

[0036] Still according to particular embodiments, each of the dependencies is a direct dependency to another sample.

[0037] According to another aspect of the disclosure, it is provided a method of encapsulating media data in a media file, in a processing device, the method comprising: generating a track comprising a sequence of samples based on the media data; generating descriptive metadata for reconstructing a list of references for each sample of the sequence of samples, each reference comprising a sample identifier code and a list of dependencies to another sample; and generating the media file comprising the generated track and the descriptive metadata; wherein the descriptive metadata comprise an indication indicating whether the sample identifier code of a reference is an absolute sample identifier or a difference with the sample identifier of the preceding reference in the reconstructed list of references.

[0038] Accordingly, the method of the disclosure makes it possible to improve encapsulation efficiency, in particular by improving mutualization of metadata used to describe different sets of media data.

[0039] According to particular embodiments, the sample identifier is a picture order count of the sample.

[0040] Still according to particular embodiments, the descriptive metadata further comprises pattern information for reconstructing the list of references.

[0041] Still according to particular embodiments, the pattern information comprises an offset indicating the start of a pattern from the reconstructed list of references

[0042] Still according to particular embodiments, the pattern information comprises a number of references in the pattern starting from the offset. Still according to particular embodiments, the media file complies with the ISO Base Media File Format, ISOBMFF.

[0043] At least parts of the methods according to some embodiments of the disclosure may be computer implemented. Accordingly, some embodiments of the present disclosure may take the form of an entirely hardware embodiment, an entirely software embodiment (including firmware, resident software, micro-code, etc.) or an embodiment combining software and hardware aspects that may all generally be referred to herein as a "circuit", a "module", or a "system". Furthermore, some embodiments of the present disclosure may take the form of a computer program product embodied in any tangible medium of expression having computer usable program code embodied in the medium.

[0044] Since some embodiments of the present disclosure can be implemented in software, some embodiments of the present disclosure can be embodied as computer readable code for provision to a programmable apparatus on any suitable carrier medium. A tangible carrier medium may comprise a storage medium such as a floppy disk, a CD-ROM, a hard disk drive, a magnetic tape device or a solid state memory device, and the like. A transient carrier medium may include a signal such as an electrical signal, an electronic signal, an optical signal, an acoustic signal, a magnetic signal or an electromagnetic signal, e.g. a microwave or RF signal.

[0045] BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Embodiments of the disclosure will now be described, by way of example only, and with reference to the following drawings in which:

[0047] Figure 1 illustrates an example of transmission of media data from an encapsulation device to a processing device;

[0048] Figure 2a illustrates a compact sample to group definition for associating a property description with a sample as defined by ISO / IEC 14496-12;

[0049] Figure 2b illustrates an example of use of a compact sample to group definition for associating a property description with a sample as defined by ISO / IEC 14496-12;

[0050] Figure 3a illustrates an example of a syntax structure for describing properties associated with samples, according to some embodiments of the disclosure;

[0051] Figure 3b illustrates an example of use of the syntax structure illustrated in Figure 3a for describing properties associated with samples, according to some embodiments of the disclosure; Figure 4 illustrates an example of a syntax structure for describing the sample references of samples according to some embodiments of the disclosure;

[0052] Figure 5 illustrates an example of describing sample references of a sequence of samples using the syntax structure illustrated in Figure 4, according to some embodiments of the disclosure;

[0053] Figure 6 illustrates an example of a syntax structure for describing sample references of samples according to some embodiments of the disclosure;

[0054] Figure 7 illustrates an example of describing sample references of a sequence of samples using the syntax structure illustrated in Figure 6, according to some embodiments of the disclosure;

[0055] Figure 8 is a block diagram illustrating an example of steps carried out by an encapsulating device to encapsulate media data in a media file, according to some embodiments of the disclosure;

[0056] Figure 9 is a block diagram illustrating an example of steps carried out by a processing device to process a media file, according to some embodiments of the disclosure; and,

[0057] Figure 10 schematically illustrates an example of a processing device configured to implement at least one embodiment of the present disclosure.

[0058] DETAILED DESCRIPTION OF THE DISCLOSURE

[0059] According to some embodiments, the disclosure makes it possible to improve the description of attributes of samples of encapsulated media data, for example sample properties or characteristics, and / or sample references or dependencies between samples of encapsulated media data.

[0060] Still according to some embodiments, such a description comprises a first indication indicating a relative position of a sample reference relative to a given sample, for example in terms of POC (Picture Order Count), in a sequence of samples, and a second indication indicating a number of samples following the sample reference in the sequence of samples, to form a pattern that may be used to describe following samples in the sequence of samples, the pattern being repeated once or more. The description may also comprise the number of samples described by the pattern.

[0061] Figure 1 illustrates an example of transmission of media data from an encapsulation device to a processing device. As illustrated, a server or encapsulation device 100 comprises an encapsulation module 105 and processes media data 125, e.g. video and / or audio data, for streaming or for storage. To that end, the server or encapsulation device 100 generates, obtains or receives media data 125 comprising, for example, an original sequence of images. Optionally, it can encode the sequence of images into encoded media data (or bit-streams) using a media encoder (e.g. a video encoder), not represented. It encapsulates the media data, possibly encoded, in one or more media files or media segments 130 using encapsulation module 105. The encapsulation process mainly consists in storing the media data in ISOBMFF boxes and generating and / or storing associated metadata in other ISOBMFF boxes describing the media data. Encapsulation module 105 comprises at least one of a writer or a packager to encapsulate the media data. When being needed, the media encoder may be implemented within encapsulation module 105 to encode received media data or may be separate from encapsulation module 105. The server or encapsulation device 100 may be connected, via a network interface (not represented), to a communication network 110 to which is also connected, via a network interface (not represented), a client or processing device 120 and may transmit the one or several media files or media segments 130, or portions thereof, to the client or processing device 120 via the communication network 110. In a variant, the one or several media files or media segments 130 may be exchanged between the encapsulation device 100 and the processing device 120 via a storage device or via a memory. A portion of a media file represents a byte-range of the media file and may comprise metadata only, metadata and one or several samples, or one or several samples. Possibly the last entity of a portion of a media file (e.g., the last sample or the last box in an ISOBMFF metadata) may be incomplete.

[0062] Client or processing device 120 is used for processing media file(s), or portions thereof, received from communication network 110, or read from a storage device or a memory, for example for processing media file 130. The one or several media files or media segments 130, or portions thereof, may be filtered or repaired by the server or encapsulation device 100 before being transmitted or may be filtered or repaired by the client or processing device 120 after being received.

[0063] After the received media file has been de-encapsulated in de-encapsulation module 115 (also known as a parser), the de-encapsulated data (or parsed data), corresponding to media data or to a bit-stream, are optionally decoded, forming, for example, audio and / or video data that may be stored, rendered (e.g. play or display), or output. When being needed, the media decoder may be implemented within deencapsulation module 115 or it may be separate from de-encapsulation module 115. The media decoder may be configured to decode media data or one or more bit-streams in parallel.

[0064] In a variant, the encapsulation device 100 and the processing device 120 can be the same device.

[0065] It is noted that media file 130 may be communicated to de-encapsulation module 115 via a communication network 110 in different ways. For instance, media data encapsulated with ISOBMFF can be used for adaptive streaming with HTTP. In such case, encapsulation module 105 may generate media file 130 with a media description (e.g. DASH MPD or HTTP Live Streaming (HLS) manifest) and communicates (or streams) it directly to de-encapsulation module 115 upon receiving a request from client 120. For example, MPEG DASH (for “Dynamic Adaptive Streaming over HTTP”), HTTP Live Streaming (HLS), and Smooth Streaming are well-known HTTP adaptive streaming protocols enabling segments or fragment-based delivery of media files. The MPEG DASH standard (see “ISO / IEC 23009-1 , Dynamic adaptive streaming over HTTP (DASH), Parti : Media presentation description and segment formats”) makes it possible to establish a link between a compact description of the content(s) of a media presentation and the HTTP addresses. Usually, this association is described in a file called a manifest file or description file. In the context of DASH, this manifest file is a file also called the MPD file (for Media Presentation Description). When a client device gets the MPD file, the description of each encoded and deliverable version of a media content component (representing a single continuous encapsulated timed media data) can easily be determined by the client. By reading or parsing the manifest file, the client is aware of the kind of media content components proposed in the media presentation and is aware of the HTTP addresses for downloading the associated media content components. Therefore, it can decide which media content components to download (via HTTP requests) and to play (decoding and playing after reception of the segments). DASH defines several types of segments, mainly initialization segments, media segments, or index segments. Initialization segments contain setup information and metadata describing the media content component, typically at least the ‘ftyp’ and ‘moov1boxes of an ISOBMFF media file. A media segment contains the media data corresponding to a media content component. It can be for example one or more ‘moof plus ‘mdaf or ‘imda’ boxes of an ISOBMFF file or a byte range in the ‘mdaf or ‘imda’ box of an ISOBMFF file. A media segment may be further subdivided into sub-segments (also corresponding to one or more complete ‘moof plus ‘mdaf or ‘imda’ boxes). The DASH manifest may provide segment URLs or a base URL to the file with byte ranges to segments for a streaming client to address these segments through HTTP requests. The byte range information of portions of an ISOBMFF media file or a media segment may be provided by specific ISOBMFF boxes such as the Segment Index box ‘sidx’ or the SubSegment Index box ‘ssix’. These boxes can be present in an index segment or in a media segment.

[0066] For the sake of illustration, media file or media segment 130 may encapsulate media data (e.g. encoded audio or video) into boxes according to standards compliant with ISO Base Media File Format (ISOBMFF, ISO / IEC 14496-12 and ISO / IEC 14496-15 standards). In such a case, media file 130 may correspond to one or more media files (indicated by a FileTypeBox ‘ftyp’), or one or more segment files corresponding to one initialization segment (when indicated by a FileTypeBox ‘ftyp’) and / or one or more media segments (when indicated by a SegmentTypeBox ‘styp’). Optionally, the segment files may also contain one or more Segment Index boxes ‘sidx’ and SubSegment Index boxes ‘ssix’ providing indexation and possibly dependence information on media segments. According to ISOBMFF, media file 130 may include two kinds of boxes, “media data boxes” (e.g. ‘mdaf or ‘imda’) containing the media data and “metadata boxes” (e.g. ‘mooV, ‘moof, ‘sidx’, ‘ssix’) containing metadata defining placement and timing of the media data.

[0067] According to some embodiments of the disclosure, the media files 130 may include a description of sample attributes or features.

[0068] In particular, media files 130 may include a description of sample properties with an enhanced mechanism for associating properties with samples as described hereafter.

[0069] In particular also, for example to avoid describing sample references or dependencies of samples using a complex hierarchy of subsegments and samples with an indexation based on ‘sidx’ and ‘ssix’ boxes and / or to cope with low-latency streaming, the media files 130 may further include an enhanced description for reconstructing, for each sample, a list of references, including e.g. sample references or dependencies to samples as described hereafter.

[0070] It is noted here that when doing low-latency streaming, transmitting or pushing data to a client should be done as fast as possible without having to wait for the complete segment to be generated. However, using a SubsegmentlndexBox 'ssix' to signal a dependence between samples mandates the use of a SegmentlndexBox 'sidx' (to get the number of entries in the SubsegmentlndexBox 'ssix' that is equal to the number of entries in the SegmentlndexBox 'sidx'), and the SegmentlndexBox 'sidx' mandates a size and a duration per entry, which are not known until the end of the generation of the segment. In other words, SegmentlndexBox 'sidx' and SubsegmentlndexBox 'ssix' are not adapted to signal sample dependencies for low- latency DASH / HLS.

[0071] It is also observed that when a server and a client are doing broadcast or multicast adaptive bit rate (ABR), some of the transmitted data may be lost or corrupted during the transmission. Therefore, the client or parser must decide whether the received data require to be repaired or not. This may be based on sample references or dependencies. In particular, when a byte-range or portion of the media file assigned to one level value by a SubsegmentlndexBox 'ssix' comprises multiple samples, if this portion is incomplete or corrupted, the client or parser needs to understand the samples impacted by the losses to take a decision (either to repair the losses or to drop the sample(s)), it being noted that some bytes missing or corrupted in a byte-range or portion assigned to one level by a SubsegmentlndexBox 'ssix' can invalidate one of the following: the entire byte range(s) for that level, a subset of the samples in that level, or only the last (in decoding order) sample in that level.

[0072] Therefore, providing a description for reconstructing a list of references including sample references or dependencies to samples for each sample of a set of samples allows a client, file reader, or parser to know exactly which samples are impacted when one sample cannot be decoded by choice or due to losses or corruption in the transmission.

[0073] Figure 2a illustrates a compact sample to group definition for associating a property description with a sample as defined by ISO / IEC 14496-12.

[0074] ISOBMFF and its extensions specify a grouping mechanism that allows defining group of samples and associating a description, typically a property of the group, with each sample within the defined group of samples.

[0075] This grouping mechanism relies on the association of two boxes, a sample group description box ’sgpd’ (the first box, not represented) providing a list of possible values for the description, and a compact sample-to-group box ‘csgp’ (the second box, referenced 200 in Figure 2a) describing the association of a sample with an index into the list defined by the associated sample group description box ’sgpd’. The ‘sgpd’ and ‘csgp’ boxes are associated one with the other via the same grouping_type parameter 210 represented by a four-character code, optionally in combination with a grouping_type_parameter. The group of samples defined by the association of these two boxes is composed of all the samples that are actually associated with an index into the list. Samples that are not associated with an index or that are associated with the value 0 are not part of the group of samples defined by the association of the ‘sgpd’ and ‘csgp’ boxes for a given grouping_type parameter 210, optionally in combination with a grouping_type_parameter.

[0076] The compact sample to group box ‘csgp’ 200 provides a compact way to represent the mapping of samples to descriptions within a group of samples by describing these samples as a number of concatenated patterns (each pattern being a set of one or more samples that description may apply to one or more other samples), each pattern being possibly repeated several times, entirely or partially. The number of patterns (excluding the repeated patterns) is given by the pattern_count parameter 220.

[0077] The first loop 230 defines the mapping of samples to patterns. The sum of all the sample_count values 232 in the first loop 230 indicates the number of mapped samples. The parameter pattern_length[i] 231 gives the length in samples of the ithpattern. If pattern_length[i] is equal to sample_count[i] for a ithpattern, this means that the ithpattern is not repeated. If the sample_count[i] is an integer multiple of pattern_length[i], this means that the ithpattern is entirely repeated a number of times equal to sample_count[i] / pattern_length[i]. If the sample_count[i] is not an integer multiple of pattern_length[i], this means that the last occurrence of the ithpattern is truncated.

[0078] The second loop 240 defines the mapping of an index, sample_group_description_index[j][k] 241 , to a description in the associated sample group description box ‘sgpd’ with the kthsample within the jthpattern defined by the first loop 230.

[0079] The sample grouping mechanism based on the sample group description box ‘sgpd’ (not represented) and the compact sample to group box ‘csgp’ 200 has proven to be efficient to describe the mapping of samples with a description within a group of samples, especially in the cases where there are repeating patterns. But this mechanism has also proven to be suboptimal in terms of cost of description in some cases, even if sample descriptions are mutualized, when possible, because it implies the cost of using two boxes, and also especially in the cases where there are samples with varying properties in-between reused patterns, and for which no repeating pattern could be computed, as illustrated by Figure 2b.

[0080] Figure 2b illustrates an example of use of a compact sample to group definition for associating a property description with a sample as defined by ISO / IEC 14496-12.

[0081] The top of Figure 2b, corresponding to patterns 250, illustrates an example of 3 patterns (pattern_count = 3) as possibly described by the compact sample to group box 200.

[0082] The first row 251 illustrates the definition of the mapping of samples to patterns corresponding to the loop 230 in the compact sample to group box 200. For instance, the first pattern is composed of two samples (pattern_length[1] = 2) and is repeated twice to map a total of four samples (sample_count[1] = 4). The second pattern consecutive to the first pattern is composed of three samples (pattern_length[2] = 3) and is not repeated since sample_count[2] = pattern_length[2]). The third pattern is composed of two samples (pattern_length[3] = 2) and is repeated three times to map a total of six samples (sample_count[3] = 6).

[0083] The second row 252 illustrates the mapping of each sample of a pattern with an index, noted idx[j][k] for the sake of illustration, and actually illustrating sample_group_description_index[j][k] 241 of the second loop 240.

[0084] The bottom of Figure 2b, corresponding to the list of samples 260, illustrates the resulting or reconstructed list of samples described by the patterns 250 with the first pattern of two samples repeated twice, from sample 1 to sample 4, followed by a second pattern of three samples (non-repeated), from sample 5 to sample 7, followed by the third pattern of 2 samples repeated three times, from sample 8 to sample 13.

[0085] The inventors have observed that if samples 8 to 13 use a pattern having the same length of patterns as samples 1 to 4 and if this pattern is associated with the same index as samples 1 to 4, i.e. in this example pattern_length[2] = pattern_length[1], idx[3][1] = idx

[0001]

[0001] and idx[3][2] = idx

[0001] [2], separated by samples 5 to 7 having a different pattern, the definition of the pattern used to describe samples 1 to 4 and samples 8 to 13 is duplicated and uses different corresponding indexes in the compact sample to group box 200, which is suboptimal.

[0086] The inventors have also observed that, in many video sequences, it is quite frequent to have frames with varying properties or references in-between reused patterns (e.g. predictive (P) or clean random access (CRA) pictures), and for which no repeating pattern could be computed, e.g.:

[0087] A - PATTERN 1 (N) - B - C - PATTERN 1(M) - D - E. where A, B, C, D, E represent individual frames, possibly with different types not represented by a reused pattern, and PATTERN 1 represents a reused pattern repeated N or M times, with possibly N is equal to M. This is a typical use case that the compact sample to group box 'csgp' cannot properly handle, since each unique entry in the middle of a repeated pattern P requires the coding of the pattern P each time it is used again, and replicating the sample description indexes.

[0088] Figure 3a illustrates an example of a syntax structure for describing properties associated with samples, according to some embodiments of the disclosure.

[0089] According to a particular embodiment, both the description of properties and their association with samples are declared in a PatternSampleDescriptionBox 300 with the four-character box type ‘psds’.

[0090] It can be noted that the name of the box and the box type are defined only for the sake of illustration and any other name or four-character box type could be used to identify this box.

[0091] Using a single box to describe both the description of properties and the association of the properties with the samples makes it possible to improve signalling costs compared to the sample grouping mechanism that requires the definition of two boxes. In addition, the use of a single box makes the list of references understandable without the initialization segment being present, which may be required for particular use cases like streaming use cases, for example in file repair operations of HTTP streaming sequences over lossy links and may be useful in trick mode optimization. Using sample grouping in this case would force having sample group descriptions and sample to group boxes defined in each fragment, costing more bytes.

[0092] PatternSampleDescriptionBox 300 may be defined in the SampleTableBox ‘stbl’ of a TrackBox ‘trak’ describing a track of the media file, or in a TrackFragmentBox ‘traf of a MovieFragmentBox ‘moof’ describing a fragment of a fragmented media file.

[0093] If PatternSampleDescriptionBox 300 is defined in a SampleTableBox ‘stbl’ of a track, it may provide the description of properties and their association with samples for all the samples of the sequence of samples of the track. If PatternSampleDescriptionBox 300 is defined in a TrackFragmentBox ‘traf in a fragment of a fragmented media file, it may provide the description of properties and their association with samples for all the samples of the sequence of samples of the track in the fragment.

[0094] PatternSampleDescriptionBox 300 is generic and allows associating any kind of properties with samples. The description of properties is defined by a data structure extending a PatternSampleDescriptionEntryO 350 that may be defined as follows : abstract class Pa tternSampleDescripti onEntry (desc type , flags) {

[0095] / / Description parameters

[0096] }

[0097] The information contained in PatternSampleDescriptionBox 300 makes it possible for a parser to reconstruct the complete list of samples in the media file or in a fragment with, for each sample, the description of the property associated with the sample for a given description_type.

[0098] PatternSampleDescriptionBox 300 comprises a list of entries, the number of entries being given by nb_entries 320, each entry defining either the description of the property 350 associated with one sample of a reconstructed list of samples, or defining a pattern 340 that indicates that a set of consecutive samples in the reconstructed list of samples have the same scheme of association with property descriptions as a set of previous samples in the reconstructed list of samples identified by the pattern. Figure 3b illustrates an example of a process of reconstruction of the list of samples from the list of entries in the PatternSampleDescriptionBox 300.

[0099] The PatternSampleDescriptionBox 300 may comprise the following syntax elements: description_type 310 that indicates the type of property associated with the samples, nb_entries 320 that indicates the number of entries in the association loop, usePattern 330 that indicates, if set to a first value (e.g., 1), that the entry is a pattern indicating that a set of samples is referencing the sample descriptions of a set of previous samples. Otherwise, if set to a second value (e.g., if set to 0), an explicit sample description follows. The first sample in a track or track fragment should have an associated use_pattern value set to the second value, offset 341 that indicates the start of the pattern in previous samples of the reconstructed list of samples, value 0 being designating the first sample in the track or track fragment, patternjength 342 that indicates the number of samples in the pattern starting from offset and num_samples 343 that indicates the number of samples described by the entry. If this value is greater than patternjength, the pattern is repeated until all the corresponding samples are described. It is noted that num_samples is not necessarily a multiple of patternjength. In such a case, the last repeated pattern may be truncated.

[0100] PatternSampleDescriptionBox 300 may comprise other fields (as illustrated by “(...)” in Figure 3a.

[0101] The number of bits to represent the above syntax elements may be fixed or variable, illustrated by the temporary non-parsable variables entry J)its and bits in Figure 3a. The values of entry_bits and bits may depend on the value of another field defined in PatternSampleDescriptionBox 300 (not represented) or depend on a value of the flags parameter as illustrated in Figure 4 or Figure 6.

[0102] In a variant, rather than defining a parameter num_samples that defines explicitly the number of samples in the reconstructed list of samples for each entry based on a pattern (use_pattern == 1), the PatternSampleDescriptionBox may indicates when an entry is a repeat of the preceding pattern using a flag repeat_pattern. The PatternSampleDescriptionBox may then be defined as follow:

[0103] Box Type: 'psds'

[0104] Container: SampleTableBox or TrackFragmentBox

[0105] Mandatory: No

[0106] Quantity: Zero or one (per container) ali gned (8 ) class PatternSampleDescriptionBox (desc type) extends Box ( 'psds ' )

[0107] { unsi gned int (8) flags; if (flags & 2) entry bi ts=32 ; else if (flags & 1 ) entry bi ts=16; else entry bi ts=8 unsi gned int (32) descripti on type = desc type; unsi gned int (entry bi ts) nb entri es ; for (i = 0;i < nb entries ; i + + ) { bi t (l) use pattern ; if (use pa ttern) { unsi gned int ( 7) offset ; unsi gned int (8) pa ttern length;

[0108] } else { bi t (l) repea t pa ttern; if (repeat pattern) { unsi gned int (6) num loops minusl ;

[0109] } else { bi t ( 6) reserved = 0;

[0110] Pa tternSampleDescripti onEntry (desc type , flags » 2) ;

[0111] }

[0112] }

[0113] }

[0114] } where description_type indicates the type of the description or property defined by PatternSampleDescriptionEntry(). description_type is a value defined by the specification or registered, or a value from a derived specification or registration entity, nb_entries indicates the number of entries in the association loop, use_pattern indicates, if set to a first value (e.g., if set to 1), that the entry is a reference to a previously coded pattern of sample descriptions. Otherwise, is set to a second value (e.g., if set to 0), an explicit sample description may follow or an instruction to repeat the last pattern. The first sample in a track or track fragment should have an associated use_pattern value set to the second value (e.g., 0), offset indicates the start of the pattern from the list of reconstructed sample descriptions, value 0 being the sample description of the first sample in the track or track fragment, patternjength indicates the number of samples in the pattern starting from offset, repeat_pattern indicates, if set to a first value (e.g., if set to 1), that the entry is a repeat of the previously coded pattern of sample descriptions. The pattern to apply is the same as previous entry. Otherwise, if set to a second value (e.g., if set to 0), repeat_pattern indicates that an explicit sample description described by a PatternSampleDescriptionEntryO follows, num_loops_minus1 indicates the number of times that the previous pattern is repeated and

[0115] PatternSampleDescriptionEntryO is a data structure of type description_type comprising the description or property associated with a sample.

[0116] In a variant, parameter num_samples 343, used for defining explicitly the number of samples in the reconstructed list of samples for each entry based on a pattern, may be conditionally present depending on the value of a flag, e.g. has_num_samples coded on 1 bit in the definition of the pattern 340. In such a case, the field patternjength 342 is coded on (bits-1) bits and the num_samples 343 is defined if has_num_samples is equal to a given value, e.g. 1 , and is omitted otherwise.

[0117] It is noted that the number of bits for coding any of the above fields may be adjustable, e.g. using flags value, e.g. on 8, 16, or 32 bits.

[0118] Figure 3b illustrates an example of use of the syntax structure illustrated in Figure 3a for describing properties associated with samples, according to some embodiments of the disclosure.

[0119] A list of entries 360 of a PatternSampleDescriptionBox 300, numbered from [1] to [6], is represented at the top of Figure 3b.

[0120] The reconstructed list of samples 370, obtained by processing the list of entries 360, is represented in the middle of Figure 3b, the samples being numbered from 1 to 11 .

[0121] The description of property 380 associated with each sample of the reconstructed list of samples 370 is represented in the bottom of Figure 3b, numbered from #1 to #3, the number representing the order of appearance of the description of property 350 in the loop over all the entries of PatternSampleDescriptionBox 300.

[0122] According to this example, the first entry, [1], in the list of entries 360 comprises a first description of property, #1 , that is associated with the first sample, sample 1 , in the reconstructed list of samples. Similarly, the second entry, [2], in the list of entries 360 comprises a second description of property, #2, that is associated with the second sample, sample 2, in the reconstructed list of samples.

[0123] The third entry, [3], in the list of entries 360 comprises a definition 340 of a pattern #1 to be used to define the association of sample 3 with a description of property and possibly also for one or more subsequent consecutive samples. The pattern #1 indicates an offset of 0 and a patternjength (denoted length) of 2. This means that the sample 3 is associated with the same description of property as the first sample identified by the pattern, i.e. the sample at the offset 0, sample 1 , in the reconstructed list of sample. Therefore, sample 3 is associated with the description #1. Since the pattern #1 has a length of 2, this means that the subsequent sample to sample 3, i.e. sample 4, is associated with the same description as the subsequent sample of the sample 1 , i.e., the description of property of sample 2, i.e., description #2.

[0124] The same logic applies to following samples 5 to 11 .

[0125] In the case of samples 6 to 9, there is a repeat of the pattern #2 with offset to 0 and patternjength of 2 because the number of samples (denoted num and corresponding to num_samples 343) described by the pattern #2 is 4, meaning that the pattern of length of 2 is repeated num_samples / patternjength times, i.e. 4 / 2 = 2 times.

[0126] In a variant, for some use cases, it may still be useful to rely on the sample grouping mechanism with separate boxes, a first box for describing the descriptions of properties and a second box for declaring the association of these descriptions with samples. This may be useful, for example, for encapsulating fragmented media files in a case where the description of properties that may be associated with samples in a track are known from the beginning. In such a case, all the descriptions of properties may be defined once in a first segment comprising the MovieBox ‘moov’ and only association information are defined in the subsequent fragments. In such a case, the descriptions of properties may be defined in sample group description entries in a SampleGroupDescriptionBox ‘sgpd’ with a grouping_type indicating the type of property. According to some embodiments, this SampleGroupDescriptionBox ‘sgpd’ would be signaled with a flag static_group_description in its flags parameter indicating that there are no SampleGroupDescriptionBoxes of this grouping_type in any TrackFragmentBox of this track. The information for reconstructing the list of samples and for indicating the description of property associated with a sample of the reconstructed list of samples may be defined as follow:

[0127] Box Type: 'psgp'

[0128] Container: SampleTableBox or TrackFragmentBox

[0129] Mandatory: No

[0130] Quantity: Zero or more aligned (8 ) class PatternSampleToGroupBox extends FullBox ( ' psgp ' , version, flags) { unsigned int (32) grouping type; if (grouping type parameter present == 1) { unsigned int (32) grouping type parameter;

[0131] } if (flags & 2) bits=32 ; else if (flags & 1) bits=16; else bits=8 unsigned int (bits) nb entries; for (i = 0;i < nb entries ; i+ + ) { bit(l) use pattern; if (use pattern) { unsigned int(bits-l) offset; unsigned int (bits) pattern length; unsigned int (bits) num samples;

[0132] } else { unsigned int (32) sample group description index;

[0133] }

[0134] }

[0135] } where nb_entries, use_pattern, offset, patternjength and num_samples have the same semantics as described for PatternSampleDescriptionBox 300 described in reference to Figure 3 and sample_group_description_index is an integer that indicates the index of the sample group description entry which describes the samples in this group. The index may range from 1 to the number of sample group description entries in the SampleGroupDescriptionBox, inclusive, or takes the value 0 to indicate that this sample is a member of no group of this type.

[0136] According to another aspect of the disclosure, a specific description of property associated with samples, that is the description of sample references or dependencies between samples of an encapsulated media data, is improved.

[0137] In this disclosure, value by difference is also denoted differential value and the approach based on values by differences is also denoted differential coding. And a value directly referring to a value associated with another sample is also denoted absolute or direct value and the approach based on direct reference to another value is denoted absolute or direct coding.

[0138] It can be noted that in absolute or direct coding, an absolute value may be negative or positive, e.g. when an absolute value refers to a sample identifier and sample identifiers (sample IDs) can use signed values. For instance, this may happen when a Picture Order Count (POC) is used as a sample identifier, since some coding formats, e.g. AVC (Advanced Video Coding), allow using positive value or negative value for a POC.

[0139] A sample reference or dependency is a reference from a first sample to a second sample being possibly used as a reference in the processing of the first sample. For instance, the second sample may be a reference for inter prediction when decoding the first sample. The sample reference or dependency is a reference to a previous sample in decoding order, it being noted that the sample reference or dependency may be to a previous sample or to a subsequent sample in rendering order, also denoted composition or presentation order. A sample reference of a sample may be an absolute value of the identifier of a referenced sample or a difference between the identifiers of the sample and the referenced sample, or the difference between the identifiers of the sample and another sample used as a base. In a variant, the syntax structure described hereafter may comprise an indication of the type of sample references, either based on an absolute identifier, a difference between the identifiers of the sample and the referenced sample, or the difference between the identifier of the sample and another sample used as a base sample. In the latter case, the syntax structure may further comprise an indication of the identifier of the sample used as a base.

[0140] According to some embodiments, the list of references comprising sample references for each sample may be described without using pattern. In such a case, a syntax structure SampleReferencesBox extending a fullbox may be defined as follows:

[0141] Box Type: 'GPSR' (GPAC Sample Reference)

[0142] Container: SampleTableBox or TrackFragmentBox

[0143] Mandatory: No

[0144] Quantity: Zero or one (per container) class Sampl eReferencesBox extends FullBox ( ' GPSR ' versi on=0rflags) { if (flags & 2) bi ts = 32; el se if (flags & 1 ) bi ts = 1 6; el se bi ts = 8 unsigned int (32) nb_sampl es ; for (i = 0; i < nb_sampl es ; i ++) { signed int (bi ts) sampl e_ID; / / <=> to ROC unsigned int (bi ts) nb_refs ; signed int (bi ts) ref_IDs [nb_refs] ; / / <=> to ROC of ref pi cs

[0145] }

[0146] } where nb_samples indicates the number of samples described by this box, sampleJD indicates the sample identifier (or sample ID), nb_refs indicates the number of sample references in the list of sample references associated with the sample and refJDs indicates the sample identifiers of the referenced samples.

[0147] Comparing the efficiency of a sample grouping approach with the GPSR approach, i.e. the approach of using the ‘GPSR’ box, is quite simple since entries per sample for the GPSR approach are usually smaller without the overhead of using two boxes as in sample grouping but because it does not use patterns, it can be a bit costly compared to a sample grouping approach with, e.g., a CompactSampleToGroupBox.

[0148] In other embodiments, the list of sample references for each sample may be described using the sample grouping mechanism of ISOBMFF and by expressing values as differences.

[0149] In such embodiments, a specific grouping_type value is defined, e.g. 'msdp', with a corresponding sample group description entry defined as follows: class Sampl eDependencyGroupEntry extends Sampl eGroupDescripti onEntry ( 'msdp ' ) {

[0150] / / diff to previ ous sampl e ID signed int (bi ts) sampl e_ID; / / 0 for first , diff wi th prev ID for others unsigned int (bi ts) nb_refs ; signed int (bi ts) ref_diff_IDs [nb_refs] ; / / <=> diff between ref sampl e ID and sampl e_ID for thi s entry } where sampleJD indicates the sample identifier (or sample ID) of the sample mapped into this entry coded as a difference with the sample identifier of the previous sample mapped to this entry. The value may be equal to 0 for the first sample, nb_refs indicates the number of sample references in the list of sample references associated with the sample and ref_diff_l Ds indicates the sample identifier of a direct sample reference coded as a difference between the identifier of the referenced sample and the identifier of the sample being described by this entry.

[0151] Using differences instead of absolute values to code the sample identifier and the list of sample references allows mapping the same SampleDependencyGroupEntry to multiple samples in the sequence of samples. The SampleDependencyGroupEntry may be defined in a SampleGroupDescriptionBox ‘sgpd’ and mapped to samples using either a CompactSampleToGroupBox ‘csgp’ or a SampleToGroupBox ‘sbgp’ having the same grouping_type as the SampleGroupDescriptionBox.

[0152] When the SampleDependencyGroupEntry of this disclosure is used in combination with a CompactSampleToGroupBox ‘csgp’, this approach is a bit more efficient than the GPSR approach because what it gains in reusing the same sample group description entry, it loses in coding, in the CompactSampleToGroupBox, the patterns which are not always consecutive, hence higher size of the CompactSampleT oGroupBox.

[0153] Hereafter, another embodiment is described using both the coding of values as differences and patterns for optimizing the signaling of sample dependencies or sample references.

[0154] Figure 4 illustrates an example of a syntax structure for describing the sample references of samples, according to some embodiments of the disclosure.

[0155] CompactDirectSampleReferencesBox 400 is an example of a syntax structure or box dedicated to describe the sample references or dependencies of samples according to the following principles: signaling a sample identifier (or sample ID) as corresponding to the Picture Order Count (POC) and using differential coding to previous sample ID, coding reference IDs in dependency lists as a difference with the sample ID.

[0156] CompactDirectSampleReferencesBox 400 provides explicit coding dependencies of samples towards other samples. It associates with each sample of a track or track fragment:

[0157] - a sample identifier (sample ID) coded as an absolute sample ID or as a difference with the sample ID of a previous sample;

[0158] - a dependency list of sample references coded as a difference with the sample ID. The listed dependencies only contain the direct dependencies, i.e. if sample A depends on sample B which in turn depends on sample C, only sample B is listed as a dependency to sample A.

[0159] CompactDirectSampleReferencesBox 400 is structured as a list of entries for reconstructing the list of samples and their sample references, each entry defining either the sample identifier and the dependency list associated with a sample or a reference to a pattern of previously coded sample IDs and sample references in the reconstructed list of samples and their sample references.

[0160] The sample ID may be any identifier. It does not need to be unique, and the identifiers (IDs) used by the list of sample references refer to the last sample defined with the given ID in the CompactDirectSampleReferencesBox. A sample ID used in a dependency list but not present in the track or past track fragments indicate a broken dependency, i.e. that the sample cannot be decoded.

[0161] It is noted that, for video track, the sample ID could be the Picture Order Count (POC) of a sample. Broken dependencies typically happen when tuning a stream on a stream access point sample of type 3 (SAP 3 sample), e.g. in an open-GOP case (open-Group of Picture case).

[0162] CompactDirectSampleReferencesBox 400 comprises the following syntax elements: flags 410 indicates flags value for the box. It may be used to indicate the number of bits used to code some of the following fields, nb_entries referenced 420 that indicates the number of entries in the loop for reconstructing the list of references. The entries make it possible for a parser to reconstruct the list of samples with their associated coded sample references, is_ref referenced 430 that indicates, if set to a first value (e.g. 1), that the entry is a reference to a pattern 440 of previously coded sample references in the reconstructed list of references. Otherwise, if set to a second value (e.g., 0), an explicit list of sample references 450 follows. According to some embodiments, the first sample in a track or track fragment has an associated is_ref value of 0, offset referenced 441 that indicates the start of the pattern in the reconstructed list of references. According to some embodiments, value 0 designates the list of sample references of the first sample in the track or track fragment, patternjength referenced 442 that indicates the number of samples in the pattern starting from offset, num_samples referenced 443 that indicates the number of samples described. If this value is greater than patternjength, the pattern is looped over until all samples indicated by the number of samples num_samples are described. It is not necessarily the case that num_samples is a multiple of patternjength. If num_samples is not a multiple of patternjength, the last repeated pattern may be truncated, nb_refs referenced 451 that indicates the number of direct sample references for this sample. If it is set to 0, the sample has no direct references (i.e. the sample is a sync sample). It is noted that the number of direct sample references nb_refs can represent a number of sample references possibly used for processing the sample, and does not necessarily represent a number of sample references actually used in a codec reference list for processing the sample, is_abs referenced 452 that indicates, if set to a first value (e.g., 1), that sample_ID_code is the absolute value of the sample identifier. If set to a second value (e.g., 0), it indicates that sample_ID_code is the difference between the sample identifier and the preceding sample identifier in the reconstructed list of references. The first sample in a track or track fragment should have an associated is_abs value set to the first value, sample_ID_code referenced 453 that indicates the difference or absolute value of the sample identifier coded as specified by is_abs flag and ref_diff_l Ds referenced 454 that indicates the sample identifier of a direct sample reference coded as a difference between the identifier of the sample being described by this entry and the identifier of the referenced sample, i.e. samplelD - referenceSamplelD.

[0163] In a variant, a reserved value of sample J D_code, e.g. 2<bits’1) - 1 if sample_ID_code is coded on (bits-1) bits, may indicate that the sample ID is unknown for the current sample. This reserved value may be useful in combination with is_abs equal to 1 to increase the number of patterns that can be described and possibly repeated in the CompactDirectSampleReferencesBox, for instance when a sample is associated with an absolute sample ID and is not referenced by any other samples.

[0164] CompactDirectSampleReferencesBox 400 may be defined in the SampleTableBox ‘stbl’ of a TrackBox ‘trak’ describing a track of the media file, or in a TrackFragmentBox ‘traf of a MovieFragmentBox ‘moof’ describing a fragment of a fragmented media file. In summary, CompactDirectSampleReferencesBox 400 has the following properties:

[0165] Box Type: 'cdrf

[0166] Container: SampleTableBox or TrackFragmentBox Mandatory: No

[0167] Quantity: Zero or one (per container)

[0168] If CompactDirectSampleReferencesBox 400 is defined in a SampleTableBox ‘stbl’ of a track, it provides the sample identifier and the list of sample references for all or for a part of the samples of the sequence of samples of the track. If CompactDirectSampleReferencesBox 400 is defined in a TrackFragmentBox ‘traf’ in a fragment of a fragmented media file, it provides the sample identifier and the list of sample references for all or for a part of the samples of the sequence of samples of the track in the fragment.

[0169] When present in a SampleTableBox (respectively a TrackFragmentBox), the number of samples described by CompactDirectSampleReferencesBox 400 is equal to the number of samples present in the track (respectively in the track fragment).

[0170] According to some embodiments, absolute values of sample identifiers are equal to the Picture Order Count (POC) of the associated samples. When the Picture Order Count is used to express a sample identifier, it is possible to optimize the coding of both sample identifiers and sample references using patterns, since the differentially coded sample identifier may follow the same pattern as the list of sample references.

[0171] In a variant, a sample identifier may be any identifier as long as the identifier is unique within the track, if the CompactDirectSampleReferencesBox is defined in a SampleTableBox, or within a track fragment if the CompactDirectSampleReferencesBox is defined in a track fragment.

[0172] It is noted that associating a unique sample identifier with each sample in a track or a track fragment and using it to describe the sample references as defined by the semantics of ref_diff_l Ds brings the benefit of supporting also the description of interlayer dependencies, as the identifiers of sample references, when mapped to POC numbers, are unique across the layers within a layered bitstream in a track or a track fragment.

[0173] The reconstruction process of the list of samples and their sample references using the CompactDirectSampleReferencesBox 400 should produce the same result as the following model: initializing an empty list f iat_refs,

[0174] - for each entry in the box, o if is_ref is false, appending to the f iat_ref s list an entry containing {nb refs, is abs , sample ID code, ref diff IDs}, o otherwise (is_ref is true), for each K ranging from 0 to num_sampies - 1 of the entry, appending to fiat_refs the entry flat refs[offset + K%pattern length],

[0175] - validating that the number of entries in f iat_refs is the same as the number of samples in the track or track fragment,

[0176] - for each sample J in the track or track run: o assigning sample ID as follows:

[0177] ■ if flat_ref s [ J] . is_abs is true, setting the sample samplelD to flat ref s [ J] . sample ID code and

[0178] ■ otherwise, setting the sample samplelD to flat refs [J] .sample ID code + sample [J- 1 ] . samplelD, o assigning sample reference IDs by removing from sample [J] .samplelD the value flat refs [J] .ref diff IDs, i.e. ref erenceSamplelD = sample [ J] . samplelD flat refs [J] . ref diff IDs.

[0179] In another embodiment, the sample references or dependencies of samples may be described by using the PatternSampleDescriptionBox 'psds' described above with a dedicated description_type, e.g. ‘sdpe’. In such case the PatternSampleDescriptionEntryO may be described as follows: aligned(8) class SampleDependenciesPatternEntry (flags) extends PatternSampleDescriptionEntry ('sdpe', flags) { if (flags & 2) bits=32 ; else if (flags & 1) bits=16; else bits=8 unsigned int (bits-l) nb_refs; bit (1) is_abs; signed int (bits-l) sample_ID_code ; unsigned int (bits) ref_diff_IDs [nb_refs] ;

[0180] } where nb refs, i s abs, sample ID code and ref di ff IDs have the same semantics as previously described for the CompactDirectSampleReferencesBox.

[0181] It is noted that nb_refs is coded on (bits-2) bits instead of (bits-1) bits if the variant of PatternSampleDescriptionBox 'psds' including the repeat_pattern parameter is used.

[0182] Figure 5 illustrates an example of describing sample references of a sequence of samples using the syntax structure illustrated in Figure 4, according to some embodiments of the disclosure.

[0183] Table 500 illustrates a sequence of samples in decoding order represented by the column entitled Frame Type and referenced 520 where each row corresponds to a sample comprising a frame with the given frame type, here I for an intra frame, P for a predictive frame and B for a bi-predictive frame. An intra frame is a frame with no dependency to another frame in a sequence of frames. A predictive frame is a frame that may have dependency with a previous frame in the sequence of frames, in the presentation order, and a bi-predictive frame may have dependencies with previous and forward frames in the sequence of frames, in the presentation order.

[0184] The column entitled sampleJD and referenced 530 illustrates the absolute sample identifier (or sample ID) associated with each sample of the column 520. According to the illustrated example, the absolute sample ID is set to the Picture Order Count (POC) that uniquely identifies the associated frame in the sample among all frames of the sequence of frames that are encapsulated in the sequence of samples. For instance, the first sample corresponds to an I frame with a sample ID equal to 0, representing the first frame in composition order, the second sample corresponds to a P frame with a sample ID equal to 4, representing the fifth frame in composition order, meaning it is rendered after rendering the samples with sample IDs 0 to 3, and so on.

[0185] The column entitled refs and referenced 540 illustrates the list of sample references or dependencies associated with each sample of column 520. According to the illustrated example, the list of references is expressed in absolute sample IDs (i.e. POCs). For instance, the second sample, corresponding to a P frame with POC equal to 4 depends on the first sample, corresponding to a I frame with POC equal to 0. Likewise, the third sample, corresponding to a B frame with POC equal to 1 , depends on the first and second samples with POCs equal to 0 and 4 respectively. The column entitled sample_ID_diff and referenced 550 and the column entitled ref_diff_l Ds and referenced 560 illustrate the same items of information as the columns sampleJD 530 and refs 540, respectively, but using relative references instead of absolute references.

[0186] Accordingly, each value in column sample_ID_diff 550 represents the relative sample ID of a sample, that is to say the difference between the absolute sample ID of the sample with regard to the absolute sample ID of the previous sample according to the decoding order. For instance, the third row of the column sample_ID_diff 550 having the value -3 is the relative sample ID of the third sample (having the frame type B and the absolute sample ID equal to 1), corresponding to its absolute sample ID (i.e., 1) minus the absolute sample ID of the preceding sample (having the frame type P and the absolute sample ID 4).

[0187] Likewise, each cell of the column ref_diff_l Ds 560 contains the list of relative sample references computed as the difference between its absolute sample ID and the absolute sample ID of the sample it depends on, i.e., in other words, sample references coded as differences from the current sample ID (corresponding to its POC) are used in the list of sample references. For instance, the list of relative sample references of the third sample with frame type B and sample ID 1 is equal to {1 , -3} corresponding to the absolute sample ID 1 of this sample minus each reference of the list of references {0, 4}.

[0188] As apparent from columns sample_ID_diff 550 and ref_diff_l Ds 560, the use of relative references, indicated in the columns sample_ID_diff 550 and ref_diff_l Ds 560 makes it possible to: reuse the description of the dependency lists of column ref_diff_l Ds 560 and reuse the description of the sample identifiers of column sample_ID_diff 550, which can be used to define an entry describing sample dependencies, either an entry in a box, e.g. in a CompactDirectSampleReferencesBox or PatternSampleDescriptionBox, or a sample group description entry in a sample group , e.g. the sample group 'psgp' described above, describing or relying on patterns and reuse patterns, which can be used to repeat or refer to a previous pattern. The part 510 on the right of the Figure 5 illustrates the gain obtained with the use of patterns according to some embodiments of the disclosure. More precisely, reference 510 illustrates the transformation of the sequence of samples and its list of sample references from the table 500 to a CompactDirectSampleReferencesBox like CompactDirectSampleReferencesBox 400 in Figure 4. For example, reference 570 illustrates the transformation of the first row of table 500 in a first entry described as an explicit sample reference (is_ref = 0) in entries loop of the CompactDirectSampleReferencesBox, reference 580 represents the transformation of the sixth row of table 500 in a sixth entry described as an explicit sample reference (is_ref = 0) in entries loop of the

[0189] CompactDirectSampleReferencesBox, and reference 590 represents the transformation of all the remaining rows of table 500 with a single entry in a seventh entry described as a pattern (is_ref = 1).

[0190] It can be noted that using a sample ID mapped to a POC would also bring the benefit of solving inter-layer dependencies, as sample reference IDs, when mapped to POC numbers, are unique within the layered bitstream. This is much simpler than defining two type of sample references, one for intra-track dependencies and one for inter-track dependencies. It can be noted that since the differentially coded sample ID, illustrated in column sample_ID_diff 550, follows the same pattern as for column ref_diff_l Ds 560, it is possible to optimize the coding of both sample IDs and sample references in the same entry of the entries loop in the CompactDirectSampleReferencesBox like CompactDirectSampleReferencesBox 400 in Figure 4.

[0191] It is noted that when a sample ID is used to identify a sample in fragmented files, the sample ID may be scoped by the track fragment, i.e. the sample ID may be reset at each closed GOP start. In other words, for fragmented cases, it is assumed that the sample ID is scoped by the track fragment, i.e. it is possible to reset sample IDs at each closed GOP start.

[0192] Figure 6 illustrates an example of a syntax structure for describing sample references of samples according to some embodiments of the disclosure.

[0193] More and more live services are using low-latency approaches for media content distribution, e.g. in live services using Common Media Application Format (CMAF) as the format for media content distribution, where an ISOBMF segment is divided in several fragments. With very low fragment durations, embodiments as described by reference to Figure 4 may not be efficient enough to minimize the cost of signalling sample dependencies. It was observed that this may happen in sub-GOP fragmentation, i.e., if fragment duration is less than the duration of a Group of Pictures (GOP). This is because the reused entries, in the reconstructed list of samples and associated sample references, that can be described by a pattern may happen mostly at the end of the GOP and refer to entries corresponding to the beginning or middle of the GOP. It is noted that an entry can be reused if the list of sample references is equivalent whatever is the order of the sample references. For instance {0, 2} and {2, 0} are considered as being equivalent lists of sample references. The more the fragment duration is short compared to a GOP duration, and the more reused entries may lie in different fragments preventing the use of patterns.

[0194] According to an embodiment devised to address one or more of the foregoing concerns, the CompactDirectSampleReferencesBox 400 described by reference to Figure 4 may be modified as illustrated by CompactDirectSampleReferencesBox 600.

[0195] CompactDirectSampleReferencesBox 600 is an example of a syntax structure or box that provides explicit coding dependencies of samples towards other samples. It associates with each sample of a track or track fragment:

[0196] - a sample identifier (sample ID) coded either as an absolute sample ID or as a difference with the sample ID of a previous sample and

[0197] - a dependency list of sample references coded either as a difference from the sample ID or as an absolute or direct value, in other words as the identifier of another sample.

[0198] In this disclosure, value by difference is also denoted differential value and the approach based on values by differences is also denoted differential coding. And a value directly referring to a value associated with another sample is also denoted absolute or direct value and the approach based on direct reference to another value is denoted absolute or direct coding.

[0199] It can be noted that in absolute or direct coding, an absolute value may be negative or positive, e.g. when an absolute value refers to a sample identifier and sample identifiers (sample IDs) can use signed values. For instance, this may happen when a Picture Order Count (POC) is used as a sample identifier, since some coding formats, e.g. AVC (Advanced Video Coding), allow using positive value or negative value for a POC.

[0200] The listed dependencies should only contain direct dependencies, i.e. if sample A depends on sample B which in turn depends on sample C, only sample B should be listed as a dependency to sample A.

[0201] CompactDirectSampleReferencesBox 600 is structured as a list of entries for reconstructing the list of samples and their sample references, each entry defining either the sample identifier and the dependency list associated with a sample or a reference to a pattern of previously coded sample IDs and sample references in the reconstructed list of samples and their sample references.

[0202] The order of samples in the reconstructed list of samples corresponds to the order of the samples within the sequence of samples of the track. This may correspond to the decoding order of the samples within a coded bitstream.

[0203] The sample ID may be any identifier. It does not need to be unique, and the IDs used by the list of sample references refer to the previous sample (in other words, the previously occurring sample) defined with the given ID in the CompactDirectSampleReferencesBox. A sample ID used in a dependency list but not present in the track or past track fragments indicate a broken dependency, i.e. that the sample cannot be decoded.

[0204] It is noted that, for video track, the sample ID could be the Picture Order Count (POC) of a sample. Broken dependencies may happen when tuning a stream on a stream access point sample of type 3 (SAP 3 sample), e.g. in an open-GOP case (open-Group of Picture case).

[0205] According to the illustrated example, CompactDirectSampleReferencesBox 600 comprises an indication that indicates whether the dependencies and sample ID associated with a sample are coded as absolute or direct values, in other words as sample ID values, or possibly as differential values, in other words as a difference with dependencies or sample ID of a previous sample. This indication, represented by the no_diff_mode variable 615, may be a syntax element of CompactDirectSampleReferencesBox 600 or, as illustrated in Figure 6 a temporary, nonparsable variable comprising the value of a flag coded in the flags parameter 610.

[0206] It is observed that coding the dependencies of the sample IDs of the samples the current sample depends on and the sample ID of the current sample as an absolute or direct value, in other words as a sample ID value, may be more efficient than using a differential coding of the dependencies and sample IDs, to identify patterns in the reconstructed list of samples and their dependencies within a small fragment, in particular when the fragment duration corresponds to a sub-GOP fragmentation. It is also observed that some entries can be reused within a fragment duration, as consecutive frames in a fragment use the same references.

[0207] The variable no_diff_mode referenced 615 allows configuring CompactDirectSampleReferencesBox 600 to use only absolute values or possibly differential values for coding the sample dependencies and the sample identifiers (IDs). Variable no_diff_mode 615 indicates, if set to a first value (e.g. 1), that the sample references in ref_l Ds[] and the sample IDs are coded as direct values, and if set to a second value (e.g. 0), that the sample references in ref_IDs[] are coded as differential values and sample IDs may be coded as differential or direct values depending on the value of is_abs parameter 653.

[0208] CompactDirectSampleReferencesBox 600 may also comprise the following syntax elements: flags 610, that indicates the flags value for the box. It may be used to indicate the number of bits used to code some of the following fields and to code the value of no_diff_mode 615, nb_entries referenced 620, that indicates the number of entries in the loop for reconstructing the list of references. The entries make it possible for a parser to reconstruct the list of samples with their associated coded sample references, is_ref referenced 630, that indicates, if set to a first value (e.g., 1), that the entry is a reference to pattern 640 of previously coded sample references in the reconstructed list of references. Otherwise, if set to a second value (e.g., 0), an explicit list of sample references 650 follows. According to some variants, the first sample in a track or track fragment has an associated is_ref value of 0, offset referenced 641 , that indicates the start of the pattern in the reconstructed list of references. According to some variants, value 0 designates the list of sample references of the first sample in the track or track fragment, patternjength referenced 642, that indicates the number of samples in the pattern starting from offset, num_samples referenced 643, that indicates the number of samples described. If this value is greater than patternjength 642, the pattern is looped over until all samples indicated by the number of samples num_samples are described. It is not necessarily the case that num_samples is a multiple of patternjength, the last repeated pattern may be truncated, sample J Ds referenced 644, that is an array that indicates the list of values of the sample identifiers coded in a pattern. According to some embodiments, this parameter is only present when absolute or direct values, in other words direct references, are used to code the sample IDs, i.e. when not any no_diff_mode 615 is set to its first value (e.g., 1). It can be noted that in such a case, it is impossible to reuse sample IDs of samples from referenced entries, contrary to the case according to which sample IDs are coded as differential values. Therefore, this requires the presence of this array to carry explicit coding of the sample IDs of the num_samples samples described by the pattern, nb_refs referenced 651 , that indicates the number of direct sample references for this sample. If it is set to a first value (e.g., 0), the sample has no direct references (i.e. the sample is a sync sample). It is noted that the number of direct sample references nb_refs can represent a number of sample references possibly used for processing the sample, and does not necessarily represent a number of sample references actually used in a codec reference list for processing the sample, is_abs referenced 653, that indicates, if set to a first value (e.g., 1), that sample_ID_code 654 is the absolute or direct value of the sample identifier, in other words, the sample_ID_code is the value of the sample identifier. If set to a second value (e.g., 0), it indicates that sample_ID_code 654 is the difference between the sample identifier and the preceding sample identifier (or, in other words, the sample identifier of the preceding sample) in the reconstructed list of references. The first sample in a track or track fragment should have an associated is_abs value set to the first value. When no_diff_mode is set to its first value (e.g. 1), the is_abs field is not coded and takes its first value (e.g., 1), sample_ID_code referenced 652 or 654, that indicates the value of the sample identifier. If no_diff_mode is set to its first value (e.g. 1), sample_ID_code 652 indicates the absolute or direct value of the sample identifier, in other words the value of the sample identifier. If no_diff_mode is set to its second value (e.g. 0), sample_ID_code 654 indicates the difference or direct value of the sample identifier coded as specified by is_abs flag 653, and ref_IDs referenced 655, that is an array that indicates the sample identifiers of the direct sample references, if any. If no_diff_mode is set to its second value (e.g. 0), the identifier is coded as a difference between the identifier of the sample being described by this entry and the identifier of the referenced sample, i.e. sampieiD - referenceSampieiD. Otherwise (no_diff_mode is set to its first value (e.g., 1), the identifier is coded as the absolute identifier of the reference sample, in other words, as the value of the sample identifier of the reference sample.

[0209] In a variant, a reserved value of sample_ID_code, e.g. 2(bits'1)- 1 if sample_ID_code is coded on (bits-1) bits, or 2(bits)- 1 if sample_ID_code is coded on (bits) bits, may indicate that the sample ID is unknown for the current sample. This reserved value may be useful, possibly in combination with is_abs equal to 1 , to increase the number of patterns that can be described and possibly repeated in the CompactDirectSampleReferencesBox, for instance when a sample is associated with an absolute or direct sample ID and is not referenced by any other samples.

[0210] CompactDirectSampleReferencesBox 600 may be defined in the SampleTableBox ‘stbl’ of a TrackBox ‘trak’ describing a track of the media file, or in a TrackFragmentBox ‘traf’ of a MovieFragmentBox ‘moot’ describing a fragment of a fragmented media file. In summary, CompactDirectSampleReferencesBox 600 may have the following properties:

[0211] Box Type: 'cdrf'

[0212] Container: SampleTableBox or TrackFragmentBox

[0213] Mandatory: No

[0214] Quantity: Zero or one (per container)

[0215] If CompactDirectSampleReferencesBox 600 is defined in a SampleTableBox ‘stbl’ of a track, it provides the sample identifier and the list of sample references for all or for a part of the samples of the sequence of samples of the track. If CompactDirectSampleReferencesBox 600 is defined in a TrackFragmentBox ‘traf’ in a fragment of a fragmented media file, it provides the sample identifier and the list of sample references for all or for a part of the samples of the sequence of samples of the track in the fragment.

[0216] When present in a SampleTableBox (respectively a TrackFragmentBox), the number of samples described by CompactDirectSampleReferencesBox 600 is equal to the number of samples present in the track (respectively in the track fragment).

[0217] According to some embodiments, values of sample identifiers, in other words absolute or direct values of sample identifiers, are equal to the Picture Order Count (POC) of the associated samples. When the Picture Order Count is used to express a sample identifier, it is possible to optimize the coding of both sample identifiers and sample references using patterns, since the differentially coded sample identifier may follow the same pattern as the list of sample references.

[0218] In a variant, a sample identifier may be any identifier as long as the identifier is unique within the track, if the CompactDirectSampleReferencesBox is defined in a SampleTableBox, or within a track fragment if the CompactDirectSampleReferencesBox is defined in a track fragment.

[0219] It is noted that associating a unique sample identifier with each sample in a track or a track fragment and using it to describe the sample references as defined by the semantics of refJDs 655 brings the benefit of supporting also the description of interlayer dependencies, as the identifiers of sample references, when mapped to POC numbers, are unique across the layers within a layered bitstream in a track or a track fragment.

[0220] The reconstruction process of the list of samples and their sample references using CompactDirectSampleReferencesBox 600 should produce the same result as the following model: initializing an empty list f iat_refs,

[0221] - for each entry in the box, o if is_ref is set to its second value (e.g., 0), appending to the flat refs list an entry containing {nb refs, is abs, sample ID code, ref IDs}, o otherwise (is_ref is set to its first value (e.g., 1)), for each K ranging from 0 to num samples — 1 of the entry,

[0222] ■ if no_dif f_mode is set to its second value (e.g., 0), appending to flat refs the entry flat refs[offset + K%pattern length],

[0223] ■ otherwise (no_diff_mode is set to its first value (e.g., 1)), appending to flat refs a copy of flat refs[offset + K%pattern length] with sample ID code equal to sample IDs[K],

[0224] - validating that the number of entries in fiat_refs is the same as the number of samples in the track or track fragment,

[0225] - for each sample J in the track or track run: o assigning the sample identifier samplelD as follows:

[0226] ■ if fiat_refs [ J] . is_abs is set to its first value (e.g., 1), setting sample lD to flat ref s [ J] . sample ID code ,

[0227] ■ otherwise, setting sample lD to flat refs [ J] . sample ID code + sample [ J- 1 ] . samplelD, o assigning sample reference IDs for each K in range [0, flat refs [J] . nb ref s] as follows:

[0228] ■ if no_diff_mode is set to its second value (e.g., 0), removing from sample [ J] . samplelD the value flat refs [ J] . ref IDs [K] , i.e. ref erenceSamplelD = sample [ J] . samplelD - flat refs [ J] . ref IDs [ K] , otherwise (no_dif f_mode is set to its first value (e.g., 1)), using the value flat refs [J] , ref IDs [K] ,

[0229] In another embodiment, the sample references or dependencies of samples may be described by using the PatternSampleDescriptionBox 'psds' described by reference to Figure 3 with a dedicated description_type, e.g. ‘sdpe’. In such a case, PatternSampleDescriptionBox 300 may be modified to include the following code in pattern 340: unsigned int no_diff_mode = (flags & 16) ? 1 : 0; if (no_diff_mode && description_type == 'sdpe') { unsigned int (bits) sample_IDs [ num_samples ] ;

[0230] } and the PatternSampleDescriptionEntryO 350 may be described as follows: aligned(8) class SampleDependenciesPatternEntry (flags) extends PatternSampleDescriptionEntry ('sdpe' flags) { if (flags & 2) bits=32 ; else if (flags & 1) bits=16; else bits=8 unsigned int no_diff_mode = (flags & 16) ? 1 : 0; unsigned int (bits-l) nb_refs; if (no_diff_mode) { signed int (bits) sample_ID_code ;

[0231] } else { bit (1) is_abs; signed int (bits-l) sample_ID_code ;

[0232] } signed int (bits) ref_IDs [nb_refs ] ;

[0233] } where nb refs, is abs, sample ID code and ref IDS have the same semantics as previously described for CompactDirectSampleReferencesBox 600. It is noted that nb_refs is coded on (bits-2) bits instead of (bits-1) bits if the variant of PatternSampleDescriptionBox 'psds' including the repeat_pattern parameter is used.

[0234] In another embodiment, the sample references or dependencies of samples may be described as a sample group description as defined in ISOBMFF, e.g. by using the SampleToGroupBox, the SampleGroupDescriptionBox and by defining a specific value of grouping_type and defining a sample group description entry comprising similar information as in CompactDirectSampleReferencesBox 600.

[0235] This embodiment using a sample group instead of a global box allows mixing the description of dependencies of some samples using a differential coding (no_diff_mode being set to its second value, e.g. 0) and the description of dependencies of some other samples using an absolute coding only (no_diff_mode being set to its first value, e.g. 1).

[0236] Figure 7 illustrates an example of describing sample references of a sequence of samples using the syntax structure illustrated in Figure 6, according to some embodiments of the disclosure.

[0237] Table 700 illustrates a sequence of samples in decoding order represented by the column entitled Frame Type and referenced 720 where each row corresponds to a sample comprising a frame with the given frame type, here I for an intra frame, P for a predictive frame and B for a bi-predictive frame. An intra frame is a frame with no dependency to another frame in a sequence of frames. A predictive frame is a frame that may have a dependency with a previous frame in the sequence of frames, in the presentation order, and a bi-predictive frame may have dependencies with previous and following frames in the sequence of frames, in the presentation order.

[0238] The column entitled sampleJD and referenced 730 illustrates the absolute sample identifier (or sample ID) associated with each sample of column 720. According to the illustrated example, the absolute sample ID is set to the Picture Order Count (POC) that uniquely identifies the associated frame in the sample among all the frames of the sequence of frames that are encapsulated in the sequence of samples. For instance, the first sample corresponds to an I frame with a sample ID equal to 0, representing the first frame in composition order, the second sample corresponds to a P frame with a sample ID equal to 3, representing the fourth frame in composition order, meaning it is rendered after rendering the samples with sample IDs 0 to 2, and so on. The column entitled refs and referenced 740 illustrates the list of sample references or dependencies associated with each sample of column 720. According to the illustrated example, the list of references is expressed in absolute sample IDs (i.e. POCs). For instance, the second sample, corresponding to a P frame with POC equal to 3 depends on the first sample, corresponding to a I frame with POC equal to 0. Likewise, the third sample, corresponding to a B frame with POC equal to 2, depends on the first and second samples with POCs equal to 0 and 3 respectively.

[0239] The column entitled refJDs 750 contains the list of absolute sample references as coded when no_diff_mode is set to its first value (e.g., 1). Therefore, this column contains the same values as the column refs 740. For instance, the list of absolute sample references of the third sample with frame type B and sample ID 2 is equal to {0, 3} corresponding to a dependence with the samples with sample ID 0 and 3.

[0240] The large black lines 780 on table 700 indicate the boundaries of the fragments, which may correspond, for example, to a fragment duration of 200ms at 25 frames per second (FPS). As illustrated, each fragment contains 5 frames or samples. For instance, the second fragment contains the samples with sample IDs 6, 4, 5, 7, 13.

[0241] As apparent from column refs 740 or refJDs 750, the use of absolute references makes it possible to reuse the description of some dependency lists within a same fragment and define patterns that refer and repeat a previously defined description of dependency lists for instance with a CompactDirectSampleReferencesBox as defined by reference to Figure 6.

[0242] Part 710 on the right of the Figure 7 illustrates the gain obtained with the use of patterns according to some embodiments of the disclosure. More precisely, reference 710 illustrates the transformation of the sequence of samples and its list of sample references from table 700 to a CompactDirectSampleReferencesBox like CompactDirectSampleReferencesBox 600 in Figure 6 when no_diff_mode is set to its first value (e.g., 1) indicating that only absolute values are used for coding the sample IDs and the sample references.

[0243] For example, reference 760 illustrates the transformation of the first row of table 700 in a first entry described as an explicit sample reference (is_ref = 0) in the entry loop of the CompactDirectSampleReferencesBox, reference 765 represents the transformation of the fifth row of table 700 in a fifth entry described as a pattern (is_ref = 1) that reuses the directly preceding entry. Reference 770 represents the transformation of the eighth and ninth rows of table 700 in an entry described as a pattern (is_ref = 1) that repeat twice (num_samples = 2 and patternjength = 1) the sample references of the directly preceding entry. It can be noted that even if, in this example, the patternjength remains to 1 , the pattern description is less costly than coding a complete entry due to the number of sample dependences (4) that do not need to be repeated.

[0244] Figure 8 is a block diagram illustrating an example of steps carried out by an encapsulating device to encapsulate media data in a media file, according to some embodiments of the disclosure.

[0245] As illustrated, a first step is directed to obtaining media data (step 800), such as a sequence of images, that may be, for example, of any format mentioned above (MPEG-4, AVC, HEVC, L-HEVC, MV-HEVC, LCEVC, VVC, G-PCC, V3C, V-DMC, etc.).

[0246] Next, a track comprising a sequence of samples based on the media data is created (step 805) and metadata describing attributes, for example sample properties or characteristics and / or sample references or dependencies, associated with samples are generated (step 810). Such metadata may comprise a list of sample properties and / or references or dependencies, and sample identifier information for each sample of the sequence.

[0247] Optionally, for metadata generated according to embodiments described by reference to Figure 6, it may be determined whether the sequence of samples is organised in fragments, each fragment comprising one or more samples of the sequence of samples and whether the fragment durations are less than a threshold representative of a sub-GOP fragmentation, i.e. representative of fragment durations that would be shorter than a Group Of Picture (GOP) duration. As result of this determination, the value of the no_diff_mode variable is selected.

[0248] Preferably, the value of the no_diff_mode variable is set to its second value (e.g. 0) when one GOP or more is coded in a fragment, i.e. the fragments durations is greater or equal to the threshold), and to its first value (e.g. 1) when it is determined that the fragment durations are less than the threshold.

[0249] It is observed that the gain in description cost may be dependent of the GOP structure used by the encoder. In a variant, the no_diff_mode variable may be set depending on the GOP structure used by the encoder, possibly in combination with the ratio of the fragmentation compared to the GOP duration or size.

[0250] Next, it is determined, for sets of contiguous samples, whether the attributes (e.g. sample references or dependencies, or properties) for the samples of this set may be defined on the basis of attributes of samples of a set of previous continuous samples (step 815). For the sake of illustration, this may be done by identifying a previous sample having the same attributes (e.g., sample references or dependencies, or properties) as the considered sample and determining recurrently whether the attributes of the sample following the considered sample are the same as the one of the sample following the previous identified. This results as a pattern that may be repeated for describing samples to be described, as described above.

[0251] If a set of N samples starting from a considered sample may be defined as a function of a pattern of previous samples (step 820), that may be repeated, the set of P samples (P < N) is described as a pattern in an entry i.e., as a pattern starting from the first previous identified sample, having a determined length, and possibly being repeated a number of times. According to some embodiments, the length N of the set of samples to be described is provided in the description so as to make it possible to determine how many times the pattern is to be repeated. Next, the new sample to be described (i.e., the new considered sample) is defined as being the sample located at the Nth position after the current position (step 825). Next, a test is carried out to determine whether all the samples have been described (step 830).

[0252] According to some embodiments, descriptive metadata comprising the description of the samples are generated. As described above, such metadata describing attributes associated with samples may comprise an offset from the considered sample to make it possible to locate the first sample reference of the pattern and the length of the pattern. Still according to some embodiments, the metadata may also comprise the length of the set of samples being described. Alternately, the metadata may comprise the number of times the pattern is to be repeated.

[0253] On the contrary, if a considered sample (and possibly the following samples) cannot be described by reference to a pattern (step 815), the attributes (e.g. sample references or dependencies, or the properties) of the considered sample are described either on a standard basis using absolute coding of the values, or on using values coded as a difference with values associated with a preceding sample as described by reference to Figure 4 or Figure 6 (step 835).

[0254] After having described one or several samples as a function of a pattern or as absolute coding or differential coding of values associated with the current sample, and if a least another sample is to be processed, the algorithm loops at step 810 to process the next sample.

[0255] On the contrary, if all the samples have been processed, the media file is generated (step 845). As described above, the media file comprises the track, that in turn comprises the sequence of samples, and the metadata. Figure 9 is a block diagram illustrating an example of steps carried out by a processing device to process a media file, according to some embodiments of the disclosure.

[0256] As illustrated, a first step is directed to obtaining a track from a media file to be parsed (step 900). The media file is for example a media file generated according to a method like the one described by reference to Figure 8. The track comprises a sequence of samples based on media data.

[0257] Next, descriptive metadata describing attributes (e.g., sample references or dependencies, or sample properties or sample characteristics) associated with samples of the track are obtained (step 905) and lists of samples or references associated with samples of the sequence are reconstructed (step 910). Next, it is determined whether the processed entry is pattern (step 915).

[0258] If the processed entry is a pattern starting from a previous sample in the list of samples, the sample attributes for the N samples to be parsed are obtained, based on the pattern (step 920). Next, the new sample to be parsed (i.e., the new sample to process) is defined as being the sample located at the Nth position after the current position (step 925). Next, a test is carried out to determine whether all the entries have been described (step 930).

[0259] If the processed entry is not a pattern, sample attributes associated with the current (or considered) sample to parse are obtained from the current entry in descriptive metadata (step 935).

[0260] After having obtained the attributes of an entry, and if a least another entry is to be processed, the algorithm loops at step 910 to process the next entry.

[0261] On the contrary, if all the entries have been processed, the track based on the reconstructed list of sample attributes is processed (step 945).

[0262] Figure 10 is a schematic block diagram of a computing device 1000 for implementation of one or more embodiments of the disclosure. The computing device 1000 may be a device such as a micro-computer, a workstation, or a light portable device. The computing device 1000 comprises a communication bus 1002 connected to:

[0263] - a central processing unit (CPU) 1004, such as a microprocessor;

[0264] - a random access memory (RAM) 1008 for storing the executable code of the method of embodiments of the disclosure as well as the registers adapted to record variables and parameters necessary for implementing the method for encapsulating, indexing, de-encapsulating, and / or accessing data, the memory capacity thereof can be expanded by an optional RAM connected to an expansion port for example;

[0265] - a read only memory (ROM) 1006 for storing computer programs for implementing embodiments of the disclosure;

[0266] - a network interface 1012 that is, in turn, typically connected to a communication network 1014 over which digital data to be processed are transmitted or received. The network interface 1012 can be a single network interface, or composed of a set of different network interfaces (for instance wired and wireless interfaces, or different kinds of wired or wireless interfaces). Data are written to the network interface for transmission or are read from the network interface for reception under the control of the software application running in the CPU 1004;

[0267] - a user interface (Ul) 1016 for receiving inputs from a user or to display information to a user;

[0268] - a hard disk (HD) 1010; and / or

[0269] - an I / O module 1018 for receiving / sending data from / to external devices such as a video source or display.

[0270] The executable code may be stored either in read only memory 1006, on the hard disk 1010 or on a removable digital medium for example such as a disk. According to a variant, the executable code of the programs can be received by means of a communication network, via the network interface 1012, in order to be stored in one of the storage means of the communication device 1000, such as the hard disk 1010, before being executed.

[0271] The central processing unit 1004 is adapted to control and direct the execution of the instructions or portions of software code of the program or programs according to embodiments of the disclosure, which instructions are stored in one of the aforementioned storage means. After powering on, the CPU 1004 is capable of executing instructions from main RAM memory 1008 relating to a software application after those instructions have been loaded from the program ROM 1006 or the hard-disc (HD) 1010 for example. Such a software application, when executed by the CPU 1004, causes the steps of the flowcharts shown in the previous figures to be performed.

[0272] In this embodiment, the apparatus is a programmable apparatus which uses software to implement the disclosure. However, alternatively, the present disclosure may be implemented in hardware (for example, in the form of an Application Specific Integrated Circuit or ASIC). Although the present disclosure has been described herein above with reference to specific embodiments, the present disclosure is not limited to the specific embodiments, and modifications will be apparent to a person skilled in the art which lie within the scope of the present disclosure. Many further modifications and variations will suggest themselves to those versed in the art upon making reference to the foregoing illustrative embodiments, which are given by way of example only and which are not intended to limit the scope of the disclosure, that being determined solely by the appended claims. In particular the different features from different embodiments may be interchanged, where appropriate. In the claims, the word “comprising” does not exclude other elements or steps, and the indefinite article “a” or “an” does not exclude a plurality. The mere fact that different features are recited in mutually different dependent claims does not indicate that a combination of these features cannot be advantageously used.

Claims

CLAIMS1. A method of encapsulating media data in a media file, in a processing device, the method comprising: generating a track comprising a sequence of samples based on the media data; generating descriptive metadata for reconstructing a list of references for samples of the sequence of samples, each reference comprising a sample identifier code and a list of attributes; and generating the media file comprising the generated track and the descriptive metadata; wherein the descriptive metadata comprise a first indication indicating a relative position of a reference relative to a given sample and a second indication indicating a number of samples following the reference in the sequence of samples, the list of references comprising at least one repetition of a set of references comprising the reference referred to by the relative position and as many following samples as set in the number of samples.

2. The method of claim 1 , wherein the list of attributes is a list of dependencies, each of the dependencies being a dependency to another sample.

3. The method of claim 1 , wherein the list of attributes consists in properties.

4. The method of any one of claims 1 to 3, wherein the sample identifier code is determined as a function of a picture order count of the sample.

5. The method of any one of claims 1 to 4, wherein the first and the second indication define a first pattern, the first pattern being referenced by a first and a third set of samples, a second set of samples, located between the first and third set of samples, referencing a second pattern, different from the first pattern.

6. The method of any one of claims 1 to 5, wherein the descriptive metadata further comprises a third indication indicating a number of samples to describe based on the reference defined by the first and the second indication.

7. The method of any one of claims 1 to 6, wherein a single structure signals both a description of attributes and an association of the attributes with samples.

8. The method of any one of claims 1 to 6, wherein a first structure signals descriptions of attributes and wherein a second structure signals associations of these descriptions with samples.

9. The method of any one of claims 1 to 8, wherein the descriptive metadata further comprise a reference flag to indicate whether a set of at least one sample is described as a function of the list of references comprising at least one repetition of the set of references or as an explicit reference.

10. The method of any one of claims 1 to 9, wherein the descriptive metadata further comprise an identifier flag to indicate whether a sample identifier code is a value of the sample identifier or a difference between the sample identifier and an identifier of a preceding sample in the reconstructed list of references.

11. The method of any one of claims 1 to 9, wherein the descriptive metadata further comprise an identifier flag to indicate whether both a sample identifier code is a value of the sample identifier, and the list of attributes is a list of sample identifier values.

12. The method of claim 11 , depending on claim 6, wherein if the reference flag is set to a first predetermined value and the identifier flag is set to a second predetermined value, the descriptive metadata comprises a sample identifier value for each sample indicated by the third indication.

13. The method of claim 11 or 12, depending on claim 2, wherein each of the dependencies is a direct dependency to another sample.

14. A method for processing a media file encapsulating media data, in a processing device, the media data comprising a plurality of samples, the method comprising:obtaining, from the media file, a track comprising a sequence of samples based on the media data; obtaining, from the media file, descriptive metadata for reconstructing a list of references for samples of the sequence of samples, each reference comprising a sample identifier code and a list of attributes; and wherein the descriptive metadata comprise a first indication indicating a relative position of a reference relative to a given sample and a second indication indicating a number of samples following the reference in the sequence of samples, the list of references comprising at least one repetition of a set of references comprising the reference referred to by the relative position and as many following samples as set in the number of samples.

15. The method of claim 14, wherein the list of attributes is a list of dependencies, each of the dependencies being a dependency to another sample.

16. The method of claim 14, wherein the list of attributes consists in properties.

17. A method of any one of claims 14 to 16, wherein the sample identifier code is determined as a function of a picture order count of the sample.

18. The method of any one of claims 14 to 17, wherein the first and the second indication define a first pattern, the first pattern being referenced by a first and a third set of samples, a second set of samples, located between the first and third set of samples, referencing a second pattern, different from the first pattern.

19. The method of any one of claims 14 to 18, wherein the descriptive metadata further comprise a third indication indicating a number of samples to describe based on the reference defined by the first and the second indication.

20. The method of any one of claims 14 to 19, wherein a single structure signals both a description of attributes and an association of the attributes with samples.

21. The method of any one of claims 14 to 20, wherein a first structure signals descriptions of attributes and wherein a second structure signals associations of these descriptions with samples.

22. The method of any one of claims 14 to 21 , wherein the descriptive metadata further comprise a reference flag to indicate whether a set of at least one sample is described as a function of the list of references comprising at least one repetition of the set of references or as an explicit reference.

23. The method of any one of claims 14 to 22, wherein the descriptive metadata further comprise an identifier flag to indicate whether a sample identifier code is a value of the sample identifier or a difference between the sample identifier and an identifier of a preceding sample in the reconstructed list of references.

24. The method of any one of claims 14 to 22 wherein the descriptive metadata further comprise an identifier flag to indicate whether both a sample identifier code is a value of the sample identifier, and the list of attributes is a list of sample identifier values.

25. The method of claim 24, depending on claim 19, wherein if the reference flag is set to a first predetermined value and the identifier flag is set to a second predetermined value, the descriptive metadata comprises a sample identifier value for each sample indicated by the third indication.

26. The method of claim 24 or 25, depending on claim 15, wherein each of the dependencies is a direct dependency to another sample.

27. The method of any one of claims 1 to 26, wherein the media file complies with the ISO Base Media File Format, ISOBMFF.

28. A computer program product for a programmable apparatus, the computer program product comprising a sequence of instructions for implementing each of the steps of the method according to any one of claims 1 to 27 when loaded into and executed by the programmable apparatus.

29. A non-transitory computer-readable storage medium storing instructions of a computer program for implementing each of the steps of the method according to any one of claims 1 to 27.

30. A communication device comprising a processing unit configured for carrying out each of the steps of the method according to any one of claims 1 to 27.

Images