Method, apparatus and computer readable storage medium for generating a media file

Abstract

The present application relates to a method, apparatus and computer readable storage medium for generating a media file, the method comprising: generating a first data structure that allocates a subset of samples or subsamples of a track to one or more sample groups; generating a second data structure comprising data for describing each of the one or more sample groups, the first data structure comprising a first grouping type and the second data structure comprising a second grouping type, wherein the second data structure comprises data for indicating whether the data for describing each of the one or more sample groups is time invariant; and generating a media file comprising the samples and the first and second data structures.

Description

Technical Field

[0001] The present invention relates to a method and apparatus for encapsulating and transmitting media data. Background Art

[0002] The International Organization for Standardization Base Media File Format (ISO BMFF, ISO / IEC 14496-12) is a well-known flexible and extensible file format that encapsulates and describes coded timed or non-timed media data for local storage or for transmission via a network or via other bitstream delivery mechanisms. An example of an extension is ISO / IEC 14496-15, which describes encapsulation tools for various video coding formats based on NAL (Network Abstraction Layer) units. Examples of such coding formats are AVC (Advanced Video Coding), SVC (Scalable Video Coding), HEVC (High Efficiency Video Coding), L-HEVC (Layered HEVC) and VVC (Versatile Video Coding). Another example of a file format extension is ISO / IEC 23008-12, which describes encapsulation tools for still images or sequences of still images such as HEVC still images. Another example of a file format extension is ISO / IEC 23090-2, which defines the Omnidirectional Media Application Format (OMAF). The ISO base media file format is object-oriented. It consists of building blocks called boxes that correspond to data structures characterized by unique type identifiers (usually four-character codes, also called FourCC or 4CC). A complete box is a data structure similar to a box that additionally includes version and flag value attributes. In the following, the term box can designate both complete boxes or boxes. These boxes or complete boxes are organized hierarchically or sequentially in the ISOBMFF file and define parameters for describing the encoded timed or non-timed media data, its structure and timing (if any). All data in the encapsulated media file (media data and metadata describing the media data) is contained in a box. No other data exists in the file. A file-level box is a box that is not contained in other boxes.

[0003] In the file format, an entire media presentation is called an animation. An animation is described by an animation box (with the four-character code "moov") at the top level of the file. This animation box represents an initialization information container containing a collection of various boxes describing the media presentation. It is logically divided into tracks, represented by track boxes (with the four-character code "trak"). Each track (uniquely identified by a track identifier (track_ID)) represents a timed sequence of media data belonging to the presentation (e.g., frames of video or audio samples). Within each track, each unit of timed data is called a sample; this might be a frame of video, audio, or timed metadata. Samples are implicitly numbered in a sequence in decoding order. Each track box contains a hierarchy of boxes describing the samples of the track. For example, the sample table box ("stbl") contains all the time and data indexes of the media samples in the track. The actual sample data is stored at the same level as the animation box in boxes called media data boxes (with the four-character code "mdat") or identification media data boxes (with the four-character code "imda," which are similar to the media data box but contain additional identifiers). Animations can also be temporally segmented and organized as an animation box containing information for the entire presentation, followed by a list of media fragments, i.e., a list of coupled animation fragments and media data boxes ("mdat" or "imda"). Within an animation fragment (a box with the four-character code "moof"), there is a collection of track fragments (boxes with the four-character code "traf") that describe the tracks within the media fragment, with each animation fragment being zero or more track fragments. A track fragment, in turn, contains zero or more track run boxes ("trun"), each of which records a continuous run of samples for that track fragment.

[0004] An ISOBMFF file may contain multiple coded timed media data or sub-parts of coded timed media data forming multiple tracks. When a sub-part corresponds to one or a continuous spatial portion of a video source captured over time (e.g., at least one rectangular area captured over time, sometimes referred to as a "tile" or "sub-picture"), the corresponding multiple tracks may be referred to as tile tracks or sub-picture tracks.

[0005] ISOBMFF and its extensions include several grouping mechanisms to group tracks, static items, or samples together and associate group descriptions with them. Groups typically share common semantics and / or properties.

[0006] The inventors have noticed several problems when describing and signaling information about media data to be sent, in particular problems for describing groups of samples or groups of NAL units within samples.

[0007] One example involves reducing the cost of signaling a set of samples that is constant over time.

[0008] Another example involves optimizing the signaling of NAL unit characteristics.

[0009] Existing solutions are either complex or poorly defined. Summary of the Invention

[0010] The present invention is directed to solving one or more of the aforementioned problems.

[0011] According to a first aspect of the present invention, there is provided a method for generating a media file, the method comprising:

[0012] generating a first data structure that assigns subsets of samples or subsamples of a track to one or more sample groups;

[0013] generating a second data structure comprising data describing respective sample groups of the one or more sample groups, the first data structure comprising a first grouping type and the second data structure comprising a second grouping type, wherein the second data structure comprises data indicating whether the data describing respective sample groups of the one or more sample groups is invariant over time; and

[0014] A media file is generated that includes the sample and the first and second data structures.

[0015] According to an embodiment, the first grouping type and the second grouping type indicate the same type of sample groups.

[0016] According to the embodiment:

[0017] data for indicating whether data describing each of the one or more sample groups is constant over time is a flag;

[0018] When the flag is set, it indicates that the first data structure includes a single set of property values ​​that applies to all track samples of the track.

[0019] According to an embodiment, data indicating whether data describing each of the one or more sample groups is constant over time is defined as:

[0020] The group type and default group description index of the second data structure; and

[0021] When the packet type value is set to a predetermined packet type and the default packet description index is greater than zero, then the second data structure is determined to be constant over time.

[0022] According to the embodiment:

[0023] The data for indicating whether the data describing each of the one or more sample groups is constant over time includes a two-bit flag;

[0024] The first flag value indicates that the second data structure includes a single set of property values ​​that apply to all track samples of the track;

[0025] The second flag value indicates that the second data structure is constant over time, whereas the samples or subsamples constituting the group may vary over time.

[0026] According to the embodiment:

[0027] The third flag value indicates that the second data structure is constant over time, whereas the default sample group mapping may change over time.

[0028] According to another aspect of the present invention, a method for generating a media file is provided, the method comprising:

[0029] generating a first data structure that assigns subsets of samples or subsamples of a track to one or more sample groups; and

[0030] generating a media file including the sample and the first data structure;

[0031] in:

[0032] The first data structure includes data indicating that one or more sample groups are virtual.

[0033] According to the embodiment:

[0034] The packet type value identification attribute in the first data structure;

[0035] The index to the attribute value set in the first data structure includes the attribute value set for the identified attribute.

[0036] According to an embodiment, the data indicating that one or more sample groups are virtual is a grouping type value.

[0037] According to an embodiment, the data indicating that one or more sample groups are virtual is a parameter within the data structure.

[0038] According to an embodiment, the method further comprises:

[0039] generating a second data structure comprising data describing each of the one or more sample groups;

[0040] in:

[0041] Each sample in the subset consists of one or more NAL units;

[0042] The first data structure and the second data structure include a same grouping type value indicating a mapping between a subset of samples and a sample group describing a group of one or more NAL units;

[0043] The packet type parameter value identification attribute in the first data structure;

[0044] The group identifier value in the data describing each group of one or more NAL units in the second data structure includes a set of attribute values ​​for the identified attribute.

[0045] According to an embodiment, the data indicating that one or more sample groups are virtual is a grouping type parameter value.

[0046] According to an embodiment, the data indicating that the one or more sample groups are virtual is a flag of the first data structure.

[0047] According to another aspect of the present invention, there is provided a computer program product for a programmable device, the computer program product comprising a sequence of instructions for implementing the method according to the present invention when loaded into and executed by the programmable device.

[0048] According to another aspect of the present invention, a computer-readable storage medium is provided, which stores instructions of a computer program for implementing the method according to the present invention.

[0049] According to another aspect of the present invention, there is provided a computer program which, when executed, causes the method of the present invention to be performed.

[0050] According to another aspect of the present invention, an apparatus for generating a media file is provided, the apparatus comprising a processor, wherein the processor is configured to:

[0051] generating a first data structure that assigns subsets of samples or subsamples of a track to one or more sample groups;

[0052] generating a second data structure comprising data describing each of the one or more sample groups, wherein the second data structure comprises data indicating whether the data describing each of the one or more sample groups is invariant over time; and

[0053] A media file is generated that includes the sample and the first and second data structures.

[0054] According to another aspect of the present invention, an apparatus for generating a media file is provided, the apparatus comprising a processor, wherein the processor is configured to:

[0055] generating a first data structure that assigns subsets of samples or subsamples of a track to one or more sample groups; and

[0056] generating a media file including the sample and the first data structure;

[0057] in:

[0058] The first data structure includes data indicating that one or more sample groups are virtual. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Other advantages of the present invention will become apparent to those skilled in the art upon examination of the drawings and detailed description. It is intended that any additional advantages also be incorporated herein.

[0060] Embodiments of the present invention are described below, by way of example only, with reference to the following drawings, in which:

[0061] Figure 1 An exemplary system including an encapsulation / decapsulation module suitable for embodying embodiments of the present invention is shown;

[0062] Figure 2 shows an example of the structure of a segment media file according to the ISO base media file format;

[0063] Figure 3 shows an example of the structure of a video media sample according to the ISO base media file format;

[0064] Figure 4 An embodiment showing static sample grouping without NAL unit mapping is shown;

[0065] Figure 5 An embodiment showing static sample grouping with static NAL unit mapping is shown;

[0066] Figure 6 An embodiment showing virtual sample grouping without NAL unit mapping;

[0067] Figure 7 An embodiment of virtual sample grouping with NAL unit mapping is shown;

[0068] Figure 8 is a schematic block diagram of a computing device for implementing one or more embodiments of the present invention. DETAILED DESCRIPTION

[0069] Figure 1 Exemplary systems 191 and 195 suitable for embodying embodiments of the present invention are shown. System 191 includes an encapsulation module 150 connected to a communications network 199. System 195 includes a decapsulation module 100 connected to a communications network 199.

[0070] According to an embodiment, system 191 is used to process content (e.g., video, still images, and / or audio content) for streaming or storage. System 191 obtains / receives content including an original non-timed set or timed sequence of images 151, encodes the non-timed set or timed sequence of images into coded media data using a media encoder (e.g., an image or video encoder), and encapsulates the coded media data into a media file 101 using an encapsulation module 150. Encapsulation module 150 includes at least one of a writer or a packager for encapsulating the coded media data. The media encoder can be implemented within encapsulation module 150 to encode the received content, or the media encoder can be separate from encapsulation module 150. Thus, encapsulation module 150 can be dedicated solely to encapsulating already-coded content (coded media data). The encoding step is optional, and the coded media data can correspond to the original media data.

[0071] The encapsulation module 150 may generate a plurality of media files 101. The plurality of media files may correspond to alternative versions of the media content and / or consecutive segments of the media content.

[0072] According to an embodiment, system 195 is configured to process encapsulated encoded media data for display / output to a user. System 195 obtains / receives media file 101 via communication network 199 or by reading a storage component, decapsulates encoded media file 101 using decapsulation module 100 to retrieve the media data, and decodes the encoded media data into audio and / or video content (signals) using a media decoder. Decapsulation module 100 includes at least one of a parser or a player. The media decoder can be implemented within decapsulation module 100 to decode the encoded media data, or the media decoder can be separate from decapsulation module 100.

[0073] The media file 101 is communicated to the parser or player of the module 100 in a variety of ways. For example, the media file 101 can be pre-generated by the writer or packager of the encapsulation module 150 and stored as data in a remote storage device (e.g., on a server or cloud storage) or a local storage device in the communication network 199 until the user requests the encoded content therein from the remote or local storage device. When the content is requested, the data is read / communicated / streamed from the storage device to the decapsulation module 100.

[0074] The system 191 may also include a content providing device for providing / streaming content information of content stored in the storage device to the user (for example, the content information may be described via a manifest file, which includes the title of the content and other descriptive metadata and storage location data for identifying, selecting, and requesting the content). The content providing device may also be adapted to receive and process user requests for content to be transferred / streamed from the storage device to the user terminal.

[0075] Alternatively, when a user requests content, the encapsulation module 150 may generate a media file 101 and communicate / stream it directly to the decapsulation module 100. The decapsulation module 100 then receives the media file 101 and decapsulates and decodes the media data according to an embodiment of the present invention to obtain / generate a video signal 109 and / or an audio signal, which is then used by the user terminal to provide the requested content to the user.

[0076] The user accesses the audio / video content (signals) through a user interface of a user terminal including module 100 or a user terminal having components communicating with module 100. Such a user terminal may be a computer, a mobile phone, a tablet or any other type of device capable of providing / displaying content to a user.

[0077] According to one implementation, media file 101 encapsulates coded media data (e.g., coded audio or video) into boxes according to the ISO Base Media File Format (ISOBMFF, ISO / IEC 14496-12, and ISO / IEC 14496-15 standards). Media file 101 can correspond to a single media file (prefixed with FileTypeBox "ftyp") or a media file followed by one or more segment files (possibly prefixed with SegmentTypeBox "styp"). According to ISOBMFF, media file 101 (and segment files, if present) can include two types of boxes: a "media data" box ("mdat" or "imda") containing media data, and a "metadata box" ("moov" or "moof" or "meta" box level) containing metadata that defines the placement and timing of the media data.

[0078] An image or video encoder encodes image or video content using an image or video standard to generate encoded media data. For example, image or video encoding / decoding (codec) standards include ITU-T H.261, ISO / IEC MPEG-1 Visual, ITU-T H.262 (ISO / IEC MPEG-2 Visual), ITU-T H.263 (ISO / IEC MPEG-4 Visual), ITU-T H.264 (ISO / IEC MPEG-4 AVC) (including its scalable video coding (SVC) and multiview video coding (MVC) extensions), and ITU-T H.265 (HEVC) (including its scalable (SHVC) and multiview (MV-HEVC) extensions).

[0079] The embodiments described herein primarily describe examples using the currently developing VVC standard (also known as ITU-TH.VVC, ISO / IEC MPEG-I VVC, and Versatile Video Coding). However, the techniques and systems described herein may also be applicable to other coding standards that are already available (such as HEVC) or coding standards that are not yet available or under development.

[0080] Figure 2 An example is shown in which the media file 101 is temporally organized into segment media files according to the ISO base media file format.

[0081] The media file 101 begins with a FileTypeBox "ftyp" (not shown), followed by a movie box ("moov") 201. The movie box provides a description of the content of the media file, the number of tracks, and information about their respective timelines and characteristics, such as a track with trackID equal to 1. The movie box 201 is followed by one or more media fragments, such as a first media fragment containing and describing samples 1 to N of the track with trackID equal to 1, the first media fragment consisting of a MovieFragmentBox ("moof") 202 and a media data box ("mdat") 203; and, for example, a second media fragment containing and describing samples N+1 to N+M of the track with trackID equal to 1, the second media fragment consisting of a movie fragment box ("moof") 204 and a media data box ("mdat") 205.

[0082] Among other information, a TrackBox ("trak") in a MovieBox ("moov") contains a SampleTableBox ("stbl") in its box hierarchy that contains descriptive and timing information for the media samples in the track. In particular, the SampleTableBox ("stbl") contains a SampleDescriptionBox ("stsd") that contains one or more SampleEntry boxes that give detailed information about the type of encoding used for the sample, as well as any initialization information required for encoding or decoding. For example, a SampleEntry box with a four-character type of "vvc1" or "vvi1" signals that the associated sample contains media data encoded according to the VVC format, or a SampleEntry box with a four-character type of "hvc1" or "hev1" signals that the associated sample contains media data encoded according to the HEVC format. A SampleEntry box may contain other boxes with information applicable to all samples associated with that SampleEntry box. When the media file is a non-fragmented media file, a sample is associated with a SampleEntry via the sample_description_index parameter in the SampleToChunkBox ("stsc") in the SampleTableBox ("stbl"), or via the TrackFragmentHeaderBox ("tfhd") in the TrackFragmentBox ("traf") of the MovieFragmentBox ("moof") when the media file is a fragmented media file. Even though SampleEntry can be used to change the information associated with a sample over time, it only provides a very coarse-grained granularity and does not allow the information to be changed for each sample or each non-contiguous group of samples, and its use is very limited in practice because it typically requires a reset of the decoder due to changes in the decoder configuration parameters contained in the SampleEntry.

[0083] ISOBMFF and its extensions include another grouping mechanism dedicated to sample groups.The sample grouping mechanism may be used to indicate that certain properties associated with an indicated grouping type or semantics apply to the indicated sample group within a track.

[0084] The sample grouping mechanism allows certain properties corresponding to the indicated grouping type to be associated with the sample groups within a track.

[0085] This mechanism relies on the use of two kinds of frames:

[0086] - SampleToGroupBox ("sbgp"), which describes the assignment of each sample to a sample group and its sample group description, and

[0087] - SampleGroupDescriptionBox ("sgpd"), which describes the common properties of samples within a specific sample group. The SampleGroupDescriptionBox ("sgpd") contains a list of SampleGroupEntry (VisualSampleGroupEntry for video content), each instance of which provides different values ​​for the common properties defined for a specific sample group (identified by its "grouping_type").

[0088] A specific type of sample grouping is defined by the combination of a SampleToGroupBox and a SampleGroupDescriptionBox via the type field ("grouping_type").

[0089] There can be multiple sample grouping instances (ie, SampleToGroupBox and SampleGroupDescriptionBox pairs) based on different grouping_type values ​​(corresponding to different grouping criteria).

[0090] SampleToGroupBox is defined as follows:

[0091]

[0092] Among them, grouping_type identifies the type of sample grouping (ie, the criteria for forming sample groups) and is linked to the sample group description table with the same value for the grouping type.

[0093] The grouping_type_parameter is an indication of the subtype of the grouping.

[0094] sample_count indicates the number of consecutive samples in the group associated with the sample group entry.

[0095] The group_description_index gives the index in the associated SampleGroupDescriptionBox of the SampleGroup entry that describes a sample in this group. A value of 0 indicates that the sample is not a member of a group of this type.

[0096] SampleGroupDescriptionBox is defined as follows:

[0097]

[0098] Here, grouping_type identifies the type of sample grouping (ie, the criteria used to form a sample group) and links the sample grouping to its SampleToGroupBox having the same value for the grouping type.

[0099] default_length indicates the length of each group entry if length is constant, or zero (0) if length is variable.

[0100] entry_count indicates the number of entries in the following table.

[0101] description_length indicates the length of a single group entry in the case where the length varies from entry to entry, so default_length is 0.

[0102] SampleGroupDescriptionEntry is a data structure (eg, a structure inherited from VisualSampleGroupEntry for video content) containing a set of parameters (ie, attributes or attribute values) characterizing a sample group of type grouping_type.

[0103] When the version of the box is higher than or equal to 2, the SampleGroupDescriptionBox contains an additional parameter default_group_description_index. This parameter specifies the index of the sample group description entry that applies to all samples in the track for which no sample-to-group mapping is provided via the SampleToGroupBox. The default value of this field is zero (indicating that the sample is not mapped to a group description of this type). This allows samples to be assigned to entries with a sample group description of a given grouping_type by default, without having to define an associated SampleToGroupBox with the same grouping_type. This mechanism is also called default sample grouping. This is useful for reducing the cost of signaling when all or almost all samples share the same set of attribute values.

[0104] In the fragment media file 101, an initial list of SampleGroupDescriptionEntries for each sample grouping type can be defined in the SampleTableBox ("stbl") of the MovieBox ("moov") by defining a SampleGroupDescriptionBox ("sgpd") for each grouping_type value. When needed (i.e., there is no default sample grouping), a SampleToGroupBoxe providing a mapping of samples with group description entries is defined in the TrackFragmentBox ("traf") in the MovieFragment ("moof") of each media fragment 202 and 204.

[0105] It is also possible to define additional SampleGroupDescriptionBoxes ("sgpd") in each media fragment to update the list of available SampleGroupDescriptionEntries for a given grouping_type over time. This means that if a default sample grouping is defined in a MovieBox ("moov"), the default sample grouping assignment for each media fragment can be updated.

[0106] As described above, the sample grouping mechanism allows attributes to be assigned to sample groups over time with a very flexible granularity, i.e. per media fragment, per default to all samples that are not explicitly associated with a SampleGroupDescriptionEntry via a SampleToGroupBox for a given grouping_type, per sample, per sample group (for both contiguous or discontinuous sample sequences).

[0107] It would be advantageous to be able to signal properties of samples that may be static or dynamic over time depending on the properties of the encapsulated media data content.

[0108] As an example, the specification (“w19278 - ISO / IEC 14496-15 Edition 5 Amendment 2 on the carriage of VVC and EVC in ISOBMFF”) th The VVC file format under ISO / IEC 16001 (Edition Amendment 2 on Carriage of VVC and EVC in ISOBMFF") proposes to signal the identifier (ID) of a VVC sub-picture carried within a sample of a VVC track or a VVC sub-picture track. To this end, two data structures are defined:

[0109] The optional box SubpicPropertiesBox ("supp") contained in the sample entry VvcSampleEntry or VvcSubpicSampleEntry, or

[0110] Use a sample group with grouping type "spid" and define a new SampleGroupDescriptionEntry (VvcSubpicIDEntry) that inherits from VisualSampleGroupEntry.

[0111] SubpicPropertiesBox is defined as follows:

[0112]

[0113] Among them, subpic_id specifies the sub-picture identifier of the sub-picture sequence contained in the VVC sub-picture track.

[0114] Similarly, VvcSubpicIDEntry is defined as follows:

[0115]

[0116] The subpic_id of the loop counter i specifies the i-th sub-picture identifier in decoding order contained in the sample mapped to the sample group description entry.

[0117] The former structure (SubpicPropertiesBox) defined in SampleEntry is intended to signal a static configuration, and the latter structure (VvcSubpicIDEntry) defined as a sample grouping is intended to signal a dynamic configuration that may change over time.

[0118] This definition is suboptimal because similar data structures located in different places in the media file need to be defined, and the parser needs to check multiple data structures to determine whether the signaling of the sample attribute (i.e., the list of sub-picture IDs) is provided in the media file and whether the configuration is static or dynamic over time.

[0119] Although the default sample grouping mechanism without a sample group mapping (i.e., without SampleToGroupBox) allows assigning a set of property values ​​to all samples of each default track, in the case of fragment media files there is no guarantee that the default values ​​will never change over time (e.g., at the next media fragment).

[0120] A first aspect of the present invention is to address the above limitations.

[0121] according to Figure 4 In the first embodiment shown, SampleGroupDescriptionBox 401 is modified as follows to authorize the definition of the box's flag:

[0122]

[0123] The semantics of the syntax elements remain the same as before.

[0124] Additionally, a new flag value 402 is defined as follows:

[0125] sample_group_static: Indicates that this sample group is constrained to be static and never modified. The flag value is 0x000001. When this flag is set, there should be a single entry in the sample group description, no sample-to-group box should exist in this or subsequent track fragments, and each sample in this or subsequent track fragments is associated with a uniquely defined sample group description entry. If a SampleGroupDescriptionBox with the same grouping_type is defined in a subsequent track fragment, it should have exactly the same BoxPayload.

[0126] This means that the properties defined in a single entry defined in the SampleGroupDescriptionBox 403 are constant over time and apply statically to all samples in the track.

[0127] Thus, when the flag sample_group_static is set, the parser or reader is guaranteed that the default value is constant over time and will never change over time, and if such a change occurs, the parser or reader may treat the media file as an inconsistent file and raise an error, or alternatively, may ignore the change.

[0128] exist Figure 4 , the grouping_type value 404 is shown with the value "xxxx", but it can take any other four-character value corresponding to any type of attribute.

[0129] As an alternative, instead of defining a flag for the SampleGroupDescriptionBox, the semantics of the box can be modified based on the grouping_type value 404. It can be enforced that if a SampleGroupDescriptionBox with grouping_type equal to a given group type value (version greater than or equal to 2 and default_group_description_index greater than 0) exists at the TrackBox ("trak") level of a segment media file, then no SampleToGroupBox with grouping_type equal to the given group type value should exist for that track (including in any subsequent segments). In this case, the SampleGroupDescriptionEntry 403 is static for the entire track (i.e., does not change over time). Otherwise, the sample groupings may change for sample groups over time. This alternative has the advantage of not changing the existing SampleGroupDescriptionBox definition.

[0130] Using the above alternative, it is not possible to signal that only the list of SampleGroupDescriptionEntry 403 does not change over time while the sample group mapping may change over time (by defining a new SampleToGroupBox), but the default sample group assignment and the list of SampleGroupDescriptionEntry cannot change.

[0131] To fix this, the two-bit flags are defined (for example, in the flags of the SampleGroupDescriptionBox) with the following values:

[0132] sample_group_static, with flags value 0x000001 and the same semantics as above.

[0133] all_sample_group_description_static: Indicates that this sample group description is constrained to be static and the list of SampleGroupDescriptionEntry and the default sample group map (when present) are never modified. The flag value is 0x000002. When this flag is set, multiple entries may exist in the sample group description, there may be a SampleToGroupBox with the same grouping_type, and there should not be a SampleGroupDescriptionBox with the same grouping_type in subsequent track fragments, or they should have exactly the same BoxPayload. This means that the group description data structure is static (i.e., it does not change over time), while the track samples that make up the group may change over time.

[0134] Other values ​​are reserved.

[0135] In another alternative, a third value of a two-bit flag is defined to allow the default sample group mapping to change over time, but still signal that the SampleGroupDescriptionEntry list remains unchanged over time. This third flag is defined as follows:

[0136] sample_group_description_list_static: Indicates that the list of SampleGroupDescriptionEntry in this sample group description is constrained to be static. The flag value is 0x000003. When this flag is set, multiple entries can exist in the sample group description, a SampleToGroupBox with the same grouping_type can exist, and a SampleGroupDescriptionBox with the same grouping_type can also exist in subsequent track fragments. When a SampleGroupDescriptionBox with the same grouping_type exists in subsequent media fragments, the list of SampleGroupDescriptionEntry should be exactly the same. This means that the group description data structure is static, while the default sample group mapping may change over time.

[0137] In another alternative, a single flag sample_group_static is defined as follows:

[0138] sample_group_static: Indicates that this sample group is constrained to be static and never modified. The flag value is 0x000001. If a SampleGroupDescriptionBox with the same grouping_type is defined in a subsequent track fragment, it should have exactly the same BoxPayload. When this flag is set and there is a single entry in the sample group description, there should be no SampleToGroupBox for this grouping type in this track or subsequent track fragments, and each sample in this track or subsequent track fragments is associated with a uniquely defined sample group description entry. When this flag is set and there are several entries in the sample group description, the SampleToGroupBox mechanism should be used (possibly using the default sample grouping mechanism) to map samples to sample group descriptions.

[0139] In another alternative, the sample_group_static, all_sample_group_description_static, and sample_group_description_list_static values ​​may be defined as separate 1-bit flag values.

[0140] When the present invention is applied to a sub-picture ID sample group (with a grouping type "spid" and a sample group description entry VvcSubpicIDEntry as defined above), the sample group "spid" can exist in a VVC sub-picture track that contains a VVC sub-picture. The sample group will not exist in a track that is not a VVC sub-picture track or does not contain a VVC sub-picture. Each sample group description entry indicates a list of one or more sub-picture ID values, in decoding order, contained in the sample mapped to the sample group description entry. When the sample group is used as the default sample group and the flag sample_group_static is set, all samples in the track contain a VVC sub-picture with the indicated sub-picture ID value.

[0141] When a VVC sub-picture track is referenced by a VVC base track containing a "spor" sample group description entry with subpic_id_info_flag equal to 1, the VVC sub-picture track shall contain a sub-picture ID sample group or a sub-picture ID default sample group. When a sub-picture ID default sample group with flag sample_group_static is present, all samples contain a VVC sub-picture with the indicated sub-picture ID value.

[0142] According to another aspect of the invention, the sample grouping mechanism is useful for associating attribute descriptions with groups of samples or with individual samples, but can also be extended to allow associating attribute descriptions with groups of sub-portions within a sample.

[0143] Figure 3 A structure example of a media (video) sample 300 of an ISO base media file format according to a video format based on a NAL (network abstraction layer) unit based video coding format, e.g. HEVC or VVC coding format, is shown.

[0144] A media sample is a unit of audio / video data with a single time (e.g. an audio sample or a video frame). According to ISO / IEC 14496-15, a sample is a set of one or more NAL units 302 corresponding to an access unit or a part of an access unit. Before each NAL unit 302 is a NAL unit length field 301 providing the byte length of that NAL unit 302. For example, for single-layer video, a sample corresponds to a coded picture. For layered video, a sample can correspond to a part of an access unit, e.g. to NAL units of a base layer. For video coded in HEVC, a sample can correspond to an access unit or a part of an access unit consisting of slices, each slice being encapsulated into one or more NAL units. As another example, for VVC video containing sub-pictures (spatially rectangular sub-sections of a picture), a sample can correspond to one or more sub-pictures, each sub-picture consisting of one or more NAL units 302.

[0145] The ISOBMFF and its extensions include in the file format for HEVC a mechanism based on a sample grouping mechanism to assign an identifier, called groupID, to each NAL unit. A sample group with grouping_type equal to “nalm” is defined.

[0146] A SampleGroupDescriptionBox (“sgpd”) with grouping_type equal to “nalm” contains a list of NALUMapEntry (inherited from VisualSampleGroupEntry) defined as follows:

[0147]

[0148] where NALU_start_number is the 1-based NAL unit index in the sample of the first NAL unit in current run associated with the groupID.

[0149] The groupID specifies a unique identifier of the group. More information about the group is provided by the sample group description entry with that groupID and grouping_type_parameter of the SampleToGroupBox with grouping_type equal to type “nalm”.

[0150] Thus, each NALUMapEntry provides a mapping of NAL units in a sample with a groupID assigned to the NAL units in each sample.

[0151] Thus, NALUMapEntry is a data structure that assigns NAL units to one or more sample groups similarly to the SampleToGroup box.

[0152] A SampletoGroupBox of grouping type "nalm" provides an index to a NALUMapEntry assigned to each sample group, and a grouping_type_parameter of grouping_type that identifies a SampleGroupDescriptionBox containing semantics describing the NAL units with the same groupID assigned to each NAL unit.

[0153] In addition, we note that there is currently no way to identify the subpicture ID of NAL units in samples of a track that mixes several subpictures.

[0154] According to embodiments, this NAL unit mapping mechanism can be re-used in the VVC file format, for example to associate each subpicture carried in a sample with its subpicture ID.

[0155] In a first alternative, it can be allowed to use NALUMapEntry with "spid" sample groups. The "spid" sample group description can be used together with the NALU mapping sample group to describe the subpicture ID of NAL units within a sample. In this case, the groupID defined by the NALU mapping sample group description entry identifies a 1 -based index to a "spid" type of sample group description entry of the track. If no groupID is assigned or if the groupID has the value 0, this indicates that the subpicture ID of the associated NAL units is unknown.

[0156] As an alternative, if no groupID is assigned or if the groupID is equal to 0, this indicates that the subpicture ID of the NAL units mapped to the groupID or to the groupID with the value 0 is provided by the entry VvcSubpicIDEntry whose 1 -based index is given by the default_group_description_index parameter of the SampleGroupDescriptionBox of grouping_type "spid".

[0157] By default, in the case of NALU mapping, NAL units are not assigned to a groupID or are assigned to a groupID with a value of 0. This indicates that the set of property values ​​associated with these NAL units is provided by a SampleGroupDescriptionEntry identified by the 1-based index default_group_description_index in a SampleGroupDescriptionBox whose grouping_type is equal to the grouping_type_parameter in a SampleToGroupBox with a grouping_type of "nalm". This default rule may be overridden by the semantics of the grouping_type indicated by the grouping_type_parameter of a SampleToGroupBox of type "nalm".

[0158] In a second alternative, an additional groupID parameter may be added to the sample group entry VvcSubpicIDEntry as follows:

[0159]

[0160] Where subpic_id of loop counter i specifies the i-th sub-picture identifier in decoding order contained in the sample mapped to this sample group description entry. groupID is a unique identifier for the sub-picture ID group described by this sample group entry. When a SampleToGroupBox of type "nalm" and grouping_type_parameter equal to "spid" is present, a SampleGroupDescriptionBox of type "spid" shall be present, and the groupID value in the entry of the NALUMapEntry shall be equal to the groupID in one of the sub-picture ID group entries.

[0161] like Figure 5 As shown, NAL unit mapping sample grouping can also be combined with the above-mentioned static signaling.

[0162] When applied to NAL unit mapped sample grouping, flags signaling static properties of the sample grouping (sample_group_static, all_sample_group_description_static, or sample_group_description_list_static) can be set on a SampleGroupDescriptionBox with grouping_type of "nalm" that describes the NAL unit groups within the sample, or can be set to a SampleGroupDescriptionBox that provides a set of property values ​​associated with individual NAL unit groups.

[0163] When any of the new flag values ​​sample_group_static, all_sample_group_description_static, or sample_group_description_list_static is set to a SampleGroupDescriptionBox with grouping_type = "nalm", this means that the static properties apply to the mode of NAL unit groups within a sample (i.e., the mode where a groupID is associated with a NAL unit):

[0164] sample_group_static: indicates that the mode of NAL unit groups within a sample is constrained to be static and never modified. When this flag value is set in a SampleGroupDescriptionBox with grouping_type "nalm", there shall be a single entry in the sample group description, a SampleToGroupBox with grouping_type "nalm", and each sample in a track or subsequent track fragment is associated with a uniquely defined sample group description entry NALUMapEntry either explicitly via group_description_index in the SampleToGroupBox with grouping_type "nalm", or by default via default_group_description_index in the SampleGroupDescriptionBox with grouping_type "nalm". If a SampleGroupDescriptionBox with the same grouping_type is defined in a subsequent track fragment, it shall have exactly the same BoxPayload.

[0165] all_sample_group_description_static: Indicates that the list of mode descriptions for NAL unit groups within a sample is constrained to be static, and the list of SampleGroupDescriptionEntry and the default sample group map (when present) are never modified. When this flag value is set in a SampleGroupDescriptionBox with grouping_type "nalm", there may be multiple NALUMapEntry entries in the sample group description box, there may be (possibly several) SampleToGroupBoxes with grouping_type "nalm", and there should not be a SampleGroupDescriptionBox with the same grouping_type in subsequent track fragments, or if present, it should have exactly the same BoxPayload. This means that the list of mode descriptions for NAL unit groups within a sample is static (i.e., does not change over time), while the association of track samples with mode descriptions of NAL unit groups may change over time.

[0166] sample_group_description_list_static: Indicates that the mode description list of NAL unit groups within a sample is constrained to be static. When this flag value is set in a SampleGroupDescriptionBox with a grouping_type of "nalm", there may be multiple entries of type NALUMapEntry in the sample group description box, there are (possibly several) SampleToGroupBoxes with a grouping_type of "nalm", and there may also be SampleGroupDescriptionBoxes with the same grouping_type in subsequent track fragments. When a SampleGroupDescriptionBox with the same grouping_type is present in subsequent media fragments, the list of entries of type NALUMapEntry should be exactly the same. This means that the sample group description data structure is static, while the default sample group map (when it exists) may change over time.

[0167] When any of the new flag values ​​sample_group_static, all_sample_group_description_static, and sample_group_description_list_static is set to a SampleGroupDescriptionBox with grouping_type equal to the grouping_type_parameter of a SampleToGroupBox with grouping_type being "nalm", this means that static properties apply to the description of the property value set associated with the NAL unit.

[0168] When the flag value sample_group_static or all_sample_group_description_static is set in the SampleGroupDescriptionBox of a given grouping_type_parameter of a SampleToGroupBox with grouping_type equal to "nalm", and when a SampleGroupDescriptionBox with grouping_type equal to the same given grouping_type_parameter exists in subsequent media fragments, it may have exactly the same BoxPayload (i.e., the list of SampleGroupDescriptionEntry and the default_group_description_index (when it exists) shall be exactly the same). This means that the sample group description data structure is static, while the association of SampleGroupDescriptionEntry with NAL unit groups may change over time.

[0169] When the flag value sample_group_description_list_static is set in a SampleGroupDescriptionBox for a given grouping_type_parameter of a SampleToGroupBox with grouping_type equal to "nalm", and when a SampleGroupDescriptionBox with grouping_type equal to the same given grouping_type_parameter is present in subsequent media fragments, the list of SampleGroupDescriptionEntry may be exactly the same. This means that the group description data structure is static, while the association of SampleGroupDescriptionEntry with NAL unit groups may change over time.

[0170] Additionally, we note that there is currently no way to identify the coordinates (horizontal and vertical offsets in luma pixels) and size (width and height in luma pixels) of the rectangular region formed by NAL units (representing a sub-picture or slice) in a VVC sample.

[0171] According to an embodiment, NAL unit mapping may be used with TileRegionGroupEntry defined in ISO / IEC 14496-15 as follows:

[0172] A rectangular region of space can be composed of any of the following:

[0173] - A sequence of one or more VVC sub-pictures consecutive in decoding order.

[0174] = - a sequence of one or more complete strips that form a rectangular area and are consecutive in decoding order.

[0175] The rectangular spatial regions cover the rectangle without holes. The rectangular spatial regions within the image do not overlap with each other.

[0176] A rectangular spatial region may be described by a rectangular spatial region visual sample group description entry (ie, an instance of TileRegionGroupEntry) with tile_region_flag equal to 1.

[0177] If each sample of a track consists of NAL units of only one rectangular spatial region, a SampleToGroupBox of type "trif" can be used to associate the sample with the rectangular spatial region, but if the default sample grouping mechanism is used (i.e., when the version of the SampleGroupDescriptionBox of type "trif" is equal to or greater than 2), the SampleToGroupBox of type "trif" can be omitted. Otherwise, the sample, NAL unit, and rectangular spatial region are associated with each other through a SampleToGroupBox of type "nalm" with grouping_type_parameter equal to "trif" and a SampleGroupDescriptionBox of type "nalm". The TileRegionGroupEntry is described as follows:

[0178] Rectangular space area,

[0179] Encoded dependencies between this rectangular spatial region and other rectangular spatial regions.

[0180] Each TileRegionGroupEntry is assigned a unique identifier called groupID. This identifier can be used to associate NAL units in a sample with a specific TileRegionGroupEntry.

[0181] Identifies the location and size of a rectangular region of space using luma sample coordinates.

[0182] When used with a movie clip, a TileRegionGroupEntry can be defined for the duration of the movie clip by defining a new SampleGroupDescriptionBox in the track segment box. However, there should not be any TileRegionGroupEntry in the track segment with the same groupID as an already defined TileRegionGroupEntry.

[0183] NAL units mapped to rectangular spatial regions can be carried in the VVC track as usual, or in a separate track called the VVC sub-picture track.

[0184] Similarly, compared to the signaling of the sub-picture ID of the sample group "spid" with or without the NAL unit mapping sample group "nalm", the static signaling in the SampleGroupDescriptionBox can also be used with the tile region sample group "trif" or other similar sample grouping types that provide the characteristics of the rectangular region (position, size, coding properties and / or dependencies with other rectangular regions) to signal a rectangular region that does not change over time.

[0185] In an alternative embodiment, the method of signaling sub-picture IDs associated with NAL units can be further simplified by defining a virtual sample grouping type for sub-pictures and mapping groupID to sub-picture ID plus 1. This will avoid carrying sample group descriptions.

[0186] According to this embodiment, information in the group data structure indicates that the group is a virtual group. Depending on the type of virtual group, attribute values ​​are associated with the elements of the group within the group data structure. Therefore, in addition to the group data structure, the attribute values ​​can be associated without the need for a group description box containing the attributes.

[0187] According to this embodiment, a new "virtual" sample group is defined with a grouping type of "spid" or with another new four-character value (e.g., "spx"). A "virtual" sample group is a sample group that does not have a SampleGroupDescriptionBox and a SampleGroupDescriptionEntry associated with it. Each attribute value set describing a "virtual" sample group can only be stored as a single 32-bit value. For example, the attribute value set of a "virtual" sample group consists of two attribute values, each of which is encoded as a 16-bit value, or as an 8-bit value and a 24-bit value. The attribute value set describing a "virtual" sample group can consist of any number of attribute values, as long as the sum of the attribute value lengths in bits does not exceed 32 bits.

[0188] like Figure 7 As shown, when the SampleToGroupBox 701 with grouping_type of "nalm" contains a grouping_type_parameter equal to a virtual sample group of type "spid" or "spix", the groupID in the NALUMapEntry 702 and 703 in the SampleGroupDescriptionBox with grouping_type of "nalm" directly provides the sub-picture ID value associated with the (one or more) NAL units mapped to the groupID plus one. Figure 7In the example shown, NAL unit 3 (NALU 3) is mapped to the sub-picture with ID value 3, and NAL unit 4 (NALU 4) is mapped to the sub-picture with ID value 5. In this case, this means that there is no need to define a SampleGroupDescriptionBox with grouping_type equal to "spid" or "spix".

[0189] In an alternative, the flag of SampleToGroupBox is used to signal that the grouping_type_parameter contains the grouping_type value of the virtual sample group. The flag values ​​are defined as follows:

[0190] is_virtual_sample_group: Indicates that grouping_type_parameter should be interpreted as the group type of a virtual sample group. The flag value is 0x000001. When this flag value is set, there should not be a SampleGroupDescriptionBox with grouping_type equal to grouping_type_parameter. The semantics of the groupID in the NALUMapEntry associated with the sample in this SampleToGroupBox is defined by the semantics of the virtual sample group identified by grouping_type_parameter.

[0191] According to the present invention, signaling virtual sample groups can be advantageously combined with signaling static sample groups. For example, if a SampleGroupDescriptionBox with grouping_type "nalm" is signaled as static by setting the flag value sample_group_static of the box, the parser is informed that the list of NAL unit groups within the sample does not change over time, that is, the pattern of NAL unit groups is the same for all samples of the track, and each NAL unit in the pattern is directly associated with a given attribute value of the virtual sample group in the groupID parameter of the NALUMapEntry in the SampleGroupDescriptionBox with grouping_type "nalm".

[0192] According to the present invention, when there is no Figure 6The concept of "virtual" sample groups can also be useful when the NAL unit mapping is shown. Similarly, when the set of attribute values associated with a sample group by the sample grouping (corresponding to an entry) can be encoded on a single value of the entire set of attribute values in the representation entry, and this single value does not exceed 32 bits, the SampleGroupDescriptionBox associated with the SampleToGroupBox having the same grouping_type can not be declared. In this case, only the SampleToGroupBox having the grouping type corresponding to the "virtual" sample group (e.g., grouping_type "spix" in Figure 6 ). The group_description_index directly provides the 32-bit value of the set of attribute values associated with the sample. For example, in Figure 6 , the group_description_index having the value 2 in the first entry indicates that the sample associated with this entry has a subpicture ID equal to 2. In an alternative, a flag of the SampleToGroupBox is used to signal that the grouping_type_parameter contains the grouping_type value of a virtual sample group, or the group_description_index contains the attribute values associated with this sample group. The flag values are defined as follows:

[0193] is_virtual_sample_group: indicates whether the SampleToGroupBox refers to a virtual sample group. The flag value is 0x000001:

[0194] When this flag is set and grouping_type is equal to "nalm" (i.e., this is a NAL unit mapping), the grouping_type_parameter shall be interpreted as the group type of the virtual sample group. There shall be no SampleGroupDescriptionBox with grouping_type equal to the grouping_type_parameter. The semantics of the groupID in the NALUMapEntry associated with the sample in this SampleToGroupBox are defined by the semantics of the virtual sample group identified by the grouping_type_parameter.

[0195] When this flag is set and grouping_type is not equal to "nalm" (ie this is not a NAL unit map), the semantics of group_description_index is defined by the semantics of the virtual sample group identified by grouping_type. There shall be no SampleGroupDescriptionBox with grouping_type equal to that of SampleToGroupBox.

[0196] According to another aspect, a NAL unit mapping mechanism is used in the VVC file format to describe rectangular spatial regions carried in the VVC bitstream.

[0197] A rectangular spatial region in VVC is composed of any of the following:

[0198] - A sequence of one or more VVC sub-pictures consecutive in decoding order.

[0199] - A sequence of one or more complete slices forming a rectangular extent and contiguous in decoding order.

[0200] The rectangular spatial regions cover the rectangle without holes. The rectangular spatial regions within the image do not overlap with each other.

[0201] A rectangular spatial region can be described by a rectangular spatial region visual sample group description entry (i.e., an instance of VvcRectSpatialRegionGroupEntry). This descriptor does not allow the indication of whether the rectangular spatial region is composed of a slice or a sub-picture. In the case where the rectangular spatial region is composed of sub-pictures, the descriptor does not allow the indication of the sub-picture identifiers associated with the sub-pictures that constitute the rectangular spatial region.

[0202] According to a third aspect, it is proposed to improve the descriptor for describing rectangular spatial regions by introducing a "has_subpictures" parameter indicating whether the rectangular spatial region is composed of slices or sub-pictures. In an advantageous embodiment, if the "has_subpictures" parameter indicates that the rectangular spatial region is composed of sub-pictures, a list of sub-picture identifiers of the sub-pictures that constitute the rectangular spatial region is provided to the descriptor.

[0203] If each sample of a track consists of NAL units of only one rectangular spatial region, a SampleToGroupBox of type "vrsr" may be used to associate the sample with the rectangular spatial region, but if the default sample grouping mechanism is used (i.e., when the version of the SampleGroupDescriptionBox of type "vrsr" is equal to or greater than 2), the SampleToGroupBox of type "vrsr" may be omitted. Otherwise, samples, NAL units, and rectangular spatial regions are associated with each other through SampleToGroupBoxes of type "nalm" with grouping_type_parameter equal to "vrsr" and SampleGroupDescriptionBox of type "nalm". VvcRectSpatialRegionGroupEntry describes:

[0204] Rectangular space area,

[0205] The encoding dependencies between this rectangular space region and other rectangular space regions, and

[0206] When VVC sub-pictures are present, a list of one or more sub-picture ID values.

[0207] Each VvcRectSpatialRegionGroupEntry is assigned a unique identifier called groupID. This identifier can be used to associate NAL units in a sample with a specific VvcRectSpatialRegionGroupEntry.

[0208] The position and size of a rectangular spatial region are identified using the coordinates of the brightness samples.

[0209] When used with animation clips, a VvcRectSpatialRegionGroupEntry may be defined for the duration of the animation clip by defining a new SampleGroupDescriptionBox in the TrackFragment box as defined in clause 8.9.4 of ISO / IEC 14496-12. However, there shall not be any VvcRectSpatialRegionGroupEntry in the TrackFragment with the same groupID as an already defined VvcRectSpatialRegionGroupEntry.

[0210] NAL units mapped to rectangular spatial regions can be carried in the VVC track as usual, or in a separate track called the VVC sub-picture track.

[0211] The rectangular spatial region sample group is defined as follows:

[0212] Rectangular spatial region sample groups can exist in tracks containing VVC sub-pictures or slice groups that form a rectangular region and are contiguous in decoding order. Each sample group description entry indicates the rectangular spatial region and the coding dependencies between it and other rectangular spatial regions. If sub-pictures are present, it also indicates a list of one or more sub-picture ID values ​​in decoding order.

[0213]

[0214] Where groupID is a unique identifier for the rectangular spatial region group described by the sample group entry. The value of groupID in the rectangular spatial region group entry should be greater than 0. The value 0 is reserved for special purposes.

[0215] When there is a SampleToGroupBox of type "nalm" with grouping_type_parameter equal to "vrsr", a SampleGroupDescriptionBox of type "vrsr" shall be present and the following apply:

[0216] - The value of groupID in the spatial region group entry shall be equal to the groupID in one of the NALUMapEntry entries.

[0217] - A NAL unit mapped to groupID 0 by NALUMapEntry means that this NAL unit is needed to decode any rectangular spatial region in the same coded picture as this NAL unit.

[0218] There may be multiple rectangular spatial region group entries having the same values ​​for horizontal_offset, vertical_offset, region_width, and region_height, respectively, but having different groupID values ​​to describe different dependencies.

[0219] When a multi-layer bitstream is carried in one or more tracks, for any two layers layerA and layerB of the bitstream, the following constraint applies: when the NAL unit of layerA is associated with the groupID value gIdA and the NAL unit of layerB is associated with the groupID value gIdB, gIdA shall not be equal to gIdB.

[0220] has_subpictures equal to 0 specifies that the area within the picture covered by the NAL unit and associated with this rectangular spatial region group entry is an area consisting of one or more complete slices forming a rectangular area and contiguous in decoding order as specified in ISO / IEC 23090-3. A value of 1 specifies that the area within the picture covered by the NAL unit and associated with this rectangular spatial region group entry is an area consisting of one or more complete sub-pictures contiguous in decoding order as specified by ISO / IEC 23090-3.

[0221] independent_idc specifies the coding dependencies between each rectangular spatial region associated with this rectangular spatial region group entry and other rectangular spatial regions in the same picture or in reference pictures of the same layer. Inter-layer dependencies, if any, are indicated by dependencyRectSpatialRegionGroupID list (when has_dependency_list is equal to 1).

[0222] This field takes the following values:

[0223] If independent_idc is equal to 0, the coding dependencies between this rectangular spatial region and other rectangular spatial regions in the same picture or in reference pictures of the same layer are described by the dependencySpatialRegionGroupID list (when has_dependency_list is equal to 1) or are unknown (when has_dependency_list is equal to 0).

[0224] -If independent_idc is equal to 1, there is no temporal dependency between the rectangular spatial region and any rectangular spatial region with a different groupID in any reference picture of the same layer, but there can be a coding dependency between the rectangular spatial region and the rectangular spatial region with the same groupID in the reference picture of the same layer.

[0225] If independent_idc is equal to 2, there is no coding dependency between this rectangular spatial region and any rectangular spatial region in the reference pictures of the same layer.

[0226] - Value 3 is reserved.

[0227] full_picture, when set, indicates that each rectangular spatial region associated with this rectangular spatial region group entry is a full picture, in which case region_width and region_height should be set to the width and height of the full picture respectively, and independent_idc should be set to 1 or 2.

[0228] filtering_disabled, when set, indicates that for each rectangular spatial region associated with the rectangular spatial region group entry, the in-loop filtering operation does not need to access pixels adjacent to the rectangular spatial region, that is, the rectangular spatial region can be bit-accurately reconstructed without decoding the adjacent rectangular spatial regions.

[0229] has_dependency_list, when it is set to 1, indicates dependency_region_count, and when dependency_region_count is greater than 0, there is a list of dependencyRectSpatialRegionGroupID. When it is set to 0, there is no dependency_region_count, and there is no dependencyRectSpatialRegionGroupID.

[0230] has_subpictures, when it is set to 1, it indicates that the rectangular spatial region is composed of VVC subpictures and there is a list of subpicture IDs subpic_id. When it is set to 0, there is no list of subpicture IDs subpic_id.

[0231] horizontal_offset and vertical_offset give the horizontal offset and vertical offset, respectively, of the upper left pixel of the rectangular area covered by the slice in each rectangular area associated with the rectangular spatial region group entry in luma samples relative to the upper left pixel of the base region. For VVC tracks and VVC sub-picture tracks as defined in this part of ISO / IEC 14496-15, the base region used in VvcRectSpatialRegionGroupEntry is the picture to which the slice in the area associated with the rectangular spatial region group entry belongs.

[0232] region_width and region_height give the width and height, respectively, of the rectangular region covered by the strip in each rectangular region associated with this rectangular spatial region group entry in luma samples.

[0233] For VVC streams using spatial scalability and rectangular spatial regions on both the base layer and enhancement layers, when each layer is carried in its own track, the sample description VvcRectSpatialRegionGroupEntry of the base layer will give the coordinates expressed in luma samples of the base layer, while the sample description VvcRectSpatialRegionGroupEntry of the enhancement layer will give the coordinates expressed in luma samples of the enhancement layer.

[0234] num_subpics_minus1 plus 1 specifies the number of VVC sub-pictures contained in the sample region or sample mapped to the sample group description entry.

[0235] The subpic_id of loop counter i specifies the i-th sub-picture identifier in decoding order contained in the sample region or sample mapped to this sample group description entry.

[0236] dependency_region_count indicates the number of rectangular spatial regions on which each rectangular spatial region associated with this rectangular spatial region group entry depends.

[0237] dependencyRectSpatialRegionGroupID gives the groupID of the rectangular spatial region (as defined by VvcRectSpatialRegionGroupEntry) that this rectangular spatial region depends on. For a specific rectangular region associated with this rectangular spatial region group entry, the rectangular spatial region it depends on can be from the same layer or the same reference layer.

[0238] Figure 8 FIG1 is a schematic block diagram of a computing device 800 for implementing one or more embodiments of the present invention. The computing device 800 may be a device such as a microcomputer, a workstation, or a lightweight portable device. The computing device 800 includes a communication bus connected to:

[0239] - Central processing unit (CPU) 801, such as a microprocessor;

[0240] A random access memory (RAM) 802 for storing executable code of the method of an embodiment of the present invention and registers suitable for recording variables and parameters necessary for implementing the method for reading and writing manifests and / or encoding video and / or reading or generating data in a given file format. The memory capacity of the random access memory can be expanded by, for example, an optional RAM connected to an expansion port;

[0241] - a read-only memory (ROM) 803 for storing computer programs for implementing embodiments of the present invention;

[0242] - A network interface 804, which in turn is typically connected to a communications network through which digital data to be processed is sent or received. The network interface 804 can be a single network interface, or comprised of a set of different network interfaces (e.g., wired and wireless interfaces, or different types of wired or wireless interfaces). Data is written to the network interface for transmission or read from the network interface for reception under the control of a software application running in the CPU 801;

[0243] - User interface (UI) 805, for receiving input from the user or displaying information to the user;

[0244] - Hard disk (HD) 806;

[0245] - I / O module 807, used to receive / send data from / to external devices (such as a video source or a display, etc.).

[0246] The executable code may be stored in the read-only memory 803, on the hard disk 806, or on a removable digital medium such as a disk. According to a variant, the executable code of the program may be received via the network interface 804 by means of a communication network so that the executable code of the program is stored in one of the storage components of the communication device 800 (such as the hard disk 806) before being executed.

[0247] The central processing unit 801 is adapted to control and direct the execution of instructions or portions of software code of instructions of one or more programs according to embodiments of the present invention, which instructions are stored in one of the aforementioned storage components. After being powered on, the CPU 801 is capable of executing instructions from the main RAM memory 802 after having loaded these instructions from, for example, a program ROM 803 or a hard disk (HD) 806, related to software applications. Such software applications, when executed by the CPU 801, cause the steps of the flowcharts shown in the previous figures to be performed.

[0248] In this embodiment, the device is a programmable device that implements the invention using software. Alternatively, however, the invention may be implemented in hardware (for example in the form of an application specific integrated circuit or ASIC).

[0249] Although the present invention has been described above with reference to a specific embodiment, the present invention is not limited to the specific embodiment, and modifications within the scope of the invention will be apparent to those skilled in the art.

[0250] For example, the present invention can be embedded in a device such as a camera, a smartphone, a head-mounted display, or a tablet used as a remote control for a TV or multimedia display, for example, to zoom in on a specific area of ​​interest. It can also be used from the same device to personalize the viewing experience of a multimedia presentation by selecting a specific area of ​​interest. Another use for users using these devices and methods is to share certain selected sub-portions of their preferred video with other connected devices. If a surveillance camera supports the method for providing data according to the present invention, it can also be used in conjunction with a smartphone or tablet to monitor what is happening in a specific area of ​​a building under surveillance.

[0251] Many further modifications and variations will suggest themselves to those skilled in the art upon reference to the foregoing illustrative embodiments, which are given by way of example only and are not intended to limit the scope of the invention, which is determined solely by the appended claims. In particular, different features from different embodiments may be interchanged where appropriate.

Claims

1. A method of generating a media file from a timed sequence of media data comprising samples, the method comprising: generating a first data structure comprising data describing characteristics of each of one or more sample groups in a track, or each of one or more subsamples in the one or more sample groups in the track, the first data structure comprising a first packet type value, wherein the first data structure comprises data indicating whether another first data structure having the same first packet type value does not exist in a subsequent fragment of the track; and A media file including the sample and the first data structure is generated.

2. The method according to claim 1, wherein The method further comprises: generating a second data structure that assigns subsets of samples to the one or more sample groups and their characteristics, the second data structure including a second grouping type value; and including the second data structure in the generated media file, The second grouping type value is equal to the first grouping type value.

3. The method according to claim 1, wherein The method further comprises: generating a second data structure that assigns subsets of samples to the one or more sample groups; generating another first data structure for describing one or more subsamples in the one or more sample groups and assigning the one or more subsamples in the one or more sample groups to characteristics described in the first data structure, the another first data structure including a third grouping type value different from the first grouping type value; and including the second data structure and the further first data structure including the third packet type value in the generated media file, The second data structure includes a second grouping type value equal to the third grouping type value, and a grouping type parameter value equal to the first grouping type value.

4. The method according to claim 1, wherein: The data indicating whether another first data structure having the same first packet type value does not exist in a subsequent fragment of the track includes one or more flags.

5. The method according to claim 2 or 3, wherein: In the case where the data in the first data structure for indicating whether there is no another first data structure with the same first grouping type value in the subsequent fragment of the track has a first value, the data indicates that there is no another first data structure with the same first grouping type value in the subsequent fragment of the track, and the assignment of the subset of the samples to the characteristics described in the first data structure, or the assignment of the one or more subsamples to the characteristics described in the first data structure may change over time.

6. The method according to claim 5, wherein: In a case where data in the first data structure indicating whether another first data structure having the same first packet type value does not exist in a subsequent segment of the track has a second value, the data indicates that another first data structure having the same first packet type value can exist in a subsequent segment of the track, and a default sample group mapping may change over time.

7. The method according to claim 6, wherein: In a case where the data in the first data structure for indicating whether there is no another first data structure having the same first packet type value in a subsequent fragment of the track has a third value, the data indicates that there is no another first data structure having the same first packet type value in a subsequent fragment of the track, and the first data structure includes a single set of attribute values ​​applicable to all samples or subsamples of the track.

8. The method according to claim 3, wherein: The second grouping type value and the third grouping type value are equal to nalm.

9. The method according to claim 1, wherein: The first data structure includes data for indicating that no sample-to-group box having the same first grouping type value exists in any fragment of the track.

10. A method of parsing a media file encapsulating a timed sequence of media data including samples, the method comprising: parsing a first data structure comprising data describing characteristics of each of one or more sample groups in a track, or each of one or more subsamples in the one or more sample groups in the track, the first data structure comprising a first grouping type value, wherein the first data structure comprises data indicating whether another first data structure having the same first grouping type value does not exist in a subsequent fragment of the track; and The media data is retrieved based on data indicating whether another first data structure having the same first packet type value does not exist in a subsequent fragment of the track.

11. The method according to claim 10, wherein: The first data structure includes data for indicating that no sample-to-group box having the same first grouping type value exists in any fragment of the track.

12. A program product for a programmable device, the program product comprising a sequence of instructions for implementing the method according to any one of claims 1 to 9 when loaded into and executed by the programmable device.

13. A non-transitory computer-readable storage medium storing instructions of a computer program for implementing the method according to any one of claims 1 to 9.

14. An apparatus for generating a media file from a timed sequence of media data comprising samples, the apparatus comprising a processor configured to: generating a first data structure comprising data describing characteristics of each of one or more sample groups in a track, or each of one or more subsamples in the one or more sample groups in the track, wherein The first data structure includes a first packet type value, wherein the first data structure includes data indicating whether another first data structure having the same first packet type value does not exist in a subsequent fragment of the track; and A media file including the sample and the first data structure is generated.

15. An apparatus for parsing a media file encapsulating a timed sequence of media data including samples, the apparatus comprising a processor configured to: Parsing a first data structure including data describing characteristics of each of one or more sample groups in a track or each of one or more subsamples in the one or more sample groups in the track, the first data structure including a first grouping type value, wherein: the first data structure comprising data for indicating whether another first data structure having the same first packet type value does not exist in a subsequent fragment of the track; as well as The media data is retrieved based on data indicating whether another first data structure having the same first packet type value does not exist in a subsequent fragment of the track.

16. An apparatus for generating a media file from a timed sequence of media data including samples, the apparatus comprising: a memory device storing an instruction set; as well as At least one processor that executes the instruction set to perform the following steps: generating a track comprising at least one of the samples; generating a first data structure that assigns subsets of samples or subsamples of the track to one or more sample groups; and generating a media file comprising the track, the sample and the first data structure, The first data structure includes the following data: when the data is set to a first value, it indicates that the group description index value associated with each sample group in the first data structure is an index of one or more attribute value sets in the second data structure; and when the data is set to a second value, it indicates that the group description index value associated with each sample group in the first data structure contains one or more attribute values.

17. An apparatus for parsing a media file encapsulating a timed sequence of media data including samples, the apparatus comprising: a memory device storing an instruction set; as well as At least one processor that executes the instruction set to perform the following steps: Parsing a track of the media file, the track including at least one of the samples; and parsing a first data structure of the media file, the first data structure assigning a subset of samples or subsamples of the track to one or more sample groups, The first data structure includes the following data: when the data is set to a first value, it indicates that the group description index value associated with each sample group in the first data structure is an index of one or more attribute value sets in the second data structure; and when the data is set to a second value, it indicates that the group description index value associated with each sample group in the first data structure contains one or more attribute values.

18. A method of generating a media file from a timed sequence of media data comprising samples, the method comprising: generating a track comprising at least one of the samples; generating a first data structure that assigns subsets of samples or subsamples of the track to one or more sample groups; as well as generating a media file comprising the track, the sample, and the first data structure for streaming via a communications network or for storage on a computing device, The first data structure includes the following data: when the data is set to a first value, it indicates that the group description index value associated with each sample group in the first data structure is an index of one or more attribute value sets in the second data structure; and when the data is set to a second value, it indicates that the group description index value associated with each sample group in the first data structure contains one or more attribute values.

19. A method of parsing a media file comprising a timed sequence of media data of a sample, the method comprising: Parsing a track of the media file, the track including at least one of the samples; as well as parsing a first data structure of the media file, the first data structure assigning a subset of samples or subsamples of the track to one or more sample groups, The first data structure includes the following data: when the data is set to a first value, it indicates that the group description index value associated with each sample group in the first data structure is an index of one or more attribute value sets in the second data structure; and when the data is set to a second value, it indicates that the group description index value associated with each sample group in the first data structure contains one or more attribute values.

Images