Coded multi-source media format (CMMF) instantiation for transport media data

Abstract

An example apparatus for retrieving media data includes a memory configured to store media data; and a processing system comprising a decoder implemented in circuitry, the processing system configured to: retrieve a manifest file indicative of a plurality of network locations for at least a first section of media data and a second section of media data, each of the plurality of network locations hosted by a separate physical server device; retrieving at least a first portion of the first section from a first one of the network locations; retrieving at least a second portion of the second section from a second one of the network locations; and providing at least a first portion of the first segment and at least a second portion of the second segment to a decoder.

Description

[0001] This application claims priority to U.S. Patent Application No. 18 / 796,817, filed August 7, 2024, and U.S. Provisional Application No. 63 / 518,302, filed August 8, 2023, the entire contents of each of which are incorporated herein by reference. U.S. Patent Application No. 18 / 796,817, filed August 7, 2024, claims the benefit of U.S. Provisional Application No. 63 / 518,302, filed August 8, 2023. Technical Field

[0002] This disclosure relates to the transmission of media data. Background Technology

[0003] Digital video capabilities can be incorporated into a wide range of devices, including digital televisions, digital direct broadcasting systems, wireless broadcasting systems, personal digital assistants (PDAs), laptops or desktop computers, digital cameras, digital recording devices, digital media players, video game devices, video game consoles, cellular or satellite radio phones, video conferencing equipment, and more. Digital video devices implement video compression technologies (such as those described in standards defined by MPEG-2, MPEG-4, ITU-T H.263 or ITU-T H.264 / MPEG-4, Part 10, Advanced Video Decoding (AVC), ITU-T H.265 (also known as High Efficiency Video Decoding (HEVC)), and extensions to these standards) to transmit and receive digital video information more efficiently.

[0004] Video compression techniques perform spatial and / or temporal prediction to reduce or remove inherent redundancy in video sequences. For block-based video decoding, video frames or slices can be divided into blocks. Each block can be further subdivided. Intra-decoded (I) blocks in frames or slices are encoded using spatial prediction relative to adjacent blocks. Inter-decoded (P or B) blocks in frames or slices can use spatial prediction relative to adjacent blocks in the same frame or slice, or temporal prediction relative to other reference frames.

[0005] After video data is encoded, it can be grouped for transmission or storage. Video data can be assembled into video files that conform to any of various standards, such as the International Organization for Standardization (ISO) Basic Media File Format and its extensions, such as AVC. Summary of the Invention

[0006] Generally, this disclosure describes techniques for streaming media data. The media data may be encapsulated according to a decoded multi-source media format (CMMF). The CMMF may include techniques for forward error correction (FEC) encoding of the media data, such that a repair object is provided in addition to a basic media object (also called an application object). The application object and the repair object may be stored separately, for example, on different physical server devices. Therefore, a client device may receive an application object from one server and one or more repair objects for that application object from one or more other servers. In this way, the techniques of this disclosure can support multi-source media streaming.

[0007] In one example, a method for retrieving media data includes: retrieving a manifest file indicating a plurality of network locations for at least a first segment and a second segment of the media data, each of the plurality of network locations being hosted by a separate physical server device; retrieving at least a first portion of the first segment from the first network location of the plurality of network locations; retrieving at least a second portion of the second segment from the second network location of the plurality of network locations; and providing the at least first portion of the first segment and the at least second portion of the second segment to a decoder.

[0008] In another example, an apparatus for retrieving media data includes: a memory configured to store media data; and a processing system including a decoder implemented in circuitry, the processing system being configured to: retrieve a manifest file indicating a plurality of network locations for at least a first segment and a second segment of the media data, each of the plurality of network locations being hosted by a separate physical server device; retrieve at least a first portion of the first segment from the first network location of the plurality of network locations; retrieve at least a second portion of the second segment from the second network location of the plurality of network locations; and provide the at least first portion of the first segment and the at least second portion of the second segment to the decoder.

[0009] In another example, a method for transmitting media data includes: performing forward error correction (FEC) encoding on at least a first portion of a first segment to form a first FEC-encoded first portion of the first segment, and performing forward error correction (FEC) encoding on at least a second portion of a second segment to form a second FEC-encoded second portion of the second segment; storing the first FEC-encoded first portion in a first physical server device; storing the second FEC-encoded second portion in a second physical server device; and forming a manifest file indicating that the first FEC-encoded first portion is available from the first physical server device and that the second FEC-encoded second portion is available from the second physical server device.

[0010] In another example, an apparatus for transmitting media data includes: a memory configured to store media data; and a processing system implemented in circuitry and configured to: perform forward error correction (FEC) encoding on at least a first portion of a first segment to form a first FEC-encoded first portion of the first segment, and perform forward error correction (FEC) encoding on at least a second portion of a second segment to form a second FEC-encoded second portion of the second segment; store the first FEC-encoded first portion in a first physical server device; store the second FEC-encoded second portion in a second physical server device; and generate a manifest file indicating that the first FEC-encoded first portion is available from the first physical server device and that the second FEC-encoded second portion is available from the second physical server device.

[0011] Details of one or more examples are set forth in the accompanying drawings and the following description. Other features, objects, and advantages will be apparent from these descriptions and drawings, and from the claims. Attached Figure Description

[0012] Figure 1 This is a block diagram illustrating an example system for implementing a technology for streaming media data over a network.

[0013] Figure 2 A more detailed example Figure 1 A block diagram of an example set of components for the retrieval unit.

[0014] Figure 3 This is a conceptual diagram illustrating the elements of example multimedia content.

[0015] Figure 4 This is a block diagram illustrating the elements of a sample video file, which may correspond to the represented segments, such as... Figure 3 One segment of the segment.

[0016] Figure 5 This is a conceptual diagram illustrating a delivery session model for decoded multi-source media format (CMMF).

[0017] Figure 6 This is a conceptual diagram illustrating an example process for generating a transport object.

[0018] Figure 7 It is a conceptual diagram depicting the relationship between the source object and the object being repaired.

[0019] Figure 8 This is a conceptual diagram illustrating the super object generation technique.

[0020] Figure 9This is a conceptual diagram illustrating another example technique used for superobject generation.

[0021] Figure 10 This is a conceptual diagram illustrating an example CMMF delivery session that includes a collection of source and repair objects organized in different transport sessions.

[0022] Figure 11 This is a conceptual diagram illustrating the CMMF reference architecture.

[0023] Figure 12 This is a conceptual diagram illustrating the deployment option of a CMMF receiver being able to access encoded transmission objects only via CMMF-R based on information in CMMF-ETI.

[0024] Figure 13 This is a conceptual diagram illustrating the deployment options for creating repair objects, which includes several processes in a CMMF transmitter.

[0025] Figure 14 This is a conceptual diagram illustrating the CMMF encoding process.

[0026] Figure 15 This is a conceptual diagram illustrating an example of the CMMF decoding process.

[0027] Figure 16 This is a conceptual diagram illustrating an example of a CMMF receiver and decoder for a single application object.

[0028] Figure 17 This is a conceptual diagram illustrating an example CMMF architecture.

[0029] Figure 18 This is a conceptual diagram illustrating another example of a CMMF architecture.

[0030] Figure 19 This is a conceptual diagram illustrating another CMMF architecture.

[0031] Figure 20 This is a conceptual diagram illustrating an example mapping between Transport Object Identifier (TOI) and Transport Session Identifier (TSI) and a URL or source data in a file delivery manifest.

[0032] Figure 21 This is a flowchart illustrating an example of static operation of the technology according to this disclosure.

[0033] Figure 22 This is a flowchart illustrating an example of dynamic operation according to the technology disclosed herein.

[0034] Figure 23 This is a conceptual diagram illustrating an example FLUTE transmitting unit.

[0035] Figure 24This is a conceptual diagram illustrating a ROUTE system that includes an example ROUTE receiving unit and a ROUTE transmitter.

[0036] Figure 25 This is a conceptual diagram illustrating an example of a FECFRAME protocol stack.

[0037] Figures 26A to 26C This is a conceptual diagram illustrating various example FEC groupings.

[0038] Figures 27A to 27C This is a conceptual diagram illustrating various example Raptor / Raptor Q code formats for FECFRAME.

[0039] Figure 28 This is a block diagram illustrating an example set of devices capable of performing the techniques of this disclosure.

[0040] Figure 29 This is a flowchart illustrating an example method for transmitting media data according to the technology of this disclosure.

[0041] Figure 30 This is a flowchart illustrating an example method for retrieving media data according to the technology of this disclosure. Detailed Implementation

[0042] Generally speaking, this disclosure describes techniques related to the transmission of media data encapsulated according to a decoded multi-source media format (CMMF). CMMF is an extensible container format designed to facilitate the management and exchange of audiovisual media and metadata in one or more decoded representations (encoded using forward error correction (FEC) codes). Multiple FEC codes, such as xCD-1, Raptor, RaptorQ, and / or Reed-Solomon, can be supported.

[0043] CMMF allows source data to be encoded and packaged without any modification and provides redundant objects that can be used in conjunction with the source data to recover the source data at the receiver, even though the receiver only receives a portion of the original source data and a portion of the redundant data. This disclosure describes techniques that can be employed within a general framework for generating repair objects based on the existing FEC framework and provides specifications for the repair object format and the external transport information required by the receiver to recover the source object.

[0044] CMMF enables efficient use of multi-source, multi-path, and / or multi-access connectivity for network- or cloud-driven processing and playback applications. CMMF supports signaling and encapsulation of metadata such as decoding type, media information, integrity information, encoder universally unique identifier (UUID), or time.

[0045] Media bitstreams containing media data encapsulated according to CMMF can serve as a standard encapsulation format, enabling a single container format to be used for a wide variety of content and use cases. For example, this container format can be used to encapsulate still image (picture) data, video data, audio data, and / or extended reality (XR) data, such as extended reality (AR), mixed reality (MR), and / or virtual reality (VR) data.

[0046] CMMF provides a universal container format that supports multimedia delivery (e.g., video and audio streaming, broadcasting, XR, video conferencing, and online gaming) by decoding the underlying content. The format supports multiple codecs (including xCD-1, RaptorQ, and Reed-Solomon) and is optimized for a range of networks and use cases. CMMF enables efficient decentralized multi-source and multi-path content delivery, suitable for use cases such as audio and video streaming that require high availability / robustness but also have strict latency and bandwidth constraints. CMMF is designed to work with existing and future streaming services (e.g., HTTP Real-Time Streaming (HLS), MPEG-DASH (HTTP Dynamic Adaptive Streaming), CMAF, etc.) and network protocols (e.g., HTTP, TCP, UDP, WebRTC, etc.), providing a flexible and scalable framework for managing the delivery of encoded multimedia content.

[0047] Standardized container formats (rather than decoding types) enable industry collaboration by creating a common exchange format for distributing and delivering encoded content. This allows service providers (such as mobile network operators or media platforms) to distribute media in encoded formats that can be interpreted and decoded by their partners.

[0048] CMMF can be used to support multipath audio / video streaming, ultra-low latency XR gaming / teleconferencing, or other such use cases. CMMF streaming can be provided via TCP, UDP, or WebRTC and other network protocols.

[0049] CMMF can be used with delivery frameworks such as: File Delivery over Unidirectional Transport (FLUTE), described in "FLUTE—File Delivery over Unidirectional Transport" by T. Paila et al., Network Working Group, RFC 6726, November 2012, available at www.rfc-editor.org / rfc / rfc6726; Real-Time Transport Object Delivery over Unidirectional Transport (ROUTE), described in "Real-Time Transport Object Delivery over Unidirectional Transport (ROUTE)" by Zia et al., RFC 9223, April 2022, available at www.rfc-editor.org / rfc / rfc9223; and / or FECFRAME, described in "Raptor Forward Error Correction (FEC) Schemes for FECFRAME" by Watson et al., Internet Engineering Task Force (IETF), RFC 6726. RFC 6681, August 2012, is available at www.rfc-editor.org / rfc / rfc6681. Network transmission of media data encapsulated in CMMF can be performed using one or more service layers, such as Multimedia Broadcast Multicast Service (MBMS) or eMBMS service layer, group communication, 5G broadcast, ATSC service layer, DVB-I service layer, or HTTP Dynamic Adaptive Streaming (DASH). These technologies are developed based on extensive research. They have undergone rigorous testing and standardization by appropriate standards organizations such as IETF, 3GPP, ETSI, MPEG, DVB, ATSC, and DASH-IF. For many of these technologies, both open-source and proprietary implementations exist.

[0050] In the context of CMMF, the terminology related to media decoding, encapsulation, and transmission includes the following:

[0051] • Bit stream: a sequence of bits.

[0052] • Block: A data unit that uses block-based codes.

[0053] Generally speaking, decoding operations (encoding and decoding) on ​​one block are independent of all decoding operations on another block.

[0054] • Block decoding: A decoding technique that first divides the input into a series of blocks or chunks; then, encoding and decoding are performed independently on a block-by-block basis.

[0055] • Block size: The number of source symbols belonging to a block.

[0056] • Byte: 8 bits

[0057] • Channel: A general term for any type of communication technology.

[0058] Example: An Ethernet link, a WiFi network, or a complete path between two nodes within a network.

[0059] • Code rate: Code rate is the ratio between the number of source symbols and the number of source symbols to be decoded or repaired.

[0060] ○ The code rate can be greater than zero, less than or equal to one. In CMMF, a code rate close to 1 indicates that a small number of decoding or repair symbols are generated during the encoding process, while a code rate close to 0 indicates that a large number of decoding or repair symbols are generated.

[0061] • Decoded symbol, encoded symbol, or repaired symbol: a data unit that is the result of the decoding operation.

[0062] • Decoding or encoding: An operation that takes a source symbol as input and produces a decoded symbol as output.

[0063] • Decoding coefficients: Coefficients selected from the same finite field encoding and decoding operations are performed. Methods for selecting the coefficients may include: randomly (e.g., LT codes), from a predefined table (e.g., Reed-Solomon, etc.), or using a predefined algorithm with a seed (e.g., LDPC, RaptorQ, etc.).

[0064] • Decoding matrix, generator matrix, or coefficient matrix: A matrix (G) that transforms the input symbol set (X) into a decoded or repaired symbol set (Y): Y = X•G. The generator matrix is ​​typically defined using block codes. The input symbol set X can only consist of source symbols.

[0065] • Decoding vector or coefficient vector: A set of decoding coefficients used to generate specific decoded or repaired symbols through linear decoding.

[0066] The density is defined by the number of non-zero coefficients in the decoding vector.

[0067] • Decoding: The operation of taking the decoded symbol as input and producing the source symbol as output.

[0068] • Encoding: The operation of taking source symbols as input and producing decoded symbols as output.

[0069] • Encoding block: See Block.

[0070] • Encoding symbols: See decoding symbols.

[0071] • Encoding window or decoding window: The set of source symbols used as input for decoding operations.

[0072] ○The symbol set may change over time as the decoding window slides through the input stream.

[0073] • Encoding window size or decoding window size: The number of source symbols in the current encoding window.

[0074] ○ This size may change over time.

[0075] • Erasure: Information along the communication path is discarded or lost.

[0076] • Erasure channel: A communication path that discards or receives information without any errors.

[0077] • Finite field, Galois field, or decoded field: The finite field used in linear codes has the desired properties that all elements (except zero) are invertible with respect to the + and × operators, and that no operation on any element results in an overflow or underflow.

[0078] • Finite field size or decoded field size: The number of elements in a finite field.

[0079] Example: A binary extended field {0..2m 1} can have a size of q=2m.

[0080] • Flow or Stream: A flow (or grouping) of information logically grouped.

[0081] • Input or source symbol: A data unit that serves as input to an encoding operation or as output to a decoding operation.

[0082] • Linear decoding: The process of generating a linear combination of source symbols using a given set of coefficients and producing decoded or repaired symbols.

[0083] • Multipath decoding: Decoding that allows transmission via multiple routes, such routes having multiple (at least partially) disjoint paths from the source to each given destination.

[0084] • Multi-source decoding: Allows decoding from multiple sources to each given destination via (at least partially) disjoint paths.

[0085] • Network: A set of interconnected nodes that communicate through a collection of links or channels.

[0086] • Node: A connection point in a communication network

[0087] • Output symbol: See decoding symbol

[0088] • Packet: A data unit transmitted over a network

[0089] • Rank or coded rank: The number of coded symbols of a linearly independent linear system, or equivalently, the number of linearly independent equations in a linear system.

[0090] • Repair stream: A stream containing FEC-encoded repair packets.

[0091] • Single-path decoding: Decoding is performed using routes that have a single path from the source to each destination. In the case of multicast or broadcast traffic, this route is a tree.

[0092] • Sliding window decoding: A decoding technique that dynamically generates or repairs symbols based on the source symbol set present in the sliding coding window at that time. The sliding window can be of a fixed size (fixed sliding window) or can be of a size that can change over time (flexible sliding window).

[0093] • Slide window size, encoding window size, or decoding window size: The number of symbols in the current window. This size may change over time.

[0094] • Source decoding: The process of removing redundant and / or (perceptually) irrelevant information from the source of information, i.e., compression of data or media (audio, video).

[0095] • Source data, source file, or raw data: Data units that can be divided into blocks, where each block is the input for an encoding operation.

[0096] • Source stream: The source information stream from which the decoding will be applied, possibly along with other source streams.

[0097] • Source node: The node that generates one or more source streams.

[0098] • Source symbols, information symbols, and system symbols: Data units derived from the source, used as input for encoding operations.

[0099] • Symbol: A data unit manipulated during encoding and decoding operations.

[0100] • Symbol size: The size of each symbol used in encoding and decoding operations.

[0101] • System decoding: Decoding techniques where the source symbol is part of the output stream generated by the encoder.

[0102] The technology disclosed herein can be applied to video files that conform to video data encapsulated according to any of the following file formats: ISO Basic Media File Format, Scalable Video Decoding (SVC) File Format, Advanced Video Decoding (AVC) File Format, 3rd Generation Partnership Project (3GPP) File Format, and / or Multi-View Video Decoding (MVC) File Format or other similar video file formats.

[0103] In HTTP streaming such as HTTP Dynamic Adaptive Streaming (DASH), frequently used operations include HEAD, GET, and partial GET. The HEAD operation retrieves the header of a file associated with a given Uniform Resource Locator (URL) or Uniform Resource Name (URN), but not the payload associated with the URL or URN. The GET operation retrieves the entire file associated with a given URL or URN. The partial GET operation takes a range of bytes as input and retrieves a consecutive number of bytes from the file, where the number of bytes corresponds to the received range. Therefore, movie clips can be provided for HTTP streaming because the partial GET operation can retrieve one or more individual movie clips. Within a movie clip, there can be several track clips from different tracks. In HTTP streaming, media presentation can be a structured collection of data accessible to the client. Clients can request and download media data information to provide streaming services to users.

[0104] In examples of using HTTP streaming to stream 3GPP data, there may be multiple representations of video and / or audio data for multimedia content. As will be explained below, different representations may correspond to different decoding characteristics (e.g., different video decoding standard profiles or hierarchies), different decoding standards or extensions to those standards (such as multi-view and / or scalable extensions), or different bit rates. A list of such representations can be defined in a Media Presentation Description (MPD) data structure. A media presentation may correspond to a structured set of data accessible to the HTTP streaming client device. The HTTP streaming client device can request and download media data information to provide streaming services to its users. The media presentation can be described in an MPD data structure (which may include updates to the MPD).

[0105] A media presentation may consist of a sequence of one or more cycles. Each cycle may extend until the beginning of the next cycle, or, in the case of the last cycle, until the end of the media presentation. Each cycle may contain one or more representations of the same media content. A representation may be one of several alternative encoded versions of audio, video, timed text, or other such data. Representations may vary depending on the encoding type (e.g., bit rate, resolution, and / or codecs for video data and bit rate, language, and / or codecs for audio data). The term "representation" can be used to refer to a portion of encoded audio or video data that corresponds to a specific cycle of multimedia content and is encoded in a particular manner.

[0106] A representation for a specific period can be assigned to a group indicated by an attribute in the MPD (which indicates the adapter set to which the representation belongs). Representations within the same adapter set are generally considered interchangeable, as client devices can dynamically and seamlessly switch between these representations, for example, to perform bandwidth adaptation. For instance, each representation of video data for a specific period can be assigned to the same adapter set, allowing any representation in the set to be selected for decoding to render media data (such as video or audio data) for the multimedia content of the corresponding period. The media content within a period can be represented by one representation from group 0 (if present), or in some examples, by a combination of at most one representation from each non-zero group. Timing data for each representation of a period can be expressed relative to the start time of the period.

[0107] A representation may include one or more sections. Each representation may include an initialization section, or each section of a representation may be self-initialized. When an initialization section exists, it may contain initialization information for accessing the representation. Generally, the initialization section does not contain media data. Sections can be uniquely referenced using identifiers such as Uniform Resource Locators (URLs), Uniform Resource Names (URNs), or Uniform Resource Identifiers (URIs). The MPD can provide an identifier for each section. In some examples, the MPD may also provide byte ranges as range attributes, which can correspond to the data within a section of a file that can be accessed by a URL, URN, or URI.

[0108] Different representations can be selected to retrieve different types of media data substantially simultaneously. For example, a client device can choose to retrieve audio representations, video representations, and timed text representations of segments. In some examples, the client device can select a specific set of adapters to perform bandwidth adaptation. That is, the client device can select an adapter set that includes video representations, an adapter set that includes audio representations, and / or an adapter set that includes timed text. Alternatively, the client device can select an adapter set for certain types of media (e.g., video) and directly select representations for other types of media (e.g., audio and / or timed text).

[0109] Figure 1 This is a block diagram illustrating an example system 10 for implementing techniques for streaming media data over a network. In this example, system 10 includes a content preparation device 20, a server device 60, and a client device 40. Client device 40 and server device 60 are communicatively connected via a network 74, which may include the Internet. In some examples, content preparation device 20 and server device 60 may also be connected via network 74 or another network, or may be directly communicatively connected. In some examples, content preparation device 20 and server device 60 may include the same device.

[0110] exist Figure 1 In the example, content preparation device 20 includes an audio source 22 and a video source 24. Audio source 22 may include, for example, a microphone that generates electrical signals representing captured audio data to be encoded by audio encoder 26. Alternatively, audio source 22 may include: a storage medium storing previously recorded audio data; an audio data generator, such as a computerized synthesizer; or any other audio data source. Video source 24 may include: a video camera that generates video data to be encoded by video encoder 28; a storage medium encoding previously recorded video data; a video data generation unit, such as a computer graphics source; or any other video data source. Content preparation device 20 is not necessarily communicatively connected to server device 60 in all examples, but multimedia content may be stored on a separate medium that is read by server device 60.

[0111] The raw audio and video data may include analog or digital data. Analog data may be digitized before being encoded by audio encoder 26 and / or video encoder 28. While a speaker is speaking, audio source 22 may obtain audio data from that speaker, and video source 24 may simultaneously obtain video data of that speaker. In other examples, audio source 22 may include a computer-readable storage medium containing stored audio data, and video source 24 may include a computer-readable storage medium containing stored video data. Thus, the techniques described in this disclosure can be applied to live, streaming, real-time audio and video data, or to archived, pre-recorded audio and video data.

[0112] An audio frame corresponding to a video frame is typically an audio frame containing audio data, which is simultaneously captured (or generated) by audio source 22 and video data captured (or generated) by video source 24 and contained within the video frame. For example, when a speaker typically generates audio data by speaking, audio source 22 captures the audio data, and video source 24 simultaneously (i.e., while audio source 22 is capturing audio data) captures the speaker's video data. Therefore, an audio frame can temporally correspond to one or more specific video frames. Thus, an audio frame corresponding to a video frame typically corresponds to the following situation: in which audio data and video data are captured simultaneously, and in this situation, the audio frame and video frame respectively include the simultaneously captured audio data and video data.

[0113] In some examples, audio encoder 26 may encode a timestamp into each encoded audio frame, where the timestamp indicates the time when the audio data for the encoded audio frame was recorded, and similarly, video encoder 28 may encode a timestamp into each encoded video frame, where the timestamp indicates the time when the video data for the encoded video frame was recorded. In such examples, the audio frame corresponding to the video frame may include: an audio frame including a timestamp, and a video frame including the same timestamp. Content preparation device 20 may include an internal clock, which audio encoder 26 and / or video encoder 28 may use to generate timestamps, or audio source 22 and video source 24 may use the internal clock to associate audio and video data with timestamps, respectively.

[0114] In some examples, audio source 22 may transmit data corresponding to the time when the audio data was recorded to audio encoder 26, and video source 24 may transmit data corresponding to the time when the video data was recorded to video encoder 28. In some examples, audio encoder 26 may encode sequence identifiers into the encoded audio data to indicate the relative time order of the encoded audio data, rather than the absolute time when the audio data was recorded; similarly, video encoder 28 may use sequence identifiers to indicate the relative time order of the encoded video data. Similarly, in some examples, sequence identifiers may be mapped or otherwise associated with timestamps.

[0115] Audio encoder 26 typically produces encoded audio data streams, while video encoder 28 produces encoded video data streams. Each individual data stream (whether audio or video) can be referred to as an elementary stream. An elementary stream is a single, digitally decoded (and possibly compressed) component. For example, a decoded video or audio portion can be represented as an elementary stream. Elementary streams can be converted into packetized elementary streams (PES) before being encapsulated into a video file. Within the same representation, stream IDs can be used to distinguish PES packets belonging to one elementary stream from those belonging to other elementary streams. The basic unit of data in an elementary stream is the packetized elementary stream (PES) packet. Therefore, decoded video data typically corresponds to an elementary video stream. Similarly, audio data corresponds to one or more corresponding elementary streams.

[0116] Many video decoding standards, such as ITU-T H.264 / AVC and the upcoming High Efficiency Video Decoding (HEVC) standard, define the syntax, semantics, and decoding process for error-free bitstreams, each conforming to a profile or level. Video decoding standards typically do not specify an encoder, but the encoder's task is to ensure that the generated bitstream conforms to the standard for the decoder. In the context of video decoding standards, a "profile" corresponds to an algorithm, feature, or tool, and a subset of constraints imposed on that algorithm, feature, or tool. For example, as defined by the H.264 standard, a "profile" is, for instance, a subset of the entire bitstream syntax specified by the H.264 standard. A "level" corresponds to limitations on decoder resource consumption, such as decoder memory and computation, which are related to image resolution, bit rate, and block processing rate. Profiles can be signaled using the `profile_idc` (profile indicator) value, and levels can be signaled using the `level_idc` (level indicator) value.

[0117] For example, the H.264 standard recognizes that, within the limits imposed by the syntax of a given profile, the performance requirements of the encoder and decoder can still vary significantly depending on the values ​​of the syntax elements in the bitstream (such as the specified size of the decoded image). The H.264 standard further recognizes that in many applications, implementing a decoder capable of handling all the assumptions within the syntax of a particular profile is neither practical nor economical. Therefore, the H.264 standard defines a “level” as a specified set of constraints imposed on the values ​​of syntax elements in the bitstream. These constraints can be simple restrictions on the values. Alternatively, these constraints can take the form of constraints on arithmetic combinations of values ​​(e.g., image width multiplied by image height multiplied by the number of images decoded per second). The H.264 standard also provides that individual implementations can support different levels for each supported profile.

[0118] A decoder conforming to a profile typically supports all the features defined in that profile. For example, B-picture decoding, as a decoding feature, is not supported in the baseline H.264 / AVC profile but is supported in other H.264 / AVC profiles. A decoder conforming to a hierarchy should be able to decode any bitstream that does not require exceeding the limits defined in that hierarchy. The definitions of profiles and hierarchies can contribute to interpretability. For example, during a video transmission, a pair of profile and hierarchy definitions can be negotiated and agreed upon for the entire transmission session. More specifically, in H.264 / AVC, a hierarchy can define the number of blocks to be processed, the size of the decoded picture buffer (DPB), the size of the decoded picture buffer (CPB), the range of vertical motion vectors, the maximum number of motion vectors per two consecutive MBs, and whether B-blocks can have sub-blocks smaller than 8x8 pixels. In this way, the decoder can determine whether it is capable of properly decoding the bitstream.

[0119] exist Figure 1 In one example, the encapsulation unit 30 of the content preparation device 20 receives a base stream of video data including decoded data from the video encoder 28 and a base stream of audio data including decoded data from the audio encoder 26. In some examples, both the video encoder 28 and the audio encoder 26 may include a grouper for forming PES packets based on the encoded data. In other examples, both the video encoder 28 and the audio encoder 26 may interface with a corresponding grouper for forming PES packets based on the encoded data. In other examples, the encapsulation unit 30 may include a grouper for forming PES packets based on the encoded audio and video data.

[0120] Video encoder 28 can encode video data of multimedia content in various ways to produce different representations of the multimedia content at various bit rates and utilizing various characteristics such as pixel resolution, frame rate, compliance with various decoding standards, compliance with various profiles and / or profile levels used for various decoding standards, representations with one or more views (e.g., for two-dimensional or three-dimensional playback), or other such characteristics. As used in this disclosure, a representation may include one of the following: audio data, video data, text data (e.g., for closed captions), or other such data. A representation may include a primary stream, such as an audio primary stream or a video primary stream. Each PES packet may include a stream_id, which identifies the primary stream to which the PES packet belongs. Encapsulation unit 30 is responsible for assembling the primary streams into video files (e.g., segments) of various representations.

[0121] Encapsulation unit 30 receives PES packets for representing the basic stream from audio encoder 26 and video encoder 28, and forms corresponding Network Abstraction Layer (NAL) units based on the PES packets. Encoded video segments can be organized into NAL units that provide a “network-friendly” video representation for addressing applications such as video telephony, storage, broadcasting, or streaming. NAL units can be categorized as Video Decoding Layer (VCL) NAL units and non-VCL NAL units. VCL units may contain the core compression engine and may include block, macroblock, and / or slice-level data. Other NAL units may be non-VCL NAL units. In some examples, a picture of the decoded data in a temporal instance (typically presented as a picture of the primary decoded data) may be included in an access unit, which may include one or more NAL units.

[0122] Non-VCL NAL units can include parameter set NAL units and SEI NAL units, etc. Parameter sets can contain sequence-level header information (in the Sequence Parameter Set (SPS)) and picture-level header information that changes very little (in the Picture Parameter Set (PPS)). Using parameter sets (e.g., PPS and SPS), it is not necessary to repeat information that changes very little for each sequence or picture; therefore, decoding efficiency can be improved. Furthermore, the use of parameter sets allows for out-of-band transmission of important header information, thus avoiding redundant transmissions required for error recovery. In an example of out-of-band transmission, parameter set NAL units can be transmitted on a different channel than other NAL units (such as SEI NAL units).

[0123] Supplemental Enhancement Information (SEI) can contain information that is not essential for decoding image samples from VCL NAL units but can assist in processes related to decoding, display, error recovery, and other purposes. SEI messages can be included in non-VCL NAL units. SEI messages are a specification of some standards and are therefore not always mandatory for specific implementations of standards-compliant decoders. SEI messages can be sequence-level SEI messages or image-level SEI messages. Some sequence-level information can be included in SEI messages, such as the scalability information SEI message in the SVC example and the view scalability information SEI message in MVC. These example SEI messages can convey information about, for example, the extraction and characteristics of operation points. Additionally, the encapsulation unit 30 can form a manifest file, such as a Media Rendering Descriptor (MPD) describing the characteristics of the representation. The encapsulation unit 30 can format the MPD according to Extensible Markup Language (XML).

[0124] The encapsulation unit 30 can provide data for one or more representations of multimedia content, along with a manifest file (e.g., MPD), to the output interface 32. The output interface 32 may include a network interface or an interface for writing to storage media, such as a Universal Serial Bus (USB) interface, a CD or DVD writer or burner, an interface to magnetic or flash memory storage media, or other interfaces for storing or transmitting media data. The encapsulation unit 30 can provide data for each representation of the multimedia content to the output interface 32, which can transmit data to the server device 60 via network transmission or storage media. Figure 1 In the example, server device 60 includes storage medium 62 for storing various multimedia content 64, wherein each multimedia content includes a corresponding manifest file 66 and one or more representations 68A-68N (representations 68). In some examples, output interface 32 can also send data directly to network 74.

[0125] In some examples, representation 68 can be separated into adaptation sets. That is, each subset of representation 68 may include a corresponding set of common characteristics, such as codecs, profiles and hierarchies, resolution, number of views, file format of segments, text type information (which may identify the language or other characteristics of the text to be displayed using the representation and / or the audio data to be decoded and presented by, for example, a speaker), camera angle information (which may describe the camera angle or real-world camera view of the scene for the representation in the adaptation set), and hierarchical information describing the suitability of the content for a particular audience, etc.

[0126] Manifest file 66 may include a subset of representations 68 corresponding to a particular adaptation set, as well as data indicating common characteristics of the adaptation set. Manifest file 66 may also include data indicating individual characteristics (such as bit rate) of individual representations for each adaptation set. In this way, adaptation sets can provide simplified network bandwidth adaptation. Sub-elements of the adaptation set element in manifest file 66 can be used to indicate representations within the adaptation set.

[0127] Server device 60 includes request processing unit 70 and network interface 72. In some examples, server device 60 may include multiple network interfaces. Furthermore, any or all of the features of server device 60 may be implemented on other devices in the content delivery network, such as routers, bridges, proxy devices, switches, or other devices. In some examples, intermediate devices in the content delivery network may cache data of multimedia content 64 and include components that generally conform to the components of server device 60. Generally, network interface 72 is configured to transmit and receive data via network 74.

[0128] Request processing unit 70 is configured to receive network requests for data for storage medium 62 from a client device (such as client device 40). For example, request processing unit 70 may implement Hypertext Transfer Protocol (HTTP) version 1.1, as described by R. Fielding et al. in RFC 2616, “Hypertext Transfer Protocol – HTTP / 1.1,” Networking Working Group, IETF, June 1999. That is, request processing unit 70 may be configured to receive HTTP GET or partial GET requests and, in response to such requests, provide data for multimedia content 64. The request may, for example, use a segment URL to specify a segment representing one of 68. In some examples, the request may also specify one or more byte ranges of a segment, thus including partial GET requests. Request processing unit 70 may also be configured to service HTTP HEAD requests to provide header data for a segment representing one of 68. In any case, the request processing unit 70 can be configured to process the request to provide the requested data to the requesting device (such as the client device 40).

[0129] Additionally or alternatively, request processing unit 70 may be configured to deliver media data via a broadcast or multicast protocol (such as eMBMS). Content preparation device 20 may create DASH segments and / or subsegments in a manner substantially the same as described, but server device 60 may use eMBMS or another broadcast or multicast network transport protocol to deliver these segments or subsegments. For example, request processing unit 70 may be configured to receive multicast group join requests from client device 40. That is, server device 60 may advertise the Internet Protocol (IP) address associated with the multicast group to client devices (including client device 40) associated with specific media content (e.g., broadcast of a live event). Client device 40 may then submit a request to join the multicast group. The request may be propagated throughout network 74 (e.g., routers constituting network 74), causing routers to direct traffic destined for the IP address associated with the multicast group to subscribing client devices (such as client device 40).

[0130] like Figure 1 As shown in the example, multimedia content 64 includes a manifest file 66, which may correspond to a Media Presentation Description (MPD). The manifest file 66 may contain descriptions of different alternative representations 68 (e.g., video services with different qualities), and this description may include, for example, codec information, profile values, layer values, bitrates, and other descriptive characteristics of representation 68. Client device 40 may retrieve the MPD of the media presentation to determine how to access segments of representation 68.

[0131] Specifically, the retrieval unit 52 can retrieve configuration data (not shown) of the client device 40 to determine the decoding capabilities of the video decoder 48 and the rendering capabilities of the video output 44. The configuration data may also include any or all of the following: the language preference selected by the user of the client device 40, one or more camera angles corresponding to the depth preference set by the user of the client device 40, and / or the grading preference selected by the user of the client device 40. The retrieval unit 52 may include, for example, a web browser or media client configured to submit HTTP GET and partial GET requests. The retrieval unit 52 may correspond to software instructions executed by one or more processors or processing units (not shown) of the client device 40. In some examples, all or part of the functionality described for the retrieval unit 52 may be implemented in hardware, or a combination of hardware, software, and / or firmware, wherein the necessary hardware may be provided to execute the instructions for the software or firmware.

[0132] The retrieval unit 52 can compare the decoding and rendering capabilities of the client device 40 with the characteristics of the representation 68 indicated by the information in the manifest file 66. The retrieval unit 52 can initially retrieve at least a portion of the manifest file 66 to determine the characteristics of the representation 68. For example, the retrieval unit 52 can request a portion of the manifest file 66 describing the characteristics of one or more adapter sets. The retrieval unit 52 can select a subset (e.g., an adapter set) of representations 68 having characteristics that can be satisfied by the decoding and rendering capabilities of the client device 40. The retrieval unit 52 can then determine the bit rate for the representations in the adapter set, determine the amount of currently available network bandwidth, and retrieve a segment from one of the representations having a bit rate that the network bandwidth can satisfy.

[0133] Generally speaking, a higher bit rate representation can produce higher quality video playback, while a lower bit rate representation can provide sufficient quality video playback when available network bandwidth decreases. Therefore, when available network bandwidth is relatively high, the retrieval unit 52 can retrieve data from a relatively high bit rate representation, and when available network bandwidth is low, the retrieval unit 52 can retrieve data from a relatively low bit rate representation. In this way, the client device 40 can stream multimedia data through network 74 while adapting to changes in network bandwidth availability.

[0134] Additionally or alternatively, the retrieval unit 52 may be configured to receive data according to broadcast or multicast network protocols (such as eMBMS or IP multicast). In such an example, the retrieval unit 52 may submit a request to join a multicast network group associated with specific media content. After joining the multicast group, the retrieval unit 52 may receive data from the multicast group without issuing further requests to the server device 60 or the content preparation device 20. When the multicast group data is no longer needed, the retrieval unit 52 may submit a request to leave the multicast group, for example, to stop playback or change the channel to a different multicast group.

[0135] Network interface 54 can receive data from the selected represented segment and provide that data to retrieval unit 52, which in turn can provide the segment to decapsulation unit 50. Decapsulation unit 50 can decapsulate the elements of the video file into a PES stream, degroup the PES stream to retrieve encoded data, and transmit the encoded data to audio decoder 46 or video decoder 48, depending on whether the encoded data is part of an audio stream or a video stream (e.g., as indicated by the PES packet header of the stream). Audio decoder 46 decodes the encoded audio data and transmits the decoded audio data to audio output 42, while video decoder 48 decodes the encoded video data and transmits the decoded video data (which may include multiple views of the stream) to video output 44.

[0136] The video encoder 28, video decoder 48, audio encoder 26, audio decoder 46, encapsulation unit 30, retrieval unit 52, and decapsulation unit 50 can all be implemented as any of a variety of suitable processing circuits, such as one or more microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), discrete logic circuits, software, hardware, firmware, or any combination thereof. Each of the video encoder 28 and video decoder 48 can be included in one or more encoders or decoders, and either the video encoder or the video decoder can be integrated as part of a combined video encoder / decoder (CODEC). Similarly, each of the audio encoder 26 and audio decoder 46 can be included in one or more encoders or decoders, and either the audio encoder or the audio decoder can be integrated as part of a combined CODEC. The apparatus including the video encoder 28, video decoder 48, audio encoder 26, audio decoder 46, encapsulation unit 30, retrieval unit 52, and / or decapsulation unit 50 can include integrated circuits, microprocessors, and / or wireless communication devices, such as cellular phones.

[0137] Client device 40, server device 60, and / or content preparation device 20 may be configured to operate according to the techniques of this disclosure. For illustrative purposes, these techniques are described with respect to client device 40 and server device 60. However, it should be understood that content preparation device 20 may also be configured to perform these techniques as an alternative to (or other than) server device 60.

[0138] Encapsulation unit 30 can form NAL units, which include a header identifying the program to which the NAL unit belongs and a payload, such as audio data, video data, or data describing the transport or program stream corresponding to the NAL unit. For example, in H.264 / AVC, a NAL unit includes a 1-byte header and a payload of varying size. NAL units whose payloads include video data can include video data at various granularities. For example, a NAL unit can include video data blocks, multiple blocks, video data slices, or entire frames of video data. Encapsulation unit 30 can receive encoded video data in PES packet format with elementary streams from video encoder 28. Encapsulation unit 30 can associate each elementary stream with its corresponding program.

[0139] The encapsulation unit 30 can also assemble access units based on multiple NAL units. Generally, an access unit may include one or more NAL units representing a video data frame, and the corresponding audio data (when the audio data is available). Access units typically include all NAL units for a single output time instance, such as all audio and video data for a single time instance. For example, if each view has a frame rate of 20 frames per second (fps), each time instance may correspond to a time interval of 0.05 seconds. During this time interval, specific frames for all views with the same access unit (same time instance) can be rendered simultaneously. In one example, an access unit may include a decoded image within a time instance, which can be rendered as the primary decoded image.

[0140] Therefore, an access unit can include all audio and video frames of a common time instance, such as all views corresponding to time X. This disclosure also refers to the encoded picture of a particular view as a "view component." That is, a view component can include pictures (or frames) encoded for a particular view at a particular time. Therefore, an access unit can be defined as including all view components of a common time instance. The decoding order of the access units need not be the same as the output or display order.

[0141] Media presentation may include a Media Presentation Description (MPD), which may contain descriptions of different alternative representations (e.g., video services with different qualities), and this description may include, for example, codec information, profile values, and hierarchical values. An MPD is an example of a manifest file (such as manifest file 66). Client device 40 may retrieve the MPD of the media presentation to determine how to access movie clips in various representations. Movie clips may reside in a movie clip box (moof box) of the video file.

[0142] The manifest file 66 (which may include, for example, an MPD) can announce the availability of segments in representation 68. Specifically, the MPD may include information indicating the clock time when a first segment of one of the representations in 68 becomes available, and information indicating the duration of segments within representation 68. In this way, the retrieval unit 52 of the client device 40 can determine when each segment is available based on the start time and duration of segments preceding a particular segment.

[0143] After the encapsulation unit 30 has assembled the NAL units and / or access units into a video file based on the received data, the encapsulation unit 30 passes the video file to the output interface 32 for output. In some examples, the encapsulation unit 30 may store the video file locally or transmit the video file to a remote server via the output interface 32, instead of transmitting the video file directly to the client device 40. The output interface 32 may include, for example, a transmitter, a transceiver, a device for writing data to a computer-readable medium (such as, for example, an optical drive, a magnetic media drive (e.g., a floppy disk drive)), a universal serial bus (USB) port, a network interface, or other output interface. The output interface 32 outputs the video file to a computer-readable medium, such as, for example, a transmitting signal, a magnetic medium, an optical medium, a memory, a flash drive, or other computer-readable medium.

[0144] Network interface 54 can receive NAL units or access units via network 74 and provide NAL units or access units to decapsulation unit 50 via retrieval unit 52. Decapsulation unit 50 can decapsulate the elements of the video file into a PES stream, degroup the PES stream to retrieve encoded data, and transmit the encoded data to audio decoder 46 or video decoder 48, depending on whether the encoded data is part of an audio stream or a video stream (e.g., as indicated by the PES packet header of the stream). Audio decoder 46 decodes the encoded audio data and transmits the decoded audio data to audio output 42, while video decoder 48 decodes the encoded video data and transmits the decoded video data (which may include multiple views of the stream) to video output 44.

[0145] Figure 2 It shows in more detail Figure 1 A block diagram of an example set of components for retrieval unit 52. In this example, retrieval unit 52 includes eMBMS middleware unit 100, DASH client 110, and media application 112.

[0146] In this example, the eMBMS middleware unit 100 further includes an eMBMS receiving unit 106, a cache 104, and a proxy server unit 102. In this example, the eMBMS receiving unit 106 is configured to receive data via eMBMS, for example, according to the following: One-way file delivery (FLUTE), described in T. Paila et al., “FLUTE—File Delivery over Unidirectional Transport,” Network Working Group, RFC 6726, November 2012, available at tools.ietf.org / html / rfc6726. Also, the eMBMS receiving unit 106 can receive files via broadcast from, for example, a server device 60, which can act as a broadcast / multicast service center (BM-SC).

[0147] When the eMBMS middleware unit 100 receives data for a file, it can store the received data in cache 104. Cache 104 may include computer-readable storage media, such as flash memory, hard disk, RAM, or any other suitable storage media.

[0148] Proxy server unit 102 can act as a server for DASH client 110. For example, proxy server unit 102 can provide DASH client 110 with an MPD file or other manifest file. Proxy server unit 102 can announce the availability of segments in the MPD file and hyperlinks from which these segments can be retrieved. These hyperlinks may include a local host address prefix corresponding to client device 40 (e.g., 127.0.0.1 for IPv4). In this way, DASH client 110 can request segments from proxy server unit 102 using an HTTP GET or partial GET request. For example, for a segment obtainable from the link http: / / 127.0.0.1 / rep1 / seg3, DASH client 110 can construct an HTTP GET request including a request for http: / / 127.0.0.1 / rep1 / seg3 and submit the request to proxy server unit 102. Proxy server unit 102 can retrieve the requested data from cache 104 and provide the data to DASH client 110 in response to such a request.

[0149] Figure 3 This is a conceptual diagram illustrating the elements of multimedia content 120. Multimedia content 120 can be combined with multimedia content 64 ( Figure 1 This corresponds to either other multimedia content stored on storage medium 62 or another multimedia content stored on storage medium 62. Figure 3In the example, multimedia content 120 includes a Media Presentation Description (MPD) 122 and multiple representations 124A-124N (representations 124). Representation 124A includes optional header data 126 and segments 128A-128N (segments 128), while representation 124N includes optional header data 130 and segments 132A-132N (segments 132). For convenience, the letter N is used to denote the last movie segment in each representation 124. In some examples, there may be a different number of movie segments between representations 124.

[0150] MPD 122 may include a data structure separate from representation 124. MPD 122 may correspond to Figure 1 The manifest file is 66. Similarly, it indicates that 124 can correspond to... Figure 1 The representation 68. Typically, MPD 122 may include data that typically describes the characteristics of representation 124 (such as decoding and rendering characteristics, adaptation set, profile corresponding to MPD 122, text type information, camera angle information, rating information, special effects mode information (e.g., information indicating representations including time subsequences) and / or information for retrieving remote cycles (e.g., for inserting targeted advertisements into media content during playback)).

[0151] Header data 126 (if present) may describe characteristics of segment 128, such as the time position of the Random Access Point (RAP, also known as the Streaming Access Point (SAP)), which segment within segment 128 includes the RAP, the byte offset to the RAP within segment 128, the Uniform Resource Locator (URL) of segment 128, or other aspects of segment 128. Header data 130 (if present) may describe similar characteristics of segment 132. Additionally or alternatively, such characteristics may be fully included within MPD 122.

[0152] Sections 128 and 132 include one or more decoded video samples, each of which may include a video data frame or slice. Each video sample in the decoded video samples of section 128 may have similar characteristics, such as height, width, and bandwidth requirements. These characteristics can be described by data from MPD 122, but such data is not included in... Figure 3 The example is shown. MPD 122 may include features as described by the 3GPP specification, plus any or all of the signaled information described in this disclosure.

[0153] Each segment in segments 128 and 132 can be associated with a unique Uniform Resource Locator (URL). Therefore, a streaming network protocol (such as DASH) can be used to retrieve each segment in segments 128 and 132 independently. In this way, a destination device (such as client device 40) can use an HTTP GET request to retrieve segment 128 or 132. In some examples, client device 40 can use a partial HTTP GET request to retrieve a specific byte range of segment 128 or 132.

[0154] Figure 4 This is a block diagram illustrating the elements of example video file 150, which may correspond to represented segments, such as... Figure 3 One of the segments in segments 128 and 132. Each segment in segments 128 and 132 may include substantially conforming to Figure 4 The example shows the data arrangement. The video file 150 encapsulates segments. As described above, video files, according to the ISO Basic Media File Format and its extensions, store data in a series of objects called "boxes". Figure 4 In the example, video file 150 includes a file type (FTYP) box 152, a movie (MOOV) box 154, a segment index (sidx) box 162, a movie clip (MOOF) box 164, and a movie clip random access (MFRA) box 166. Although Figure 4 The example shown is a video file, but it should be understood that other media files may include other types of media data (e.g., audio data, timed text data, etc.) constructed similarly to those in video file 150, depending on the ISO Basic Media File Format and its extensions.

[0155] The File Type (FTYP) box 152 generally describes the file type of the video file 150. The File Type box 152 may include data identifying the specifications for the best use of the video file 150. The File Type box 152 may optionally be placed before the MOOV box 154, the Movie Clip box 164, and / or the MFRA box 166.

[0156] In some examples, a segment such as video file 150 may include an MPD update box (not shown) preceding the FTYP box 152. The MPD update box may include information indicating that the MPD corresponding to the representation including video file 150 will be updated, as well as information for updating the MPD. For example, the MPD update box may provide a URI or URL for the resource to be used to update the MPD. As another example, the MPD update box may include data for updating the MPD. In some examples, the MPD update box may immediately follow the segment type (STYP) box (not shown) of video file 150, where the STYP box defines the segment type of video file 150.

[0157] exist Figure 4 In the example, the MOOV box 154 includes a Movie Header (MVHD) box 156, a Track (TRAK) box 158, and one or more Movie Extension (MVEX) boxes 160. Typically, the MVHD box 156 describes general characteristics of the video file 150. For example, the MVHD box 156 may include data describing when the video file 150 was initially created, when the video file 150 was last modified, the timestamp of the video file 150, the playback duration of the video file 150, or other data generally describing the video file 150.

[0158] TRAK box 158 may include track data of video file 150. TRAK box 158 may include a Track Header (TKHD) box that describes the characteristics of the track corresponding to TRAK box 158. In some examples, TRAK box 158 may include decoded video images, while in other examples, the decoded video images of the track may be included in movie clip 164, which may be referenced by data from TRAK box 158 and / or sidx box 162.

[0159] In some examples, video file 150 may include more than one track. Therefore, MOOV box 154 may include TRAK boxes in a number equal to the number of tracks in video file 150. TRAK boxes 158 may describe the characteristics of the corresponding tracks in video file 150. For example, TRAK boxes 158 may describe the temporal and / or spatial information of the corresponding tracks. When encapsulation unit 30 ( Figure 3 When a parameter set track is included in a video file (such as video file 150), a TRAK box similar to the TRAK box 158 of the MOOV box 154 can describe the characteristics of the parameter set track. The encapsulation unit 30 can signal the presence of a sequence level SEI message in the parameter set track within the TRAK box describing the parameter set track.

[0160] The MVEX box 160 can describe the characteristics of the corresponding movie clip 164, for example, by signaling to the video file 150 that, in addition to the video data (if any) included in the MOOV box 154, the movie clip 164 is also included. In the context of streaming video data, decoded video images may be included in the movie clip 164 instead of the MOOV box 154. Therefore, all decoded video samples may be included in the movie clip 164 instead of the MOOV box 154.

[0161] MOOV boxes 154 may include an equal number of MVEX boxes 160 as the number of movie segments 164 in the video file 150. Each MVEX box in the MVEX boxes 160 may describe the characteristics of a corresponding movie segment in the movie segment 164. For example, each MVEX box may include a Movie Extended Header Box (MEHD) box, which describes the time duration of a corresponding movie segment in the movie segment 164.

[0162] As mentioned above, encapsulation unit 30 may store a set of sequence data in a video sample that does not include the actual decoded video data. The video sample may typically correspond to an access unit, which is a representation of a decoded picture at a specific time instance. In the context of AVC, the decoded picture includes one or more VCL NAL units (containing information about all pixels used to construct the access unit) and other associated non-VCL NAL units (such as SEI messages). Therefore, encapsulation unit 30 may include a set of sequence data in a movie clip of movie clip 164, which may include a sequence-level SEI message. Encapsulation unit 30 may also signal the presence of the set of sequence data and / or the sequence-level SEI message as being present in a movie clip of movie clip 164 within an MVEX box of MVEX box 160 corresponding to a movie clip of movie clip 164.

[0163] SIDX box 162 is an optional element of video file 150. That is, video files conforming to 3GPP file formats or other such file formats do not necessarily need to include SIDX box 162. According to the example of the 3GPP file format, SIDX boxes can be used to identify sub-segments of a segment (e.g., a segment contained within video file 150). The 3GPP file format defines a sub-segment as "a self-contained set of one or more consecutive movie clip boxes having a corresponding media data box and a media data box containing data referenced by a movie clip box that must follow that movie clip box and precede the next movie clip box containing information about the same track." The 3GPP file format also instructs that the SIDX box "contains a series of references to sub-segments of the (sub)segment recorded by that box. The referenced sub-segments are consecutive in presentation time. Similarly, bytes referenced by the segment index box are always consecutive within the segment. The size of the reference gives a count of the number of bytes in the referenced material."

[0164] SIDX box 162 typically provides information representing one or more sub-segments of a segment included in video file 150. For example, such information may include the playback time of the start and / or end of the sub-segment, the byte offset of the sub-segment, whether the sub-segment includes (e.g., begins at) a Stream Access Point (SAP), the type of SAP (e.g., whether the SAP is an Instant Decoder Refresh (IDR) picture, a Clean Random Access (CRA) picture, a Broken Link Access (BLA) picture, etc.), the location of the SAP within the sub-segment (in terms of playback time and / or byte offset), etc.

[0165] Movie clip 164 may include one or more decoded video pictures. In some examples, movie clip 164 may include one or more groups of pictures (GOPs), where each GOP may include several decoded video pictures, such as frames or images. Additionally, as described above, in some examples, movie clip 164 may include a sequence of data. Each movie clip in movie clip 164 may include a movie clip header box (MFHD). Figure 4 (Not shown in the image). The MFHD box can describe the characteristics of the corresponding movie clip, such as the movie clip's sequence number. Movie clips 164 can be included in video file 150 in the order of their sequence numbers.

[0166] MFRA box 166 can describe random access points within movie segments 164 of video file 150. This can help perform special effects modes, such as searching for specific time positions (i.e., playback times) within segments encapsulated by video file 150. In some examples, MFRA box 166 is typically optional and does not need to be included in the video file. Similarly, client devices (such as client device 40) do not necessarily need to refer to MFRA box 166 to correctly decode and display the video data of video file 150. MFRA box 166 can include a number equal to the number of tracks in video file 150, or in some examples, a number equal to the number of media tracks (e.g., non-cue tracks) in video file 150 (TFRA) boxes (not shown).

[0167] In some examples, movie clip 164 may include one or more streaming access points (SAPs), such as IDR pictures. Similarly, MFRA box 166 can provide an indication of the location within the SAP video file 150. Therefore, a temporal subsequence of video file 150 can be formed from the SAP of video file 150. The temporal subsequence may also include other pictures, such as SAP-dependent P-frames and / or B-frames. Frames and / or slices of the temporal subsequence can be arranged within segments such that frames / slices of the temporal subsequence that depend on other frames / slices of the subsequence can be correctly decoded. For example, in a hierarchical arrangement of data, data used for prediction of other data may also be included in the temporal subsequence.

[0168] Figure 5 This is a conceptual diagram illustrating a delivery session model 200 for CMMF. Figure 5 Multiple transport sessions with corresponding Transport Session Identifiers (TSIs) are described. According to the techniques of this disclosure, application data and application objects to be delivered from a CMMF transmitter to a CMMF receiver can be organized into transport objects. Transport objects are application objects with associated metadata, which can be retrieved for use by the receiving application. Objects can be organized within a transport session.

[0169] A transport session may include one or more transport objects, each of which can be uniquely identified within the session using a Transport Object Identifier (TOI). Each transport object may be assigned to a transport session within a delivery session. A transport session flow may be uniquely identified by a Transport Session Identifier (TSI). Transport objects within a transport session may be uniquely identified by a TOI and may have a specified delivery order. Transport objects within a transport session may share common metadata and other properties.

[0170] Each TOI can be associated with corresponding metadata and a URL. The associated metadata and URL can be used (e.g., using an HTTP partial GET request) to retrieve the byte range of a single application object. The File Delivery Table (FDT) can be used to create a mapping between TOI / TSI and URL for application objects.

[0171] The file delivery table provides a mechanism for describing the various attributes associated with a file to be delivered within a file delivery session. The following list provides examples of such attributes and is not intended to be mutually exclusive or exhaustive.

[0172] Attributes related to file delivery may include:

[0173] • Represents the TOI value of the file.

[0174] • FEC object sends information (including FEC encoded ID and FEC instance ID, if relevant)

[0175] • Size of the object being sent carrying the file

[0176] • Aggregation rate for transmitting packets to all channels

[0177] Attributes related to the file itself may include:

[0178] • The file's name, identifier, and location (specified by the URI)

[0179] • Alternate file location

[0180] • Media type of the file

[0181] • File size

[0182] • File encoding

[0183] ○ Message Summary of the Document

[0184] Figure 5 An overview of the application and source data within the context of the techniques disclosed herein is provided. In this example, a delivery session comprises N transport sessions, each identified by a TSI from 0 to N-1. Each transport session comprises N[TSI] transport objects, each identified by a TOI from 0 to N[TSI]-1. The techniques disclosed herein can be applied to a single transport session with a single transport object, or multiple transport objects and / or transport sessions.

[0185] The data model and associated metadata can be provided implicitly or as part of a manifest file. A typical example is a DASH or HLS manifest that describes the delivery session model.

[0186] CMMF provides techniques for generating repair objects and for repairing object sessions against transport sessions, as discussed below. The CMMF repair framework is based on the existing FEC framework. Based on Forward Error Correction (FEC) building blocks as defined in IETF RFC 5052, CMMF can be considered as a Content Delivery Protocol (CDP) specification as defined in Clause 8 of RFC 5052. In this description, the terminology defined in Clause 2 of IETF RFC 5052 is used, specifically:

[0187] • Object: An ordered sequence of octets to be transmitted by a transport protocol. For example, a file or stream.

[0188] • Symbol: A data unit processed by forward error correction codes. A symbol is always treated as a single unit, either fully received or completely lost.

[0189] • Source symbol: A symbol containing information from the original object.

[0190] • Repair symbol: A symbol containing information generated by FEC code, which can be used to recover the lost source symbol.

[0191] • Encoding symbols: source symbols or repair symbols.

[0192] • Encoder: An FEC scheme-specific function required to convert an object into FEC-encoded data. That is, a function to generate repair symbols from source symbols.

[0193] • Decoder: The FEC scheme-specific function required to convert received FEC-encoded data into a copy of the original object.

[0194] • Receiver: A system that supports the receiving function of the CDP and FEC schemes according to this specification.

[0195] • Transmitter: A system that supports the transmission functions of the CDP and FEC schemes according to this manual.

[0196] • Source block: A portion of an object formed by a subset of the object's source symbols.

[0197] • FEC Scheme: Supporting information and processes, combined with FEC code or algorithm specifications, fully define how FEC codes can be used with content packaging or delivery systems.

[0198] CMMF can use FEC schemes defined based on the FEC building blocks in RFC5052, and CMMF can be considered as CDP as defined in RFC5052. RFC5052 also provides the definition and transmission of three types of information from transmitter to receiver, as follows:

[0199] • The encoded symbols themselves,

[0200] • Auxiliary messages associated with coded symbols (or groups of such symbols), such as groups of symbols in a repair object, and

[0201] • Auxiliary information associated with the entire object being transmitted.

[0202] FEC information can be categorized as follows:

[0203] • FEC information associated with an object can send information to the FEC object.

[0204] • FEC information associated with a specific encoded symbol of the object can be the FEC payload ID.

[0205] FEC schemes can be identified by FEC-coded IDs (e.g., integers assigned by IANA). Requirements for FEC scheme specifications are defined in Clause 7 of RFC 5052.

[0206] Additionally, for FEC, the following information is defined and provided in RFC 5052:

[0207] • FEC information associated with an object is called FEC object transmission information. This includes, for example, an FEC code ID that identifies the FEC scheme and is an integer assigned by IANA, the object's transmission length, or the code symbol length. For details, refer to clause 6.2 of RFC 5052.

[0208] • FEC information associated with a specific coded symbol of an object is called the FEC payload ID. This information indicates how to construct the associated repair symbol from the object. The semantics and encoding format (including its size) of the FEC payload ID are defined by the FEC scheme. The CDP specifies how the FEC payload ID is carried in the appropriate framework. For details, refer to clause 6.3 of RFC 5052.

[0209] The core concept of RFC 5052 is the "FEC scheme," which can be distinguished from the concepts of "FEC code" or "FEC algorithm." An FEC scheme defines auxiliary information and procedures that, in conjunction with the FEC code or algorithm specification, fully define how the FEC code can be used with CDP. Requirements for the FEC scheme specification are defined in Clause 7 of RFC 5052.

[0210] The FEC object transmission information may include an FEC-encoded ID, which can be an integer between 0 and 255, including the endpoint, identifying a specific FEC scheme. The FEC object transmission information may further include:

[0211] • Transfer-length: A non-negative integer indicating the length of the object (in octets).

[0212] •Encoding-Symbol-Length: A non-negative integer indicating the length of each encoded symbol (in octets).

[0213] •Maximum-Source-Block-Length: A non-negative integer indicating the maximum number of source symbols in the source block.

[0214] • Max-Number-of-Encoding-Symbols: A non-negative integer indicating the maximum number of encoded symbols.

[0215] FEC object transmission messages can further include scheme-specific information. FEC schemes can use scheme-specific FEC object transmission message elements to convey information that is important to the decoder but cannot be fully represented in mandatory or common FEC object transmission message elements.

[0216] The FEC payload ID contains information indicating to the FEC decoder the relationship between the coded symbols carried by a particular packet and the FEC coded transformation. For example, if a packet carries source symbols, the FEC payload ID indicates which source symbols of the object the packet carries. If a packet carries repair symbols, the FEC payload ID indicates how those repair symbols are constructed from the object. The FEC payload ID may also contain information about larger groups of coded symbols, of which the coded symbols contained in the packet are a part. For example, the FEC payload ID may contain information about the source block associated with the symbol. The encoding format of the FEC payload ID (including its size) is defined by the FEC scheme. CDP specifies how the FEC payload ID is carried within the data packet, i.e., the location of the FEC payload ID within the CDP packet format and how the FEC payload ID is associated with the coded symbols. For systematic FEC codes (i.e., those codes in which the original source data is included within the encoded data), the FEC scheme may specify two FEC payload ID formats: one for packets carrying only source symbols and another for packets carrying at least one repair symbol.

[0217] Figure 6 This is a conceptual diagram illustrating an example process for generating a transport object. A forward error correction (FEC) transport object 232 can be formed from one or more source transport objects 230, such as... Figure 6 As shown.

[0218] like Figure 6As shown, initially, one or more source transport objects 230 are combined into an FEC transport object 232. A typical case is that the source transport objects are directly passed into the FEC transport object. Next, for each FEC transport object, one or more source blocks 234 are generated, each source block containing an integer number of symbols of size T. For each source block associated with an FEC transport object, a set of repair symbols 236 can be generated, each repair symbol also of size T. The transport object forming unit 238 uses the source and repair symbols of all source blocks associated with the FEC transport object to generate one or more coded transport objects 240. CMMF coded transport objects may include source symbols only, a mixture of source symbols and repair symbols, or repair symbols only.

[0219] For FEC transport object formation, CMMF defines several instantiations and patterns. In one example, the principles defined in RFC 5052 and summarized above are reused, namely, forming source blocks from FEC transport objects. In this case, CMMF can also inherit principles from IETF RFC 9223 (ROUTE for One-Way Transport Real-Time Transport Object Delivery).

[0220] In some examples, there may be a one-to-one mapping between source data and transport objects, which is available in file delivery protocols such as FLUTE and ROUTE. In some examples, there may be a many-to-one mapping from a source object to a single transport object, which is available in ROUTE. In some examples, there may be a one-to-many mapping of byte ranges of source objects to transport objects, such that a source object can be mapped to multiple transport objects (e.g., each byte range of the source object can be mapped to a corresponding transport object).

[0221] In some examples, each repair object includes a different set of repair symbols. In other examples, each repair object contains coded symbols with the same ESI for each source block. The coded symbols for each source block can be sequential or interleaved across repair objects for optimized startup.

[0222] According to the technology disclosed herein, the object to be repaired may include any or all of the following information:

[0223] • Indicator for CMMF repair object

[0224] • Includes the header of the information sent by the FEC object, for example:

[0225] ○FEC code ID

[0226] ○ Sending information to public FEC objects

[0227] ■Transmission Length

[0228] ■ Symbol Size

[0229] ○ Sending information to specific targets in the plan

[0230] • Includes repaired TOI / TSI information and optional header information about the source object:

[0231] ○ Identifier

[0232] ○ The source object used to repair object mapping

[0233] ○Source object URL

[0234] ○Content Type

[0235] ○Timecode associated with the source object

[0236] • Source FEC Payload ID: A header that provides information about the included coded symbols and the FEC payload ID.

[0237] • Encoded symbol sequence

[0238] Static and public information may be included by a repair object that is explicitly or implicitly referenced by multiple repair objects.

[0239] The following assumptions can be made regarding CMMF:

[0240] • Fully specified FEC code

[0241] • System FEC code

[0242] • The application object can be provided without modification.

[0243] • The application objects can follow the guidelines Figure 19 The model described.

[0244] • The mapping between application objects, URLs, and FEC associations can exist in both TSI and TOI.

[0245] This mapping can be explicitly provided, for example, in the FDT or in the manifest.

[0246] • The FEC transmission object can be:

[0247] ○ A single application object with fill and size, such as, for example, regarding Figure 9 The discussion.

[0248] ○ Byte range of application objects with padding and size

[0249] ○ The aggregation of the transport object and the FEC super object used to create, for example, as per [reference to...] Figure 8 The discussion.

[0250] • The objects to be repaired can each be one of the following aggregates:

[0251] ○FEC object sends information

[0252] ○ Fix TO and TSI (based on source TOI and TSI)

[0253] ○ Source information (including FDT and possible timing information)

[0254] ○FEC Payload ID

[0255] ○ Contains repair symbols

[0256] • Repair objects can be organized in a stream / session using TSI and repair object headers.

[0257] Encoded symbols can be included in the following pattern: all encoded symbols with the same ESI are written sequentially starting from block 1. Afterward, the next ESI is written for all source blocks.

[0258] Each repair object can contain enough symbols to restore the application object.

[0259] Figure 7 This is a conceptual diagram (250) depicting the relationship between the source object and the repair object. The process for generating an FEC transport object is provided in RFC 9223, Clause 5.6. It is assumed that the transport object is as follows: Figure 7 The single application object shown is associated with an FEC transport object, which consists of a concatenation of the transport object, a padding octet (P), and the FEC object size (F) in octets, carried in four octet fields. In the FEC encoded symbol, the FEC transport object size K can be an integer multiple of the symbol size Y. The transport object size F is carried in the last four octets of the FEC transport object. A repair symbol is then generated using the FEC scheme with ESIK, K+1, etc. Information K and Y, along with the FEC OTI, FEC payload ID, and application metadata, are then added to the repair object and / or can be provided in the external transport information (ETI).

[0260] CMMF may include certain principles ported from ROUTE. FEC design can follow these design principles:

[0261] • Provide FEC-related information only when needed.

[0262] • Receivers that do not support this framework can ignore the repair group.

[0263] • FEC is symbol-based, with each protected source stream having a fixed symbol size.

[0264] • FEC Repair Stream provides protection for delivered objects from one or more source streams.

[0265] Additionally, the FEC repair flow statement may include FEC-specific information, such as:

[0266] • FEC repair flow statement including all FEC-specific information.

[0267] • An FEC transport object is formed by concatenating the delivery object, filling octets, and size information to form a chunk of data with a symbol size of N, where N≥1.

[0268] • As a concatenation of one or more FEC transport objects to bundle FEC transport objects for use as an FEC super object for FEC protection.

[0269] • Package FEC data into the repair object

[0270] The receiver can be configured to recover the application object from the repair object based on the available FEC information.

[0271] Figure 8 This is a conceptual diagram illustrating example superobject generation technique 280. ROUTE allows transport objects to be combined with FEC superobjects to generate a single FEC source block, which collectively protects all transport objects included in the virtual superobject, as described in RFC 9223, Clause 5.7. Figure 8 As shown.

[0272] Figure 9 This is a conceptual diagram illustrating another example technique 230 for superobject generation. The repaired object can be as follows: Figure 9 The setup is illustrated. In this example, objects in different TSIs include source symbols in a cyclical manner. This configuration enables parallel downloading of repair objects and the aggregation of initial symbols from each encoded object into the initial data of the application object.

[0273] Figure 10 This is a conceptual diagram illustrating a sample CMMF delivery session 350, which includes a collection of source and repair objects organized across different transport sessions. The following assumptions are made regarding the CMMF content delivery framework:

[0274] • The FEC code is specified exactly according to the definition in RFC 5052.

[0275] •FEC code is the system

[0276] • The application object is provided unmodified for access by the CMMF receiver.

[0277] • The application data can follow the principles described above.

[0278] • Using TOI and TSI, a mapping exists between application objects, repair objects, and sessions, and associated metadata can be provided in ETI.

[0279] • The FEC transfer object can follow the principles discussed above and can be one of the following:

[0280] ○ Has a basis Figure 7 A single applied object with varying fill and size.

[0281] ○ The transport object used to create and based on Figure 8 Aggregation of FEC super objects

[0282] • Encoded transmission objects may include any or all of the following information or related to them:

[0283] ○FEC object sends information

[0284] ○ Fix TOI and TSI (based on source TOI and TSI)

[0285] ○Source object information

[0286] ○FEC Payload ID

[0287] ○ Repair symbol

[0288] • Repair objects can be organized within a session with TSI and can have an assigned repair object header that includes public metadata applicable to all objects in the delivery session.

[0289] Figure 11 This is a conceptual diagram illustrating a CMMF reference architecture according to the technology of this disclosure. In this example, the CMMF reference architecture includes a source transport object 400, a CMMF transmitter 402 (e.g., Figure 1 Content preparation device 20 and / or server device 60), CMMF receiver 408 (e.g. Figure 1 The client device 402 and the restored source transmission object 414 are described. In this example, the CMMF transmitter 402 includes a repair object generation unit 404 that forms an encoded transmission object 406, which may include an application object and a repair object. In this example, the CMMF receiver 408 includes a data collection unit 410 and an object recovery unit 412. As discussed below, the object recovery unit 412 uses the application object and the repair object to form the restored source transmission object 414. The data collection unit 410 may provide back to the CMMF transmitter 402 metrics about reception statistics or the like, which can be used to perform future FEC coding (e.g., increasing or decreasing the frequency, size, number, or the like of repair objects).

[0290] CMMF defines how to generate a repair object from a source object, such as Figure 11 As shown. Such repair objects, along with the original source object and externally transmitted information (ETI) describing the location and relationship between the source and the repair object, can be provided to the CMMF receiver 408 (e.g., Figure 1 The client device 40). CMMF receiver 408 uses ETI and, by collecting source transmissions and repairing transmission objects, is able to recover the source transmission object and provide it to applications (e.g., decoders and presentation units, such as...). Figure 1 (See client device 40). Details regarding the CMMF transmitter and CMMF receiver architecture are provided below.

[0291] The reference point between CMMF transmitter 402 and CMMF receiver 408 includes:

[0292] •CMMF-ETI: This reference point provides External Transfer Information (ETI) describing the location and relationship of the source and the object being repaired, which can be provided to the CMMF receiver. This information may be provided, for example, in a streaming manifest or any other document delivery manifest.

[0293] •CMMF-S: This reference point provides the source transport object. For CMMF, this type of object has not been modified from the original data. Object recovery can use portions of this type of object to recover the source object.

[0294] •CMMF-R: This reference point provides a repair transfer object.

[0295] Various deployment options are available. In one example deployment option, only the decoded object is provided. In another example, the source object is provided along with the repair object.

[0296] Figure 12 This is a conceptual diagram illustrating a deployment option where the CMMF receiver 428, based on information in the CMMF-ETI, can only access the encoded transmission object 426 via CMMF-R. Specifically, with... Figure 11 similar, Figure 12 The diagram depicts a source transmission object 420, a CMMF transmitter 422, a repair object generation unit 424, an encoded transmission object 426, a CMMF receiver 428, a data collection unit 430, an object recovery unit 432, and a recovered source transmission object 434. The CMMF receiver 428 uses information (encoded transmission object 426 and ETI information) to recover the complete object. Sub-cases include: 1) each encoded transmission object contains sufficient information to recover the source transmission object; 2) the encoded transmission object is partial, and the source transmission object can only be recovered using a combination of multiple encoded objects.

[0297] Figure 13This is a conceptual diagram illustrating the CMMF transmitter 452, which includes several procedures for creating deployment options for repair objects. (Similar to...) Figure 11 and Figure 12 , Figure 13 The source transport object 450 and CMMF transmitter 452 are depicted. However, in this example, the CMMF transmitter 452 includes an FEC transport object generation unit 454, repair metadata source object information, an FEC symbol generation unit 458, a packing unit 460, and a repair transport object 462. Figure 13 The components can be incorporated Figure 1 The encapsulation unit 30 and / or request processing unit 70. The CMMF transmitter 452 may include several procedures to create a repair object as follows:

[0298] • One or more source transport objects are provided for FEC transport object generation. Source blocks can be formed based on the principles discussed above.

[0299] • Next, the source block, along with the FEC OTI for generating the repair symbol, is provided to the FEC symbol generation program.

[0300] • Repair symbols are provided together with repair metadata for packaging.

[0301] • The packaging function creates one or more report objects based on repair metadata and repair symbols.

[0302] To ensure proper operation of the CMMF transmitter, configuration information can be provided to the CMMF transmitter. This includes, but may not be limited to:

[0303] • FEC object sends information

[0304] • Source object included in FEC transport object

[0305] • Packaging options, such as how many repair objects are generated from the source block, how many coded symbols are included in each repair object, whether to generate a repair object header, or whether the repair objects are self-contained (i.e., have enough symbols to recover the source object).

[0306] Externally sent messages allow CMMF clients to map application requests to CMMF receiver operations. Table 1 below provides a summary of possible externally sent messages. Specific JSON instantiations can be provided to offer ETI information to the receiver. However, other cases exist in the remainder of this disclosure that only allow a subset of use cases, for which information is provided in a simpler version.

[0307] Table 1—Summary of Externally Sent Messages

[0308]

[0309]

[0310] Figure 14 This is a conceptual diagram illustrating the example CMMF encoding process 480. The decoded media representation carried within the CMMF can be provided to serve multiple current and emerging use cases for media caching, distribution, and delivery. For example:

[0311] • Highly efficient multi-source / multi-path and / or multi-access network delivery.

[0312] • Improve cache flexibility, including leveraging open caches and peer-to-peer systems for efficient operation.

[0313] • Application-layer FEC used to improve reliability in the event of packet loss.

[0314] Applying linear, network, or channel decoding to source-decoded audiovisual media can follow these basic steps:

[0315] • Divide the source decoded media data into blocks_count. The value of block_count can be very small (e.g., 1) or very large, depending on the size of the source data.

[0316] • Divide each source decoded media block into block_num_symbols source symbols, x i , where i is the index of the original source symbol in the block. The size of each symbol is block_symbol_size (in bytes), and the final symbol is padded with zero-value bytes if necessary. The value of block_num_symbols is usually chosen by the specific application and the decoding type used.

[0317] • Encode the source symbols to create at least block_num_symbols decoded symbols (depending on the decoding type), y i The number of decoded symbols generated typically varies depending on the decoding type, use case, and network.

[0318] Figure 14 A graphical representation illustrating this type of step is provided. In this example, the original source (media) data file or stream is split into three source blocks. Each source block is further split into eight source symbols x. i , i = 0, 1, ..., block_num_symbols - 1. Then, these source symbols are encoded using a supported CMMF decoding type to produce the resulting decoded symbol y. j , j≥i.

[0319] The resulting decoded symbols and associated metadata (decoding parameters) can be formatted according to the CMMF specification for storage, exchange, or delivery. Decoding parameter metadata is necessary to ensure that a compatible decoder can reconstruct the original (media) source symbols. Depending on the type of decoding used, decoding parameters may include coded symbol IDs (EIDs), coefficient vectors, pseudo-random number generator (PRNG) seeds, and parameters such as block_num and block_symbol_size.

[0320] The CMMF bitstream specifies and implements the carrying and signaling of such basic decoding parameters required to reconstruct the original source symbols in the decoder.

[0321] Figure 15 This is a conceptual diagram illustrating the CMMF decoding process in example 490. The basic steps for decoding include:

[0322] • For each block, recreate the source symbol x i The received decoded symbol y j Decode it.

[0323] • Concatenate each source symbol x i And if necessary, remove any zero padding from the last symbol to reform the original source block.

[0324] • Each source block in the concatenated source blocks is reassembled to reconstruct the original source decoded media file / data.

[0325] In some examples, FLUTE-based instantiation of the CMMF framework can be provided. Instantiation can be consistent with RFC 6726 and reuse some of the concepts defined in 3GPP TS 26.346. In this case, the following limitations may apply:

[0326] •ETI is a document delivery table (FDT) with minimal expansion to support CMMF.

[0327] • Only a single transport session is used, and different encoded objects can be generated for TOI.

[0328] • The encoded object is compatible with the ALC payload format with some additional headers and carries only the repair symbols.

[0329] • Only a single object is associated with the TOI; a super object is not used.

[0330] The FEC code ID can be set to 1 (Raptor), 5 (Reed-Solomon over GF 2^^8), or 6 (RaptorQ).

[0331] In the FLUTE example, ETI information can be provided in the extended document delivery table. To provide relevant ETI information to the CMMF receiver, a file including the extended document delivery table can be created to signal the ETI information to the receiver, as follows:

[0332] • Attributes and elements from RFC 6726 can be reused as follows:

[0333] The expiration attribute indicates the validity of the FDT instance, i.e., the duration of the binding validity.

[0334] When the complete attribute is true, the signaling for this "FDT instance" includes a set of "file" entries, which exhausts the file sets delivered so far as well as the file sets to be provided in the session.

[0335] ○ The following data can be set as default values ​​at the instance level or at the file level. If set at the file level, the instance level information will be overwritten:

[0336] ■ Includes the Content-Type attribute to express the type of the delivered file according to RFC2616.

[0337] ■ Includes the Content-Encoding attribute to indicate the encoding of the delivered file. For details, refer to RFC6726.

[0338] The FEC-OTI-FEC-Encoding-ID attribute provides the "FEC Encoding ID (FECEncoding ID)" object as defined in [RFC5052] for sending information elements.

[0339] The FEC-OTI-FEC-Instance-ID attribute provides the FEC instance ID (FECInstance ID) object sending information element as defined in [RFC5052].

[0340] ■ If required by the FEC scheme, the FEC-OTI-Maximum-Source-Block-Length property provides the “Maximum-Source-Block-Length” object as defined in [RFC5052] to send information elements.

[0341] ■ If required by the FEC scheme, the FEC-OTI-Max-Number-of-Encoding-Symbols property provides the “Max-Number-of-Encoding-Symbols” object as defined in [RFC5052] to send information elements.

[0342] ■ If required by the FEC scheme, the FEC-OTI-Max-Number-of-Encoding-Symbols property provides the “Max-Number-of-Encoding-Symbols” object as defined in [RFC5052] to send information elements.

[0343] ○The TOI value is a positive integer to represent the transfer object identifier.

[0344] ○ The Content-Location attribute is used for purposes defined in [RFC2616].

[0345] The Content-Length attribute is used for purposes defined in [RFC2616].

[0346] If the file is content-encoded before transmission (and therefore uses the "Content-Encoding" attribute), the Transfer-Length attribute is used to carry the transmission length. For example, if compression is applied before transmission to reduce the number of octets required for transmission, the transmission length will be roughly different.

[0347] The Content-MD5 attribute is used for purposes defined in [RFC2616].

[0348] • To access content, you can define the following new elements and attributes.

[0349] ○ A list of access objects assigned to the file, each defined as follows:

[0350] ■Access-URL of the accessible object.

[0351] ■ Provides the type attribute for the type of the transmitted object.

[0352] ■ The Included-Symbols property provides a list of symbols included in the encoded object. It, along with the interleaving type, provides how symbols are added.

[0353] ■ The Independent-Object property, if set to true, ensures that the file contains enough information to recover all files included in this repair object.

[0354] ■ The availability time window, expressed by the availabilityStartTime and availabilityEndTime properties, indicates when the URL is available.

[0355] The following provides an example of an extended FDT scheme that can be used to resolve such files:

[0356] <?xml version="1.0" encoding="UTF-8"?>

[0357] <xs:schema

[0358] xmlns="urn:ETSI:CMMF:2023:EFD"

[0359] xmlns:xs="http: / / www.w3.org / 2001 / XMLSchema"

[0360] targetNamespace="urn:ETSI:CMMF:2023:EFD"

[0361] elementFormDefault="qualified"

[0362] version="1">

[0363] <xs:element name="FDT-Instance" type="FDT-InstanceType" / >

[0364] <xs:complextype name="FDT-InstanceType">

[0365] <xs:sequence>

[0366] <xs:element name="File" type="FileType" maxOccurs="unbounded" / >

[0367] <xs:element name="schemaVersion" type="xs:unsignedInt" / >

[0368] <xs:element name="delimiter" type="DelimiterType" / >

[0369] <xs:any namespace="##other" processContents="skip" minOccurs="0" maxOccurs="unbounded" / >

[0370] < / xs:sequence>

[0371] <xs:attribute name="Expires" type="xs:dateTime" use="required" / >

[0372] <xs:attribute name="Complete" type="xs:boolean" use="optional" / >

[0373] <xs:attribute name="Content-Type" type="xs:string" use="optional" / >

[0374] <xs:attribute name="Content-Encoding" type="xs:string" use="optional" / >

[0375] <xs:attribute name="FEC-OTI-FEC-Encoding-ID" type="xs:unsignedLong" use="optional" / >

[0376] <xs:attribute name="FEC-OTI-FEC-Instance-ID" type="xs:unsignedLong" use="optional" / >

[0377] <xs:attribute name="FEC-OTI-Maximum-Source-Block-Length" type="xs:unsignedLong" use="optional" / >

[0378] <xs:attribute name="FEC-OTI-Encoding-Symbol-Length" type="xs:unsignedLong" use="optional" / >

[0379] <xs:attribute name="FEC-OTI-Max-Number-of-Encoding-Symbols" type="xs:unsignedLong" use="optional" / >

[0380] <xs:attribute name="FEC-OTI-Scheme-Specific-Info" type="xs:base64Binary" use="optional" / >

[0381] <xs:anyattribute processContents="skip" / >

[0382] < / xs:complextype>

[0383] <xs:complextype name="FileType">

[0384] <xs:sequence>

[0385] <xs:element name="EncodedObjects" type="EncodedObjectType" minOccurs="1" maxOccurs="unbounded" / >

[0386] <xs:element name="delimiter" type="DelimiterType" / >

[0387] <xs:any namespace="##other" processContents="skip" minOccurs="0" maxOccurs="unbounded" / >

[0388] < / xs:sequence>

[0389] <xs:attribute name="Content-Location" type="xs:anyURI" use="required" / >

[0390] <xs:attribute name="TOI" type="xs:positiveInteger" use="required" / >

[0391] <xs:attribute name="Content-Length" type="xs:unsignedLong" use="optional" / >

[0392] <xs:attribute name="Transfer-Length" type="xs:unsignedLong" use="optional" / >

[0393] <xs:attribute name="Content-Type" type="xs:string" use="optional" / >

[0394] <xs:attribute name="Content-Encoding" type="xs:string" use="optional" / >

[0395] <xs:attribute name="Content-MD5" type="xs:base64Binary" use="optional" / >

[0396] <xs:attribute name="FEC-OTI-FEC-Encoding-ID" type="xs:unsignedLong" use="optional" / >

[0397] <xs:attribute name="FEC-OTI-FEC-Instance-ID" type="xs:unsignedLong" use="optional" / >

[0398] <xs:attribute name="FEC-OTI-Maximum-Source-Block-Length" type="xs:unsignedLong" use="optional" / >

[0399] <xs:attribute name="FEC-OTI-Encoding-Symbol-Length" type="xs:unsignedLong" use="optional" / >

[0400] <xs:attribute name="FEC-OTI-Max-Number-of-Encoding-Symbols" type="xs:unsignedLong" use="optional" / >

[0401] <xs:attribute name="FEC-OTI-Scheme-Specific-Info" type="xs:base64Binary" use="optional" / >

[0402] <xs:anyattribute processContents="skip" / >

[0403] < / xs:complextype>

[0404] <xs:complextype name="EncodedObjectType">

[0405] <xs:simplecontent>

[0406] <xs:extension base="xs:anyURI">

[0407] <xs:attribute name="interleaving" type="interleavingType" / >

[0408] <xs:attribute name="includedSymbols" type="xs:string" / >

[0409] <xs:attribute name="objectType" type="ObjectTypeType" default="source" / >

[0410] <xs:attribute name="independentObject" type="xs:boolean" default="false" / >

[0411] <xs:attribute name="availabilityStartTime" type="xs:dateTime" / >

[0412] <xs:attribute name="availabilityEndTime" type="xs:dateTime" / >

[0413] <xs:anyattribute namespace="##other" processContents="skip" / >

[0414] < / xs:extension>

[0415] < / xs:simplecontent>

[0416] <xs:anyattribute processContents="skip" / >

[0417] < / xs:complextype>

[0418] <xs:simpletype name="DelimiterType">

[0419] <xs:restriction base="xs:byte" / >

[0420] < / xs:simpletype>

[0421] <xs:simpletype name="ObjectTypeType">

[0422] <xs:restriction base="xs:string">

[0423] <xs:enumeration value="source" / >

[0424] <xs:enumeration value="self-contained" / >

[0425] <xs:enumeration value="partial" / >

[0426] < / xs:restriction>

[0427] < / xs:simpletype>

[0428] <xs:simpletype name="interleavingType">

[0429] <xs:restriction base="xs:string">

[0430] <xs:enumeration value="symbol-list" / >

[0431] <xs:enumeration value="list-sbn" / >

[0432] <xs:enumeration value="sequential-sbn" / >

[0433] <xs:enumeration value="interleaved" / >

[0434] <xs:enumeration value="spread" / >

[0435] < / xs:restriction>

[0436] < / xs:simpletype>

[0437]

[0438] An Extended FDT Description (EFD) is a file that can contain CMMF client access objects and provide metadata required for a CMMF session. An EFD can be an XML file formatted according to the XML scheme provided above.

[0439] EFD can be configured such that, after removing XML attributes or elements from an XML scheme not specified above within the EFD namespace, the result is a valid XML file formatted according to that scheme and conforming to this disclosure. Additionally, EFD can be configured such that, after removing XML attributes or elements from namespaces other than the EFD namespace, the result is a valid XML file formatted according to that scheme and conforming to this file. If a CMMF client removes all XML attributes and elements from this namespace and other namespaces not specified above within the XML scheme from the EFD, the EFD will produce a valid XML file conforming to that file.

[0440] The MIME type definition for EFD files is as follows. The encoding of the EFD can be UTF-8 as defined in IETF RFC 3629. All data provided in the extended namespace can be UTF-8 as defined in IETF RFC 3629. If binary data needs to be added, it can be included in Base64 within a UTF-8 encoded element with the appropriate namespace or identifier, as described in IETF RFC 4648, so that the XML parser knows how to process or ignore the binary data.

[0441] If the EFD is delivered via HTTP, the EFD file may be transport-encoded for transmission, as described in IETF RFC 7230.

[0442] IANA registration for EFD might look like this. MIME types and subtypes can be defined as follows:

[0443] • MIME Media Type Name: Application

[0444] • MIME subtype name: cmmf-efd+xml

[0445] • Required parameters: None

[0446] • Optional parameters: None.

[0447] • Encoding considerations: UTF-8

[0448] • Security Considerations: EFDs contain references to other resources. They are decoded using XML, and there is a risk that intentionally malformed XML could lead to security issues. Additionally, EFDs can be created that allow receiving clients to access other resources; if widely distributed, this could be used to launch denial-of-service attacks.

[0449] The EFD format does not incorporate any active or executable content. However, EFDs may reference other forms of material from external sources, and this material may contain active or executable content. Such material is expected to be identified using its own MIME type, and security considerations for that format should be taken into account.

[0450] If operating in an insecure environment and required by the content / service provider, the confidentiality of EFD elements and attributes can be protected by encrypting them using the syntax and processing rules specified in the W3C recommendation "XML Encryption Syntax and Processing".

[0451] If operating in an insecure environment and required by the content / service provider, the digital signature and verification process specified in W3C Recommendation “XML Signature Syntax and Processing” can be used to protect the authenticity and integrity of EFD’s data source.

[0452] • Interoperability considerations: The specification definition document is a platform-independent expression and is designed to achieve broad interoperability.

[0453] • Published Specifications: ETSI TS XXXX

[0454] • Applications using this media type: various

[0455] •Additional Information:

[0456] • File extension: efd

[0457] •Intended use: Public

[0458] • Other information / general comments: None

[0459] • Author / Change Controller: ETSI

[0460] The format of a transport object may include an encoded object format using a well-defined ALC payload format and some additional headers, and may carry only the record's repair symbol. Example definitions for three different types of transport objects are as follows:

[0461] • The source transfer object, i.e., the unmodified source object.

[0462] • Partially encoded objects, that is, objects that only contain encoded symbols but are insufficient to recover the object.

[0463] • Self-contained encoded objects, which are objects that include encoded symbols and are sufficient to recover the object and all metadata.

[0464] If the EFD signaling is of type "Source", the object referenced in the Access-URL can be the original source object. If the application can also access the object at the original location, the value of Access-URL can be the same as the value of Content-Location. The source object, as a whole or in part, can be used by the CMMF receiver to reconstruct the application object, possibly combined with the encoded object.

[0465] The following example syntax and semantics provide details on collecting symbols in a data structure. Table 2 provides a summary of the syntax for encoding symbols, including the indexing and mapping of symbols and the set of decoded symbols.

[0466] Table 2—Encoding Symbol Syntax

[0467]

[0468] Table 3 below provides the syntax for the coded symbol header. Different header structures are defined to provide the order and details of the included symbols from different source blocks, referring to their source block number (SBN) and coded symbol ID (ESI). Specifically:

[0469] • Table 4 lists the encoded symbol headers that provide SBN and ESI separately for each symbol in the header.

[0470] • Table 5 lists the encoded symbol headers for each symbol individually included in the header of a source block having a specified SNB.

[0471] Table 6 lists the index of the first symbol and the number of symbols in the encoded symbol header for a source block with a specified SNB. The included ESIs then go from first_esi to first_esi + number_symbols – 1.

[0472] Table 7 lists the encoded symbol headers for the interleaved version of the source block. It provides the indices of the first symbol and the first SBN, as well as the number of symbols.

[0473] • Table 8 lists the coded symbol headers for the scattered versions of the source symbols, where the first source block is first scattered, followed by the expansion of the second source block, and so on.

[0474] Table 3: Encoded Symbol Header

[0475]

[0476] Table 4: List of Encoded Symbol Headers

[0477]

[0478] Table 5: List of encoded symbol headers for a source block

[0479]

[0480] Table 6: List of Encoded Symbol Headers with Sequential ESI

[0481]

[0482] Table 7: Encoded symbol headers with interleaved ESI

[0483]

[0484] Table 8: Encoded symbol headers with distributed ESI

[0485]

[0486] Partially encoded objects can be CMMF objects that include symbols associated with the source object. Information related to the object can be carried in the EFD. Table 9 shows example syntax for partially encoded objects.

[0487] Table 9: Partial Encoded Objects

[0488]

[0489] The encoded header provides EFD information in binary format. This information may include FEC OTI information and file-related information. Strings can be encoded using utf8(), referring to UTF-8 strings as defined in RFC 3629, and terminated with null. Table 10 below shows example encoded header syntax.

[0490] Table 10: Encoded symbol headers with interleaved ESI

[0491]

[0492] A self-contained encoded object can be an object that allows the recovery of the source object and associated metadata from the object itself. Thus, a self-contained encoded object can include synchronization information, an encoded header, and encoded symbols, thereby including enough encoded symbols to make the object recoverable. Table 11 shows example syntax for self-contained encoded objects.

[0493] Table 11: Self-contained encoding objects

[0494]

[0495] The example provided is as follows. In the first example, a single file including source objects and partially encoded objects is provided. In this example, a single file is provided that is an MP4 file compatible with a certain MIME type, and additionally, for this file, three partially repaired objects are provided, each containing 500 symbols of a single source block. The file is only partial, and the information is provided in the extended FDT. The FDT is complete, i.e., no new information is provided.

[0496] <?xml version="1.0" encoding="UTF-8"?>

[0497] <FDTInstance xmlns:xsi="http: / / www.w3.org / 2001 / XMLSchema-instance"

[0498] xmlns="urn:ETSI:CMMF:2023:FDT"

[0499] xsi:schemaLocation="urn:ETSI:CMMF:2023:FDTextendedFDT.xsd"

[0500] Expires="2024-05-30T09:30:10Z"

[0501] Complete="true"

[0502] ContentType="video / mp4 codecs='avc1.42c01e,mp4a.40.29'profiles='iso8'"

[0503] FEC-Encoding-ID="6"

[0504] FEC-Encoding-Symbol-Length="64">

[0505] <File ContentLocation="https: / / example.com / efd1.mp4"

[0506] TOI="0"

[0507] Content-Length="64000">

[0508] <EncodedObjects type="source"

[0509] independentObject="true">https: / / example.com / efd1.mp4

[0510] <EncodedObjects type="partial"

[0511] interleavingType="sequential-sbn"

[0512] includedSymbols="500,0,1001">https: / / example.com / part1.cmf

[0513] <EncodedObjects type="partial"

[0514] interleavingType="sequential-sbn"

[0515] includedSymbols="500,0,1501">https: / / example.com / part2.cmf

[0516] <EncodedObjects type="partial"

[0517] interleavingType="sequential-sbn"

[0518] includedSymbols="500,0,2001">https: / / example.com / part3.cmf

[0519]

[0520]

[0521] As another example, multiple files can be provided, including self-contained objects containing source symbols. In this example, two files are provided, both encoded using RaptorQ and with a symbol length of 64. In this case, only self-contained objects are provided, and such symbols are scattered such that symbols 0, 3, 6… are contained in one file, 1, 4, 7… are included in another file, and 2, 5, 8 are included in a file not yet included in the other. Each encoded object is self-contained and can represent a file.

[0522] <?xml version="1.0" encoding="UTF-8"?>

[0523] <FDTInstance xmlns:xsi="http: / / www.w3.org / 2001 / XMLSchema-instance"

[0524] xmlns="urn:ETSI:CMMF:2023:FDT"

[0525] xsi:schemaLocation="urn:ETSI:CMMF:2023:FDTextendedFDT.xsd"

[0526] Expires="2024-05-30T09:30:10Z"

[0527] Complete="true"

[0528] FEC-OTI-FEC-Encoding-ID="6"

[0529] FEC-OTI-Encoding-Symbol-Length="64">

[0530] <File ContentLocation="https: / / example.com / efd1-video.mp4"

[0531] ContentType="video / mp4 codecs='avc1.42c01e' profiles='iso8'"

[0532] TOI="0"

[0533] Content-Length="64000">

[0534] <EncodedObjects type="self-contained"

[0535] interleavingType="spread"

[0536] independentObject="true"

[0537] includedSymbols="1001,3,1,0,0,0">https: / / example.com / part1-video.cmf

[0538] <EncodedObjects type="self-contained"

[0539] interleavingType="spread"

[0540] independentObject="true"

[0541] includedSymbols="1001,3,1,0,0,1">https: / / example.com / part2-video.cmf

[0542] <EncodedObjects type="self-contained"

[0543] interleavingType="spread"

[0544] independentObject="true"

[0545] includedSymbols="1001,3,1,0,0,2">https: / / example.com / part3-video.cmf

[0546]

[0547] <File ContentLocation="https: / / example.com / efd1-audio.mp4"

[0548] ContentType="audio / mp4 codecs='mp4a.40.29' profiles='iso8'"

[0549] TOI="1"

[0550] Content-Length="4800">

[0551] <EncodedObjects type="self-contained"

[0552] interleavingType="spread"

[0553] independentObject="true"

[0554] includedSymbols="80,3,1,0,0,0">https: / / example.com / part1-audio.cmf

[0555] <EncodedObjects type="self-contained"

[0556] interleavingType="spread"

[0557] independentObject="true"

[0558] includedSymbols="80,3,1,0,0,1">https: / / example.com / part2-audio.cmf

[0559] <EncodedObjects type="self-contained"

[0560] interleavingType="spread"

[0561] independentObject="true"

[0562] includedSymbols="80,3,1,0,0,2">https: / / example.com / part3-audio.cmf

[0563]

[0564]

[0565] In another example, real-time streaming is used. In this example, an initialization segment / CMAF header is provided as four segments, two for video and two for audio. The encoding is preset to RaptorQ with a symbol length of 64. The EFD will expire and needs to be reloaded. The file records availability times. Media segments are provided in both raw and encoded versions. For video, a source object and three part objects are provided. For audio, a source object and a single part object are provided. The example also shows optimizations that can be made, such as:

[0566] • Combine information about the source object and the encoded object into the streaming list.

[0567] • Template information to avoid listing URLs

[0568] • Preset at hierarchical levels

[0569] • Provides the ability to operate using populate and difference updates.

[0570] <?xml version="1.0" encoding="UTF-8"?>

[0571] <FDTInstance xmlns:xsi="http: / / www.w3.org / 2001 / XMLSchema-instance"

[0572] xmlns="urn:ETSI:CMMF:2023:FDT"

[0573] xsi:schemaLocation="urn:ETSI:CMMF:2023:FDTextendedFDT.xsd"

[0574] Expires="2023-08-08T9:30:18Z"

[0575] Complete="false"

[0576] FEC-OTI-FEC-Encoding-ID="6"

[0577] FEC-OTI-Encoding-Symbol-Length="64">

[0578] <File ContentLocation="https: / / example.com / video / init-video.mp4"

[0579] ContentType="video / mp4 codecs='avc1.42c01e' profiles='cmfc'"

[0580] TOI="0"

[0581] Content-Length="2000">

[0582] <EncodedObjects type="source"

[0583] availabilityStartTime="2023-08-08T9:30:10Z">https: / / example.com / video / init-video.mp4

[0584]

[0585] <File ContentLocation="https: / / example.com / init-audio.mp4"

[0586] ContentType="audio / mp4 codecs='mp4a.40.29' profiles='cmfc'"

[0587] TOI="1"

[0588] Content-Length="1500">

[0589] <EncodedObjects type="source"

[0590] availabilityStartTime="2023-08-08T9:30:10Z">https: / / example.com / video / init-audio.mp4

[0591]

[0592] <File ContentLocation="https: / / example.com / video / video001.m4s"

[0593] ContentType="video / m4s"

[0594] TOI="2"

[0595] Content-Length="12000">

[0596] <EncodedObjects type="source"

[0597] independentObject="true"

[0598] availabilityStartTime="2023-08-08T9:30:10Z">https: / / example.com / video / video001.m4s

[0599] <EncodedObjects type="partial"

[0600] interleavingType="sequential-sbn"

[0601] includedSymbols="100,0,188"

[0602] availabilityStartTime="2023-08-08T9:30:10Z">https: / / example.com / video / video001-1.cmf

[0603] <EncodedObjects type="partial"

[0604] interleavingType="sequential-sbn"

[0605] includedSymbols="100,0,288"

[0606] availabilityStartTime="2023-08-08T9:30:10Z">https: / / example.com / video / video001-2.cmf

[0607] <EncodedObjects type="partial"

[0608] interleavingType="sequential-sbn"

[0609] includedSymbols="100,0,388"

[0610] availabilityStartTime="2023-08-08T9:30:10Z">https: / / example.com / video / video001-3.cmf

[0611]

[0612] <File ContentLocation="https: / / example.com / audio / audio001.m4s"

[0613] ContentType="audio / m4s"

[0614] TOI="3"

[0615] Content-Length="2000">

[0616] <EncodedObjects type="source"

[0617] independentObject="true"

[0618] availabilityStartTime="2023-08-08T9:30:10Z">https: / / example.com / audio / audio001.m4s

[0619] <EncodedObjects type="partial"

[0620] interleavingType="sequential-sbn"

[0621] includedSymbols="32,0,32"

[0622] availabilityStartTime="2023-08-08T9:30:10Z">https: / / example.com / audio / audio001-1.cmf

[0623]

[0624] <File ContentLocation="https: / / example.com / video / video002.m4s"

[0625] ContentType="video / m4s"

[0626] TOI="4"

[0627] Content-Length="14000">

[0628] <EncodedObjects type="source"

[0629] independentObject="true"

[0630] availabilityStartTime="2023-08-08T9:30:14Z">https: / / example.com / video / video002.m4s

[0631] <EncodedObjects type="partial"

[0632] interleavingType="sequential-sbn"

[0633] includedSymbols="100,0,219"

[0634] availabilityStartTime="2023-08-08T9:30:14Z">https: / / example.com / video / video001-1.cmf

[0635] <EncodedObjects type="partial"

[0636] interleavingType="sequential-sbn"

[0637] includedSymbols="100,0,319"

[0638] availabilityStartTime="2023-08-08T9:30:14Z">https: / / example.com / video / video001-2.cmf

[0639] <EncodedObjects type="partial"

[0640] interleavingType="sequential-sbn"

[0641] includedSymbols="100,0,419"

[0642] availabilityStartTime="2023-08-08T9:30:14Z">https: / / example.com / video / video001-3.cmf

[0643]

[0644] <File ContentLocation="https: / / example.com / audio / audio001.m4s"

[0645] ContentType="audio / m4s"

[0646] TOI="3"

[0647] Content-Length="2000">

[0648] <EncodedObjects type="source"

[0649] independentObject="true"

[0650] availabilityStartTime="2023-08-08T9:30:10Z">https: / / example.com / audio / audio001.m4s

[0651] <EncodedObjects type="partial"

[0652] interleavingType="sequential-sbn"

[0653] includedSymbols="32,0,32"

[0654] availabilityStartTime="2023-08-08T9:30:10Z">https: / / example.com / audio / audio001-1.cmf

[0655]

[0656] <File ContentLocation="https: / / example.com / audio / audio002.m4s"

[0657] ContentType="audio / m4s"

[0658] TOI="3"

[0659] Content-Length="3000">

[0660] <EncodedObjects type="source"

[0661] independentObject="true"

[0662] availabilityStartTime="2023-08-08T9:30:14Z">https: / / example.com / audio / audio002.m4s

[0663] <EncodedObjects type="partial"

[0664] interleavingType="sequential-sbn"

[0665] includedSymbols="48,0,48"

[0666] availabilityStartTime="2023-08-08T9:30:14Z">https: / / example.com / audio / audio002-2.cmf

[0667]

[0668]

[0669] Potential receiver operations may be as follows. After receiving an EFD, the CMMF receiver can access different versions of the same source object. Details are implementation-specific. However, some potential operations are provided below.

[0670] If multiple encoded objects are available for a single source object, the CMMF receiver can download multiple transmitted objects in parallel until a sufficient set of symbols is received to recover the source object. Downloading can be terminated once a sufficient set of symbols is available. Determining a sufficient set depends on the FEC code used, but for RaptorQ, a sufficient set for each source block is K+2 independent source symbols, where K is the source block size.

[0671] If the objects are available in a scatter format that includes the source symbols, all objects can be downloaded in parallel, and the objects can be gradually recovered from the beginning using system symbols.

[0672] If the source object is available and has been partially downloaded, the CMMF receiver can convert the available partial data into the correct source symbol and use the received repair symbol to recover the entire object.

[0673] The extended deployment options provide RFC 5052-based instantiation of the CMMF framework as defined in Clauses 4.2 through 4.4. Instantiation can be aligned to RFC 5052 and some concepts defined in the ROUTE, such as those defined in RFC 923, can be reused.

[0674] • For ETI, define a further extended XML-based manifest that mimics the data model for ROUTE and is defined above, but builds upon the above scheme.

[0675] • Multiple transport sessions can be used.

[0676] • The encoded object is compatible with the ALC payload format with some additional headers and carries only the repair symbols as documented above.

[0677] • Super objects can be used.

[0678] • Encoded object types as defined above can be reused.

[0679] • The FEC code ID can be set to 6 (RaptorQ).

[0680] Delivery based on a transport session can be provided in the transport session description (TSD). The transport session description (TSD) is a file containing CMMF client access objects and the metadata required to provide a CMMF session. The TSD is an XML file that should be formatted according to the XML scheme provided above. The XML scheme for the TSD might be as follows:

[0681] <?xml version="1.0" encoding="UTF-8"?>

[0682] <xs:schema

[0683] xmlns="urn:ETSI:CMMF:2023:TSD"

[0684] xmlns:xs="http: / / www.w3.org / 2001 / XMLSchema"

[0685] targetNamespace="urn:ETSI:CMMF:2023:TSD"

[0686] elementFormDefault="qualified"

[0687] version="1">

[0688] <xs:element name="TSD" type="TSDType" / >

[0689] <xs:complextype name="TSDType">

[0690] <xs:sequence>

[0691] <xs:element name="SourceFlow"

[0692] type="SourceFlowType" minOccurs="1" maxOccurs="unbounded" / >

[0693] <xs:element name="RepairFlow"

[0694] type="RepairFlowType" minOccurs="1" maxOccurs="unbounded" / >

[0695] <xs:element name="delimiter" type="DelimiterType" / >

[0696] <xs:any namespace="##other" processContents="skip" minOccurs="0" maxOccurs="unbounded" / >

[0697] < / xs:sequence>

[0698] <xs:attribute name="Content-Location" type="xs:anyURI" use="optional" / >

[0699] <xs:attribute name="Complete" type="xs:boolean"

[0700] use="optional" default="true" / >

[0701] <xs:attribute name="Expires" type="xs:dateTime"

[0702] use="optional" / >

[0703] < / xs:complextype>

[0704] <xs:complextype name="SourceFlowType">

[0705] <xs:choice>

[0706] <xs:element name="Object" type="ObjectType" minOccurs="0" maxOccurs="unbounded" / >

[0707] <xs:element name="Representation" type="RepresentationType" minOccurs="0" maxOccurs="1" / >

[0708] <xs:element name="delimiter" type="DelimiterType" / >

[0709] <xs:any namespace="##other" processContents="skip" minOccurs="0" maxOccurs="unbounded" / >

[0710] < / xs:choice>

[0711] <xs:attribute name="TSI" type="xs:positiveInteger"

[0712] use="required" / >

[0713] < / xs:complextype>

[0714] <xs:complextype name="ObjectType">

[0715] <xs:sequence>

[0716] <xs:element name="Access-URL" type="xs:anyURI" minOccurs="1" maxOccurs="unbounded" / >

[0717] <xs:element name="delimiter" type="DelimiterType" / >

[0718] <xs:any namespace="##other" processContents="skip" minOccurs="0" maxOccurs="unbounded" / >

[0719] < / xs:sequence>

[0720] <xs:attribute name="Content-Location" type="xs:anyURI" use="required" / >

[0721] <xs:attribute name="TOI" type="xs:positiveInteger" use="required" / >

[0722] <xs:attribute name="Content-Length" type="xs:unsignedLong" use="optional" / >

[0723] <xs:attribute name="Transfer-Length" type="xs:unsignedLong" use="optional" / >

[0724] <xs:attribute name="Content-Type" type="xs:string" use="optional" / >

[0725] <xs:attribute name="Content-Encoding" type="xs:string" use="optional" / >

[0726] <xs:attribute name="Content-MD5" type="xs:base64Binary" use="optional" / >

[0727] <xs:anyattribute processContents="skip" / >

[0728] < / xs:complextype>

[0729] <xs:complextype name="RepresentationType">

[0730] <xs:simplecontent>

[0731] <xs:extension base="xs:anyURI">

[0732] <xs:attribute name="firstToi" type="xs:unsignedLong" / >

[0733] <xs:anyattribute namespace="##other" processContents="skip" / >

[0734] < / xs:extension>

[0735] < / xs:simplecontent>

[0736] <xs:anyattribute processContents="skip" / >

[0737] < / xs:complextype>

[0738] <xs:complextype name="RepairFlowType">

[0739] <xs:choice>

[0740] <xs:element name="RepairObject"

[0741] type="RepairObjectType" minOccurs="0" maxOccurs="unbounded" / >

[0742] <xs:element name="RepairTemplate" type="RepairTemplateType" minOccurs="0" maxOccurs="1" / >

[0743] <xs:element name="delimiter" type="DelimiterType" / >

[0744] <xs:any namespace="##other" processContents="skip" minOccurs="0" maxOccurs="unbounded" / >

[0745] < / xs:choice>

[0746] <xs:attribute name="TSI" type="xs:positiveInteger"

[0747] use="required" / >

[0748] <xs:attribute name="FEC-OTI-FEC-Encoding-ID" type="xs:unsignedLong" use="optional" / >

[0749] <xs:attribute name="FEC-OTI-FEC-Instance-ID" type="xs:unsignedLong" use="optional" / >

[0750] <xs:attribute name="FEC-OTI-Maximum-Source-Block-Length" type="xs:unsignedLong" use="optional" / >

[0751] <xs:attribute name="FEC-OTI-Encoding-Symbol-Length" type="xs:unsignedLong" use="optional" / >

[0752] <xs:attribute name="FEC-OTI-Max-Number-of-Encoding-

[0753] Symbols" type="xs:unsignedLong" use="optional" / >

[0754] <xs:attribute name="FEC-OTI-Scheme-Specific-Info" type="xs:base64Binary" use="optional" / >

[0755] < / xs:complextype>

[0756] <xs:complextype name="RepairObjectType">

[0757] <xs:simplecontent>

[0758] <xs:extension base="xs:anyURI">

[0759] <xs:attribute name="includedSourceTOI" type="xs:string" / >

[0760] <xs:attribute name="includedSymbols" type="xs:string" / >

[0761] <xs:attribute name="interleaving" type="xs:interleavingType" / >

[0762] <xs:attribute name="independentObject" type="xs:boolean" default="false" / >

[0763] <xs:attribute name="availabilityStartTime" type="xs:dateTime" / >

[0764] <xs:attribute name="availabilityEndTime" type="xs:dateTime" / >

[0765] <xs:anyattribute namespace="##other" processContents="skip" / >

[0766] < / xs:extension>

[0767] < / xs:simplecontent>

[0768] <xs:anyattribute processContents="skip" / >

[0769] < / xs:complextype>

[0770] <xs:complextype name="RepairTTemplateType">

[0771] <xs:simplecontent>

[0772] <xs:extension base="xs:anyURI">

[0773] <xs:attribute name="repairTOI"

[0774] type="xs:string" / >

[0775] <xs:attribute name="includedSymbols" type="xs:string" / >

[0776] <xs:attribute name="interleaving" type="xs:interleavingType" / >

[0777] <xs:attribute name="independentObject" type="xs:boolean" default="false" / >

[0778] <xs:attribute name="availabilityStartTime" type="xs:dateTime" / >

[0779] <xs:attribute name="availabilityEndTime" type="xs:dateTime" / >

[0780] <xs:anyattribute namespace="##other" processContents="skip" / >

[0781] < / xs:extension>

[0782] < / xs:simplecontent>

[0783] <xs:anyattribute processContents="skip" / >

[0784] < / xs:complextype>

[0785] <xs:simpletype name="DelimiterType">

[0786] <xs:restriction base="xs:byte" / >

[0787] < / xs:simpletype>

[0788] <xs:simpletype name="interleavingType">

[0789] <xs:restriction base="xs:string">

[0790] <xs:enumeration value="symbol-list" / >

[0791] <xs:enumeration value="list-sbn" / >

[0792] <xs:enumeration value="sequential-sbn" / >

[0793] <xs:enumeration value="interleaved" / >

[0794] <xs:enumeration value="spread" / >

[0795] < / xs:restriction>

[0796] < / xs:simpletype>

[0797]

[0798] The TSD should be designed such that, after removing XML attributes or elements from XML schemes not documented above within the TSD namespace, the result is a valid XML file formatted according to that scheme and conforming to this document. Additionally, the TSD should be designed such that, after removing XML attributes or elements from namespaces other than the TSD namespace, the result is a valid XML file formatted according to that scheme and conforming to this document. If a CMMF client removes all XML attributes and elements from this namespace and other namespaces not documented above within the XML scheme from the TSD, the TSD will generate a valid XML file conforming to that document.

[0799] The MIME type definition for TSD files is as follows. The encoding of the TSD should be UTF-8 as defined in IETF RFC 3629. All data provided in the extended namespace should be UTF-8 as defined in IETF RFC 3629. If binary data needs to be added, it should be included in Base64 within a UTF-8 encoded element with the appropriate namespace or identifier, as described in IETF RFC 4648, so that the XML parser knows how to process or ignore the binary data.

[0800] Delivery of TSDs is beyond the scope of this document. However, if a TSD is delivered via HTTP, the TSD file may be transport-encoded for transmission, as described in IETF RFC 7230.

[0801] A formal MIME type registration for TSD might look like this. MIME types and subtypes can be defined as follows:

[0802] • MIME Media Type Name: Application

[0803] • MIME subtype name: cmmf-tsd+xml

[0804] • Required parameters: None

[0805] • Optional parameters: None.

[0806] • Encoding considerations: UTF-8

[0807] • Security Considerations: TSDs contain references to other resources. They are decoded as XML, and there is a risk that deliberately misformatted XML could lead to security issues. Additionally, TSDs can be created that allow receiving clients to access other resources; if widely distributed, this could be used to launch denial-of-service attacks.

[0808] The TSD format does not incorporate any active or executable content. However, TSDs may reference other forms of material from external sources, and this material may contain active or executable content. Such material is expected to be identified using its own MIME type, and security considerations for that format should be taken into account.

[0809] If operating in an insecure environment and required by the content / service provider, the confidentiality of the TSD can be protected by encrypting the elements and attributes of the TSD using the syntax and processing rules specified in the W3C recommendation "XML Encryption Syntax and Processing".

[0810] If operating in an insecure environment and required by the content / service provider, the digital signature and verification process specified in W3C Recommendation “XML Signature Syntax and Processing” can be used to protect the authenticity and integrity of EFD’s data source.

[0811] • Interoperability considerations: The specification definition document is a platform-independent expression and is designed to achieve broad interoperability.

[0812] • Published Specifications: ETSI TS XXXX

[0813] • Applications using this media type: various

[0814] •Additional Information:

[0815] • File extension: tsd

[0816] •Intended use: Public

[0817] • Other information / general comments: None

[0818] • Author / Change Controller: ETSI

[0819] The format of this instantiated encoded object can utilize a well-defined ALC payload format with some additional headers, and carries only the repair symbols of the records. Three different types of transport objects are defined:

[0820] • The source object being transferred, i.e., the unmodified source object.

[0821] • Partially encoded objects, that is, objects that only contain encoded symbols but are insufficient to recover the object.

[0822] • No self-contained transport object is defined, requiring TSD availability.

[0823] The mapping from the source object to the transport object is defined above using a super object.

[0824] The following describes additional example providers. An efficient DASH distribution example might look like this. Encoding can be preset to RaptorQ, where the symbol length is 64. It is assumed that the information is provided in the manifest and referenced. A repair object can be generated for each source object in the source objects, but a super object can be created for each of the first and third representations (e.g., video 1 and audio) and the second and third representations (e.g., video 2 and audio).

[0825] <?xml version="1.0" encoding="UTF-8"?>

[0826] <TSD xmlns:xsi="http: / / www.w3.org / 2001 / XMLSchema-instance"

[0827] xmlns="urn:ETSI:CMMF:2023:TSD"

[0828] xsi:schemaLocation="urn:ETSI:CMMF:2023:TSD TSD.xsd"

[0829] Expires="2024-05-30T09:30:10Z"

[0830] Complete="true"

[0831] Content-Location="http: / / www.example.com / my.tsd">

[0832] <sourceflow tsi="1">

[0833] <representation firsttoi="17"> https: / / example.com / dash.mpd#rep=1< / representation>

[0834] < / sourceflow>

[0835] <sourceflow tsi="2">

[0836] <representation firsttoi="17"> https: / / example.com / dash.mpd#rep=2< / representation>

[0837] < / sourceflow>

[0838] <sourceflow tsi="3">

[0839] <representation firsttoi="10"> https: / / example.com / dash.mpd#rep=3< / representation>

[0840] < / sourceflow>

[0841] <RepairFlow TSI="4"

[0842] FEC-OTI-FEC-Encoding-ID="6"

[0843] FEC-OTI-Encoding-Symbol-Length="64"

[0844] interleavingType="sequential-sbn"

[0845] includedSymbols="500,0,500"

[0846] repairTOI="TOI(TSI=1),TOI(TSI=3)">

[0847] <repairtemplate firsttoi="17"> https: / / example.com / repair_4_$TOI$< / repairtemplate>

[0848]

[0849] <RepairFlow TSI="5"

[0850] FEC-OTI-FEC-Encoding-ID="6"

[0851] FEC-OTI-Encoding-Symbol-Length="64"

[0852] interleavingType="sequential-sbn"

[0853] includedSymbols="500,0,1000"

[0854] repairTOI="TOI(TSI=1),TOI(TSI=3)">

[0855] <repairtemplate firsttoi="17"> https: / / example.com / repair_5_$TOI$< / repairtemplate>

[0856]

[0857] <RepairFlow TSI="6"

[0858] FEC-OTI-FEC-Encoding-ID="6"

[0859] FEC-OTI-Encoding-Symbol-Length="64"

[0860] interleavingType="sequential-sbn"

[0861] includedSymbols="500,0,1500"

[0862] repairTOI="TOI(TSI=1),TOI(TSI=3)">

[0863] <repairtemplate firsttoi="17"> https: / / example.com / repair_6_$TOI$< / repairtemplate>

[0864]

[0865] <RepairFlow TSI="7"

[0866] FEC-OTI-FEC-Encoding-ID="6"

[0867] FEC-OTI-Encoding-Symbol-Length="64"

[0868] interleavingType="sequential-sbn"

[0869] includedSymbols="500,0,500"

[0870] repairTOI="TOI(TSI=2),TOI(TSI=3)">

[0871] <repairtemplate firsttoi="17"> https: / / example.com / repair_7_$TOI$< / repairtemplate>

[0872]

[0873] <RepairFlow TSI="8"

[0874] FEC-OTI-FEC-Encoding-ID="6"

[0875] FEC-OTI-Encoding-Symbol-Length="64"

[0876] interleavingType="sequential-sbn"

[0877] includedSymbols="500,0,1000"

[0878] repairTOI="TOI(TSI=2),TOI(TSI=3)">

[0879] <repairtemplate firsttoi="17"> https: / / example.com / repair_8_$TOI$< / repairtemplate>

[0880]

[0881] <RepairFlow TSI="9"

[0882] FEC-OTI-FEC-Encoding-ID="6"

[0883] FEC-OTI-Encoding-Symbol-Length="64"

[0884] interleavingType="sequential-sbn"

[0885] includedSymbols="500,0,1500"

[0886] repairTOI="TOI(TSI=2),TOI(TSI=3)">

[0887] <repairtemplate firsttoi="17"> https: / / example.com / repair_9_$TOI$< / repairtemplate>

[0888]

[0889]

[0890] Figure 16 This is a conceptual diagram illustrating an example of a CMMF receiver 510 and decoder for a single application object. In this example, the CMMF receiver 510 includes a CMMF access client 512, a partial download unit 514, a symbol recovery unit 518, an FEC decoding unit 516, and an object recovery unit 520. According to the techniques of this disclosure, the CMMF access client 512 retrieves a file delivery manifest 500 (e.g., an extended FDT including ETI information, according to the techniques of this disclosure). The file delivery manifest 500 may include data indicating the location of the application object (e.g., application object 502) and the repair objects 504, 506. The location may be represented by a URL and may indicate a separate and different server (e.g., a separate CDN) from which each of the application object 502 and the repair objects 504, 506 can be obtained. The file delivery manifest 500 may further indicate that various application objects of the source stream can also be obtained from separate server devices / CDNs.

[0891] Therefore, the partial download unit 514 can retrieve portions of the segment in the form of application object 502 and repair objects 504, 506 in parallel. Even if the retrieved application object 502 is not a complete segment, in combination with repair objects 504, 506, the symbol recovery unit 518 and the FEC decoding unit 516 can fully recover the segment.

[0892] Figure 17 This is a conceptual diagram illustrating an example CMMF architecture. In this example, the encoded representation system and the FEC symbol / repair object are mixed. In this example, the CMMF transmitter 550 includes an FEC transport object generation unit 552, an FEC symbol generation unit 554, encoded metadata 552, and a packing unit 558. The FEC transport object generation unit 552 receives a source transport object 562, generates a source block, and provides the source block to the FEC symbol generation unit 554. The FEC symbol generation unit 554 forms encoded symbols based on the source block and provides the encoded symbols to the packing unit 558. The packing unit 558 uses the encoded symbols and encoded metadata 556 to form an encoded transport object 560.

[0893] Figure 18This is a conceptual diagram illustrating another example of a CMMF architecture. In this example, a source object is provided to the receiver, and only repair symbols are provided in the repaired transport object. In this example, the CMMF transmitter 580 includes an FEC transport object generation unit 582, an FEC symbol generation unit 584, encoded metadata 582, and a packing unit 588. The FEC transport object generation unit 582 receives a source transport object 592, generates a source block, and provides the source block to the FEC symbol generation unit 584. The FEC symbol generation unit 584 forms encoded symbols based on the source block and provides the encoded symbols to the packing unit 588. The packing unit 588 uses the encoded symbols and encoded metadata 586 to form an encoded transport object 590.

[0894] Figure 19 This is a conceptual diagram illustrating another CMMF architecture. Figure 19 The example indicates that it is allowed Figure 17 and Figure 18 This example illustrates two common transmission architectures. In this example, the CMMF transmitter 600 includes an FEC transport object generation unit 602, an FEC symbol generation unit 604, encoded metadata 602, and a packing unit 608. The FEC transport object generation unit 602 receives a source transport object 612, generates a source block, and provides the source block to the FEC symbol generation unit 604. The FEC symbol generation unit 604 forms encoded symbols based on the source block and provides the encoded symbols to the packing unit 608. The packing unit 608 uses the encoded symbols and encoded metadata 606 to form an encoded transport object 610.

[0895] Figure 20 This is a conceptual diagram 620 illustrating an example mapping 620 between Transport Object Identifiers (TOIs) and Transport Session Identifiers (TSIs) and URLs or source data in a file delivery manifest. For example, a file delivery manifest could be a File Delivery Table (FDT) or a template mechanism. A single application object can be associated with metadata and a URL. The byte range of a single application object can be associated with metadata and a URL. A single application object stream can include multiple objects, each of which can include associated metadata and a URL, as well as a Transport Object Identifier (TOI) identifying the sequence of objects. Multiple application object streams can each follow the principles of a single application stream and can be identified using a Transport Session Identifier (TSI).

[0896] Figure 21 This is a flowchart illustrating an example of static operation according to the technology disclosed herein. In this example, the application server provides configuration data and objects, as well as metadata, to the CMMF transmitter (630). The CMMF transmitter can then transmit a CMMF manifest file (including TOI and metadata) to the CMMF receiver (632). The CMMF transmitter may correspond to a request processing unit 70, and the application server may correspond to... Figure 1 Server device 60. The CMMF receiver can request one or more CMMF objects (e.g., repair objects). The CMMF receiver may correspond to retrieval unit 52, and the application may correspond to media application 112. Figure 2 ).

[0897] In this example, the CMMF receiver retrieves a CMMF object by sending a CMMF object request to the CMMF transmitter, and the CMMF transmitter sends the CMMF object to the CMMF receiver (634). The CMMF receiver recovers the object data according to the techniques of this disclosure (636), and then provides the object and its metadata to the application (638).

[0898] Figure 22 This is a flowchart illustrating an example of dynamic operation according to the technology disclosed herein. This may include continuous updates to the manifest file. In this example, the application server provides configuration data and a first object set, along with metadata (650), to the CMMF transmitter. The CMMF transmitter may then transmit a first CMMF manifest file (including TOIs and metadata) to the CMMF receiver (652). The CMMF transmitter may correspond to a request processing unit 70, and the application server may correspond to... Figure 1 Server device 60. The CMMF receiver can use a first manifest file to request a first CMMF object (654). The CMMF receiver may correspond to retrieval unit 52, and the application may correspond to media application 112. Figure 2 In this example, the CMMF receiver recovers object data for the first object according to the techniques of this disclosure (656), and then provides the first object and its metadata to the application (658).

[0899] Subsequently, the application server may provide a second CMMF object and metadata to the CMMF transmitter (660). The CMMF transmitter may then transmit the second CMMF manifest file to the CMMF receiver (662). The CMMF receiver may use the second CMMF manifest file to request the second CMMF object (664). The CMMF receiver may then recover the object data for the second object (666) and subsequently provide the second object to the application (668).

[0900] Figure 23 This is a conceptual diagram illustrating an example FLUTE transmission unit. In this example, the FLUTE transmission unit includes a padding unit 680, an intermediate symbol generation unit 628, an encoding unit 684, and a tuple generation unit 686.

[0901] Figure 24This is a conceptual diagram illustrating a route system including an example route receiver 710 and a route transmitter 700. In this example, the route receiver 710 includes a source object recovery unit 712 and a repair object recovery unit 714. The repair object recovery unit 714 recovers a repair object 716 and stores the repair object 716 in an application object cache 718. The route receiver 710 can provide the repair object from the application object cache 718 to a media player application.

[0902] Figure 25 This is a conceptual diagram illustrating an example FECFRAME protocol stack. In this example, the FECFRAME protocol stack for content delivery protocol 730 includes an application layer 732, RTP 734, an FEC framework 736, an RTP layer 750, a transport layer 738, and an IP layer 740. The FEC framework 736 represents data formed according to the techniques of this disclosure. The FEC framework 736 uses FEC scheme 742 to perform FEC decoding of media data objects. The RTP layer 750 further includes RTP processing 752 and RTP (de)multiplexing 754.

[0903] Figures 26A to 26C This is a conceptual diagram illustrating various example FEC groupings. Specifically, Figure 26A The example depicts the FEC payload ID format. The grouping includes the source block number (SBN) 760 and the encoded symbol ID (ESI) 762. Figure 26B An example encoded public FEC OTI for the RaptorQ FEC scheme is depicted. The packet includes a transmission length value (F) 7t64, a symbol size value (T) 768, and a reserved field 766. Figure 26C The example format for FEC objects to send messages, specific to the encoding scheme, is described. The groups include scheme-specific parameters Z 770, N 772, and A1 774 for implementation-specific optimization.

[0904] Figures 27A to 27C This is a conceptual diagram illustrating various example Raptor / Raptor Q code formats for FECFRAME. Figure 27A Example source FEC payload IDs are depicted, including SBN 780 and ESI 782. Figure 27B Example repair FEC payload IDs are depicted, including SBN 784, ESI 786, and source block length (SBL) value 788. Figure 27C The example FEC scheme-specific information format is described, including symbol size (T) value 790, maximum source block length (MSBL) value 792 and P value 794, and reserved field 796.

[0905] Figure 28This is a block diagram illustrating an example set of devices capable of performing the techniques of this disclosure. In this example, Figure 28 The description includes a content preparation device 800, a content delivery network 802A-802N (CDN 802), a network 804, and a client device 806. The content preparation device 800 can correspond to... Figure 1 Content preparation equipment 20 or Figure 21 and Figure 22 The application server. CDN 802 can correspond to Figure 1 Server devices 60, respectively corresponding to Figure 11 , Figure 12 , Figure 13 , Figure 17 , Figure 18 and Figure 19 CMMF transmitters 402, 422, 452, 550, 580, 600, or corresponding to Figure 21 and Figure 22 The CMMF transmitter. Client device 806 can correspond to... Figure 1 The client devices 40 correspond to respectively Figure 11 , Figure 12 and Figure 16 CMMF receivers 408, 428, and 510 correspond to Figure 21 and Figure 22 CMMF receivers and applications, or corresponding to Figure 24 The ROUTE receiver 710.

[0906] According to the technology disclosed herein, content preparation device 800 can prepare content to be streamed in the form of application objects and repair objects. Content preparation device 800 can store some application objects to, for example, CDN 802A, other application objects to, for example, CDN 802B, and repair objects to, for example, CDN 802N. In some examples, content preparation device 800 can store all application objects and / or all repair objects in each of CDN 802. For example, content preparation device 800 can store all application objects in CDN 802A and CDN 802B, and store repair objects in CDN 802N.

[0907] Content preparation device 800 can also construct manifest files indicating the locations of various application objects and repair objects. The manifest file may be or include enhanced FDTs as discussed above. Content preparation device 800 can construct manifest files to indicate source stream information and repair stream information, the source stream information indicating the location of application objects and the repair stream information indicating the location of repair objects. For example, the manifest file may indicate that application objects can be obtained from URLs associated with CDNs 802A and 802B, and repair objects can be obtained from URLs associated with CDN 802N. In some examples, content preparation device 800 may transmit the manifest file to client device 806 via network 804. In other examples, content preparation device 800 may store the manifest file in any or all of CDNs 802, and client device 806 may retrieve the manifest file from one of CDNs 802.

[0908] After retrieving the manifest file, client device 806 can determine the locations of application objects and repair objects. For example, client device 806 can determine, according to the manifest file, that application objects can be obtained from CDN 802A and 802B, and repair objects can be obtained from CDN 802N. Using this information, client device 806 can retrieve application objects from CDN 802A and 802B and repair objects from CDN 802N in parallel. As discussed above, client device 806 can use repair objects to repair incomplete application objects, i.e., application objects for which not all data has been retrieved. Therefore, client device 806 can simply retrieve sufficient application objects, rather than retrieving complete application objects from CDN 802A and 802B (for example), and combine them with the corresponding repair objects to restore the application objects to form a decorable segment of media data.

[0909] In this manner, client device 806 represents an example of a device for retrieving media data, comprising: a memory configured to store media data; and a processing system including a decoder implemented in circuitry, the processing system being configured to: retrieve a manifest file indicating a plurality of network locations for at least a first segment and a second segment of the media data, each of the plurality of network locations being hosted by a separate physical server device; retrieve at least a first portion of the first segment from the first network location; retrieve at least a second portion of the second segment from the second network location; and provide the at least first portion of the first segment and the at least second portion of the second segment to the decoder.

[0910] Similarly, content preparation device 800 represents an example of a device for transmitting media data, including: a memory configured to store media data; and a processing system implemented in circuitry and configured to: perform forward error correction (FEC) encoding on at least a first portion of a first segment to form a first FEC-encoded first portion of the first segment, and perform forward error correction (FEC) encoding on at least a second portion of a second segment to form a second FEC-encoded second portion of the second segment; store the first FEC-encoded first portion in a first physical server device; store the second FEC-encoded second portion in a second physical server device; and form a manifest file indicating that the first FEC-encoded first portion is available from the first physical server device and that the second FEC-encoded second portion is available from the second physical server device.

[0911] Figure 29 This is a flowchart illustrating an example method for transmitting media data according to the technology of this disclosure. Figure 29 Methods about Figure 28 The content preparation device 800 and CDN 802 are used for interpretation. However, other devices can be configured to perform this method or similar methods as discussed above.

[0912] Initially, the content preparation device 800 can generate a manifest file (820) for media rendering. Media rendering may include various different adaptive sets, representations, and segments (e.g., according to...). Figure 4 (Video files). Content preparation device 800 can perform FEC encoding on the segments to form application objects and repair objects. Content preparation device 800 can transmit the first segment application object to the first CDN (822), for example... Figure 28 The CDN 802A. Content preparation device 800 can transmit the first segment repair object to the second CDN (824), for example... Figure 28 The CDN 802N. Content preparation device 800 can transmit the second segment application object to the third CDN (826), for example. Figure 28 The CDN 802B. Content preparation device 800 can also transmit the second segment repair object to a fourth CDN (828), such as CDN 802N or... Figure 28 Another CDN is not shown. Content preparation device 800 can indicate the location of various application objects in the manifest file. Content preparation device 800 can transmit the manifest file to the client device (830), for example, directly or via one or more of CDNs 802.

[0913] In this way, Figure 29The method represents an example of a method for transmitting media data, comprising: performing forward error correction (FEC) encoding on at least a first portion of a first segment to form a first FEC-encoded first portion of the first segment, and performing forward error correction (FEC) encoding on at least a second portion of a second segment to form a second FEC-encoded second portion of the second segment; storing the first FEC-encoded first portion in a first physical server device; storing the second FEC-encoded second portion in a second physical server device; and forming a manifest file indicating that the first FEC-encoded first portion is available from the first physical server device and that the second FEC-encoded second portion is available from the second physical server device.

[0914] Figure 30 This is a flowchart illustrating an example method for retrieving media data according to the technology of this disclosure. For illustrative purposes, Figure 30 The method can be about Figure 28 This is interpreted by client devices 806 and CDN 802. However, other devices can be configured to perform this method or a similar method as explained above.

[0915] Initially, client device 806 retrieves a manifest file (850) for media rendering. The manifest file may indicate the location of application objects and repair objects for segments intended for media rendering. The manifest file may also indicate the adaptive set and representation corresponding to the segment. Therefore, client device 806 may selectively retrieve the appropriate representation for media rendering of the segment from it. Client device 806 may then retrieve the application object for the first segment from the first CDN (852), for example, Figure 28 CDN 802A. Client device 806 can retrieve the first segment repair object (854) from the second CDN, for example. Figure 28 CDN 802N. Client device 806 can retrieve the second segment application object (856) from a third CDN, such as CDN 802B. Client device 806 can retrieve the second segment repair object (858) from a fourth CDN, such as CDN 802N or Figure 28 Another CDN not shown in the diagram.

[0916] Client device 806 can use application objects and repair objects to reconstruct the first segment and the second segment (860). Client device 806 can then transmit the segments to a decoder (862), such as video decoder 48. Figure 1 Therefore, the media data in the first and second segments can be decoded and presented to the user of the client device 806.

[0917] In this way, Figure 30The method represents an example of a method for retrieving media data, comprising: retrieving a manifest file indicating a plurality of network locations for at least a first segment and a second segment of the media data, each of the plurality of network locations being hosted by a separate physical server device; retrieving at least a first portion of the first segment from the first network location of the plurality of network locations; retrieving at least a second portion of the second segment from the second network location of the plurality of network locations; and providing the at least first portion of the first segment and the at least second portion of the second segment to a decoder.

[0918] Various examples of the technology disclosed herein are summarized in the following clauses:

[0919] Clause 1: A method for transmitting media data, the method comprising: transmitting media data encapsulated in an extensible container format, the media data being encoded using one or more forward error correction (FEC) codes.

[0920] Clause 2: The method described in Clause 1, wherein the one or more FEC codes include xCD-1, RaptorQ, and Reed-Solomon.

[0921] Clause 3: The method according to any one of Clauses 1 and 2, wherein the extensible container format includes data representing one or more of the following: the decoding type of the media data, the media information of the media data, the integrity information of the media data, the encoder universally unique identifier (UUID), or the time of the media data.

[0922] Clause 4: A method for transmitting media data, the method comprising: transmitting one or more forward error correction (FEC) decoded media data segments via multiple sources, multiple paths and / or multiple access connections.

[0923] Clause 5: A method comprising a combination of the method according to any one of Clauses 1 to 3 and the method according to Clause 4.

[0924] Clause 6: A method for transmitting media data, the method comprising: transmitting forward error correction (FEC) decoded media data segments via one of Transmission Control Protocol (TCP), Uniform Datagram Protocol (UDP), or Web Real-Time Communication (WebRTC) protocol.

[0925] Clause 7: A method comprising a combination of the method according to any one of Clauses 1 to 5 and the method according to Clause 6.

[0926] Clause 8: The method according to any one of Clauses 6 and 7, wherein the encoded media data slice corresponds to at least one of a multipath audio / video stream or an ultra-low latency extended reality (XR) communication session.

[0927] Clause 9: The method according to any one of Clauses 1 to 8, wherein transmitting the media data comprises: transmitting the media data using HTTP-based streaming.

[0928] Clause 10: The method described in Clause 9, wherein transmitting the media data includes: transmitting the media data using HTTP Dynamic Adaptive Streaming (DASH).

[0929] Clause 11: The method described in Clause 9, wherein transmitting the media data includes: transmitting the media data using HTTP Live Streaming (HLS).

[0930] Clause 12: The method according to any one of Clauses 1 to 11, wherein transmitting the media data comprises: transmitting the media data using at least one of the One-Way Transfer File Delivery (FLUTE) or One-Way Transfer Real-Time Transfer Object Delivery (ROUTE) protocols.

[0931] Clause 13: A method for transmitting media data, the method comprising: processing feedback packets of one or more previous application data units (ADUs) of media data for a media streaming session; and transmitting forward error correction (FEC) data volumes of the application data units (ADUs) of the media data for the media streaming session, the FEC data volumes being associated with data from the feedback packets.

[0932] Clause 14: A method comprising a combination of the methods according to any one of Clauses 1 to 12 and the methods according to Clause 13.

[0933] Clause 15: A method for retrieving media data, the method comprising: retrieving a manifest file indicating a plurality of network locations for at least a first segment and a second segment of the media data, each of the plurality of network locations being hosted by a separate physical server device; retrieving at least a first portion of the first segment from the first network location of the plurality of network locations; retrieving at least a second portion of the second segment from the second network location of the plurality of network locations; and providing the at least first portion of the first segment and the at least second portion of the second segment to a decoder.

[0934] Clause 16: A method comprising a combination of the methods according to any one of Clauses 1 to 14 and the methods according to Clause 15.

[0935] Clause 17: The method according to any one of Clauses 15 and 16, wherein retrieving the at least first portion comprises: transmitting a first request for the at least first portion of the first segment to a first Uniform Resource Location (URL); and transmitting a second request for a third portion of the first segment to a second URL.

[0936] Clause 18: The method according to Clause 17 further includes: using FEC decoding to combine the at least first part and the third part.

[0937] Clause 19: The method according to any one of Clauses 17 and 18, wherein the first part includes an application object, and wherein the third part includes a repair object.

[0938] Clause 20: A method for transmitting media data, the method comprising: performing forward error correction (FEC) encoding on at least a first portion of a first segment to form a first FEC-encoded first portion of the first segment, and performing forward error correction (FEC) encoding on at least a second portion of a second segment to form a second FEC-encoded second portion of the second segment; storing the first FEC-encoded first portion and the second FEC-encoded second portion in a first physical server device; storing the first FEC-encoded first portion and the second FEC-encoded second portion in a second physical server device; and forming a manifest file indicating that the first FEC-encoded first portion and the second FEC-encoded second portion are available from the first physical server device and the second physical server device.

[0939] Clause 21: A method comprising a combination of the method according to any one of Clauses 1 to 14 and the method according to Clause 20.

[0940] Clause 22: The method according to any one of Clauses 20 and 21, the method further comprising: transmitting the manifest file to the first physical server device and the second physical server device.

[0941] Clause 23: The method according to any one of Clauses 20 to 22, the method further comprising: transmitting the manifest file to a client device.

[0942] Clause 24: The method according to any one of Clauses 20 to 23, the method further comprising: by the first physical server device: receiving from the client device a first request for the first FEC-encoded first portion; and in response to the first request transmitting the first FEC-encoded first portion to the client device; and by the second physical server device: receiving from the client device a second request for the second FEC-encoded second portion; and in response to the second request transmitting the second FEC-encoded second portion to the client device.

[0943] Clause 25: An apparatus for transmitting media data, the apparatus comprising one or more components for performing the method according to any one of Clauses 1 to 24.

[0944] Clause 26: The device pursuant to Clause 25, wherein the one or more components include one or more processors implemented in a circuit.

[0945] Clause 27: The device according to Clause 25, the device further includes: a memory configured to store media data.

[0946] Clause 28: The device pursuant to Clause 25, wherein the device includes at least one of: an integrated circuit; a microprocessor; and a wireless communication device.

[0947] Clause 29: A system for transmitting media data, the system comprising: a computing device including one or more processors; a first physical server device; and a second physical server device, wherein the one or more processors of the computing device are configured to: perform forward error correction (FEC) encoding on at least a first portion of a first segment to form a first FEC-encoded first portion of the first segment, and perform forward error correction (FEC) encoding on at least a second portion of a second segment to form a second FEC-encoded second portion of the second segment; store the first FEC-encoded first portion and the second FEC-encoded second portion in the first physical server device; store the first FEC-encoded first portion and the second FEC-encoded second portion in the second physical server device; and form a manifest file indicating that the first FEC-encoded first portion and the second FEC-encoded second portion are available from the first physical server device and the second physical server device.

[0948] Clause 30: The system according to Clause 29, wherein the first physical server device is configured to: receive a first request from a client device for the first FEC-encoded first portion; and transmit the first FEC-encoded first portion to the client device in response to the first request, and wherein the second physical server device is configured to: receive a second request from the client device for the second FEC-encoded second portion; and transmit the second FEC-encoded second portion to the client device in response to the second request.

[0949] Clause 31: An apparatus for transmitting media data, the apparatus including components for transmitting media data encapsulated in an extensible container format, the media data being encoded using one or more forward error correction (FEC) codes.

[0950] Clause 32: An apparatus for transmitting media data, the apparatus comprising components for transmitting one or more forward error correction (FEC) decoded media data segments via multiple sources, multiple paths, and / or multiple access connections.

[0951] Clause 33: An apparatus for transmitting media data, the apparatus comprising components for transmitting forward error correction (FEC) decoded media data segments via one of the Transmission Control Protocol (TCP), Uniform Datagram Protocol (UDP), or Web Real-Time Communication (WebRTC) protocol.

[0952] Clause 34: An apparatus for transmitting media data, the apparatus comprising: means for processing feedback packets of one or more previous application data units (ADUs) of media data for a media streaming session; and means for transmitting forward error correction (FEC) data amounts of the application data units (ADUs) of the media data for the media streaming session, the FEC data amounts being associated with data from the feedback packets.

[0953] Clause 35: An apparatus for transmitting media data, the apparatus comprising: means for retrieving a manifest file indicating a plurality of network locations for at least a first segment and a second segment of the media data, each of the plurality of network locations being hosted by a separate physical server device; means for retrieving at least a first portion of the first segment from a first network location of the plurality of network locations; means for retrieving at least a second portion of the second segment from a second network location of the plurality of network locations; and means for providing the at least first portion of the first segment and the at least second portion of the second segment to a decoder.

[0954] Clause 36: An apparatus for transmitting media data, the apparatus comprising: means for performing forward error correction (FEC) encoding on at least a first portion of a first segment to form a first FEC-encoded first portion of the first segment, and performing forward error correction (FEC) encoding on at least a second portion of a second segment to form a second FEC-encoded second portion of the second segment; means for storing the first FEC-encoded first portion and the second FEC-encoded second portion in a first physical server device; means for storing the first FEC-encoded first portion and the second FEC-encoded second portion in a second physical server device; and means for forming a manifest file indicating that the first FEC-encoded first portion and the second FEC-encoded second portion are available from the first physical server device and the second physical server device.

[0955] Clause 37: A method for retrieving media data, the method comprising: retrieving a manifest file indicating a plurality of network locations for at least a first segment and a second segment of the media data, each of the plurality of network locations being hosted by a separate physical server device; retrieving at least a first portion of the first segment from the first network location of the plurality of network locations; retrieving at least a second portion of the second segment from the second network location of the plurality of network locations; and providing the at least first portion of the first segment and the at least second portion of the second segment to a decoder.

[0956] Clause 38: The method according to Clause 37, wherein retrieving the at least first portion of the first segment comprises: transmitting a first request for the at least first portion of the first segment to a first uniform resource location (URL), the method further comprising: transmitting a second request for a third portion of the first segment to a second URL.

[0957] Clause 39: The method according to Clause 38, wherein the at least first portion includes an application object including media data, and wherein the third portion includes a repair object for repairing the application object according to forward error correction (FEC) in the event of loss of a portion of the application object.

[0958] Clause 40: The method according to Clause 38 further includes: using forward error correction (FEC) decoding to combine the at least first portion and the third portion.

[0959] Clause 41: The method according to Clause 38, wherein the manifest file includes source stream information of a media stream, the media stream including the at least first portion of the first segment, the manifest file further including: an identifier of the source stream, an identifier of an object corresponding to the at least first portion of the first segment and included in the source stream, a transport object identifier (TOI) of the object, the size of the object, the content type of the object indicating the media type of the object, encoding information of the object, a message digest of the object, an associated Uniform Resource Identifier (URI) of the object, a Uniform Resource Locator (URL) of the object, and an availability start time indicating when the object can be retrieved.

[0960] Clause 42: The method according to Clause 38, wherein the manifest file includes repair flow information of a repair flow, the repair flow including the third portion of the first segment, the manifest file further including: an identifier of a source flow corresponding to the repair flow, an identifier of an object of the source flow corresponding to the at least first portion of the first segment, an identifier of a repair object of the repair flow corresponding to the object of the source flow, forward error correction (FEC) object transmission information of the repair object including an FEC encoded identifier, data associating a transport session identifier (TSI) with a transport object identifier (TOI) of the repair object, the media type of the repair object, the number of symbols included in the repair object, the Uniform Resource Locator (URL) of the repair object, and an availability start time indicating when the repair object can be retrieved.

[0961] Clause 43: The method according to Clause 37, wherein the inventory file includes data indicating an expiry date when the inventory file needs to be updated, the method further comprising: updating the inventory file according to the expiry date.

[0962] Clause 44: The method according to Clause 37 further comprises: transmitting a feedback packet for one or more application data units (ADUs) of media data for a media streaming session receiving the first segment.

[0963] Clause 45: An apparatus for retrieving media data, the apparatus comprising: a memory configured to store media data; and a processing system including a decoder implemented in circuitry, the processing system being configured to: retrieve a manifest file indicating a plurality of network locations for at least a first segment and a second segment of the media data, each of the plurality of network locations being hosted by a separate physical server device; retrieve at least a first portion of the first segment from the first network location of the plurality of network locations; retrieve at least a second portion of the second segment from the second network location of the plurality of network locations; and provide the at least first portion of the first segment and the at least second portion of the second segment to the decoder.

[0964] Clause 46: The device according to Clause 45, wherein, in order to retrieve the at least first portion of the first segment, the processing system is configured to: transmit a first request for the at least first portion of the first segment to a first Uniform Resource Location (URL), and the processing system is further configured to: transmit a second request for a third portion of the first segment to a second URL.

[0965] Clause 47: The device according to Clause 46, wherein the at least first portion includes an application object including media data, and wherein the third portion includes a repair object for repairing the application object according to forward error correction (FEC) in the event of loss of a portion of the application object.

[0966] Clause 48: The device according to Clause 46, wherein the manifest file includes source stream information of a media stream, the media stream including the at least first portion of the first segment, the manifest file further including: an identifier of the source stream, an identifier of an object corresponding to the at least first portion of the first segment and included in the source stream, a transport object identifier (TOI) of the object, the size of the object, the content type of the object indicating the media type of the object, encoding information of the object, a message digest of the object, an associated Uniform Resource Identifier (URI) of the object, a Uniform Resource Locator (URL) of the object, and an availability start time indicating when the object can be retrieved.

[0967] Clause 49: The device according to Clause 46, wherein the manifest file includes repair stream information of a repair stream, the repair stream including the third portion of the first segment, the manifest file further including: an identifier of a source stream corresponding to the repair stream, an identifier of an object of the source stream corresponding to the at least first portion of the first segment, an identifier of a repair object of the repair stream corresponding to the object of the source stream, forward error correction (FEC) object transmission information of the repair object including an FEC encoded identifier, data associating a transport session identifier (TSI) with a transport object identifier (TOI) of the repair object, the media type of the repair object, the number of symbols included in the repair object, the Uniform Resource Locator (URL) of the repair object, and an availability start time indicating when the repair object can be retrieved.

[0968] Clause 50: The device according to Clause 45, wherein the inventory file includes data indicating an expiration time when the inventory file needs to be updated, and wherein the processing system is further configured to update the inventory file according to the expiration time.

[0969] Clause 51: The apparatus according to Clause 45, wherein the processing system is further configured to transmit feedback packets for one or more application data units (ADUs) of media data for a media streaming session receiving the first segment.

[0970] Clause 52: A method for transmitting media data, the method comprising: performing forward error correction (FEC) encoding on at least a first portion of a first segment to form a first FEC-encoded first portion of the first segment, and performing forward error correction (FEC) encoding on at least a second portion of a second segment to form a second FEC-encoded second portion of the second segment; storing the first FEC-encoded first portion in a first physical server device; storing the second FEC-encoded second portion in a second physical server device; and forming a manifest file indicating that the first FEC-encoded first portion is available from the first physical server device and that the second FEC-encoded second portion is available from the second physical server device.

[0971] Clause 53: The method according to Clause 52 further includes: transmitting the manifest file to the first physical server device and the second physical server device.

[0972] Clause 54: The method according to Clause 52, wherein FEC encoding of the at least first portion of the first segment comprises: forming an object for the at least first portion of the first segment and a repair object for a third portion of the first segment, the method further comprising: storing the repair object to a third physical server device.

[0973] Clause 55: The method according to Clause 54, wherein the manifest file includes source stream information of a media stream, the media stream including the at least first portion of the first segment, the manifest file further including: an identifier of the source stream, an identifier of an object corresponding to the at least first portion of the first segment and included in the source stream, a transport object identifier (TOI) of the object, the size of the object, the content type of the object indicating the media type of the object, encoding information of the object, a message digest of the object, an associated Uniform Resource Identifier (URI) of the object, a Uniform Resource Locator (URL) of the object, and an availability start time indicating when the object can be retrieved.

[0974] Clause 56: The method according to Clause 54, wherein the manifest file includes repair flow information of the repair flow, the repair flow including the third portion of the first segment, the manifest file further including: an identifier of the source flow corresponding to the repair flow, an identifier of the object of the source flow corresponding to the at least first portion of the first segment, an identifier of the repair object of the repair flow corresponding to the object of the source flow, forward error correction (FEC) object transmission information of the repair object including an FEC encoded identifier, data associating a transport session identifier (TSI) with a transport object identifier (TOI) of the repair object, the media type of the repair object, the number of symbols included in the repair object, the Uniform Resource Locator (URL) of the repair object, and an availability start time indicating when the repair object can be retrieved.

[0975] Clause 57: The method described in Clause 52, wherein the inventory file includes data indicating the expiration date when the inventory file needs to be updated.

[0976] Clause 58: The method according to Clause 52 further comprises: receiving from a client device a feedback packet of one or more application data units (ADUs) for media data of a media streaming session to which the first segment is transmitted to the client device.

[0977] Clause 59: An apparatus for transmitting media data, the apparatus comprising: a memory configured to store media data; and a processing system implemented in circuitry and configured to: perform forward error correction (FEC) encoding on at least a first portion of a first segment to form a first FEC-encoded first portion of the first segment, and perform forward error correction (FEC) encoding on at least a second portion of a second segment to form a second FEC-encoded second portion of the second segment; store the first FEC-encoded first portion in a first physical server device; store the second FEC-encoded second portion in a second physical server device; and form a manifest file indicating that the first FEC-encoded first portion is available from the first physical server device and that the second FEC-encoded second portion is available from the second physical server device.

[0978] Clause 60: The device according to Clause 59, wherein the processing system is further configured to transmit the manifest file to the first physical server device and the second physical server device.

[0979] Clause 61: The apparatus according to Clause 59, wherein, in order to perform FEC encoding on the at least first portion of the first segment, the processing system is configured to: form an object for the at least first portion of the first segment and a repair object for a third portion of the first segment, and wherein the processing system is further configured to: store the repair object to a third physical server device.

[0980] Clause 62: The device according to Clause 61, wherein the manifest file includes source stream information of a media stream, the media stream including the at least first portion of the first segment, the manifest file further including: an identifier of the source stream, an identifier of an object corresponding to the at least first portion of the first segment and included in the source stream, a transport object identifier (TOI) of the object, the size of the object, the content type of the object indicating the media type of the object, encoding information of the object, a message digest of the object, an associated Uniform Resource Identifier (URI) of the object, a Uniform Resource Locator (URL) of the object, and an availability start time indicating when the object can be retrieved.

[0981] Clause 63: The device according to Clause 61, wherein the manifest file includes repair flow information of a repair flow, the repair flow including the third portion of the first segment, the manifest file further including: an identifier of a source flow corresponding to the repair flow, an identifier of an object of the source flow corresponding to the at least first portion of the first segment, an identifier of a repair object of the repair flow corresponding to the object of the source flow, forward error correction (FEC) object transmission information of the repair object including an FEC encoded identifier, data associating a transport session identifier (TSI) with a transport object identifier (TOI) of the repair object, the media type of the repair object, the number of symbols included in the repair object, the Uniform Resource Locator (URL) of the repair object, and an availability start time indicating when the repair object can be retrieved.

[0982] Clause 64: The device as described in Clause 59, wherein the inventory file includes data indicating the expiration date when the inventory file needs to be updated.

[0983] Clause 65: The method according to Clause 59, wherein the processing system is further configured to: receive from the client device a feedback packet of one or more application data units (ADUs) for media data of a media streaming session that transmits the first segment to the client device.

[0984] In one or more examples, the described functionality may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functionality may be stored as one or more instructions or code on a computer-readable medium or transmitted via a computer-readable medium and executed by a hardware-based processing unit. A computer-readable medium may include a computer-readable storage medium (which corresponds to a tangible medium such as a data storage medium) or a communication medium, including, for example, any medium that facilitates the transfer of a computer program from one place to another according to a communication protocol. In this way, a computer-readable medium may generally correspond to (1) a non-transitory tangible computer-readable storage medium, or (2) a communication medium such as a signal or carrier wave. A data storage medium may be any available medium that can be accessed by one or more computers or one or more processors to extract instructions, code, and / or data structures for implementing the techniques described in this disclosure. Computer program products may include computer-readable media.

[0985] By way of example, and not limitation, such computer-readable storage media may include RAM, ROM, EEPROM, CD-ROM or other optical disc storage devices, magnetic disk storage devices or other magnetic storage devices, flash memory, or any other medium capable of storing desired program code in the form of instructions or data structures and accessible by a computer. Furthermore, any connection is appropriately referred to as a computer-readable medium. For example, if instructions are transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies (such as infrared, radio, and microwave), then coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies (such as infrared, radio, and microwave) are included in the definition of medium. However, it should be understood that computer-readable storage media and data storage media do not include connections, carrier waves, signals, or other transient media, but instead refer to non-transient tangible storage media. As used herein, disks and optical discs include: compact optical discs (CDs), laser optical discs, optical discs, digital versatile optical discs (DVDs), floppy disks, and Blu-ray discs, wherein disks typically reproduce data magnetically, while optical discs reproduce data optically using lasers. The combinations described above should also be included within the scope of computer-readable media.

[0986] Instructions can be executed by one or more processors, such as one or more digital signal processors (DSPs), general-purpose microprocessors, application-specific integrated circuits (ASICs), field-programmable arrays (FPGAs), or other equivalent integrated or discrete logic circuits. Therefore, the term "processor" as used herein can refer to any of the foregoing structures or any other structure suitable for implementing the techniques described herein. Additionally, in some aspects, the functionality described herein may be provided within dedicated hardware and / or software modules configured for encoding and decoding, or incorporated into combined codecs. Furthermore, these techniques may be fully implemented in one or more circuit or logic elements.

[0987] The techniques disclosed herein can be implemented in a wide variety of devices or apparatuses, including wireless mobile phones, integrated circuits (ICs), or a set of ICs (e.g., chipsets). Various components, modules, or units are described in this disclosure to emphasize functional aspects of a device configured to perform the disclosed techniques, but implementation by different hardware units is not necessarily required. Specifically, as described above, various units may be combined in a codec hardware unit, or may be provided by a collection of interoperable hardware units (including one or more processors as described above) combined with appropriate software and / or firmware.

[0988] Various examples have been described. These and other examples are within the scope of the following claims.

Claims

1. A method for retrieving media data, the method comprising: The retrieval instructions include a list of multiple network locations for at least a first segment and a second segment of the media data, each of the multiple network locations being hosted by a separate physical server device. Retrieve at least a first portion of the first segment from a first network location among the plurality of network locations; Retrieve at least a second portion of the second segment from a second network location among the plurality of network locations; as well as The at least first portion of the first segment and the at least second portion of the second segment are provided to the decoder.

2. The method of claim 1, wherein retrieving the at least first portion of the first segment comprises: The method further includes sending a first request for at least a first portion of the first segment to a first uniform resource location (URL).

3. The method of claim 2, wherein the at least first portion includes an application object comprising media data, and wherein the third portion includes a repair object for repairing the application object according to forward error correction (FEC) in the event of loss of a portion of the application object.

4. The method according to claim 2, wherein the method further comprises: Forward error correction (FEC) decoding is used to combine the at least first part and the third part.

5. The method of claim 2, wherein the manifest file includes source stream information of the media stream, the media stream includes the at least first portion of the first segment, and the manifest file further includes: The source stream identifier, the identifier of the object corresponding to the at least first part of the first segment and included in the source stream, the transport object identifier (TOI) of the object, the size of the object, the content type of the object indicating the media type of the object, the encoding information of the object, the message digest of the object, the associated Uniform Resource Identifier (URI) of the object, the Uniform Resource Locator (URL) of the object, and the availability start time indicating when the object can be retrieved.

6. The method of claim 2, wherein the manifest file includes repair flow information of the repair flow, the repair flow including the third portion of the first segment, and the manifest file further includes: The identifier of the source stream corresponding to the repair stream, the identifier of the object of the source stream corresponding to the at least first part of the first segment, the identifier of the repair object of the repair stream corresponding to the object of the source stream, the forward error correction (FEC) object transmission information of the repair object including the FEC encoded identifier, the data that associates the transport session identifier (TSI) with the transport object identifier (TOI) of the repair object, the media type of the repair object, the number of symbols included in the repair object, the Uniform Resource Locator (URL) of the repair object, and the availability start time indicating when the repair object can be retrieved.

7. The method of claim 1, wherein the manifest file includes data indicating the expiration time when the manifest file needs to be updated, the method further comprising: The inventory file is updated according to the stated expiration date.

8. The method according to claim 1, wherein the method further comprises: Transmit one or more Application Data Units (ADUs) for media data received in a media streaming session of the first segment.

9. An apparatus for retrieving media data, the apparatus comprising: A memory configured to store media data; and A processing system, comprising a decoder implemented in a circuit, is configured to: The retrieval instructions include a list of multiple network locations for at least a first segment and a second segment of the media data, each of the multiple network locations being hosted by a separate physical server device. Retrieve at least a first portion of the first segment from a first network location among the plurality of network locations; Retrieve at least a second portion of the second segment from a second network location among the plurality of network locations; as well as The decoder is provided with at least the first portion of the first segment and at least the second portion of the second segment.

10. The device of claim 9, wherein, in order to retrieve the at least first portion of the first segment, the processing system is configured to: transmit a first request for the at least first portion of the first segment to a first uniform resource location (URL), and the processing system is further configured to: transmit a second request for a third portion of the first segment to a second URL.

11. The device of claim 10, wherein the at least first portion includes an application object comprising media data, and wherein the third portion includes a repair object for repairing the application object according to forward error correction (FEC) in the event of loss of a portion of the application object.

12. The device of claim 10, wherein the manifest file includes source stream information of the media stream, the media stream including the at least first portion of the first segment, and the manifest file further includes: The source stream identifier, the identifier of the object corresponding to the at least first part of the first segment and included in the source stream, the transport object identifier (TOI) of the object, the size of the object, the content type of the object indicating the media type of the object, the encoding information of the object, the message digest of the object, the associated Uniform Resource Identifier (URI) of the object, the Uniform Resource Locator (URL) of the object, and the availability start time indicating when the object can be retrieved.

13. The device of claim 10, wherein the manifest file includes repair flow information of a repair flow, the repair flow including the third portion of the first segment, and the manifest file further includes: The identifier of the source stream corresponding to the repair stream, the identifier of the object of the source stream corresponding to the at least first part of the first segment, the identifier of the repair object of the repair stream corresponding to the object of the source stream, the forward error correction (FEC) object transmission information of the repair object including the FEC encoded identifier, the data that associates the transport session identifier (TSI) with the transport object identifier (TOI) of the repair object, the media type of the repair object, the number of symbols included in the repair object, the Uniform Resource Locator (URL) of the repair object, and the availability start time indicating when the repair object can be retrieved.

14. The apparatus of claim 9, wherein the inventory file includes data indicating an expiration time when the inventory file needs to be updated, and wherein the processing system is further configured to update the inventory file according to the expiration time.

15. The apparatus of claim 9, wherein the processing system is further configured to: transmit feedback packets for one or more application data units (ADUs) of media data for a media streaming session receiving the first segment.

16. A method for transmitting media data, the method comprising: Forward error correction (FEC) encoding is performed on at least a first portion of the first segment to form a first FEC-encoded first portion of the first segment, and forward error correction (FEC) encoding is performed on at least a second portion of the second segment to form a second FEC-encoded second portion of the second segment; The first portion encoded by FEC is stored in the first physical server device; The second part, encoded by FEC, is stored in the second physical server device; as well as A manifest file is generated, the manifest file indicating that the first FEC-encoded first portion can be obtained from the first physical server device, and the second FEC-encoded second portion can be obtained from the second physical server device.

17. The method of claim 16, further comprising: The manifest file is transmitted to the first physical server device and the second physical server device.

18. The method of claim 16, wherein performing FEC encoding on the at least first portion of the first segment comprises: The method further includes: forming an object for at least a first portion of the first segment and a repair object for a third portion of the first segment, wherein the method further includes: storing the repair object to a third physical server device.

19. The method of claim 18, wherein the manifest file includes source stream information of the media stream, the media stream including the at least first portion of the first segment, and the manifest file further includes: The source stream identifier, the identifier of the object corresponding to the at least first part of the first segment and included in the source stream, the transport object identifier (TOI) of the object, the size of the object, the content type of the object indicating the media type of the object, the encoding information of the object, the message digest of the object, the associated Uniform Resource Identifier (URI) of the object, the Uniform Resource Locator (URL) of the object, and the availability start time indicating when the object can be retrieved.

20. The method of claim 18, wherein the manifest file includes repair flow information of the repair flow, the repair flow including the third portion of the first segment, and the manifest file further includes: The identifier of the source stream corresponding to the repair stream, the identifier of the object of the source stream corresponding to the at least first part of the first segment, the identifier of the repair object of the repair stream corresponding to the object of the source stream, the forward error correction (FEC) object transmission information of the repair object including the FEC encoded identifier, the data that associates the transport session identifier (TSI) with the transport object identifier (TOI) of the repair object, the media type of the repair object, the number of symbols included in the repair object, the Uniform Resource Locator (URL) of the repair object, and the availability start time indicating when the repair object can be retrieved.

21. The method of claim 16, wherein the manifest file includes data indicating the expiration time when the manifest file needs to be updated.

22. The method of claim 16, further comprising: Receive feedback packets from the client device for one or more application data units (ADUs) of media data for a media streaming session that transmits the first segment to the client device.

23. An apparatus for transmitting media data, the apparatus comprising: A memory configured to store media data; and A processing system, implemented in a circuit and configured to: Forward error correction (FEC) encoding is performed on at least a first portion of the first segment to form a first FEC-encoded first portion of the first segment, and forward error correction (FEC) encoding is performed on at least a second portion of the second segment to form a second FEC-encoded second portion of the second segment; The first portion encoded by FEC is stored in the first physical server device; The second part, encoded by FEC, is stored in the second physical server device; as well as A manifest file is generated, the manifest file indicating that the first FEC-encoded first portion can be obtained from the first physical server device, and the second FEC-encoded second portion can be obtained from the second physical server device.

24. The apparatus of claim 23, wherein the processing system is further configured to transmit the manifest file to the first physical server device and the second physical server device.

25. The apparatus of claim 23, wherein, in order to perform FEC encoding on the at least first portion of the first segment, the processing system is configured to: form an object for the at least first portion of the first segment and a repair object for a third portion of the first segment, and wherein the processing system is further configured to: store the repair object to a third physical server device.

26. The device of claim 25, wherein the manifest file includes source stream information of the media stream, the media stream including the at least first portion of the first segment, and the manifest file further includes: The source stream identifier, the identifier of the object corresponding to the at least first part of the first segment and included in the source stream, the transport object identifier (TOI) of the object, the size of the object, the content type of the object indicating the media type of the object, the encoding information of the object, the message digest of the object, the associated Uniform Resource Identifier (URI) of the object, the Uniform Resource Locator (URL) of the object, and the availability start time indicating when the object can be retrieved.

27. The device of claim 25, wherein the manifest file includes repair flow information of a repair flow, the repair flow including the third portion of the first segment, and the manifest file further includes: The identifier of the source stream corresponding to the repair stream, the identifier of the object of the source stream corresponding to the at least first part of the first segment, the identifier of the repair object of the repair stream corresponding to the object of the source stream, the forward error correction (FEC) object transmission information of the repair object including the FEC encoded identifier, the data that associates the transport session identifier (TSI) with the transport object identifier (TOI) of the repair object, the media type of the repair object, the number of symbols included in the repair object, the Uniform Resource Locator (URL) of the repair object, and the availability start time indicating when the repair object can be retrieved.

28. The device of claim 23, wherein the inventory file includes data indicating the expiration date when the inventory file needs to be updated.

29. The method of claim 23, wherein the processing system is further configured to: receive from the client device a feedback packet of one or more application data units (ADUs) for media data of a media streaming session that transmits the first segment to the client device.

Images