Multi-source method and system for encoding media

Abstract

Attached herein are embodiments of a multi-source method and system for encoding media. In some embodiments, a method includes: at a first device: receiving media data representing a media asset; obtaining a first plurality of data elements comprising at least one of bitstream identification data, content-specific encoded data, and media segment data; encoding at least a portion of the media data into encoded data corresponding to the media assets according to a first encoding process; generating a second plurality of data elements different from the first plurality of data elements based on information associated with the first encoding process; combining the first plurality of data elements and the second plurality of data elements into one or more encoded bitstreams representing the media asset; and transmitting the one or more encoded bit streams to one or more second devices using one or more network paths.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims the benefit of priority to U.S. Provisional Patent Application No. 63 / 332,210 filed on April 18, 2022 (reference number: D20228AUSP1), U.S. Provisional Patent Application No. 63 / 332,212 filed on April 18, 2022 (reference number: D20228BUSP1), and U.S. Provisional Patent Application No. 63 / 443,933 filed on February 7, 2023 (reference number: D20228AUSP2), each of which is incorporated herein by reference in its entirety. Technical Field

[0003] The present disclosure relates generally to encoding and decoding of media. Background Art

[0004] Application layer coding is a method that can be used to mitigate the impact of low-level protocols, networks, and channels on the overall quality of data transmission from one or more sources to one or more receivers. The data transfer rate through a network depends largely on the characteristics of the network (e.g., congestion, packet loss, latency, etc.). Application layer coding aims to optimize data transmission by selecting the appropriate type of coding given known network characteristics (e.g., network capacity, latency, etc.), specific use cases (e.g., online streaming), and overall goals (e.g., minimizing latency). Summary of the invention

[0005] Attached hereto are embodiments of a multi-source method and system for encoding media.

[0006] In some embodiments, a method includes: at a first device: receiving media data representing a media asset; obtaining a first plurality of data elements including at least one of bitstream identification data, content-specific encoding data, and media segment data; encoding at least a portion of the media data into encoded data corresponding to the media asset according to a first encoding process; generating a second plurality of data elements different from the first plurality of data elements based on information associated with the first encoding process; combining the first plurality of data elements and the second plurality of data elements into one or more encoded bitstreams representing the media asset; and transmitting the one or more encoded bitstreams to one or more second devices using one or more network paths.

[0007] In some embodiments, the first encoding process is a linear encoding process to obtain encoded data corresponding to the media asset.

[0008] In some embodiments, the first encoding process is a non-linear encoding process that transforms the media data into encoded data consistent with another linear encoding process.

[0009] In some embodiments, the data element is a data structure.

[0010] In some embodiments, the first plurality of data elements includes synchronization data elements, and each of the one or more encoded bit streams begins with a synchronization data element.

[0011] In some embodiments, the synchronization data element includes at least one of the following: identification data that identifies a corresponding one of the one or more encoded bitstreams as being generated from a common source media asset; synchronization data for synchronizing the corresponding one of the one or more encoded bitstreams with other encoded bitstreams generated from the common source media asset; and lineage data of the corresponding one of the one or more encoded bitstreams.

[0012] In some embodiments, the lineage data includes data identifying original or encoded symbols used to generate encoding symbols included in the corresponding encoded bit stream, data identifying how the original or encoded symbols are combined to generate encoding symbols included in the corresponding encoded bit stream, and data identifying encoding coefficients used to generate encoding symbols included in the corresponding encoded bit stream.

[0013] In some embodiments, the method further includes: interleaving synchronization data elements in the one or more encoded bit streams, wherein the synchronization data elements enable a second electronic device receiving one of the one or more encoded bit streams to start decoding from multiple points within the one of the one or more encoded bit streams.

[0014] In some embodiments, the first plurality of data elements or the second plurality of data elements are generated based at least in part on system data associated with the received media data.

[0015] In some embodiments, the method further comprises obtaining the first plurality of data elements comprises generating the first plurality of data elements from the received media data or system data associated with the received media data.

[0016] In some embodiments, encoding the media data according to the first encoding process includes segmenting the media data into a plurality of original symbols and generating a plurality of encoded symbols based on the plurality of original symbols.

[0017] In some embodiments, the encoded symbols are generated by linearly combining subsets of the plurality of original symbols according to a network coding technique.

[0018] In some embodiments, the media data comprises a linearly encoded bitstream.

[0019] In some embodiments, obtaining the first plurality of data elements comprises: extracting metadata from the media data; and calculating the first plurality of data elements based on the extracted metadata.

[0020] In some embodiments, metadata is extracted from the media data without performing linear decoding.

[0021] In some embodiments, encoding the media data according to a first encoding process includes re-encoding linear encoding symbols in the encoded bitstream into new linear encoding symbols representing the media asset.

[0022] In some embodiments, the method further comprises, prior to combining, splitting one or more data elements having a first size into data elements having a second size that is smaller than the first size.

[0023] In some embodiments, the method further comprises adding hash data to one or more data elements of the second plurality of data elements to enable at least one of verification, authentication or integrity checking by a device receiving the one or more bitstreams.

[0024] In some embodiments, the method further includes adding hash data to one or more data elements of the second plurality of data elements, the second plurality of data elements being associated with one or more of: a file corresponding to the media asset, a block of the media asset, a fragment of the media asset, and a data packet of the media asset.

[0025] In some embodiments, combining comprises inserting the directory data elements into the one or more encoded bitstreams.

[0026] In some embodiments, the directory data elements are used to help a receiving device identify subatoms in each bitstream.

[0027] In some embodiments, the one or more coded bitstreams include a first coded bitstream including data elements of a first type and a second coded bitstream not including data elements of the first type.

[0028] In some embodiments, the data elements or subatoms used to initialize the decoder on the client are not latency sensitive data.

[0029] In some embodiments, the bitstream includes a sync() subatom and at least one of a bitstream_header(), block_header(), encoder_content_info(), or media_segment_info() subatoms.

[0030] In some embodiments, the data elements or subatoms carry encoded data or latency sensitive data.

[0031] In some embodiments, the bitstream includes the packet() subatom and the sync() subatom, but does not include at least one of the bitstream_header(), block_header(), encoder_content_info(), or media_segment_info() subatoms.

[0032] In some embodiments, transmitting comprises: transmitting the first encoded bitstream using a first type of communication channel; and transmitting the second encoded bitstream using a second type of communication channel different from the first type.

[0033] In some embodiments, the second type of communication channel is at least one of a higher latency channel, a lower bandwidth channel, a reliable channel, or a channel that uses a second type of transport protocol different than the first type of transport protocol.

[0034] In some embodiments, a method includes: at a first device: receiving one or more encoded bitstreams associated with a media asset; storing the one or more encoded bitstreams in a buffer; processing the one or more encoded bitstreams stored in the buffer, including: extracting a first plurality of data elements including at least one of bitstream identification data, content-specific encoding data, and media segment data from the one or more encoded bitstreams; and extracting a second plurality of data elements including information associated with a corresponding encoding process used to generate the one or more encoded bitstreams from the one or more encoded bitstreams; and performing one of the following operations: generating decoded data representing the media asset based on the second plurality of data elements; and generating new encoded data representing the media asset based on the second plurality of data elements.

[0035] In some embodiments, generating the decoded data comprises decoding the encoded data with a corresponding decoder of a plurality of decoders based on a determination of an encoding type used to generate the encoded data.

[0036] In some embodiments, the encoding type is determined based on the extracted first plurality of data elements.

[0037] In some embodiments, the encoding type is determined based on the extracted second plurality of data elements.

[0038] In some embodiments, the encoding type is determined from the first plurality of data elements without using the extracted second plurality of data elements.

[0039] In some embodiments, the encoding type is determined from the second plurality of data elements without using the extracted first plurality of data elements.

[0040] In some embodiments, the method further comprises generating consumable data corresponding to the media asset from the decoded data and the first plurality of data elements.

[0041] In some embodiments, the consumable data is compatible with an application executing on the first electronic device or a media file type supported by the application.

[0042] In some embodiments, generating the decoded data further comprises: processing the first type of encoded data into a second type of encoded data without performing linear encoding; and decoding the second type of encoded data using a decoder capable of decoding the second type of encoded data.

[0043] In some embodiments, processing the coded data is performed by a code unifier that aligns coefficients or projects the coded data into a common finite field.

[0044] In some embodiments, generating new encoded data comprises processing the encoded data with a corresponding re-encoder of a plurality of re-encoders in accordance with a determination of an encoding type used to generate the encoded data included in the second plurality of data elements.

[0045] In some embodiments, the encoding type is determined based on the extracted first plurality of data elements.

[0046] In some embodiments, the encoding type is determined based on the extracted second plurality of data elements.

[0047] In some embodiments, the encoding type is determined from the first plurality of data elements without using the extracted second plurality of data elements.

[0048] In some embodiments, the encoding type is determined from the second plurality of data elements without using the extracted first plurality of data elements.

[0049] In some embodiments, generating new encoded data includes: processing the first type of encoded data into a second type of encoded data without performing linear decoding; and re-encoding the second type of encoded data using a re-encoder capable of encoding the second type of encoded data.

[0050] In some embodiments, processing the coded data is performed by a code unifier that aligns coefficients or projects the coded data into a common finite field.

[0051] In some embodiments, the method further comprises storing the new encoded data or generating one or more bitstreams from the new encoded data.

[0052] In some embodiments, the method further comprises, before processing the one or more encoded bitstreams stored in the buffer, combining a first number of data elements (each having a first size) in the first plurality of data elements or the second plurality of data elements into a smaller number of data elements having a second size greater than the first size.

[0053] In some embodiments, the grouping is based on data within the CHUNKED_SUBATOM subatom.

[0054] In some embodiments, the method further includes: extracting integrity data from the second plurality of data elements before decoding or re-encoding; and performing at least one of decoding or re-encoding based on a determination of successful verification of the corresponding data element based on the corresponding integrity data; and not performing at least one of decoding or re-encoding based on a determination of unsuccessful verification of the corresponding data element based on the corresponding integrity data.

[0055] In some embodiments, the integrity data comprises hash data and the verification of the hash data is based on key information. In some embodiments, the key information comprises at least one of a public key or a private key.

[0056] In some embodiments, the key information is received from an authentication server.

[0057] In some embodiments, the method further comprises, prior to decoding or re-encoding, decrypting the corresponding encrypted data using a decryption key based on a determination that the second plurality of data elements includes corresponding encrypted data.

[0058] In some embodiments, the key information or hash data is associated with a bitstream_encryption_key_id.

[0059] In some embodiments, the owner or creator of a media asset applies encryption to one or more received encoded bitstreams associated with the media asset to prevent unauthorized access / consumption / decoding / re-encoding.

[0060] In some embodiments, the method also includes: transmitting identification information associated with the first electronic device to a second electronic device that is different from the first electronic device; and receiving a decryption key from the second electronic device, wherein the decryption key enables access, consumption, decoding, or re-encoding of the one or more encoded bit streams associated with the media asset.

[0061] In some embodiments, a method includes: at a first electronic device: obtaining, using at least one processor, a first encoded bit stream of a first type; generating, using the at least one processor, a second encoded bit stream of a second type different from the first type; and transmitting, using the at least one processor, the second encoded bit stream to one or more second electronic devices via one or more network paths.

[0062] In some embodiments, the first bitstream of the first type is generated according to a first type of linear encoding protocol or standard.

[0063] In some embodiments, the first bitstream or the second bitstream includes systematic data packets associated with the data segments and non-systematic data packets associated with the data segments.

[0064] In some embodiments, the first bitstream or the second bitstream includes system data packets associated with the data segment and does not include non-system data packets associated with the data segment.

[0065] In some embodiments, the first bitstream or the second bitstream includes non-system data packets associated with the data segment and does not include system data packets associated with the data segment.

[0066] In some embodiments, the second bitstream of the second type is generated according to a second type of linear encoding protocol or standard.

[0067] In some embodiments, the one or more network paths include more than one network, more than one network type, more than one network, more than one network transmission medium, or communication channels having different performance characteristics.

[0068] In some embodiments, a method includes: at a first electronic device: receiving a first bitstream associated with a media asset and encoded with a first type of encoding; based on a determination that the first bitstream should be decoded using a decoder associated with a second type of encoding, processing the first bitstream into a second bitstream of a second type of encoding, including: extracting data from a set of fields of the first bitstream and mapping the data to a second set of corresponding fields of the second bitstream; extracting metadata associated with the media asset from the first bitstream; and decoding the second bitstream into an unencoded representation of the media asset using a decoder associated with the second type of encoding; based on a determination that the first bitstream should not be decoded using a decoder associated with the second type of encoding, processing the first bitstream, including: decoding the first bitstream into an unencoded representation of the media asset using a decoder associated with the first type of encoding and metadata associated with the media asset; and passing the unencoded representation of the media asset and the extracted metadata associated with the media asset to an application.

[0069] In some embodiments, a non-transitory computer-readable storage medium stores instructions that, when executed by a computing device, cause the computing device to perform any of the foregoing methods.

[0070] In some embodiments, a computing device includes: at least one processor; and a memory storing instructions that, when executed by the at least one processor, cause the computing device to perform any of the foregoing methods.

[0071] Other embodiments disclosed herein relate to a system, an apparatus, and a computer-readable medium. The details of the disclosed embodiments are described in the following drawings and description. Other features, purposes, and advantages are apparent from the description, drawings, and claims.

[0072] Certain embodiments disclosed herein provide one or more of the following advantages. Data elements associated with an encoding process (e.g., a linear encoding process) are combined with data elements including at least one of bitstream identification data, content-specific encoding data, and media segment data prior to transmission, enabling more efficient delivery of media data to / from devices with various processing capabilities (e.g., encoding / decoding capabilities) over networks with different performance characteristics. The combined data provides more efficient decoding and reduces the need to generate and transmit multiple redundant types of bitstreams to ensure compatibility, thereby saving network and processing resources.

[0073] The disclosed multi-source method and system supports application layer encoding using a unique bitstream container format (also referred to as the "disclosed bitstream" below), which supports different applications with a common representation and interface. The disclosed bitstream implements a scalable and interoperable method that supports the use of one or more linear codes to improve network-oriented efficiency and reliability of multimedia processing, storage, distribution, delivery, and playback for cloud and network connections. The disclosed bitstream is also designed to carry metadata and other information (e.g., encoding symbols, block counts, number of symbols per block, information about the encoding matrix, field size of the encoding matrix coefficients) in a compatible format to be compatible with standardized encoder and decoder implementations of different encoding types.

[0074] The disclosed methods and systems also support an exchange bitstream format that enables novel use of a variety of code types and use cases, including single-source and multi-source media distribution over one or more networks and / or access paths.

[0075] The disclosed methods and systems also support embodiments where a single processing entity can seamlessly process and / or decode multiple signaled code types on the fly. BRIEF DESCRIPTION OF THE DRAWINGS

[0076] Figure 1A and Figure 1B The concepts of linear encoding and decoding are illustrated separately.

[0077] Figure 2A and Figure 2B Block diagrams of bitstream encoding and decoding, respectively, according to some embodiments.

[0078] Figure 3A Illustrated is a hierarchical organization of data elements of a disclosed bitstream in accordance with some embodiments.

[0079] Figure 3B Illustrated are example data elements in a subatom of a disclosed bitstream in accordance with some embodiments.

[0080] Figure 4 is a block diagram of a media processing and delivery system according to some embodiments.

[0081] Figure 5 According to some embodiments Figure 4 Block diagram of the bitstream generator shown in .

[0082] Figure 6 According to some embodiments Figure 4 Block diagram of a bitstream decoder shown in .

[0083] Figure 7 is a block diagram of a bitstream re-encoder according to some embodiments.

[0084] Fig. 8A is a block diagram of a unified decoder according to some embodiments.

[0085] Figure 8B is a block diagram of a unified decoder with a decoder switch according to some embodiments.

[0086] Fig. 9A is a block diagram of a unified bitstream re-encoder according to some embodiments.

[0087] Fig. 9B is a block diagram of a unified bitstream recoder with a recoder switch according to some embodiments.

[0088] Fig.10 A bitstream mapping for compatibility with other coding bitstream standards according to some embodiments is illustrated.

[0089] Fig.11 is an example of a bitstream delivered by multiple sources according to some embodiments.

[0090] Fig.12 A multi-source multi-path delivery system is illustrated in accordance with some embodiments.

[0091] Fig.13 A low latency unicast bitstream delivery system is illustrated in accordance with some embodiments.

[0092] Fig.14A is a block diagram of a system for low-latency unicast encoding and decoding in accordance with some embodiments.

[0093] Fig. 14B Illustrated is the handling of variable data sizes out-of-band in accordance with some embodiments.

[0094] Fig.15A Illustrated is low-latency unicast encoding using a separate linear encoder-decoder pair in accordance with some embodiments.

[0095] Fig. 15B Illustrated is low latency unicast encoding using cross-generation decoders in accordance with some embodiments.

[0096] Fig.16 The use of the disclosed bitstream for digital rights management (DRM) according to some embodiments is illustrated.

[0097] Fig.17 is a block diagram of a decentralized media delivery system with data authentication implemented by the disclosed bitstream, according to some embodiments.

[0098] Fig.18is a flow chart of a process for generating a bitstream from non-linear or linearly encoded data according to some embodiments.

[0099] Fig.19 is a flow chart of a decoding / re-encoding process according to some embodiments.

[0100] Fig. 20 is a flow diagram of a process for converting new code types for downstream compatibility according to some embodiments.

[0101] Fig.21 is a flow diagram of another process for converting new code types for downstream compatibility according to some embodiments.

[0102] Fig. 22 is suitable for implementing reference figures 1 to Fig.21 A block diagram of an example hardware architecture for the described systems and methods.

[0103] In the accompanying drawings, for ease of description, specific arrangements or orderings of schematic elements are shown, such as those representing devices, units, instruction blocks, and data elements. However, it should be understood by those skilled in the art that the specific ordering or arrangement of schematic elements in the accompanying drawings is not meant to imply that a specific processing order or sequence or process separation is required. Further, the inclusion of a schematic element in the accompanying drawings is not meant to imply that such an element is required in all embodiments, or that in some embodiments, the features represented by such an element may not be included in or combined with other elements.

[0104] Further, in the accompanying drawings, when connecting elements such as solid lines or dotted lines or arrows are used to illustrate the connection, relationship or association between two or more other schematic elements, the absence of any such connecting elements does not mean that it is impossible to imply that there is a connection, relationship or association. In other words, some connections, relationships or associations between elements are not shown in the accompanying drawings to avoid blurring the present disclosure. In addition, for ease of illustration, a single connecting element is used to represent multiple connections, relationships or associations between elements. For example, in the case where a connecting element represents the communication of a signal, data or instruction, it will be understood by those skilled in the art that such an element represents one or more signal paths that may be needed to affect the communication.

[0105] The same reference numbers used in the various drawings indicate similar elements. DETAILED DESCRIPTION

[0106] In the following detailed description, many specific details are set forth to provide a comprehensive understanding of the various embodiments described. It will be apparent to those of ordinary skill in the art that the various embodiments described can be practiced without these specific details. In other examples, well-known methods, processes, components, and circuits are not described in detail to avoid unnecessarily obscuring various aspects of the embodiments. Several features are described below, each of which can be used independently of one another or used in combination with any of the other features.

[0107] Explanation of terms

[0108] As used herein, the term "including" and its variations should be understood as open-ended terms meaning "including but not limited to". Unless the context clearly indicates otherwise, the term "or" should be understood as "and / or". The term "based on" should be understood as "based at least in part on". The terms "an example embodiment" and "example embodiment" should be understood as "at least one example embodiment". The term "another embodiment" should be understood as "at least one other embodiment". The term "determine" should be understood as obtain, receive, calculate, estimate, estimate, predict or obtain. In addition, in the following description and claims, unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present disclosure belongs.

[0109] In addition to the above definitions, the following additional definitions shall apply to the following description.

[0110] "Encoding" means encoding a source coded / non-linearly coded media data input, or re-encoding a linearly coded media input using a linear encoding.

[0111] A "finite field" is a field of finite order where the numbers in the field are arithmetically closed and complete.

[0112] "Homomorphic hashing" is a family of cryptographic hashing algorithms in which the hash of a composite block of data can be derived using the hashes of the individual blocks of data (eg, by linearly combining or concatenating the individual blocks of data).

[0113] "Linear codes" are a family of redundant codes in which linear combinations of data segments are created as redundant information. Examples of linear codes include, but are not limited to: Reed-Solomon codes, RaptorQ codes, random linear (network) codes, Hamming codes, convolutional codes, low-density parity-check (LDPC) codes, and Bose-Chaudhuri-Hocquenghem (BCH) codes.

[0114] "Linear encoding" is the operation of generating linear redundancy by linearly combining data segments.

[0115] "Linear decoding" is the operation of solving a linear equation to obtain the original data.

[0116] "Linear recoding" is an operation that generates new linear redundancy by first decoding and then encoding or further linearly combining linear combinations.

[0117] "Metadata" is high-level descriptive data that helps the receiver process media data, such as media type, frame rate, bit rate, etc.

[0118] "Post-processing" is the process of converting the delivered data into consumable media by, for example, applying unpacking and / or decompression.

[0119] "Preprocessing" is processing raw media data into a suitable format. For example, source encoding techniques (e.g., compression) are applied to raw media data, and / or media data is packaged using certain representations such as HTTP Streaming (HLS) and Dynamic Adaptive Streaming over HTTP (DASH) .

[0120] Application layer coding overview

[0121] To help readers better understand the disclosed embodiments, an overview of application layer coding, particularly linear coding, is discussed below.

[0122] The vast majority of application layer codes are linear codes, and encoding data usually follows three basic steps: 1) split the data into multiple blocks, where the number of blocks can be small (e.g., 1) or large, depending on the size of the data; 2) split each block into multiple source symbols X i , where i is the index of the original symbol in the block, each source symbol has a specified size (e.g., in bytes), the final source symbol is padded with zero-valued bytes if necessary, and the number of source symbols is typically selected based on the application and the encoding type used; and 3) the source symbols are encoded to create an equal or greater number of encoded symbols (depending on the encoding type) Y i .

[0123] The mathematical representations of the linear code encoding and linear code decoding steps are as follows: Figure 1A and Figure 1B In the example shown, the original data file or stream is shown as being segmented into a plurality of blocks. Although the example described focuses on the linear encoding of one block, the linear code encoding / decoding steps are repeated for each block in the data file or stream.

[0124] refer to Figure 1A ,The steps to encode data given a general linear code are as follows.

[0125] Step 1: Split each data block into multiple source symbols. In this example, 4 source symbols are created from the input block data.

[0126] Step 2: Formulate the vector of source symbols

[0127] Step 3: Generate a coding matrix (G), where each element in the matrix is ​​selected from the Galois field GF{q}, where q = 2m. In this example, the coding matrix is ​​6×4.

[0128] Step 4: By combining the encoding matrix (G) with the input source symbol Multiply to calculate the coded symbol (Y). If the Galois field GF{2} is used, and The coefficient vector of 1，1 , a 1，2 , a 1，3 , a 1，4 ]=[1 0 0 0], then the first coded symbol y1 is equivalent to the first source symbol x1, because the coefficient vector has only one non-zero coefficient. In this case, y1 can be called a systematic symbol.

[0129] After encoding, the encoding symbols and encoding coefficients of the encoding matrix are transmitted to the decoder. In other embodiments, the encoding symbols can be cached in the network, CDN, etc. and delivered upon request from one or more client devices including appropriate decoders. The encoding coefficients of the encoding matrix can be: (a) transmitted to the decoder; (b) the device for generating the encoding coefficients is parameterized and transmitted to the decoder; or (c) the coefficients are known and assumed in both the encoder and the decoder.

[0130] Knowing the coding coefficients at the decoder, the original symbols can be solved from the coded symbols using methods such as Gaussian elimination. The process is simplified when systematic symbols exist because the systematic symbols are guaranteed to be linearly independent. At a minimum, the decoder should receive a number of coded symbols equal to or greater than the number of source symbols to decode. In some cases, more coded symbols are required if the generated coding coefficients are not linearly independent of each other.

[0131] refer to Figure 1B , the steps to decode the above example linear code are as follows.

[0132] Step 1: Determine the coefficient vector associated with each coded symbol and generate the received coding coefficient matrix A.

[0133] Step 2: Calculate the inverse of A -1 .

[0134] Step 3: By taking the inverse of the received coding coefficient matrix (A-1 ) and the vector of received coded symbols Multiply, restore the vector of original sign

[0135] Step 4: Each symbol in is concatenated to re-form the original block data.

[0136] The published bitstream as a container

[0137] Figure 2A and Figure 2B are block diagrams of bitstream encoding and decoding according to some embodiments. Figure 3A , Figure 3B ) can be thought of as a container that supports different application layer encoding types. A specific encoding type can be chosen for a specific use case. While many encoding types have their own formats that can produce different kinds of bitstreams, the goal of the exposed bitstream is to support different applications with a common representation and interface.

[0138] When application layer coding is used, information is transmitted to the receiver to recover the original data. For example, in addition to the coding symbols, the block count, the number of symbols per block, information about the coding matrix, and the field size of the coding matrix coefficients are also sent to the receiver. Figure 3A , Figure 3B The disclosed bitstream shown in is designed to carry metadata and this information in a compatible format to be compatible with standardized encoder and decoder implementations of different encoding types. The bitstream mapper is used to translate the information produced by the application layer encoder into the data element types supported by the disclosed bitstream.

[0139] refer to Figure 2A , the system 200a includes a media file 201, an application layer encoder 202a ... 202n, a bitstream mapper 203, and a bitstream encoder 204. The bitstream mapper 203 translates the information generated by the bitstream encoders 202a ... 202n into data element types supported by the disclosed bitstream generated by the bitstream encoder 204. The bitstream encoder 204 generates the disclosed bitstream, which includes the supported data element types and content metadata 205 output by the bitstream mapper 203.

[0140] refer to Figure 2B, system 200b includes a bitstream decoder / parser 206, a demapper 207, and application layer encoders 208a...208n. The bitstream decoder / parser 206 decodes the received disclosed bitstream using content metadata 205 and parses the data elements in the disclosed bitstream. The bitstream demapper 207 translates the data element type back to the type required by the standardized application layer decoder 208a...208n. The output of the application layer decoder 208...208n is combined with the content media data 205 into a media file 209.

[0141] Example of the disclosed bitstream structure

[0142] Figure 3A A hierarchical organization 300 of a disclosed bitstream 301 is illustrated in accordance with some embodiments. The disclosed bitstream 301 provides an extensible container that enables several new, more efficient and more reliable storage, caching, distribution and delivery paradigms for multimedia applications than currently provided by existing approaches. These new multimedia applications are unable to utilize today's source encoding and highly repetitive storage approaches to distribute media over single and / or multiple communication channels / networks. More specifically, the disclosed bitstream 301 implements an extensible and interoperable approach that supports the use of one or more linear codes to improve network-oriented efficiency and reliability of multimedia processing, storage, distribution, delivery and playback for cloud and network connections.

[0143] In an example embodiment, a source-coded multimedia asset is mapped to one or more functionally equivalent binary representations or "bitstreams" through one or more linear encoding processes to generate a binary representation of the source-coded multimedia asset according to the Figure 3A The format of the disclosed bitstream 301 shown in is used to generate multiple functionally equivalent (FE) bitstreams. In some embodiments, each FE bitstream generated by the linear encoding process begins with a SYNC subatomic data structure 302. The information defined in the SYNC data structure 302 is used by subsequent processes to synchronize to, identify, and interpret the disclosed bitstream 301. In some embodiments, the SYNC data structure 302 and the information defined in the SYNC data structure 302 can also be included in the subatomic (bitstream data structure) and injected at any point within the disclosed bitstream 301 for applications that require resynchronization. The information contained in the SYNC data structure 302 provides a common entry point for bitstreams generated by the applications described in the disclosed embodiments, including but not limited to forward compatibility with future applications with future linear code types and subatomics.

[0144] Following the SYNC data structure 302, each disclosed bitstream 301 includes a plurality of subatomic data elements 303a ... 302n (hereinafter referred to as "subatoms") that contain information required to implement one or more applications. Each subatom 303a ... 303n includes a subatom_id field (not shown) that enables the disclosed bitstream 301 to describe and carry one or more unique subatoms and information required to implement one or more applications. As described above, the use of the disclosed subatom_id bit field enables the definition and use of several unique subatomic data structures for the format of the disclosed bitstream 301.

[0145] like Figure 3A As shown, each subatom 303a, 303b, 303c ... 303n is a general container that can be filled with different types of data. Because a subatom can only carry one type of data, the data type carried in the subatom defines the subatom type and the associated data structure. For example, the BITSTREAM_HEADER subatom 305 is an instance of the syntax structure of the subatom 303a, which carries the bitstream header isomorphic structure (BITSTREAM_HEADER).

[0146] Hereinafter, subatoms and their respective bit fields may be more generally referred to as "data elements". In addition, a subatom type may be referenced by its name (e.g., bitstream subatom) or its associated syntax structure (BITSTREAM_HEADER()). In some embodiments, each subatom 303a, 303b, 303c ... 303n includes the following syntax structures: SYNC 304, BITSTREAM_HEADER 305, MEDIA_SEGMENT INFO 306, ENCODER_CONTENT_INFO 307, BLOCK_HEADER 308, PACKET 309, BLOCK_DIRECTORY 310, CHUNKED_SUBATOM 311, and PACKET_HEADER_ONLY 312. Although a valid disclosed bitstream 301 may contain a single subatom, the decoder receives multiple subatoms including BITSTREAM_HEADER, BLOCK_HEADER, and PACKET to successfully decode the disclosed bitstream 301. The syntax allows for other subatom data types to be defined in the future for new applications.

[0147] Figure 3BThe data elements in each sub-atom of the disclosed bitstream 301 container format according to some embodiments are further illustrated. The data carried by the unique instance of the sub-atom described below can be independent or dependent on other sub-atoms. Correct parsing / interpretation and / or decoding of a sub-atom may require correct parsing and / or processing of data from a previous sub-atom instance. The sub-atomic data elements are described as follows.

[0148] The SYNC subatom 304 is used to identify and synchronize two bitstreams generated from the same source media asset. Some examples of the semantics / usage of the SYNC subatom 304 data element (eg, bit field) are described below.

[0149] Sync word: a unique byte sequence used to identify that the disclosed bit stream is compatible with the format of the disclosed bit stream 301 .

[0150] Version: The specific version of the bitstream syntax that follows. This bit field enables the bitstream to be extended in a backward and forward compatible manner.

[0151] content_encode_uuid: A unique identifier that, in one example, enables a multi-source media system (e.g., including encoders, decoders, re-encoders, parsers, processors, etc.) to implicitly signal a specific 'encoding' of a media asset. In one example, this information is useful for downstream processes to efficiently (and independently) determine the lineage of an encoded bitstream generated by one or more encoders or re-encoders. Encoded bitstreams with this level of implicit lineage signaling are advantageous because it enables a multi-source media system to operate most efficiently by ensuring that all coded media sources (distributed and available across multiple nodes, including storage devices / caches in a network) include unique and functionally equivalent encoded representations of the media.

[0152] The BITSTREAM_HEADER subatom 305 contains high-level general file information about the linear encoded data, such as the linear encoding method used, bitstream profile information, and an indication of whether parts of the bitstream are encrypted. Some examples of the semantics / usage of the bitstream_header bit field are as follows.

[0153] · content_source_size: Indicates the size of the linear coded media in bytes.

[0154] content_source_type: Indicates more information about the source of the following linearly encoded media, such as whether the encoded stream originates from the original content, an encoded variant, or a live stream.

[0155] · linear_code_type: Indicates which type of linear code is used to encode the content.

[0156] profile_information: enables processing of bitstreams conforming to a specific profile, which is useful in applications to efficiently determine whether a specific bitstream is suitable for a specific decoder / parser with restricted or limited capabilities.

[0157] encrypted_coefficients: enables implicit signaling that the coded coefficients are encrypted.

[0158] Encryption of the coding coefficients can be viewed as a form of digital rights management, reducing or eliminating the need to fully encrypt the linear coded media itself.

[0159] The MEDIA_SEGMENT_INFO subatom 306 is used for media streaming applications, where, in network and set-top box (OTT) media streaming applications / services, content is divided into segments, such as audio and video segments. Hereinafter, MEDIA_SEGMENT_INFO is also more generally referred to as "media segment data". The structure of the MEDIA_SEGMENET_INFO subatom 306 provides information about the media in the segment, such as its audio / video properties (e.g., frame rate, bit rate, codec type, etc.). This information can typically be found in a manifest file (e.g., a DASH manifest or an HLS manifest). The information is included in the structure to avoid having to decode the bitstream to check the linearly encoded data. An example bit field of the MEDIA_SEGMENT_INFO subatom 306 is as follows.

[0160] segment_index: identifies the media index of the segment represented in the current subatom.

[0161] content_type: indicates the type of multimedia contained in the media segment, such as audio, video, text, or application data.

[0162] aspect_ratio: The ratio of the video's width to its height.

[0163] frame_rate: The number of video frames per second.

[0164] audio_channel_config: Configuration of the audio (e.g., channel, position, object, virtualization, etc.).

[0165] The ENCODER_CONTENT_INFO subatom 307 identifies the specific linear encoder used to create the bitstream and a unique identifier representing the underlying media content. The ENCODER_CONTENT_INFO subatom 307 also carries information that enables verification of the integrity (trustworthiness) of the underlying stream / file / chunk information, as well as information about the duration of the underlying stream / file / chunk information when it is decoded. Some examples of the semantics / usage of the ENCODER_CONTENT_INFO subatom 307 bit field are as follows.

[0166] encoder_uuid: a unique identifier that, in one example, enables a multi-source media system (e.g., including encoders, decoders, recorders, parsers, processors, etc.) to intrinsically signal the lineage of the encoded bitstreams generated by one or more encoders or recorders. Encoded bitstreams with intrinsic lineage signaling is a key feature of the disclosed embodiments that enables a multi-source media system to operate most efficiently by ensuring that all encoded media sources (distributed and available across multiple nodes, including storage devices / caches in a network) include unique and functionally equivalent encoded representations of the media.

[0167] content_id_type: Indicates the type of identification format used to identify the content, such as Entertainment Identification Registry (EIDR), Advertising Digital Identifier (Ad-iD), or a user-defined value.

[0168] · content_id: The content_id bit field carries an ID value.

[0169] In some embodiments, the above data elements enable processing entities throughout a distributed system (including content-centric networks) to efficiently identify information about the underlying coded media without having to decode the media, communicate with a centralized server, or locate a stream / segment manifest (if applicable). This feature is one of several key enablers of future media distribution paradigms that will not use HTTP delivery.file_integrity: A unique data structure that provides a mechanism to verify that a received and decoded stream / block matches the stream / block of the original source. The delivery.file_integrity structure maps to the fb_integrity() bit field, which further specifies information, including, for example, the hash function used to hash the file / stream / block, the hash size, and the actual hash value. In one example, the fb_integrity() bit field enables any processing node in the network to independently verify the authenticity of the coded multi-source bitstream. This functionality also enables the use of traditionally untrusted media sources such as mobile devices, personal computers (PCs), set-top boxes, digital media adapters, and home gateways for storing multi-source coded media.

[0170] The BLOCK_HEADER 308 contains information about the linear encoding parameters applied to the data within the data block, such as block size, symbol size, number of symbols, and finite field size. Some examples of the semantics / usage of the BLOCK_HEADER data element are as follows.

[0171] ·block_index: A multi-source coded media segment may be segmented into blocks, where each block is coded separately. This field indicates the index of the block.

[0172] symbol_size: The size of system and line encoding symbols in bytes.

[0173] number_of_symbols: indicates the number of coding symbols (for linear coding) that the block is divided into.

[0174] field_size: The size of the finite field on which the linear encoding operates.

[0175] · encryption parameters: These fields are used to decrypt the encrypted coding coefficients.

[0176] prng_seed: Linear codes can use a pseudo-random number generator to generate randomized coding decisions for the current block. prng_seed is the seed used to initialize the pseudo-random number generator in a decoder or re-encoder.

[0177] block_integrity: A structure that provides a mechanism to verify that a received and decoded block matches the block of the original source. The structure may contain information such as the hash function used to hash the block, the hash size, and the actual hash. In one example, an implementation may use SHA1 or homomorphic hashing to hash all systematic symbols in the current block, and then use a hash aggregator such as a Merkle tree to aggregate the symbol hashes and obtain the block hash.

[0178] The PACKET subatom 309 contains a piece of basic linearly encoded data, the encoding symbol. It also contains a packet header data structure that provides information about the encoding symbol, such as whether it is a systematic symbol, the window size, and the encoding coefficients. Some example PACKET subatom 309 data elements are:

[0179] packet_header: A set of bit fields that help in decoding packets, such as: 1) block_index: The fragment can be divided into blocks, and each block is encoded separately. This field indicates the index of the block to which the packet belongs; 2) b_systematic: Indicates whether the packet carries systematic symbols or linearly coded symbols; 3) poly_index: This is a general field used to index packets. It can be used to identify the original symbol index of the systematic packet. It can also be used to synchronize the packet header subatom and the packet subatom; 4) coeff_vector: A finite field scalar sequence used to linearly combine the systematic packets to generate the symbol; 5) window_start_index: A packet can be generated using a continuous subset (i.e., window) of systematic symbols. This field indicates the index of the first systematic symbol of the window; 6) window_start_index: A packet can be generated using a continuous subset (i.e., window) of systematic symbols. This field indicates the index of the last systematic symbol of the window; 7) prng_seed: the pseudo-random number generator seed used to generate the coefficients of this symbol; and 8) packet_integrity: a structure that provides a mechanism to verify that a received packet was generated correctly before it is used for decoding or re-encoding. The structure can contain information such as the hash function used to hash the packet, the hash size, and the actual hash.

[0180] In some embodiments, a minimal packet_header may be included in the PACKET subatom 309, but the full packet_header is stored and delivered separately / independently (e.g., out-of-band, separate network session, etc.). If only a minimal packet_header is available, the payload may not be useful for decoding the data. This method implements an additional simple digital rights management method by controlling access to the full packet_header, for example.

[0181] The BLOCK_DIRECTORY subatom 310 contains fields that indicate the start of a block header subatom instance within the disclosed bitstream 301. In some embodiments, the block directory subatom 310 includes one or more of the following fields: block_header_subatom_offsets(), which describes the offset (in bytes) from the end of the BLOCK_DIRECTORY subatom 310 to the beginning of the BLOCK_HEADER subatom 308 in the same disclosed bitstream 301.

[0182] The CHUNKED_SUBATOM 311 provides a mechanism for splitting large subatomic data into smaller chunks. For example, if the encoded symbol size in the PACKET subatom 309 is too large for the transport protocol being used (e.g., the payload of a UDP packet is approximately 1400 bytes), the PACKET subatom 309 can be split into smaller chunks that fit the transport requirements. Each chunk is then carried by one CHUNKED_SUBATOM 311. Some example bit fields are as follows. chunked_segment_id: Identifies a series of chunked_subatom instances as containing different parts of the same subatomic data. When subatomic data is chunked, all created CHUNKED_SUBATOM 311 instances should have the same chunk_segment_id field.

[0183] chunked_segment_index: identifies the index of the chunked segment of the subatomic data contained in the chunked_subatom instance. The parser can use this information to reassemble the chunked subatomic data into its original order.

[0184] num_chunked_segments: describes the number of chunk segments into which the subatomic data (associated with a given chunk_segment_id) is split.

[0185] original_subatom_id: specifies the subatom_id of the segment data contained in this chunked_subatom instance.

[0186] chunked_data: The chunked_subatom instance carries the data chunks.

[0187] The PACKET_HEADER_ONLY subatom 312 is identical to the PACKET subatom 309 except that it does not contain the actual encoding symbols.

[0188] Example media processing and delivery system

[0189] Figure 4 402n, a communication network 403, a bitstream decoder 404, and a post-processing block 405. The network / cloud 402 includes a reprocessor 406 and a bitstream regenerator 407. In some embodiments, the system 400 may be partially or completely implemented in one or more of the following: a server (e.g., a content delivery network (CDN) server, a point of presence (PoP) server, an application server (e.g., a server hosting an electronic news gathering application, a content broadcast application, a unicast application, etc.)), a content cache, a peer device, a local cache device, an edge device (e.g., a 5G / 6G edge application server), or a client device (e.g., a media consumption device, a computer, a mobile device, a gaming system, a streaming device, an audio / video equipment, etc.).

[0190] In this example embodiment, media data is fed to one or more servers. Media data can be obtained from media assets, including but not limited to: images, audio, video, game state, game commands, etc. In some embodiments, the media data is source encoded data, data packaged according to a streaming protocol (e.g., HLS, DASH, etc.). In some embodiments, the media data is linear encoded data or an encoded bitstream associated with the media asset (e.g., an encoded bitstream including encoded content and associated metadata). In some embodiments, media data representing the media asset is received from multiple network paths.

[0191] The preprocessing blocks 401a...401n in the server preprocess the original media data. Examples of preprocessing include, but are not limited to, applying source compression techniques and / or packaging the original media data, as defined, for example, in the HLS and DASH standards. The server also includes corresponding bitstream generators 402a...402n to generate bitstreams. Different bitstream generators 402a...402n can generate different bitstreams, such as bitstreams with different formats or different payloads. The bitstream is then transmitted using a communication network 403, which can cache, route and / or forward the bitstream. The communication network 403 may also include an entity that can further modify the bitstream before retransmitting the bitstream, such as reprocessing 406 and / or regenerating 407 the bitstream.

[0192] The receiver of the bitstream (e.g., a consumer device) downloads one or more bitstreams and uses a bitstream decoder 404 to recover the processed media data. The processed media data is then post-processed by a post-processor 405 and is ready for use, e.g., ready to be consumed by an end user. Optionally, the bitstream decoder 404 sends feedback 408 to the bitstream generators 402a...402n to help upstream processes in the system 400 adapt to dynamic conditions in the communication network 403.

[0193] Example bitstream generation

[0194] Figure 5 is a bitstream generator 500 according to some embodiments (e.g., Figure 4 The pre-processed media data and optional system information that can help decision making are fed to the bitstream generator 500. The bitstream generator 500 generates a bitstream including a reference Figure 3A and Figure 3B 301. In this example embodiment, metadata generator 501 analyzes the pre-processed media data to generate SYNC subatom 304 and MEDIA_SEGMENT_INFO subatom 306. Linear encoder 502 performs linear encoding on the media data, for example, by segmenting the media data into original (source) symbols and then creating linear combinations of the original symbols as linear encoding symbols, as described in reference to Figure 1A and Figure 1B The linear encoding information is referred to together with the encoded data. Figure 3A and Figure 3B The data elements described are carried by BITSTREAM_HEADER, ENCODER_CONTENT_INFO, BLOCK_HEADER, and PACKET subatoms, where each packet subatom corresponds to a linear encoding symbol.

[0195] In some embodiments, the security sealer 503 electronically signs the linear encoded media data by adding hash information about the file, data block, and data packet to the corresponding ENCODER_CONTENT_INFO, BLOCK_HEADER, and PACKET subatoms. This step enables intrinsic signaling of information required for subsequent processing throughout the media system to independently verify / authenticate the encoded bitstream / file generated according to the disclosed invention.

[0196] In another example embodiment, the bitstream generator 500 can also separate the packet header information from the data packet subatom and use a dedicated PACKET_HEADER_ONLY subatom 312 to carry the packet header information to control access to the corresponding data packet subatom 309. The coding coefficients in the PACKET subatom 309 or the PACKET_HEADER_ONLY subatom 312 are encrypted to add a layer of protection to the media content. In some embodiments, each subatom can be further truncated by the subatom truncation device 504 and carried by multiple CHUNKED_SUBATOM subatoms 311 to reduce the size of the subatom. This embodiment is particularly useful when the transport protocol has restrictions on the data unit size.

[0197] The subatoms are then multiplexed by multiplexer 505 ("Mux") to form one or more bitstreams 301. Different bitstreams 301 may carry different subatoms and may be processed (e.g., cached, passed) differently. To help the recipient navigate the subatoms in each bitstream, multiplexer 505 may generate BLOCK_DIRECTORY subatoms 310 on the fly and insert it into each bitstream.

[0198] Example of bitstream decoding disclosed

[0199] Figure 6 is a bitstream decoder 600 according to some embodiments (e.g., Figure 44 ). The bitstream decoder 600 receives data from one or more bitstreams having the disclosed bitstream format, wherein each bitstream includes one or more subatoms. The downloading of the bitstream is performed by a downloader 601, which manages the download (e.g., using a buffer), including but not limited to discovering and selecting media sources, establishing, listening to, and terminating connections, and controlling the speed and amount of data downloaded from each media source. In some embodiments, a subatomic parser 602 parses the bitstream to extract and temporarily cache the received CHUNKED_SUBATOM subatoms 311, and a chunked subatomic merger 603 merges the CHUNKED_SUBATOM subatoms 311. When security / authentication / integrity data is present for the media asset, the merged CHUNKED_SUBATOM subatoms 311 are then verified by a security checker 604.

[0200] The verified subatoms are demultiplexed by the demultiplexer 605, wherein the media metadata is extracted and provided to the post-processor 607 (e.g., an audio / video decoder), and the linear encoded data is fed to the linear decoder 606 to recover the original media data. The decoded media data together with the extracted media metadata are fed to the post-processor 607 for post-processing, so that consumable media data can be obtained at the output of the post-processor 607.

[0201] Example of bitstream re-encoding disclosed

[0202] Figure 7 is a block diagram of a bitstream recoder 700 according to some embodiments. The recoder 700 downloads data from one or more public bitstreams 701, each bitstream 701 including one or more subatoms 304 to 309, 312. The download is performed by a downloader 702, which manages the download (e.g., using a buffer), including but not limited to discovering and selecting media content sources, establishing, listening to, and terminating connections, and controlling the speed and amount of data downloaded from each media source. A subatomic parser 703 parses the bitstream 701 to extract and temporarily cache the received CHUNKED_SUBATOM subatoms 311, and a chunked subatomic merger 704 merges the CHUNKED_SUBATOM subatoms 311. When security / authentication / integrity data is present for the media asset, the merged CHUNKED_SUBATOM subatoms 311 are then verified by a security checker 705.

[0203] The verified subatoms are then demultiplexed by demultiplexer 706, where the media metadata is used to recreate the SYNC subatoms 304 and the MEDIA_SEGMENT_INFO subatoms 306, while the linearly encoded data is fed to a linear reencoder 707 to create new linearly encoded data.

[0204] The creation of new linearly coded data can be implemented in various ways. For example, the recoder 700 can first apply linear decoding to decode the original symbols, and then create a new linear combination of the original symbols as a new linearly coded symbol. For another example, the recoder 700 can directly linearly combine the linearly coded symbols to create a new linearly coded symbol.

[0205] In some embodiments, the linear encoding information is carried along with the linear encoding data by the BITSTERAM_HEADER subatom, the ENCODER_CONTENT_INFO subatom, the BLOCK_HEADER subatom, the PACKET subatom, and the PACKET_HEADER_ONLY subatom, where each packet subatom corresponds to a new linear encoding symbol. The security sealer 708 then electronically signs the media data by adding hash information about the file, data block, and data packet to the corresponding encoder_content_info, block_header, and packet subatoms described above.

[0206] In some embodiments, each subatom can be further truncated by a subatom truncation device 709 and carried by multiple chunked_subatom subatoms to reduce the size of the subatom. This is particularly useful when the transport protocol has restrictions on the data unit size. The subatoms are then multiplexed by a multiplexer 710 to form and output one or more bitstreams 711. Different bitstreams can carry different subatoms and can be processed differently (e.g., cached, passed). To help the recipient navigate the subatoms in each bitstream, the multiplexer 710 creates a block_directory subatom and inserts it into each bitstream.

[0207] Sample Unicode

[0208] Fig. 8A800a according to some embodiments, which can recover media data from multiple bitstreams that have implemented different linear encoding methods. Different bitstreams 801 may use different linear codes, which are signaled according to the code_type bit field in the BITSTREAM_HEADER subatom 305 of the bitstream 801. Downloading is performed by a downloader 802, which manages the download (e.g., using a buffer), including but not limited to discovering and selecting media sources, establishing, listening to, and terminating connections, and controlling the speed and amount of data downloaded from each media source. The subatomic parser 803 parses the bitstream 801 to extract and temporarily cache the received CHUNKED_SUBATOM subatoms 311, which are merged by the chunked subatomic merger 804. When security sealing is enabled for the media asset, the merged CHUNKED_SUBATOM subatom 311 is then verified by the security checker 805.

[0209] The verified subatoms are demultiplexed by a demultiplexer 806, where the media metadata is extracted and provided to a post-processor 809 (e.g., an audio / video decoder), and the linearly encoded data using different linear codecs is fed to an encoding unifier 807. The encoding unifier 807 unifies the linearly encoded data from different linear codecs, for example by aligning coefficients and / or projecting the encoded data onto the same finite field. The unified linearly encoded data is then fed to a linear decoder 808 to recover the original media data. The decoded media data is then fed to a post-processor 809 for post-processing using the extracted media metadata, so that consumable media data can be obtained at the output of the post-processor 809.

[0210] Figure 8B 800b with a decoder switch 810 according to some embodiments. The unified decoder 800b includes a linear decoder module 808 that encapsulates multiple different decoder types. The module 808 allows access to a specific type of linear codec via the decoder switch 810. More specifically, the decoder 800b receives data from one or more bitstreams 801, where each bitstream 801 is composed of one or more subatoms 303 (see Figure 3A and Figure 3B ). Different bitstreams 801 use different linear codecs. Downloading is performed by a downloader 802, which manages the download (e.g., using a buffer), including but not limited to discovering and selecting media sources, establishing, listening, and terminating connections, and controlling the speed and amount of data downloaded from each media source.

[0211] In some embodiments, sub-atom parser 803 parses bitstream 801 to extract sub-atoms and temporarily caches received CHUNKED_SUBATOM sub-atoms 311, which are merged by chunked sub-atom merger 804. When security sealing is enabled for the media asset, the merged CHUNKED_SUBATOM sub-atom 311 is then verified by security checker 805. Security checker 805 then verifies the complete sub-atom when security sealing is enabled for the media asset.

[0212] The verified subatoms are then demultiplexed by the demultiplexer 806, where the media metadata is extracted and provided to a post-processor 809 (e.g., an audio / video decoder), and the linear encoded data using a specific linear codec is fed to the unified linear decoder module 808. The unified linear decoder module 808 identifies the type of linear codec used (the type is signaled according to the code_type bit field in the bitstream_header() of the disclosed bitstream) and dynamically calls and / or switches the decoder switch 810 to the appropriate linear decoder type to decode the original media data. The decoded media data is then fed to the post-processor 809 for post-processing with the help of the extracted media metadata, so that consumable media data can be obtained from the output of the post-processor 809.

[0213] Example unified bitstream recoder

[0214] Fig. 9A 900a according to some embodiments. The recoder 900a can create a new (including linearly independent) bitstream from one or more bitstreams using different linear codecs. More specifically, the recoder 900a downloads data from one or more bitstreams 901, where each bitstream 901 includes one or more subatoms 303 (see Figure 3A and Figure 3B ). The download is performed by a downloader 902, which manages the download (e.g., using a buffer), including but not limited to discovering and selecting sources, establishing, listening to, and terminating connections, and controlling the speed and amount of data downloaded from each media source. A subatomic parser 903 parses the bitstream 901 to extract and temporarily cache the CHUNKED_SUBATOM subatoms 311, which are then merged by a chunked subatomic merger 904. When security sealing is enabled for a media asset, the merged CHUNKED_SUBATOM subatoms 311 are then verified by a security checker 905.

[0215] The verified subatoms are then demultiplexed by a demultiplexer 906, where the media metadata is used to recreate the SYNC subatoms 304 and the MEDIA_SEGMENT_INFO subatoms 306, and the linearly coded data using different linear codecs is fed to an encoding unifier 907. The encoding unifier 907 unifies the linearly coded data using different linear codecs, for example, by aligning coefficients and / or projecting the coded data onto the same finite field. The unified linearly coded data is fed to a linear recoder 908 to create new linearly coded data.

[0216] The creation of new linearly coded data can be implemented in various ways. For example, the recoder 900a can first apply linear decoding to decode the original symbols, and then create a new linear combination of the original symbols as a new linearly coded symbol. For another example, the recoder 900a can directly linearly combine the linearly coded symbols to create a new linearly coded symbol.

[0217] In some embodiments, the linear encoding information is carried together with the linear encoding data in BITSTREAM_HEADER subatom 306, ENCODER_CONTENT_INFO subatom 307, BLOCK_HEADER subatom 308, PACKET subatom 309, and PACKET_HEADER_ONLY subatom 310, where each PACKET subatom 309 corresponds to a linear encoding symbol. Then, the security sealer 909 seals the security of the media data by adding hash information about the file, data block, and data packet to the corresponding ENCODER_CONTENT_INFO subatom 307, BLOCK_HEADER subatom 308, and PACKET subatom 309.

[0218] In some embodiments, each subatom can be truncated by a subatom truncation device 910 and carried by multiple CHUNKED_SUBATOM subatoms 311 to reduce the size of the subatom. This is particularly useful when the transport protocol has restrictions on the data unit size. The subatoms are then multiplexed by a multiplexer 911 to form the output of one or more bitstreams 912. Different bitstreams 912 can carry different subatoms and can be processed differently (e.g., cached, passed). To help the recipient navigate the subatoms in each bitstream 901, the multiplexer 911 can create a BLOCK_DIRECTORY subatom 310 and insert it into each bitstream 901.

[0219] Fig. 9B is a block diagram of a unified bitstream recoder 900b with a recoder switch 914 according to some embodiments. The unified bitstream recoder 900b shows Fig. 9AAnother example embodiment of the disclosed unified bitstream recoder 900a shown in , which implements recoders for multiple types of linear codecs and encapsulates them within a linear recoder module 913. The linear recoder module 913 allows access to a specific type of linear codec via a switch 914.

[0220] More specifically, the re-encoder 900b downloads data from one or more bitstreams 901, each of which includes one or more sub-atoms 303 (see Figure 3A and Figure 3B ). Downloading is performed by a downloader 902, which manages the download (e.g., using a buffer), including but not limited to discovering and selecting sources, establishing, listening, and terminating connections, and controlling the speed and amount of data downloaded from each media source.

[0221] In some embodiments, subatomic parser 903 parses bitstream 901 to extract and temporarily cache CHUNKED_SUBATOM subatoms 311 from bitstream 901, which are then merged by chunked subatomic merger 904. When security sealing is enabled for a media asset, the merged CHUNKED_SUBATOM subatoms 311 are then verified by security checker 905.

[0222] The verified subatoms are then demultiplexed by the demultiplexer 906, where the media metadata is used to recreate the SYNC subatom 304 and the MEDIA_SEGMENT_INFO subatom 306, and the linear encoder module 913 data using a different linear codec is fed to the linear reencoder module 913. The linear reencoder module 913 calls the reencoder switch 914 to select a specific reencoder type based on, for example, a switch signal in the bitstream metadata.

[0223] The creation of new linearly coded data can be implemented in various ways. For example, the recoder 900b can first apply linear decoding to decode the original symbols, and then create a new linear combination of the original symbols as a new linearly coded symbol. For another example, the recoder 900b can directly linearly combine the linearly coded symbols to create a new linearly coded symbol.

[0224] In some embodiments, the linear encoding information is carried together with the linear encoding data in BITSTREAM_HEADER subatom 306, ENCODER_CONTENT_INFO subatom 307, BLOCK_HEADER subatom 308, PACKET subatom 309, and PACKET_HEADER_ONLY subatom 312, where each PACKET subatom 309 corresponds to a linear encoding symbol. Then, the security sealer 909 seals the security of the media data by adding hash information about the file, data block, and data packet to the corresponding ENCODER_CONTENT_INFO subatom 307, BLOCK_HEADER 308, and PACKET subatom 309.

[0225] In some embodiments, each subatom can be truncated by a subatom truncation device 910 and carried by multiple CHUNKED_SUBATOM subatoms to reduce the size of the subatom. This is particularly useful when the transport protocol has restrictions on the data unit size. The subatoms are then multiplexed by a multiplexer 911 to form the output of one or more bitstreams 912. Different bitstreams 912 can carry different subatoms and can be processed differently (e.g., cached, passed). To help the recipient navigate the subatoms in each bitstream 901, the multiplexer 911 can create a block_directory subatom and insert it into each bitstream 901.

[0226] Example bitstream mapping for standards compliance

[0227] Fig.10 A bitstream mapping process 1000 for compatibility with other coded bitstream standards according to some embodiments is illustrated. The disclosed bitstream 301 and its implementations can also be used as a container format to carry linear coded data generated using various linear coding techniques, even if they have been previously specified or standardized. The process 1000 supports existing linear codes by using a bitstream mapper 1003 and a bitstream demapper 1007. The disclosed mapping process 1000 enables the disclosed bitstream to maintain compatibility with existing linear coding methods that have been standardized and implemented in existing media products.

[0228] refer to Fig.10 , the media data to be protected is first encoded using an existing standardized linear encoder 1002 for the linear coding technique under consideration. The output is a standardized bitstream for that technique / method. A bitstream mapper 1003 then extracts the fields of this bitstream and maps them to a bitstream with reference Figure 3A , Figure 3BThe disclosed bitstream 301 is in the format described in the disclosure. The bitstream mapper 1003 also extracts rich media metadata from the application layer 1001 to fill metadata related fields in the disclosed bitstream 301. Then, the bitstream is ready to be delivered through the communication network 1004.

[0229] On the receiving side, there are two options depending on whether to use an existing standardized decoder. First, if an existing standardized decoder 1005 is used, the bitstream demapper 1007 maps the encoding-related data back to an existing standardized bitstream, which is input into a standardized linear decoder 1008, which outputs the media data to an application layer 1009. At the same time, the bitstream demapper 1007 also extracts rich media metadata from the disclosed bitstream 301 and passes it to the receiving side's application layer 1009. If the existing standardized decoder 1005 does not have to be used, or it does not exist, the receiving side uses a bitstream decoder 1006 to decode the media data and metadata for the receiving side's application layer 1009.

[0230] Fig.11 An example implementation of two FE encoded copies of a media file generated using the disclosed bitstream and bit generator according to some embodiments is illustrated. Each encoded copy includes instances of the subatoms disclosed above, including all or a subset of the SYNC subatom 304, the BITSTREAM_HEADER subatom 305, the MEDIA_SEGMENT_INFO subatom 306, the ENCODER_CONTENT_INFO subatom 307, the BLOCK_HEADER subatom 308, the PACKET subatom 309, the BLOCK_DIRECTORY subatom 310, and the PACKET_HEADER_ONLY 312 subatom.

[0231] In particular, the BLOCK_HEADER subatom 310 and PACKET subatom 309 within different copies are generated in a manner that provides maximum linear independence between the PACKET subatoms 309 across all used media sources. This is achieved by uniquely configuring the disclosed bitstream generator for different media sources. Example configurations that can produce linear independence include applying different random seeds at different bitstream generators or using different coding coefficient vectors. In this way, any coded media PACKET subatom 309 from any generated copy will be used for recovery with high probability and is therefore a FE copy (also referred to as "one or more FE copies" hereinafter). If a FE copy carries sufficiently linearly independent PACKET subatoms 309 that allow media stream / file recovery, the copy is called a complete FE encoded copy. Otherwise, it is called a partial FE encoded copy.

[0232] Fig.12 A decentralized multi-source multi-path delivery system 1200 implemented by the disclosed bitstream format according to some embodiments is illustrated. The system 1200 includes: 1) an origin server 1201, which has source-encoded original media files; 2) multiple cache clusters 1202, 1203, 1204, each of which contains connected caches that can store, process (e.g., such as re-encoding and decoding) and serve data files. Some examples of clusters include, but are not limited to: a CDN, a group of edge computing nodes, an ISP node with caching capabilities, a group of peer users, a mix of different types of caches; and an end user 1205 who wants to obtain and consume the original media file.

[0233] In the existing configuration of the system 1200, the source server 1201 distributes the original media file to all connected caches 1202, 1203, 1204 for replication. In contrast, in the disclosed system using the disclosed bitstream 301, the source server 1201 uses the disclosed bitstream generator (see Figure 4 ) to create different FE-encoded copies of the original media file, and then distribute each copy to a different connection cache 1202, 1203, 1204.

[0234] In the existing configuration of the system 1200, a cache that is not connected to the source server 1201 will connect to one of its neighbors to create a copy of the original media file. In contrast, in the disclosed system using the disclosed bitstream 301, such a cache connects to multiple neighbors and uses the disclosed bitstream re-encoder described with reference to FIG. 9 to efficiently use the neighbor's FE copy to create its own FE encoded copy. The newly created FE copy can also be a complete or partial copy. The cache 1202, 1203, 1204 can even be an end user 1205 who wishes to contribute as a peer.

[0235] System 1200 is a decentralized media delivery system in which all caches store different but FE-encoded copies of the same original media file. This enables efficient multi-source multi-path downloading: end-user 1205 can download from multiple caches 1202, 1203, 1204 in parallel via different communication links, and as long as the total amount of data downloaded is slightly higher than the original media file (due to header overhead, etc.), end-user 1205 can terminate all downloads and use Figure 4 The disclosed bitstream decoder 404 shown recovers the original media file. In this way, the end user 1205 can enjoy the aggregate bandwidth and stability of all communication links without increasing the download cost.

[0236] The above download process is superior to existing single-source single-path download processes, in which the end user 1205 only connects to one of the caches 1202, 1203, 1204 to download the original media file. The performance of existing download processes is limited by the single link between the cache and the end user 1205, and is susceptible to cache or link failures. Note that the disclosed download process is also superior to existing replication-based multi-source multi-path download processes, at least because existing replication-based multi-source multi-path download processes require time-consuming and error-prone scheduling between downloads to avoid downloading duplicate data from the media source.

[0237] Fig.13 A low latency unicast bitstream delivery system 1300 is illustrated in accordance with some embodiments. System 1300 illustrates an example use of the disclosed bitstream 301 in a low latency unicast use application where a recipient requires low latency and reliable data stream delivery via a lossy communication channel. For this use case, the media stream is preprocessed by a preprocessor unit 1301 into small chunks, such as video frames, that can be sent to a recipient in a short time. A bitstream generator 1302 is applied to the preprocessed bitstream to create the disclosed bitstream 301. Subatoms 1303 (such as SYNC subatoms 304, BITSTREAM_HEADER subatoms 305, MEDIA_SEGMENT_INFO subatoms 306) that carry metadata and do not change frequently are then delivered to the recipient via a reliable control channel, which may have higher latency and lower bandwidth than other channels.

[0238] On the other hand, linear coding related subatoms 1304 (such as BLOCK_HEADER subatomic 308 and CHUNKED_PACKET_SUBATOM subatomic 311) are sent to the receiver via a low latency lossy data channel to achieve low latency decoding of data chunks for post-processing. Optionally, the bitstream decoder 1305 can observe the channel conditions and send feedback to the bitstream generator 1302 via a feedback channel to help the bitstream generator 1302 make wise coding decisions.

[0239] Fig.14A1 is a block diagram of a system 1400 for low latency unicast encoding and decoding according to some embodiments, with emphasis on the PACKET subatom 309. The pre-processed media stream 1401 is fed to a bitstream generator 1402 in time scales, and each scale includes one or more equal-length data symbols to be linearly encoded. In the example shown, scales 1, 2, and 4 each include 1 data symbol, while scale 3 includes 2 data symbols (3.2, 3.1). The bitstream generator 1402 creates a PACKET subatom 309 for each data symbol as a system PACKET subatom 309, and also creates a linearly encoded data packet by applying linear encoding to these data symbols. The bitstream decoder 1403 on the receiving side uses the received system PACKET subatom 309 and the encoded data packet and other supporting subatoms to decode the scale data and pass the data to the receiving side's application layer.

[0240] Fig. 14B The variable data size is handled out of band via size marker 1404 according to some embodiments. Preprocessor 1401 generates data in units of scale, wherein the data of each scale can have different sizes. In some embodiments, size marker 1404 measures the size of each scale data, and pre-adds size information to the scale data before the scale data is sent to bitstream generator 1405. From the perspective of bitstream generator 1405, this size information is included in the data and is linearly encoded. In the example shown, there are 3 data scales: P1, P2 and P3. Their sizes are pre-added by size marker 1404 and become part of the data.

[0241] Within the bitstream generator 1405, a zero filler 1406 fills the data with zeros to equalize its length for linear encoding. In some embodiments, the length is the largest scale data among the scales to be encoded together. In some embodiments, a global maximum length is applied. In the example shown, 3 scales are linearly encoded together, and since P1 is the largest, the symbol length is set to the length of P1 with the size pre-added. Therefore, P2 and P3 are filled with zeros before linear encoding.

[0242] When the tick data is transmitted as the system PACKET subatom 309, the padding zeros do not need to be transmitted. This is because when the bitstream decoder 1408 receives the system PACKET subatom 309, it can directly pass the symbols within the PACKET subatom 309 to the padding stripper 1409 described below. The padding stripper 1409 parses the size and realizes that there are no zeros to strip. The padding stripper 1409 removes the size prefix and passes the tick data back to the application.

[0243] In the example shown, it is assumed that the transmission of P3 as the systematic PACKET subatom 309 is successful, so the bitstream decoder 1408 receives the pre-sized P3 without zero padding and passes it to the padding stripper 1407. The b_systematic flag in the PACKET subatom 309 is set to true for making the decision. When the bitstream decoder 1408 receives the encoded PACKET subatom 309, by the size of the encoded symbol, it knows how many zeros it should pad to the systematic symbol it receives to enable decoding.

[0244] If the systematic symbol is obtained via decoding, the bitstream decoder 1408 does not know how many zeros are filled. In this case, the bitstream decoder 1408 passes the zero-filled symbol to the padding stripper 1409, and the padding stripper 1409 parses the size information and strips accordingly. In the example shown, assuming that the transmission of the systematic PACKET subatom 309 of P2 fails, P2 is obtained via decoding. The decoded P2 is filled with zeros, and the padding stripper 1409 will remove these zeros after parsing the size information.

[0245] Fig.15A A low latency unicast, separate linear encoder-decoder pair is illustrated according to some embodiments. In particular, Fig.15A A method for protecting multiple scales of data 1501 using the disclosed bitstream implementation is shown, with an emphasis on linear encoding and decoding. In the method, for each scale, a linear encoder 1502 is created. The linear encoder 1502 treats the data from that scale and several earlier scales as a block and applies a linear encoding to the block. On the receiving side, each linear encoder 1502 is paired with a linear decoder 1503, which decodes the data of the corresponding block and passes the decoded scale data directly to the application or to a buffer 1504, which reorders the scale data and removes duplicates.

[0246] The above method can be implemented in an adaptive manner. For example, the number of scales to be encoded together can be dynamically changed based on factors such as network quality, system latency budget, and other suitable factors. For another example, the scale at which the encoder should be created can also be dynamically determined based on factors such as network quality and system latency budget, rather than one encoder per scale.

[0247] Fig. 15B Illustrated is low latency unicast using cross-generation decoders according to some embodiments. In particular, Fig. 15BA method of using the disclosed bitstream 301 to protect multiple scales of data 1501 is presented, with an emphasis on linear encoding and decoding. In the method, for each scale, a linear encoder 1502 is created. The linear encoder treats the data from that scale and several earlier scales as a block and applies a linear encoding to the block. On the receiving side, a cross-block linear decoder 1505 is implemented that collects PACKET subatoms 309 from one or more blocks and decodes them together, for example by solving a set of extended linear equations. The cross-block decoder 1505 also reorders the scale data and removes duplicates, and then passes the data to the receiving application layer.

[0248] In some embodiments, the above method can be implemented in an adaptive manner. For example, the number of ticks to be encoded together can be dynamically changed based on factors such as network quality and system latency budget. For another example, the number of blocks that the cross-block decoder 1505 will decode together can also be dynamically determined based on factors such as network quality and system latency budget.

[0249] Fig.16 A system 1600 for using the disclosed bitstream 301 for digital rights management (DRM) according to some embodiments is illustrated. In particular, Fig.16 It is shown how the proposed bitstream can implement simple digital copyright protection. In order to decode or re-encode linearly encoded media data, the receiver 1602 obtains linear encoding information such as coding coefficients, generation size, packet size, random seed, etc. Otherwise, the receiver 1602 cannot recover the media data. Therefore, by encrypting such information in the corresponding subatom of the proposed bitstream, the content owner 1601 can prevent unauthorized access to the media data. This enables the implementation of a simple DRM system using the proposed bitstream.

[0250] In some embodiments, when preparing the content, the content owner 1601 encrypts the linear encoding information using a cryptographic encryption scheme (e.g., such as AES). The encrypted bitstream is then distributed to the recipient 1602 via a communication network. The content owner 1601 then stores the decryption key to the authentication server 1603. The authentication server 1603 authenticates the recipient 1602 based on its ID, and if the authentication is successful, sends the decryption key to the recipient 1602. The recipient 1602 then decrypts the linear encoding information using the decryption key and uses the information to decode or re-encode the media data.

[0251] Fig.17 is a block diagram of a decentralized media delivery system 1700 with data authentication according to some embodiments. In particular, Fig.17It is demonstrated how the disclosed bitstream can enable data authentication in a decentralized media delivery system where untrusted caching nodes (e.g., peers) may exist.

[0252] In the example embodiment shown, the content creator 1701 encodes the media data into multiple bitstreams using the disclosed bitstream with the security sealing feature enabled. For example, file integrity may be included in the ENCODER_CONTENT_INFO subatom 307, block integrity is included in the BLOCK_HEADER subatom 308, and packet integrity is included in the PACKET subatom 309. The content creator 1701 then distributes the bitstream to a network 1703 of cache nodes 1704 to 1706. Although three cache nodes are shown in this example, multiple cache nodes may be included in the network 1703.

[0253] Cache nodes 1704 to 1706 can store the bitstream intact, or create a new bitstream via a re-encoding function implemented by the disclosed bitstream format. Among them, untrusted cache nodes 1705, 1706 may be hostile and generate corrupted bitstreams, especially PACKET subatoms 309 that are not correctly linearly encoded.

[0254] The receiving node 1707 (which may be an end user or a new cache node) may authenticate the PACKET subatom 309 it downloads from the untrusted cache nodes 1705, 1706 by first downloading the correct ENCODER_CONTENT_INFO subatom 307 and BLOCK_HEADER subatom 308 to obtain the file integrity information and block integrity information. When such information is appropriately generated (e.g., using homomorphic hashing), it may be used to verify the integrity of the PACKET subatom 309 generated by the untrusted node.

[0255] In order to download the correct ENCODER_CONTENT_INFO subatom 307 and BLOCK_HEADER subatom 308, the recipient node 1707 can use a trusted cache node 1704, such as a cache node operated by an authorized party. Alternatively, since the ENCODER_CONTENT_INFO subatom 307 and the BLOCK_HEADER subatom 308 are static for each media asset, the content creator 1701 can cryptographically sign these two subatoms so that the recipient 1707 can download these two subatoms from anywhere and verify them through the content creator's cryptographic signature.

[0256] Additionally, when a receiving node 1707 discovers that a PACKET subatom 309 received from a node is corrupted, it can report this violation to the regulatory manager 1702 of the system 1700 by submitting, for example, a forensic fraud certificate. The regulatory manager 1702 can then punish the node that did the wrong thing by, for example, imposing a fine, canceling a subscription, and blacklisting the node from the network 1703.

[0257] Example Process

[0258] Fig.18 is a flow chart of a process 1800 for generating a bitstream from non-linear or linearly encoded data according to some embodiments. Process 1800 may use, for example, reference Fig. 22 The disclosed electronic device architecture is implemented.

[0259] Process 1800 includes the following steps: at a first device: receiving media data representing a media asset (1801); obtaining a first plurality of data elements including at least one of bitstream identification data, content-specific encoding data, and media segment data (1802); encoding at least a portion of the media data into encoded data corresponding to the media asset according to a first linear encoding process (1803); generating a second plurality of data elements different from the first plurality of data elements based on information associated with the first linear encoding process (1804); combining the first plurality of data elements and the second plurality of data elements into one or more encoded bitstreams representing the media asset (1805); and transmitting the one or more encoded bitstreams to one or more devices using one or more network paths (1806).

[0260] In some embodiments, process 1800 is implemented by one or more devices, including but not limited to: one or more servers (e.g., content delivery network (CDN) servers, point of presence (PoP) servers, one or more application servers (e.g., servers hosting electronic news gathering applications, content broadcast applications, unicast applications, etc.), content caches, peer devices, local caching devices, edge devices (e.g., 5G / 6G edge application servers), or client devices (e.g., media consumption devices, computers, mobile devices, gaming systems, streaming devices, audio / video equipment, etc.).

[0261] In some embodiments, media assets include, but are not limited to, images, videos, game states, game command data, and the like.

[0262] In some embodiments, the media data includes, but is not limited to, source encoded data, data packaged according to a streaming protocol (e.g., HLS (HTTP-like streaming), DASH (Dynamic Adaptive Streaming over HTTP), etc. In some embodiments, the media data is linear encoded data or an encoded bitstream associated with a media asset (e.g., an encoded bitstream including encoded content and associated metadata). In some embodiments, media data representing a media asset is received from multiple network paths.

[0263] In some embodiments, the media segment data identifies characteristics of the media asset in the corresponding segment or chunk (e.g., MEDIA_SEGMENT_INFO). In some embodiments, the first plurality of data elements includes metadata associated with (e.g., derived from, corresponding to, received with) the media asset (e.g., the SYNC subatom and the MEDIA_SEGMENT_INFO subatom). In some embodiments, the first plurality of data elements does not include linearly encoded payload data.

[0264] In some embodiments, encoding according to a first linear encoding process (e.g., a linear encoding process corresponding to a supported linear code type) includes applying a linear encoding technique to a data packet including at least a first portion of the media data (e.g., including generating an encoded data packet that is a combination of multiple data packets corresponding to at least a portion of the media data).

[0265] In some embodiments, encoding according to the first linear encoding process is performed without applying a linear encoding technique to at least a portion of the media data (e.g., without including a process to create an encoded data packet by combining a plurality of data packets corresponding to at least a portion of the media data). For example, the encoding process can include applying a mapping function (e.g., one or more mapping rules or tables) associated with the first linear encoding process to transform at least a portion of the media data into encoded data without applying the first linear encoding process.

[0266] In some embodiments, the second plurality of data elements includes encoded payload data and metadata associated with the media asset (e.g., one or more of the BITSTREAM_HEADER subatom 305, the ENCODER_CONTENT_INFO subatom 307, the BLOCK_HEADER subatom 308, the PACKET subatom 309, or the PACKET_HEADER_ONLY subatom 312 and their associated bit fields).

[0267] In some embodiments, combining includes interleaving, multiplexing, or otherwise converting to a sequential signal or representation for transmission or storage.

[0268] In some embodiments, one or more network paths include more than one network, more than one network type (e.g., LAN, VLAN, WAN, SAN, WLAN, VPN, etc.), more than one network standard (e.g., 1G / 2G / 3G / 4G / 5G / 6G mobile, Bluetooth, 802.11-based WIFI, etc.), more than one network transmission medium (e.g., wireless, wired, fiber optic, etc.), and communication channels with different performance characteristics (e.g., utilizing lossy protocols, reliable protocols, and channels with lower latency).

[0269] Fig.19 is a flow chart of a decoding / re-encoding process 1900 according to some embodiments. The process 1900 may be performed using, for example, reference Fig. 22 The disclosed electronic device architecture is implemented.

[0270] Process 1900 includes the following steps: at a first electronic device: receiving one or more encoded bit streams associated with a media asset (1901); storing the one or more encoded bit streams in a buffer (1902); processing the one or more encoded bit streams stored in the buffer, the processing including: extracting a first plurality of data elements including at least one of a bit stream identifier, content-specific encoding data, and media segment data from the one or more encoded bit streams (1903); extracting a second plurality of data elements including information associated with a corresponding linear encoding process used to generate the one or more encoded bit streams from the one or more encoded bit streams (1904); and performing at least one of the following: generating decoded data representing the media asset based on the second plurality of data elements, or generating new encoded data representing the media asset based on the second plurality of data elements (1905).

[0271] In some embodiments, the first device is a server (e.g., a content delivery network (CDN) server, a point of presence (PoP) server, an application server (e.g., a server hosting an electronic news gathering application, a content broadcast application, a unicast application, etc.)), a content cache, a peer device, a local cache device, an edge device (e.g., a 5G / 6G edge application server), a client device (e.g., a media consumption device, a computer, a mobile device, a gaming system, a streaming device, audio / video equipment, etc.).

[0272] In some embodiments, the one or more encoded bitstreams include a bitstream encoded according to one of at least two different linear coding protocols / standards.

[0273] In some embodiments, the processing includes demultiplexing and / or parsing.

[0274] In some embodiments, the media segment data is used to identify characteristics of the media asset in the corresponding segment or chunk (eg, MEDIA_SEGMENT_INFO subatom 306).

[0275] In some embodiments, the first plurality of data elements includes metadata (eg, SYNC subatom 304 and MEDIA_SEGMENT_INFO subatom 306 ) associated with (eg, derived from, corresponding to, received with) the media asset.

[0276] In some embodiments, the first plurality of data elements does not include linearly encoded payload data.

[0277] In some embodiments, the second plurality of data elements includes encoded payload data and metadata associated with the media asset (eg, one or more of BITSTREAM_HEADER subatom 305, ENCODER_CONTENT_INFO subatom 307, BLOCK_HEADER subatom 308, PACKET 309, PACKET_HEADER_ONLY 312 subatom, etc.).

[0278] In some embodiments, the encoding data corresponds to different ones of the at least two different linear encoding protocols / standards.

[0279] Fig. 20 is a flow chart of a process 2000 for converting a new code type to achieve downstream compatibility according to some embodiments. The process 2000 may use, for example, reference Fig. 22 The disclosed electronic device architecture is implemented.

[0280] Process 2000 includes the following steps: at a first electronic device: obtaining a first encoded bit stream of a first type (2001); generating a second encoded bit stream of a second type different from the first type (2002); and transmitting the second encoded bit stream to one or more second electronic devices via one or more network paths (2003).

[0281] In some embodiments, the first bitstream of the first type is encoded according to a first type of linear encoding protocol or standard (e.g., RaptorQ TM ) generated by

[0282] In some embodiments, the first bitstream or the second bitstream includes systematic data packets associated with the data segments and non-systematic data packets associated with the data segments.

[0283] In some embodiments, the first bitstream or the second bitstream includes system data packets associated with the data segment and does not include non-system data packets associated with the data segment.

[0284] In some embodiments, the first bitstream or the second bitstream includes non-system data packets associated with the data segment and does not include system data packets associated with the data segment.

[0285] In some embodiments, the second bitstream of the second type is generated according to a second type of linear encoding protocol or standard (eg, Reed Solomon code).

[0286] In some embodiments, one or more network paths include more than one network, more than one network type (e.g., LAN, VLAN, WAN, SAN, WLAN, VPN, etc.), more than one network standard (e.g., 1G / 2G / 3G / 4G / 5G / 6G mobile, Bluetooth, 802.11-based WIFI, etc.), more than one network transmission medium (e.g., wireless, wired, fiber optic, etc.), communication channels with different performance characteristics (e.g., channels utilizing lossy protocols, reliable protocols, lower latency protocols, etc.), etc.).

[0287] Fig.21 is a flow chart of another process for converting new code types to achieve downstream compatibility according to some embodiments. Process 2100 may use, for example, reference Fig. 22 The disclosed electronic device architecture is implemented.

[0288] Process 2100 includes the following steps: at a first electronic device: receiving a first bitstream associated with a media asset encoded using a first type of linear encoding (2101); based on a determination that the first bitstream should be decoded using a decoder associated with a second type of linear encoding, processing the first bitstream into a second bitstream of a second type of linear decoding, including: extracting data from a set of fields of the first bitstream and mapping the data to a second set of corresponding fields of the second bitstream (2102); extracting metadata associated with the media asset from the first bitstream (2103); decoding the second bitstream into an unencoded representation of the media asset using a linear decoder associated with the second type of linear encoding (2104); based on a determination that the first bitstream should not be decoded using a decoder associated with the second type of linear encoding, processing the first bitstream, including: decoding the first bitstream into an unencoded representation of the media asset using a linear decoder associated with the first type of linear encoding and metadata associated with the media asset (2105); and passing the unencoded representation of the media asset and the extracted metadata associated with the media asset to an application (2106).

[0289] Each of the processes 1800, 1900, 2000, and 2100 disclosed above allows combining data elements associated with a linear encoding process with data elements including at least one of bitstream identification data, content-specific encoding data, and media segment data prior to transmission to enable more efficient delivery of media data to / from devices having various processing capabilities (e.g., encoding / decoding capabilities) over networks having different performance characteristics. Combining data provides more efficient decoding and reduces the need to generate and transmit multiple redundant types of bitstreams to ensure compatibility, thereby saving network and processing resources.

[0290] Example system architecture

[0291] Fig. 22 A block diagram of an example electronic device architecture 2200 suitable for implementing an example embodiment of the present disclosure is shown. The architecture 2200 includes, but is not limited to, a server and a client device, as previously described with reference to FIGS. Fig.21 as described.

[0292] As shown, the architecture 2200 includes a central processing unit (CPU) 2201 that can execute various processes according to a program stored in, for example, a read-only memory (ROM) 2202 or a program loaded from, for example, a storage unit 2208 to a random access memory (RAM) 2203. In the RAM 2203, data required when the CPU 2201 executes various processes is also stored as needed. The CPU 2201, the ROM 2202, and the RAM 2203 are connected to each other via a bus 2204. An input / output (I / O) interface 2205 is also connected to the bus 2204.

[0293] The following components are connected to the I / O interface 2205: an input unit 2206, which may include a keyboard, a mouse, etc.; an output unit 2207, which may include a display such as a liquid crystal display (LCD) and one or more speakers; a storage unit 2208, which includes a hard disk or another suitable storage device; and a communication unit 2209, which includes a network interface card such as a network card (e.g., wired or wireless).

[0294] In some embodiments, the input unit 2206 includes one or more microphones located at different locations (depending on the host device) that enable capture of audio signals in various formats (e.g., mono, stereo, spatial, immersive, and other suitable formats).

[0295] In some embodiments, the output unit 2207 includes a system with various numbers of speakers. The output unit 2207 (depending on the capabilities of the host device) can render audio signals in various formats (e.g., mono, stereo, immersive, binaural, and other suitable formats).

[0296] In some embodiments, the communication unit 2209 is configured to communicate with other devices (e.g., via a network). The drive 2210 is also connected to the I / O interface 2205 as needed. A removable medium 2211 (such as a magnetic disk, an optical disk, a magneto-optical disk, a flash memory drive, or another suitable removable medium) is installed on the drive 2210 so that a computer program read therefrom is installed into the storage unit 2208 as needed. It will be understood by those skilled in the art that although the system 2200 is described as including the above components, in actual applications, some of these components may be added, removed, and / or replaced, and all such modifications or changes fall within the scope of the present disclosure.

[0297] According to example embodiments of the present disclosure, the processes described above may be implemented as a computer software program or on a computer-readable storage medium. For example, an embodiment of the present disclosure includes a computer program product, the computer program product including a computer program tangibly embodied on a machine-readable medium, the computer program including program code for executing the method. In such an embodiment, the computer program may be downloaded and installed from a network via the communication unit 2209, and / or installed from the removable medium 2211, such as Fig. 22 shown.

[0298] In general, various exemplary embodiments of the present disclosure may be implemented in hardware or dedicated circuits (e.g., control circuits), software, logic, or any combination thereof. For example, the units discussed above may be implemented by a control circuit (e.g., CPU 2201 and Fig. 22 Other components of the present disclosure) are executed, and thus, the control circuit can perform the actions described in the present disclosure. Some aspects may be implemented in hardware, while other aspects may be implemented in firmware or software (e.g., control circuit) that can be executed by a controller, microprocessor, or other computing device. Although various aspects of the example embodiments of the present disclosure are illustrated and described as block diagrams, flow charts, or using some other graphical representation, it should be understood that the blocks, devices, systems, techniques, or methods described herein may be implemented in hardware, software, firmware, dedicated circuits or logic, general-purpose hardware or controllers, or other computing devices, or some combination thereof, as non-limiting examples.

[0299] In addition, the various blocks shown in the flowchart can be viewed as method steps, and / or operations generated by the operation of computer program code, and / or multiple coupled logic circuit elements configured to perform associated functions. For example, embodiments of the present disclosure include a computer program product, the computer program product including a computer program tangibly embodied on a machine-readable medium, the computer program including program code configured to perform the method described above.

[0300] In the context of the present disclosure, a machine-readable medium may be any tangible medium that may contain or store a program for use by an instruction execution system, device or equipment or in combination with an instruction execution system, device or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may be non-transient and may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0301] The computer program code for performing the disclosed method can be written in any combination of one or more programming languages. These computer program codes can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device with a control circuit, so that the program code, when executed by the processor of the computer or the processor of the other programmable data processing device, implements the function / operation specified in the flow chart and / or block diagram. The program code can be executed completely on a computer, partially on a computer, as an independent software package, partially on a computer and partially on a remote computer, or completely on a remote computer or server, or distributed on one or more remote computers and / or servers.

[0302] Other embodiments

[0303] In some embodiments, encoding according to a first linear encoding process (e.g., a linear encoding process corresponding to a supported linear code type) includes applying a linear encoding technique to a data packet including at least a first portion of the media data (e.g., including generating an encoded data packet that is a combination of multiple data packets corresponding to at least a portion of the media data).

[0304] In some embodiments, encoding according to a first linear encoding process is performed without applying a linear encoding technique to at least a portion of the media data (e.g., without including a process for creating an encoded data packet by combining a plurality of data packets corresponding to at least a portion of the media data). For example, the encoding process can include applying a mapping function (e.g., one or more mapping rules or tables) associated with the first linear encoding process to transform at least a portion of the media data into encoded data without applying the first linear encoding process.

[0305] Advantages of the Disclosed Embodiments

[0306] The disclosed bitstream provides the following advantages.

[0307] 1. The disclosed embodiments support the use of multiple linear encoding techniques in the same bitstream, including any linear code, including but not limited to block codes.

[0308] 2. The disclosed embodiments contain metadata that allows media assets to be encoded, distributed, re-encoded, cached, and then decoded at the delivery point on a multi-source multi-path media data delivery system.

[0309] 3. The disclosed bitstream 301 (subatomic and metadata) allows multi-source multi-path delivery without the need for centralized control or coordination between sources, and enables minimal redundancy at the delivery points.

[0310] 4. The disclosed embodiments allow subatomic systems to use different methods to send different types of data, which enables flexible and heterogeneous delivery of different subatomics. For example, some subatomics (such as metadata) can go into reliable channels, while other subatomics (such as media essence) can go into unreliable channels. Some subatomics can be encrypted, and some can be unencrypted.

[0311] 5. The disclosed embodiments allow flexible bitstream subatom size to suit the size of the communication protocol data unit by using the CHUNKED_SUBATOM subatom.

[0312] 6. The disclosed embodiments allow the media asset to be chunked into smaller chunks, and then linear codes are applied to different chunks separately by using the block_index field of the BLOCK_HEADER subatom to indicate the different chunks.

[0313] 7. The disclosed embodiments allow linear re-encoding of encoded data within a particular subatom without changing the subatom structure (e.g., the output is the same subatom structure and can be consumed by a decoder without change).

[0314] 8. The disclosed embodiments allow the creation of functionally equivalent code copies of various sizes to meet the storage constraints of different devices without changing the subatomic structure.

[0315] 9. The disclosed embodiments support the ability to encode a media asset with multiple different linear codes and deliver it via multiple sources (e.g., CDNs), and when the multiple bitstreams are available at the decoding location, they can be decoded using the same bitstream decoder.

[0316] 10. The disclosed embodiments support the ability to linearly re-encode media assets that have been encoded with multiple different linear codes and delivered via multiple sources (e.g., CDNs), and re-encode the multiple bitstreams using the same bitstream codec when they are available at the re-encoding location.

[0317] 11. The disclosed embodiments allow for the conversion of media assets that have been encoded into a bitstream using an existing standardized linear code format (e.g., RaptorQ, RS), which can then be mapped to the proposed bitstream via a mapping function. A demapping function is applied at the decoder of the proposed bitstream to restore the original standardized bitstream back to a standardized format.

[0318] 12. The disclosed embodiments support the selection and use of linear codes, which are either sliding window or block codes. If a sliding window code is used at the encoder, the format allows the decoder to use block decoding or sliding window decoding.

[0319] 13. The disclosed embodiments support delivery of low latency lossy media data unicast systems by selecting appropriate linear codes (e.g., supporting sliding window codes) and subatomic assets, which can be sent over a reliable channel while the media assets are sent via an unreliable channel.

[0320] 14. The disclosed embodiments enable encryption of selected subatomic elements and fields in a bitstream to prevent unauthorized decoders from recovering media assets (eg, DRM).

[0321] 15. The disclosed embodiment supports different levels of data integrity checking (e.g., file level, block level, and packet level checking) using fields that carry authentication information. In particular, it supports the use of special hashing algorithms (such as homomorphic hashing) to implement integrity checking of linearly encoded data packets without decoding.

[0322] In addition, the disclosed bitstream generator provides the following advantages.

[0323] 1. The disclosed bitstream generator may chunk a subatom into smaller chunks (CHUNKED_SUBATOM subatom 311) to fit the size of a communication protocol data unit.

[0324] 2. The disclosed bitstream generator may chunk the media asset into smaller chunks, and then apply linear codes to different chunks respectively and use the block_index field of the BLOCK_HEADER subatom to indicate the different chunks.

[0325] 3. The disclosed bitstream generator can use a subatomic multiplexer to create multiple different bitstreams, and different bitstreams can be delivered to different network entities.

[0326] 4. The disclosed bitstream generator can receive channel information and use it to adapt the coding scheme.

[0327] 5. The disclosed bitstream generator selects a linear coding technique from a plurality of linear coding techniques for encoding the media into a bitstream by signaling with the subatomic BITSTREAM_HEADER using the field linear_code_type.

[0328] 6. The disclosed bitstream generator can generate multiple linear encoding variants of the same media asset by placing the same media information (such as the same SYNC, BITSTREAM_HEADER, ENCODER_CONTENT_INFO and MEDIA_SEG_INFO subatoms) at different variants to uniquely identify the media asset, and placing different linear code information (such as different prng_seed in the BLOCK_HEADER subatom, different poly_index and different coefficients in the PACKET subatom) to distinguish the variants. In this way, the variants can be distributed through a multi-source data delivery system and can all contribute to the decoding of the media asset.

[0329] 7. The disclosed bitstream generator can signal whether to use block code or sliding window code through various fields, such as setting the linear_code_type field of the BITSTREAM_HEADER subatom to different values, setting the content_source_size field of the BITSTREAM_HEADER subatom to zero for sliding window code and / or using dynamic window_start_index and window_end_index of the PACKET subatom for sliding window code.

[0330] 8. The disclosed bitstream generator can dynamically change linear coding techniques and parameters based on receiver feedback.

[0331] 9. The disclosed bitstream generator may encrypt the coding information (eg, the coefficient_vector field of the PACKET subatom) and carry the coding information separately (eg, using the PACKET_HEADER_ONLY subatom) to prevent unauthorized decoding of the media asset.

[0332] 10. The disclosed bitstream generator can use various hashing techniques (SHA2, SHA256, homomorphic hashing, etc.) to hash media assets in various ways (hashing files, hashing blocks, hashing raw symbols, etc.) to implement data integrity checks. In particular, it can use special hashing algorithms (such as homomorphic hashing) to implement integrity checks on linearly encoded data packets without decoding.

[0333] Additionally, the disclosed bitstream re-encoder embodiments provide the following advantages.

[0334] 1. The disclosed bitstream recoder can be implemented in different network entities to perform bitstream recoder and generate functionally equivalent fully encoded copies and partially encoded copies without coordinating with each other.

[0335] 2. The disclosed bitstream reencoder can identify and re-encode using different linear encoding variants of the same media asset through the same media information from different variants (such as the same SYNC, BITSTREAM_HEADER, ENCODER_CONTENT_INFO and MEDIA_SEG_INFO subatoms) and different linear code information (such as different prng_seed in the BLOCK_HEADER subatom, different poly_index and different coefficients in the PACKET subatom).

[0336] 3. The disclosed bitstream re-encoder can unify and re-encode different bitstreams of the same media asset generated using different linear codes.

[0337] 4. The disclosed bitstream encoder can decrypt (for re-encoding) and re-encrypt the encoded information (such as the coefficient_vector field of the PACKET subatom), and carry the encoded information separately (such as using the PACKET_HEADER_ONLY subatom) to prevent unauthorized decoding of the media assets.

[0338] 5. The disclosed bitstream re-encoder can verify the integrity of media assets in various ways (file verification, block verification, packet verification, etc.) based on hashing techniques (SHA2, SHA256, homomorphic hashing, etc.) and data integrity fields specified in subatomic. In particular, when using special hashing techniques such as homomorphic hashing, it can verify linear encoded packets without decoding.

[0339] Additionally, the disclosed bitstream decoder embodiments provide the following features / advantages.

[0340] 1. The disclosed bitstream decoder can identify and download multiple different bitstreams of the same media asset and use them to decode the media asset.

[0341] 2. The disclosed bitstream decoder can decode different blocks of the same media asset separately and connect them in the correct order to restore the original media asset by using the block_index field of the BLOCK_HEADER subatom as an indicator.

[0342] 3. The disclosed bitstream decoder can use multiple CHUNKED_SUBATOM subatoms to recover the complete subatoms by using associated fields in the CHUNKED_SUBATOM subatoms (such as original_subatom_id, chunk_size, chunked_segment_index, num_chunked_segments) as a connector, by using a buffer to temporarily store the CHUNKED_SUBATOM subatoms that cannot be recovered, and by using a merger to try to recover.

[0343] 4. The disclosed bitstream decoder can unify and decode media assets using bitstreams generated using different linear codes.

[0344] 5. The disclosed bitstream decoder supports various types of decoding, such as block decoding and sliding window decoding based on the coding scheme and latency requirements.

[0345] 6. The disclosed bitstream decoder can identify whether to use block code or sliding window code through various fields (such as the linear_code_type field of the BITSTREAM_HEADER subatom, the content_source_size field of the BITSTREAM_HEADER subatom, and / or the window_start_index and window_end_index of the PACKET subatom).

[0346] 7. The disclosed bitstream decoder can identify the linear code type by reading the BITSTREAM_HEADER subatom and the field linear_code_type to correctly decode the bitstream from a variety of linear coding techniques.

[0347] 8. The disclosed bitstream decoder can identify and decode different linear encoding variants of the same media asset through the same media information between different variants (e.g., the same SYNC, BITSTREAM_HEADER, ENCODER_CONTENT_INFO, and MEDIA_SEG_INFO subatoms) and different linear code information (such as different prng_seed in the BLOCK_HEADER subatom, different poly_index and different coefficients in the PACKET subatom).

[0348] 9. The disclosed bitstream decoder uses a combination of window size and generation size to identify whether a linear code is a sliding window code or a block code.

[0349] 10. The disclosed bitstream decoder can analyze channel information based on the received subatoms and report the information to the generator in order to make intelligent encoding decisions.

[0350] 11. The disclosed bitstream decoder can decrypt the encoded information (such as the coefficient_vector field of the PACKET subatom) to achieve decoding of the media asset.

[0351] 12. The disclosed bitstream decoder can verify the integrity of media assets in various ways (file verification, block verification, packet verification, etc.) based on hashing techniques (SHA2, SHA256, homomorphic hashing, etc.) and data integrity fields specified in subatomic. In particular, when using special hashing techniques such as homomorphic hashing, it can verify linear encoded packets without decoding.

[0352] Additionally, the disclosed embodiments provide:

[0353] 1. A mechanism for handling variable size packets using preliminary processing of the data prior to the linear code. Thus, the bitstream is not aware that the media is in variable size packets.

[0354] 2. A system that can achieve low-latency delivery of media streams by sending certain time-insensitive but decoding-critical subatomic signals over a reliable but latency-insensitive connection (TCP) and sending the encoded media stream over an unreliable (and low-latency) connection.

[0355] 3. A multi-path, multi-source media delivery system in which a source server generates different functionally equivalent linearly encoded copies of the same media asset for different caches, new caches may re-encode existing copies to obtain new functionally equivalent linearly encoded copies, and a recipient downloads data from multiple sources / caches to recover the original media asset;

[0356] 4. A DRM system in which the content creator encrypts the linear encoded information and provides the decryption key only to authorized parties. In this way, only authorized parties can re-encode and decode the content;

[0357] 5. A decentralized media delivery system that allows data integrity authentication, where network entities can use the data integrity field of the disclosed bitstream to verify the integrity of received data and submit fraud proofs against adversarial nodes to a regulatory manager for punishment;

[0358] 6. Enabling coded multi-source delivery of media and / or data over wireless uplink and downlink channels. Example embodiments of uplink channels include electronic news gathering applications; and

[0359] 7. An end-to-end solution is implemented that supports encoded multi-source media generated and formatted according to the disclosed invention for efficient / scalable processing on a 5G / 6G edge application server, while also allowing the same 5G / 6G edge application server to support real-time translation of encoded unicast or broadcast traffic for latency-sensitive applications.

[0360] Although this document contains many specific implementation details, these details should not be interpreted as limitations on the scope of things that may be claimed, but rather as descriptions of features that may be specific to a particular embodiment. The specific features described in the context of a separate embodiment in this specification may also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment may also be implemented in multiple embodiments individually or in any suitable sub-combination. In addition, although the features may be described above as working in certain combinations and even initially claimed, one or more features of the claimed combination may be removed from the combination in some cases, and the claimed combination may involve a variant of a sub-combination or a sub-combination. The logical flow depicted in the accompanying drawings does not require the specific order or ordered order shown to achieve the desired result. In addition, other steps may be provided from the described process, or steps may be deleted from it, and other components may be added to the described system, or other components may be removed from it. Therefore, other embodiments are within the scope of the appended claims.

Claims

1. A method comprising: At the first device: Receiving, with at least one processor, media data representing a media asset; obtaining, with the at least one processor, a first plurality of data elements including at least one of bitstream identification data, content specific coding data, and media segment data; encoding, with the at least one processor, at least a portion of the media data according to a first encoding process to obtain encoded data corresponding to the media asset; generating, with the at least one processor, a second plurality of data elements different from the first plurality of data elements based on information associated with the first encoding process; combining, with the at least one processor, the first plurality of data elements and the second plurality of data elements into one or more encoded bitstreams representing the media asset; as well as The one or more encoded bitstreams are transmitted, with the at least one processor, to one or more second devices using one or more network paths.

2. The method of claim 1, wherein: The first encoding process is a linear encoding process for obtaining encoded data corresponding to the media asset.

3. The method of claim 1, wherein: The first encoding process is a non-linear encoding process that transforms the media data into encoded data consistent with another linear encoding process.

4. The method of claim 1, wherein: The first plurality of data elements includes at least one of content specific encoding data or media segment data and bitstream identification data.

5. The method of claim 1, wherein: The data element is a data structure.

6. The method according to any one of claims 1 to 5, wherein: The first plurality of data elements includes a synchronization data element, and each of the one or more coded bit streams begins with a synchronization data element.

7. The method according to any one of claims 1 to 6, wherein: The synchronization data element includes at least one of the following: identification data identifying a corresponding one of the one or more encoded bitstreams as being generated from a common source media asset; synchronization data for synchronizing the corresponding one of the one or more encoded bitstreams with other encoded bitstreams generated from the common source media asset; as well as Lineage data for the corresponding one of the one or more coded bitstreams.

8. The method according to claim 7, wherein: The lineage data includes data identifying original or encoded symbols used to generate the encoded symbols included in the corresponding encoded bit stream, data identifying how the original or encoded symbols are combined to generate the encoded symbols included in the corresponding encoded bit stream, and data identifying encoding coefficients used to generate the encoding symbols included in the corresponding encoded bit stream.

9. The method according to any one of claims 1 to 8, further comprising: Synchronization data elements are interleaved in the one or more coded bit streams, the synchronization data elements enabling a second electronic device receiving one of the one or more coded bit streams to begin decoding from multiple points within the one of the one or more coded bit streams.

10. The method according to any one of claims 1 to 9, wherein: The first plurality of data elements or the second plurality of data elements are generated based at least in part on system data associated with the received media data.

11. The method according to any one of claims 1 to 9, wherein: Obtaining the first plurality of data elements includes generating the first plurality of data elements from the received media data or system data associated with the received media data.

12. The method according to any one of claims 1 to 11, wherein: Encoding the media data according to the first encoding process includes segmenting the media data into a plurality of original symbols and generating a plurality of encoded symbols based on the plurality of original symbols.

13. The method according to any one of claims 12, wherein: The encoded symbols are generated by linearly combining subsets of the plurality of original symbols according to a network coding technique.

14. The method according to any one of claims 1 to 13, wherein: The media data comprises a linearly encoded bit stream.

15. The method according to any one of claims 1 to 14, wherein: Obtaining a first plurality of data elements comprises: extracting metadata from the media data; and The first plurality of data elements is calculated from the extracted metadata.

16. The method according to any one of claims 1 to 15, wherein: The metadata is extracted from the media data without performing linear decoding.

17. The method according to any one of claims 1 to 16, wherein: Encoding the media data according to a first encoding process includes re-encoding linear encoding symbols in the encoded bitstream into new linear encoding symbols representing the media asset.

18. The method according to any one of claims 1 to 17, further comprising: Prior to combining, one or more data elements having a first size are split into data elements having a second size that is smaller than the first size.

19. The method according to any one of claims 1 to 18, further comprising: Adding hash data to one or more data elements of the second plurality of data elements enables at least one of verification, authentication, or integrity checking by a device receiving the one or more bitstreams.

20. The method according to any one of claims 1 to 19, further comprising: The hash data is added to one or more data elements of the second plurality of data elements, the second plurality of data elements being associated with one or more of: a file corresponding to the media asset, a block of the media asset, a fragment of the media asset, and a data packet of the media asset.

21. The method according to any one of claims 1 to 20, wherein: Combining includes inserting the directory data elements into the one or more encoded bitstreams.

22. The method according to claim 21, wherein: The directory data elements are used to help receiving devices identify the subatoms in each bitstream.

23. The method according to any one of claims 1 to 22, wherein: The one or more coded bitstreams include a first coded bitstream including data elements of a first type and a second coded bitstream not including data elements of the first type.

24. The method of any one of claims 1 to 23, wherein: The data elements or subatomic elements used to initialize the decoder on the client are not latency sensitive data.

25. The method of any one of claims 1 to 24, wherein: The bitstream includes a sync() subatom and at least one of a bitstream_header(), a block_header(), an encoder_content_info(), or a media_segment_info() subatom.

26. The method according to any one of claims 1 to 25, wherein: The data elements or subatoms carry coded data or latency sensitive data.

27. The method according to any one of claims 1 to 26, wherein: The bitstream includes the packet() subatom and the sync() subatom, but does not include at least one of the bitstream_header(), block_header(), encoder_content_info(), or media_segment_info() subatoms.

28. The method according to any one of claims 1 to 27, wherein: The transmission includes: transmitting the first encoded bit stream using a first type of communications channel; and The second encoded bit stream is transmitted using a second type of communication channel different from the first type.

29. The method according to claim 28, wherein: The second type of communication channel is at least one of a higher latency channel, a lower bandwidth channel, a reliable channel, or a channel that uses a second type of transmission protocol different than the first type of transmission protocol.

30. A method comprising: At the first device: Receiving, with at least one processor, one or more encoded bitstreams associated with a media asset; storing, with the at least one processor, the one or more encoded bitstreams in a buffer; Processing the one or more encoded bitstreams stored in the buffer comprises: extracting, with the at least one processor, a first plurality of data elements comprising at least one of bitstream identification data, content-specific encoding data, and media segment data from the one or more encoded bitstreams; and extracting, with the at least one processor, from the one or more coded bitstreams, a second plurality of data elements comprising information associated with a corresponding encoding process used to generate the one or more coded bitstreams; and Do one of the following: generating, with the at least one processor, decoded data representing the media asset based on the second plurality of data elements; and New encoded data representing the media asset is generated, with the at least one processor, based on the second plurality of data elements.

31. The method according to claim 30, wherein: Generating the decoded data includes decoding the encoded data using a corresponding decoder of a plurality of decoders according to the encoding type determined to be used to generate the encoded data.

32. The method according to claim 31, wherein: The encoding type is determined from the extracted first plurality of data elements.

33. The method according to claim 31, wherein: The encoding type is determined from the extracted second plurality of data elements.

34. The method of claim 31, wherein: The encoding type is determined from the first plurality of data elements without using the extracted second plurality of data elements.

35. The method of claim 31, wherein: The encoding type is determined from the second plurality of data elements without using the extracted first plurality of data elements.

36. The method of any one of claims 31 to 35, further comprising generating consumable data corresponding to the media asset from the decoded data and the first plurality of data elements.

37. The method of claim 33, wherein: The consumable data is compatible with an application executing on the first electronic device or a media file type supported by the application.

38. The method according to any one of claims 30 to 37, wherein: Generating decoded data also includes: processing the first type of encoded data into a second type of encoded data without performing linear encoding; and The second type of encoded data is decoded using a decoder capable of decoding the second type of encoded data.

39. The method of claim 38, wherein: Processing the coded data is performed by a code unifier that aligns coefficients or projects the coded data into a common finite field.

40. The method according to any one of claims 30 to 39, wherein: Generating new encoded data includes processing the encoded data using a corresponding re-encoder of a plurality of re-encoders in accordance with an encoding type determined to be used to generate the encoded data included in the second plurality of data elements.

41. The method of claim 40, wherein: The encoding type is determined from the extracted first plurality of data elements.

42. The method of claim 40, wherein: The encoding type is determined from the extracted second plurality of data elements.

43. The method of claim 40, wherein: The encoding type is determined from the first plurality of data elements without using the extracted second plurality of data elements.

44. The method of claim 40, wherein: The encoding type is determined from the second plurality of data elements without using the extracted first plurality of data elements.

45. The method of claim 40, wherein: Generating new encoded data includes: processing the first type of encoded data into a second type of encoded data without performing linear decoding; and The second type of encoded data is re-encoded using a re-encoder capable of encoding the second type of encoded data.

46. ​​The method according to any one of claims 30 to 45, wherein: Processing the coded data is performed by a code unifier that aligns coefficients or projects the coded data into a common finite field.

47. The method according to any one of claims 30 to 46, further comprising: The new encoded data is stored or one or more bitstreams are generated from the new encoded data.

48. The method according to any one of claims 30 to 47, further comprising: Prior to processing the one or more encoded bitstreams stored in the buffer, a first number of data elements in the first plurality of data elements or the second plurality of data elements are grouped into a smaller number of data elements having a second size greater than the first size, each data element in the first number of data elements having the first size.

49. The method of claim 48, wherein: The grouping is based on the data within the CHUNKED_SUBATOM subatom.

50. The method according to any one of claims 30 to 48, further comprising: extracting integrity data from the second plurality of data elements prior to decoding or re-encoding; as well as performing at least one of decoding or re-encoding based on determining successful verification of the corresponding data element based on the corresponding integrity data; as well as Based on determining an unsuccessful verification of the corresponding data element based on the corresponding integrity data, at least one of decoding or re-encoding is not performed.

51. The method of claim 50, wherein: The integrity data includes hash data and verification of the hash data is based on key information.

52. The method of claim 51, wherein: The key information includes at least one of a public key or a private key.

53. The method of claim 51, wherein: The key information is received from the authentication server.

54. The method according to any one of claims 51 to 53, further comprising: Prior to decoding or re-encoding, based on determining that the second plurality of data elements includes corresponding encrypted data, the corresponding encrypted data is decrypted using a decryption key.

55. The method of claim 51, wherein: The key information or hash data is associated with the bitstream_encryption_key_id.

56. The method of claim 30, wherein: Encryption is applied by the owner or creator of a media asset to one or more received encoded bitstreams associated with the media asset to prevent unauthorized access, consumption, decoding, or re-encoding.

57. The method of claim 30, further comprising: transmitting identification information associated with the first electronic device to a second electronic device different from the first electronic device; as well as A decryption key is received from the second electronic device, the decryption key enabling access, consumption, decoding, or re-encoding of the one or more encoded bitstreams associated with the media asset.

58. A method comprising: At the first electronic device: obtaining, using at least one processor, a first encoded bit stream of a first type; generating, with the at least one processor, a second encoded bitstream of a second type different from the first type; as well as The second encoded bitstream is transmitted, using the at least one processor, to one or more second electronic devices via one or more network paths.

59. The method of claim 58, wherein: The first bitstream of the first type is generated according to a first type of linear encoding protocol or standard.

60. The method of claim 58 or 59, wherein: The first bitstream or the second bitstream includes a system data packet associated with a data segment and a non-system data packet associated with the data segment.

61. The method according to any one of claims 58 to 60, wherein: The first bitstream or the second bitstream includes system data packets associated with the data segment and does not include non-system data packets associated with the data segment.

62. The method according to any one of claims 58 to 61, wherein: The first bitstream or the second bitstream includes non-system data packets associated with the data segment and does not include system data packets associated with the data segment.

63. The method according to any one of claims 59 to 62, wherein: The second bitstream of the second type is generated according to a second type of linear encoding protocol or standard.

64. The method according to any one of claims 59 to 63, wherein: The one or more network paths include more than one network, more than one network type, more than one network, more than one network transmission medium, or communication channels having different performance characteristics.

65. A method comprising: At the first electronic device: Receiving, with at least one processor, a first bitstream associated with a media asset and encoded using a first type of encoding; Based on determining that the first bitstream should be decoded using a decoder associated with a second type of encoding, processing the first bitstream into a second bitstream of a second type of linear decoding includes: extracting, with the at least one processor, data from a set of fields of the first bitstream and mapping the data to a second set of corresponding fields of the second bitstream; extracting, with the at least one processor, metadata associated with the media asset from the first bitstream; and decoding, with the at least one processor, the second bitstream into an unencoded representation of the media asset using a decoder associated with the second type of encoding; Based on determining that the first bitstream should not be decoded using a decoder associated with a second type of encoding, processing the first bitstream comprises: decoding, with the at least one processor, the first bitstream into an unencoded representation of the media asset using a decoder associated with the first type of encoding and metadata associated with the media asset; and The unencoded representation of the media asset is communicated to an application along with the extracted metadata associated with the media asset, utilizing the at least one processor.

66. A non-transitory computer-readable storage medium storing instructions that, when executed by a computing device, cause the computing device to perform the method of any one of claims 1 to 65.

67. A computing device comprising: at least one processor; as well as A memory storing instructions which, when executed by the at least one processor, cause the computing device to perform a method as claimed in any one of claims 1 to 65.

Images